particle-qwen35-ngram-research

Does learned n-gram memory help a small Qwen model?

A learned lookup-memory adapter adds 273 MB to Qwen3.5-0.8B. We compare it with the unmodified model, same-data LoRA, and stock Qwen3.5-2B.

Results

50 questions per model per benchmark; thinking enabled. Scores are correct answers, using the same questions for each model.

Takeaways

• A tradeoff, not a general upgrade: memory scores higher than stock 0.8B on coding (9 vs 4) and instructions (32 vs 27), nearly the same on reasoning (18 vs 17), and lower on math (16 vs 25 under strict scoring).
• The gains need confirmation. Memory changes which questions succeed; the paired intervals below show the uncertainty in the net gains.
• It does not replace a 2B model in these tests. Stock 2B leads all four panels. Memory-equipped 0.8B uses less storage: 2.04 GB versus 4.57 GB.
• Memory exceeds same-data LoRA on three of four panels, but its adapter is 273 MB versus 22 MB. Training exposure is matched; parameter counts are not.

What changed versus stock 0.8B?

Gained/lost counts compare the same questions. Intervals covering zero leave the direction uncertain; they cover question sampling, not training or decoding-seed variation.

Math scoring matters. Strict GSM8K extraction requires ####; the fixed secondary extractor takes the last numeric match. On the same answers:

The secondary rule changes the size of the math gap. Neither extraction rule is selected per model.

Storage and speed

Total files include vision/tokenizer components. These are disk sizes, not VRAM.

Matched inference work on NVIDIA GeForce RTX 5090: batch 1, 1,536-token prompt, 128 forced continuation tokens; medians of 3 repetitions after warmup.

Memory reduced decode throughput by 4.3–6.1% versus stock 0.8B across the tested batch/prefix sizes.

This measures equal-length forward/cache work, excluding sampling and serving. LoRA is unmerged. Full timings, including batch 5.

Method and limits

The backbone stays frozen. Both adapters train on the same 512 conversation packets (589,624 input tokens; 196,350 scored positions), using full-vocabulary distillation from dense Qwen3.5-4B. Memory uses about 67M table parameters plus a learned reader; LoRA uses rank 8. The checkpoint is fixed before evaluation.

One sampled answer per question, with up to 32,768 output tokens and 65,536 context. No training or decoding tuning on these evaluation questions. These are benchmark subsets, not full leaderboard scores; pretraining exposure is unknown. IFEval requires all checked instructions to pass. Unfinished answers receive no credit. MMLU-Pro uses fixed Markdown-normalized extraction; LoRA scores 20/50 under official strict extraction versus 19/50 normalized. Both scores are retained.

Downloads are experimental adapter sidecars and require the base model.

Detailed results and paired comparisons · Per-answer evidence · Reproduce / load the adapters · Data sources · Attribution