template-lens-qwen3-4b

Template lens for Qwen3-4B

An implementation of the template lens from appendix A.9.1 of Gurnee et al. (2026), Verbalizable Representations Form a Global Workspace in Language Models (transformer-circuits.pub, arXiv:2607.15495), fitted for Qwen/Qwen3-4B.

The template lens reads a residual-stream direction against a vocabulary of whole words rather than single tokens, which is what lets it name concepts the logit lens and J-lens can only show as fragments — blackmail instead of black, Equestrian instead of E/quest/rian.

Status: this repository currently holds the passage corpus and vocabulary only. The fitted lens itself — per-layer template tensors and the whitening moments — is still being built and will be added here. Nothing in this repo is a lens yet; it is what a lens gets fitted from.

What the method does

For a word w at layer l, collect passages each ending at the exact point where w would naturally come next, run the model over them, and average the residual stream at the final position to get mu_w(l). Then centre and whiten:

t_w(l) = (Sigma_l + lambda*I)^-1 (mu_w(l) - mu(l))

t_w is the linear discriminant direction separating "about to say w" from "not about to say w", and is used exactly like a J-lens vector minus the unembedding step: project a direction onto it and rank over the vocabulary.

The whitening is the method. Unwhitened, mu_w - mu is an ordinary difference-in-means steering vector; multiplying by the inverse activation covariance is what puts it in the J-lens's metric, via Stein's lemma (E[grad g(x)] = Sigma^-1 E[g(x)(x - mu)], derived in the paper's appendix).

Contents

passages.jsonl is one JSON object per line:

{"w": "equestrian", "s": "equestrian", "t": "...requiring every young officer to master basic"}

w is the target word (lowercased, the vocabulary key), s its surface form, and t the passage — which ends exactly where w would follow, and never contains w.

How the passages were made

Generated with DeepSeek V4 Pro via OpenRouter, ~100 per word, at a total cost of roughly $50. The paper uses Claude and gives no passage count beyond "a few hundred forward passes per word".

The generator was chosen by a bake-off against Gemini 2.5 Flash Lite and a corpus-mined baseline, scored on the paper's own next-word diagnostic. DeepSeek won by ~2.2x at every passage count and was the only generator to cover the whole vocabulary.

Two constraints are enforced on every passage, because both change what the template measures:

• The passage never contains the target word. A context that has already said equestrian makes the continuation a copy rather than a prediction, and its activation reflects a primed model instead of one poised to say the word for the first time.
• Passage counts are balanced (~99/word). The paper describes mu as the mean of all other words' vectors in prose, and as the mean over the same passage set in the formula gloss. Under balanced counts those are the same object, so the ambiguity does not arise. Where counts do vary, both moments should be taken with per-word weights 1/P_w, keeping mu and Sigma over one reweighted population — mixing estimators breaks both the Stein and linear-discriminant readings.

Vocabulary

12,744 words: the top 12,700 English words by frequency (wordfreq), plus 44 content words from the evaluation cohort's topics that fell below that cut. 1,609 entries (13%) are multi-token under the Qwen3-4B tokenizer — those are the entries the logit lens cannot express at all.

The paper's own ~12,700-word list is not published, so this is a reconstruction rather than a copy.

Leakage

The vocabulary is built topic-blind: it comes from a published frequency list, and topic words enter only if they clear the frequency cut or are added as individual common words. equestrian is present because it is a common English word (Zipf 3.2), and the lens must still find it among 12,744 candidates.

Full topic strings are a different matter and are kept in ceiling_vocab.json, never merged into the vocabulary. Templates for them would let a readout surface an exact answer, which is straightforward leakage in open-ended evaluation. They exist only as a ceiling — separating "the vocabulary could not express the answer" from "the lens could not find it", which are different failures.

Intended use and limitations

Intended for interpretability research: fitting template-lens directions for Qwen3-4B, and as an N=1 non-parametric slice of the dictionary an oracle-lens reconstructor learns to amortise.

• English only, and frequency-weighted toward common words.
• A topic can only be read out if its words are in the vocabulary. Coverage against the evaluation cohort is in coverage.json.
• The method has known pathologies the paper reports and this corpus does not fix: readouts can "skip to the answer" in early layers (a tuned-lens pathology), and a small set of words top readouts regardless of relevance. On the latter, the cause appears to be template norm — rare words have noisy mu_w, so ||t_w|| inflates and wins the dot product against anything. Scoring by cosine rather than raw projection addresses it at the source.
• Passages are model-generated and carry that model's stylistic and factual biases.

Citation

The method is from:

@article{gurnee2026workspace,
  title  = {Verbalizable Representations Form a Global Workspace in Language Models},
  author = {Gurnee, Wes and Sofroniew, Nicholas and Pearce, Adam and Piotrowski, Mateusz
            and Kauvar, Isaac and Chen, Runjin and Soligo, Anna and Bogdan, Paul
            and Ong, Euan and Wang, Rowan and Thompson, Ben and Abrahams, David
            and Kantamneni, Subhash and Ameisen, Emmanuel and Batson, Joshua
            and Lindsey, Jack},
  journal = {arXiv preprint arXiv:2607.15495},
  year    = {2026}
}

This repository is an independent reimplementation and is not affiliated with the authors.