Before It Codes
Catching Qwen3.5-4B deciding its algorithm before it writes any code.
Explanation
Qwen3.5-4B writes one deterministic Python solution on an H100 while a pre-fitted Jacobian lens reads its residual stream. If the method you picked reaches the top 25 of the lens readout at any prompt token, layers 11 to 26, that counts as a plan signal. A size-matched shuffled method runs as the control.
In a frozen 75 problem study, this signal preceded the code in 55 of 75 transcripts against 10 of 75 for the control, a median of 87 tokens ahead. The readout is observational, it does not prove the thought caused the code.
References
- Gurnee et al. (2026). “Verbalizable Representations Form a Global Workspace in Language Models.” Transformer Circuits Thread. The Jacobian lens this artifact is built around.
- Jacobian-lens implementation: anthropics/jacobian-lens on GitHub.
- Pre-fitted lens weights: neuronpedia/jacobian-lens (n = 1000, WikiText) on Hugging Face. No fitting, no training.
- Model: Qwen3.5-4B.