The IOI Circuit: Wang, Variengien, Conmy, Shlegeris and Steinhardt (2023), replicated and extended to 2025

Read "When Mary and John went to the store, John gave a drink to" and you know the next word is Mary: the drink goes to the person who has not just been named again. Language models learn this too, and the task has a name, indirect object identification, or IOI.

Wang and coauthors did more than check that GPT-2 small gets it right. They opened the model and traced how. An attention head is a small unit inside the model that moves information from one word position to another, and GPT-2 small has 144 of them. The paper argues that just 26 of them, organized into seven classes with legible jobs, carry out IOI: some heads notice that John has appeared twice, some pass that signal forward to suppress John, and some copy Mary into the answer. A set of heads with a joint, human-readable job like this is called a circuit.

The yardstick throughout is the logit difference: how much the model's raw output score for the right name exceeds its score for the wrong one, so bigger is better and negative means the model prefers the wrong name. We replicated the paper on its own code, then asked whether the story survives on larger and newer models.

The paper's main claims

1. GPT-2 small does IOI: mean logit difference 3.56, and the right name beats the wrong one 99.3% of the time.
2. A circuit of 26 attention heads in seven classes (name movers, negative name movers, S-inhibition, induction, duplicate token, previous token, backup name movers) implements the task.
3. The circuit is faithful: keeping only those 26 heads recovers about 87% of the full model's logit difference. Faithfulness is the share of the model's task performance the circuit reproduces on its own.
4. The circuit passes completeness and minimality checks, and it explains enough to design adversarial prompts: adding an extra mention of the correct name confuses the mechanism and the logit difference collapses.

What we got when we re-ran it

We re-ran the authors' published code end to end on one GPU, all ten planned steps, and scored 0.761: of the 21 graded claims (3 appendix claims were out of the plan's scope), 12 matched, 8 were partial, and 1 did not match. All four headline claims reproduced at least qualitatively.

Table 1: The paper's headline numbers beside our re-execution of the authors' archive. Notes: baseline rows use the paper's full 100,000-prompt scale. The faithfulness ratio matches cleanly; the absolute values behind it sit 8 to 13 percent low because the shipped completeness script runs at a smaller sample size (claim C3 graded partial). The adversarial row reproduces the qualitative claim, with the collapse even stronger in our run (C22 partial). The one non-match is C21: the deliberately naive comparison circuit's faithfulness gap came out 0.574 against the paper's 0.1, though the point it supports, that the naive circuit is incomplete, did reproduce.

The code is a 2022 interactive research notebook rather than a push-button pipeline. Our replication applied 11 fixes: 7 minor (version pinning, headless plotting, batching) and 4 major, three of which were extracting analyses that exist only as notebook cells or re-enabling a completeness script whose shipped defaults skip its own searches, and one of which was a genuine off-by-one bug in the token positions of the adversarial-example code. None of the fixes touch the method: the circuit itself reproduces with the repo's own primitives.

The same analysis on today's models

We took the paper's discovery recipe, path patching, which swaps in activations from a spoiled prompt to measure how much each head contributes to the answer, made the model a parameter, and ran the full pipeline up the GPT-2 ladder (small to XL) and across the Pythia family (70M to 2.8B), with behaviour-only checks on two modern small models. Before trusting any new number we confirmed the harness reproduces the stored anchor exactly: the paper's hand-picked circuit scores 87.78% through this pipeline, matching the replication's 87.8%, and a freshly discovered circuit on GPT-2 small scores 88.6%.

The answer splits in two: the behaviour survives, the circuit does not scale.

Table 2: Baseline IOI behaviour and discovered-circuit faithfulness per model. Notes: logit differences at N=2000 prompts; faithfulness is of a freshly discovered circuit with the paper's class sizes, at N=100 for the smaller models and N=30 or 50 for the largest. GPT-2 XL's circuit covers only three of the six discoverable classes (budget), so its 28.4% is not comparable to the full rows. Where the model fails the task the faithfulness ratio is arithmetically defined but meaningless, so we report n.a. rather than a number.

The behaviour survives scale and generation, with one striking exception. Every GPT-2 size solves IOI, and modern small models solve it even better: Qwen2.5-0.5B reaches a logit difference of +5.06, meaning its score for the right name beats the wrong one by a wider margin than any GPT-2, at a third of GPT-2 medium's size. But the entire Pythia family fails: three of its four sizes score the wrong name higher on the majority of prompts, and even Pythia 1.4B manages only a weak +0.85. Since Pythia's sizes bracket GPT-2's and its architecture is newer, the difference points at training data and recipe, not scale. The paper never claimed Pythia, so this is a boundary on the behaviour, not an error in the paper.

The circuit is another story. The discovered circuit's faithfulness falls monotonically up the GPT-2 ladder, 88.6% to 41.5% to 11.5% from small to medium to large, even though every one of those models still performs the task essentially perfectly. And where the behaviour does exist across a generation gap, the method still works: Pythia 1.4B, the one Pythia that weakly does IOI, yields a discovered circuit with 87.7% faithfulness, essentially the GPT-2 small value, on a rotary-position architecture the paper never touched.

What the circuit becomes at scale

The theory update re-analyzed the per-head measurements to ask which of the paper's head classes still show strong single heads, ones whose removal moves the logit difference by at least 0.2, in each model.

Table 3: Strong single heads per class and model. Notes: n.r. means the discovery pass for that class was not run on that model for budget reasons, which is different from a true zero.

The updated theory reads: the name mover backbone is universal, the rest of the circuit is a GPT-2 small phenomenon. Name movers, the heads that copy the right name into the answer, are the only class with strong single heads at every GPT-2 size and in the working Pythia, and their negative twins are the second most persistent. S-inhibition and the upstream induction, duplicate-token and previous-token chain show strong single heads essentially only in GPT-2 small; in larger models those jobs spread across many individually weak heads. Fitting faithfulness against model size within GPT-2 gives a slope of −0.61 per tenfold increase in parameters (R² 0.75), but on four points with p = 0.13 that is a trend, not a law. Whether the circuit itself grows with model size could not be tested at all, because the discovery template fixes the head count at 26 by construction. Behaviourally, generation matters and size barely does: at matched scale, GPT-2 XL scores 3.79 where Pythia 1.4B scores 0.85.

Notes

These are the judgment calls and data limits behind our numbers, so you can decide how much weight each result deserves.

• Both extension studies are partial by design. The paper's completeness and minimality searches were deliberately not carried up the ladder (the search alone took 42 minutes on GPT-2 small), so those properties are anchor-only. GPT-2 XL's full six-class circuit was never finished (one discovery pass alone took 38 minutes), so its 28.4% is a three-class, output-side circuit and is flagged as such everywhere; Pythia 2.8B ran one discovery pass only. The faithfulness-collapse headline stands on the three complete points: small, medium, large.
• An independent adversarial QA review accepted the studies with minor issues. A launcher bug caused runaway filesystem scans that wasted wall-clock time but, per QA, corrupted no completed result. QA also verified the Pythia failure is real and not a harness bug: the models load correctly, the same pipeline scores GPT-2 highly, the failing models prefer the wrong name at below-chance rates, and Pythia 1.4B works as an internal control.
• Every extension faithfulness number is for a mechanically discovered circuit, not the paper's hand-curated one. The two agree on GPT-2 small (88.6% vs 87.8%), which is what licenses the cross-model comparison; part of the collapse at scale may still reflect the fixed-size template rather than the models.
• The scale fits rest on four points and are not statistically significant; the largest models were patched at smaller sample sizes (N = 30 or 50 against 100), so their faithfulness estimates are noisier.
• Faithfulness ratios are withheld wherever the model fails the task, since dividing by a negative baseline produces a number without meaning.
• The modern models (Qwen2.5-0.5B, SmolLM2-1.7B) were tested for behaviour only; no circuit was extracted, so they say nothing about circuit survival. OLMo-2-1B could not be loaded by the pinned library version and was skipped, and SmolLM2 stands in for the Llama and Gemma models originally sketched for this tier.
• Whether circuits grow with model size remains open: answering it needs the per-model minimality search that was descoped, a named gap rather than a silent one.

Data and model sources for the extension studies

The extension studies downloaded model weights fresh from Hugging Face between September 7 and 10, 2026. In full:

• OpenAI, GPT-2 model family: gpt2 (124M), gpt2-medium (355M), gpt2-large (774M), gpt2-xl (1.5B), retrieved from Hugging Face.
• EleutherAI, Pythia suite: EleutherAI/pythia-70m, EleutherAI/pythia-410m, EleutherAI/pythia-1.4b, EleutherAI/pythia-2.8b, retrieved from Hugging Face.
• Alibaba Cloud Qwen team, Qwen/Qwen2.5-0.5B (494M), retrieved from Hugging Face (behaviour tier only).
• Hugging Face TB, HuggingFaceTB/SmolLM2-1.7B, retrieved from Hugging Face (behaviour tier only).
• The IOI prompt distributions are the paper's own: pIOI, sentences built from the paper's mixed templates over its pool of 99 single-token names, and pABC, the matching three-distinct-name sentences used as the spoiled baseline for patching. For tokenizers other than GPT-2's, the pool was filtered to names that remain a single token: 88 of 99 for Pythia, 97 for Qwen2.5, 79 for SmolLM2.
• The authors' replication archive (the Easy-Transformer fork shipped with the paper), reused verbatim as the analysis codebase for every GPT-2 and Pythia run.

The theory update fetched no new data or models; it works entirely from the living update's stored measurements and the replication's anchors.