When Do Attention-Head Ablations Support Causal Claims? Projection-Level Confounds, Floor Effects, and Matched Controls

• Pearson r = 0.057; the natural post-projection ablation is nearly uncorrelated with corrected pre-projection ablation and selects a disjoint top-5 set.The paper identifies projection-level intervention semantics as a condition for interpreting an ablation effect.Source: §1 · p. 2
• 0.0% arithmetic accuracy and 99.5% copying accuracy show floor and near-ceiling regimes where binary accuracy can miss changes that gold-token log-probability still registers.Source: §1 · p. 2
• Spearman ρ = 0.974 across splits and Monte Carlo p = 0.001 against both matched controls support a replicated held-out effect, but task specificity is not robust.Source: §1 · p. 2
• The paper argues that a positive or null single-head result is not self-certifying; interpretation additionally requires correct projection semantics, a non-saturated continuous metric, and matched held-out controls.Source: §1 · p. 2

• The shared output projection mixes contributions from all heads, so zeroing a head-sized slice of the returned output removes a linear combination rather than one head’s contribution.The corrected intervention zeros the corresponding slice of the projection input before mixing.Source: §2 · p. 2
• The audited post-projection hook came from a pre-existing implementation, and the two methods differ in where within the block they intervene.Source: §2 · p. 2
• Prior work cautions that attention weights can be uncorrelated with feature importance and that post hoc pruning does not establish a head did nothing task-specific [2].Source: §2 · p. 2
• The paper’s narrower contribution is to isolate the projection-level confound and combine it with floor/ceiling-aware metrics and matched controls.Source: §2 · p. 3

• The study uses GPT-2 small with 12 layers and 12 heads per layer, testing synthetic induction-style copying and arithmetic as an off-target control.The 200 examples per task are split so copying heads are selected on discovery data and evaluated on held-out data; arithmetic is reserved for specificity testing.Source: §3 · p. 3
• Teacher-forced exact match is binary, while gold-token log-probability is continuous and sums token log-probabilities over the answer.Source: §3 · p. 3
• The corrected head_ablation pre-hook zeros the selected slice at the output projection input; executable tests check that other channels remain unchanged and that removing the hook restores the output.Source: §3 · p. 3
• The top five heads are selected by mean log-probability change on the discovery split, then compared with 1,000 random-head and 1,000 layer-matched control draws.The layer-matched draws preserve the candidates’ layer composition while replacing candidate identities.Source: §3 · p. 3
• Rank stability is measured with Spearman correlation across all 144 heads, and control comparisons use finite-sample-corrected one-sided Monte Carlo p-values.The pipeline records seeds and configuration and independently reproduces saved result files byte for byte.Source: §3 · pp. 3–4

4.1 RQ0: Does the ablation implementation matter?

• Pearson r = 0.057 and Spearman ρ = 0.110; naive and corrected ablation rankings are nearly uncorrelated, with no shared top-five heads.Their mean absolute per-head disagreement is 0.067 nats, comparable to the candidates’ 0.412-nat held-out effect.Source: §4.1 · p. 4

4.2 RQ1: Metric behavior at floor and near ceiling

• 0.0% arithmetic exact match and 99.5% copying exact match put GPT-2 small at a behavioral floor and near a ceiling, where accuracy can miss changes that log-probability registers.The reported mean log-probabilities are −7.44 nats on arithmetic and −0.83 nats on copying.Source: §4.2 · p. 4

4.3 RQ2: Specificity of selected-head intervention effects

• Spearman ρ = 0.974 across splits, and the top-five mean held-out effect is 0.412 nats versus random-control mean 0.029 and layer-matched mean 0.051 (Monte Carlo p = 0.001 for each).The held-out intervention effect exceeds both matched control distributions, while the separate specificity contrast is less robust.Source: §4.3 · p. 4
• Monte Carlo p = 0.078 for the primary specificity contrast is suggestive but does not clear the conventional 0.05 threshold, so generic disruption cannot be ruled out.Source: §4.3 · p. 4

Scope of the causal claim

• Monte Carlo p = 0.001 against each matched control, but specificity is only suggestive (p = 0.078), leaving GPT-2 evidence short of a task-specific causal claim.The authors place the matched-control result at the intervention-effect rung, not the mechanistic-role rung.Source: §5 · p. 5, Figure 1 · p. 5

Limitations

• Zero-ablation can partly reflect distribution shift because setting a head’s contribution to zero produces an activation the model never saw during training.Mean-ablation and resampling-based interventions are proposed as robustness checks but were not run.Source: §5 · p. 5

Model scope

• On DistilGPT2, selected heads again exceed both matched controls and the specificity control, but both tested models belong to one architecture family.The authors did not test an unrelated architecture.Source: §5 · p. 5

Task specificity

• On the pre-committed N = 500 rerun, GPT-2 specificity weakened to 0.056 nats (p = 0.274), and no further control tasks or thresholds were tested.The held-out advantage over both matched controls remained (p = 0.001 each).Source: §5 · p. 5

Limitations

• Candidate heads came from one fixed 50/50 discovery/held-out partition, so sensitivity to alternative random partitions was not tested.The discovery set selected heads; the held-out set evaluated them.Source: §5 · p. 5

Broader implication

• The authors’ checks change the interpretation: hook placement selects disjoint heads, accuracy loses information at floor or ceiling, and held-out specificity does not support a robust task-specificity claim.The broader conclusion is that a positive or null single-head ablation result is not self-certifying.Source: Figure 1 · p. 5, §5 · p. 6

Conclusion

• A head ablation is not self-certifying evidence of a specific causal role.Interpretation depends on intervention placement, metric headroom, and held-out controls.Source: §6 · p. 6
• Hook placement can ablate the wrong quantity, while accuracy is censored at a floor and coarse near a ceiling.Held-out controls distinguish a replicating effect from arbitrary disruption.Source: §6 · p. 6
• Task specificity can still fail to replicate, even when an ablation effect is held-out replicated against controls.The conclusion separates evidence for an intervention effect from evidence for a specific causal role.Source: §6 · p. 6

Supplementary figure

• Pearson r = 0.057: per-head effects from correct and naive ablations are nearly uncorrelated.The comparison covers all 144 heads on the controlled copying-task discovery split.Source: Figure 2 · p. 7
• The plotted quantities are per-head ∆log-prob under pre-projection versus post-projection ablation on the controlled copying-task discovery split.Figure 2 includes all 144 heads.Source: Figure 2 · p. 7, A Supplementary figure: RQ0 per-head… · p. 7
• A valid replication would place points near y = x; the observed methods instead show near-zero correlation.The plotted comparison corresponds to the summary statistics in Section 4.1.Source: Figure 2 · p. 7, A Supplementary figure: RQ0 per-head… · p. 7

Replication on DistilGPT2

• The identical pipeline was rerun on DistilGPT2 with only the model changed.The model has 6 layers, 12 heads per layer, and 72 heads total.Source: B Replication on a second model… · p. 7
• The DistilGPT2 protocol again separated a held-out-replicated effect from matched-control distributions.The selected heads had different indices from GPT-2 small’s.Source: B Replication on a second model… · p. 7
• Different model parameter sets mean this replication does not show that the same head performs the same role across models.The authors present it as another data point for the methodological claim, not evidence for a universal induction circuit.Source: B Replication on a second model… · p. 7

C Reproducibility details

• The experiment records include device and software versions, random seeds, split counts, bootstrap and control-draw counts, per-head effects, and baseline log-probabilities.Exact model revisions and hashes are recorded separately in the provenance document.Source: C Reproducibility details · p. 7
• DistilGPT2 reproduces the intervention-semantic and matched-control findings, but its copying accuracy is 90.0% and its specificity comparison gives Monte Carlo p = 0.001.The arithmetic floor recurs, while GPT-2’s near-ceiling copying example does not replicate.Source: Table 2 · p. 8
• Monte Carlo p = 0.001 for the selected candidate versus both random-head and layer-matched controls, and for the specificity comparison.The results are reported for held-out effects and specificity.Source: C Reproducibility details · p. 8
• The appendix makes the pipeline, figure script, README, and result files available, while task data are generated in memory rather than shipped.A claim-to-evidence map links each quantitative statement to its source field in the result files.Source: C Reproducibility details · p. 8