PaperlayerView exact revision on arXiv

Source-linked AI summary

From Error Prediction to Causal Influence: Auditing SAE Features in Multiple-Choice QA

Aria Nourbakhsh, Adelaide Danilov, Oleksandr Marchenko Breneur, Salima Lamsiyah

arXiv:2610.38067v1cs.CLcs.AI

TL;DR

SAE features associated with errors may track uncertainty or format artifacts rather than causally influence answer choice, so their predictive association alone does not establish a causal mechanism. The paper introduces a pipeline for discovering behaviorally predictive SAE features across eight datasets and six token positions, filtering features correlated with datasets, prompts, answer letters, and other artifacts. Across eight datasets, selected question- and option-span features shift correct-versus-wrong margins more and yield higher wrong-to-correct rates than random-active controls, but flips are rare and effects are unstable across equivalent prompts and answer orders.

  • Problem

    SAE features associated with errors may track uncertainty or format artifacts rather than causally influence answer choice, so their predictive association alone does not establish a causal mechanism.

    Source: §1 · p. 1

  • Method

    The paper introduces a pipeline for discovering behaviorally predictive SAE features across eight datasets and six token positions, filtering features correlated with datasets, prompts, answer letters, and other artifacts.

    Source: §1 · p. 2

  • Results

    Across eight datasets, selected question- and option-span features shift correct-versus-wrong margins more and yield higher wrong-to-correct rates than random-active controls, but flips are rare and effects are unstable across equivalent prompts and answer orders.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The selected features influence answer preference but do not provide dependable error repair.

    Source: §6 · p. 8

    The analysis focuses on multiple-choice errors; extending it to free-form wrong answers remains future work.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Sparse autoencoders (SAEs) can predict internal features associated with model errors, but association alone does not establish causal influence. We study this distinction in multiple-choice question answering, where answer preference can be measured by the logit margin between the correct answer and the competing choices. Across eight datasets, we analyze Gemma~3 1B and 4B with Gemma Scope SAEs. We propose a pipeline to filter out the features that have high correlation with format and content artifacts, such as prompt and answer permutation, and evaluate the remaining ones with causal feature interventions. Our results show that SAE activations contain predictive signals about wrongness and uncertainty across models and datasets. After filtering artifact-contaminated features, selected features for causal intervention at question and option spans produce larger absolute margin shifts and a higher wrong-to-correct rate than randomly selected features, indicating greater causal leverage. However, answer flips are rare, and we do not observe stable effect directions across meaning-preserving changes in prompt format and answer order. An adapted HalluSAE baseline produces larger final-token margin changes, but relies on substantially contaminated features and does not significantly improve correction rates. Therefore, the identified features reflect context-dependent answer competition rather than the mechanism of incorrect answer errors.

1 Introduction

The paper asks whether SAE features associated with forced-choice errors actually influence answer preferences or correct errors. It combines artifact-aware feature selection with interventions, finding predictive signals but rare, unstable corrections and substantial artifact contamination in the adapted HalluSAE baseline.

1 Introduction

  • The paper asks whether SAE activations predict errors, interventions change answer preferences, and those changes correct errors in forced-choice evaluation.It studies two instruction-tuned Gemma 3 models using behavioral probes, artifact-risk screening, contrastive DLA, and instance interventions.Source: §1 · p. 1
  • SAE activations predict forced-choice errors, but many features are entangled with format, topic, or answer-choice structure.The paper frames errors as potentially reflecting misconceptions, uncertainty, competition, or surface-form bias.Source: §1 · p. 1
  • The discovery pipeline covers eight datasets and six token positions, screening predictive SAE features for dataset, prompt, answer-letter, and other artifact contamination.It selects features for answer-choice prediction and causal evaluation.Source: §1 · p. 2
  • Question- and option-span interventions produce larger margin changes and higher wrong-to-correct rates than random-active controls, but answer flips are rare and effect directions are unstable.Instability occurs across steering methods, prompt templates, and answer permutations.Source: §1 · p. 2
  • The adapted HalluSAE baseline is predictive and produces large final-token margin shifts, yet 95.5% of its nonzero feature units fail the artifact-risk criterion.The paper argues predictive performance and logit influence alone do not establish artifact-independent causal mechanisms.Source: §1 · p. 2

2 Related Work

Prior work uses SAEs to analyze and steer language-model features, while mechanistic studies distinguish answer content from answer-symbol selection. This paper asks whether artifact-screened, predictive SAE features support reliable error correction under interventions and prompt changes.

Sparse Autoencoders for Language Model Interpretability

  • SAEs reconstruct dense model activations from a larger sparse set of latent features, with learned decoder directions and feature activations.Prior work has improved reconstruction and sparsity tradeoffs through larger-scale training and architectural variants.Source: §2 · p. 2
  • Prior studies use SAE features for steering and show sparse directions can sometimes serve as actionable control handles.Examples include refusal steering, in-context-learning improvement, and knowledge-selection control.Source: §2 · p. 2

Mechanistic analyses of multiple-choice answering

  • Mechanistic studies distinguish selecting answer content from mapping it to an output symbol, motivating separation of knowledge-related behavior from format-dependent processing.This study tests whether artifact-screened predictive SAE features support reliable error correction under interventions and prompt-format changes.Source: §2 · p. 2

Evaluation, Artifacts, and Controls

  • Prior evaluation work emphasizes task-specific benchmarks, controls, and caution about whether sparse features represent downstream concepts reliably.Feature-explanation falsification work highlights the risk that a single description overstates what a feature represents.Source: §2 · p. 2
  • The paper uses ablation and steering as diagnostic tests, with artifact controls, because predictive features may be non-causal, polysemantic, or artifact-driven.Its focus is whether features associated with wrong forced-choice answers affect correct-versus-wrong logit preference.Source: §2 · p. 3

3 Methodology

The methodology discovers and screens SAE features using behavioral measurements, prediction probes, statistical audits, artifact controls, and contrastive direct logit attribution. It then intervenes on selected features and measures changes in answer preference.

3 Methodology

  • The two-phase pipeline uses Phase 1 for feature discovery and selection, then Phase 2 for causal interventions.Figure 1 depicts behavioral measurements and SAE activations feeding predictive, statistical, and artifact-aware screening before contrastive DLA prioritizes candidates.Source: §3 · p. 3, Figure 1 · p. 4

3.1 Phase 1

  • Phase 1 pools SAE activations over token spans and tests their associations with correctness, confidence, margin, entropy, and answer logits.Candidate-ranking probes and a separate statistical audit evaluate binary and continuous behavioral targets.Source: §3.1 · p. 3
  • Artifact controls measure feature dependence on answer labels, prompt templates, answer-order permutations, and question or answer-text properties.The pipeline combines these controls into artifact risk, then applies density, support, dataset-coverage, sign-consistency, and shortlist filters.Source: §3.1 · p. 3
  • After artifact-aware selection, contrastive DLA measures whether a feature’s decoder direction aligns with the correct-versus-strongest-wrong logit contrast.Positive values support the margin, negative values support the strongest wrong answer, and magnitude measures alignment strength.Source: §3.1 · p. 4

3.2 Phase 2

  • Phase 2 applies decoder-vector updates to selected SAE features, testing zero replacement, mean replacement, and upward steering.Behavioral measurements include answer-logit and probability changes, entropy, and the correct-versus-strongest-wrong margin; positive margin change favors the correct answer.Source: §3.2 · p. 4

4 Experiments

Experiments span eight multiple-choice datasets, multiple prompt forms and answer permutations, three response modes, and Gemma 3 1B and 4B with Gemma Scope SAEs. Feature selection and interventions use separate discovery, validation, and test data.

4.1 Datasets and splits

  • Eight multiple-choice datasets are sampled, length-filtered, and rendered in a common lettered-choice interface, with original examples split approximately 60/20/20 for discovery, validation, and test.The datasets cover commonsense, truthfulness, academic knowledge, science, hallucination, factuality, medical reasoning, and claim verification tasks.Source: §4.1 · p. 5

4.2 Prompting and activation positions

  • Experiments vary three prompt templates, three answer-letter permutations, and forced-letter, reason-first, and letter-then-reason response modes.Margins, answer probabilities, and entropy are computed from answer-letter logits; SAE activations are extracted at six positions.Source: §4.2 · p. 5, Table 2 · p. 5

4.3 Models and SAEs

  • The study analyzes instruction-tuned Gemma 3 1B and 4B using corresponding Gemma Scope 2 residual-stream SAEs.It uses layers 17 and 22 for 1B and layers 22 and 29 for 4B, with 16K- and 65K-width SAEs at each layer.Source: §4.3 · p. 5

4.4 Feature Selection

  • Feature selection forms separate non-oracle and oracle candidate pools, aggregates probe evidence, penalizes artifact risk, and applies eligibility criteria before DLA filtering.The resulting shortlists contain 55 non-oracle and 22 oracle feature units, and interventions are evaluated only on test instances.Source: §4.4 · pp. 5–6

4.5 Intervention Setup

  • Interventions target active features on selected test instances, editing every token in the relevant final-prompt, question, option, or oracle-choice span.Each intervention is compared with a no-intervention pass; generated-reasoning spans are not intervention targets in this fixed-span setup.Source: §4.5 · p. 6

4.6 Baselines

  • Random-active controls use eligible active features, while the adapted HalluSAE baseline selects features at final-prompt-token positions and tests matched feature counts.The adaptation uses specified layers rather than reproducing HalluSAE’s all-layer localization stage.Source: §4.6 · p. 6

5 Results and Discussion

SAE features predict error-related outcomes, but selected interventions show a limited, context-sensitive advantage over random controls rather than a general correction effect. HalluSAE produces larger final-token margin shifts, yet most of its features fail the artifact-risk criterion.

5.1 MCQA Performance

  • Macro-averaged accuracy was 43.8% for Gemma 1B and 58.6% for Gemma 4B; WHC rates were 52.0% and 39.3%, respectively.The 4B model had higher accuracy and a lower WHC rate on every dataset.Source: §5.1 · p. 6

5.2 Statistical Audit

  • LMvHM and WHC had the highest mean AUROC (.739), while WLM had the highest F1 (.805); median R² was .285 for entropy and .117 for margin.The audit covered final-prompt-token test performance over 192 runs.Source: §5.2 · pp. 6–7, Table 3 · p. 7

5.3 Feature Selection

  • After clean filters, 2,205 non-oracle and 1,100 oracle units remained; DLA criteria yielded 55 non-oracle and 22 oracle units for intervention.The top-300 pools had mean artifact risks of .712 and .724, respectively.Source: §5.3 · p. 7

5.4 Causal Interventions

  • Intervention effects were mixed and sparse: only 51.8% of feature–prompt pairs increased monotonically, and sign-agreement differences across matched permutations and templates had intervals including zero.The selected–random margin-change difference remained distinguishable at option spans after adjustment, but not at question spans or pooled question/option spans.Source: §5.4 · pp. 7–8
  • .187 vs. .146 mean |∆m| and .189% vs. .058% W2C at Q/O favored selected over random-active features; at FP, |∆m| was .154 vs. .237.The pooled Q/O selected-minus-random differences were +.041 [.014, .067] for mean |∆m| and +.132 percentage points [+.024, +.257] for W2C; both intervals were above zero.Source: §5.4 · p. 7, Table 5 · p. 8

5.5 HalluSAE Baseline Comparison

  • HalluSAE achieved mean test AUROC/F1 of .775/.674 versus .761/.659 for random-feature controls, but 95.5% of its nonzero units failed the artifact-risk criterion.Its final-token margin shifts were larger, but its W2C rate was not detectably higher than either baseline, and its DLA-aligned margin interval included zero.Source: §5.5 · p. 8

6 Conclusion

Across eight datasets, SAE activations predict correctness, confidence, and uncertainty, while filtered question- and option-span features shift answer margins more than random-active controls. Artifact entanglement, rare flips, and unstable effects limit these features’ value for dependable error repair.

6 Conclusion

  • Across eight datasets, SAE activations predict correctness, confidence, and uncertainty, while selected question- and option-span features shift margins more than random-active controls and correct more initially wrong events.Answer flips remain rare, and effects are unstable across equivalent prompt formats and answer orders.Source: §6 · p. 8
  • Although every final candidate passes the prespecified gate, 26 of 55 non-oracle units retain an artifact or semantic tag, and 95.5% of HalluSAE nonzero units fail the artifact threshold.HalluSAE produces larger final-token margin shifts but does not improve correction rates; neither prediction nor large shifts establish clean error mechanisms.Source: §6 · p. 8
  • The findings are consistent with sparsity favoring low-dimensional surface cues while higher-level behavior remains distributed across features.Selected features influence answer preference without providing dependable error repair.Source: §6 · p. 8

Limitations

The analysis is bounded by the models, SAE layers, and multiple-choice setting studied, as well as by methodological constraints on controls and comparisons.

Limitations

  • Results cover Gemma 3 1B and 4B at four mid-to-late residual layers with 16K/65K Gemma Scope SAEs; larger models may behave differently, with substantially higher analysis and intervention costs.Source: Limitations · p. 9
  • Random-active controls are not exactly matched on activation frequency or edit magnitude, perturbation-adjusted analyses omit decoder norm, and bootstrap intervals condition on the fixed candidate shortlist.The HalluSAE comparison is a matched-layer adaptation rather than a repeat of its all-layer localization stage.Source: Limitations · p. 9
  • The study focuses on multiple-choice errors, whose constrained answer space enables precise correct-versus-wrong logit measurement and controlled feature interventions; free-form errors remain future work.Source: Limitations · p. 9

Use of AI Assistance

The authors used AI-assisted tools for code completion and language editing, while retaining responsibility for the scientific content, analyses, and conclusions.

Use of AI Assistance

  • AI-assisted tools supported code completion and language editing, while the authors take full responsibility for the work’s scientific content, analyses, and conclusions.Source: Use of AI Assistance · p. 9

A Feature Ranking, Artifact Controls, and Clean-Feature Filters

Feature selection combines predictive evidence with controls for measured artifacts and eligibility filters; DLA then identifies shortlisted features with strong, consistent answer-contrast alignment. The resulting selection includes directional and mixed-direction diagnostic categories.

A Feature Ranking, Artifact Controls, and Clean-Feature Filters

  • Predictive evidence is weighted and aggregated across datasets, targets, response modes, and token positions before eligibility filters determine the highest-scoring feature units.Evidence weights depend on probe validation quality and coefficient rank; binary probes at or below chance receive zero weight, and regression probes with negative R2 contribute zero.Source: A Feature Ranking, Artifact Controls,… · pp. 11–12
  • Artifact risk combines measured associations with answer-letter, prompt-template, permutation, and topic/text controls, with any one strong measured confound able to raise the relative score.The score is relative, not a probability of artifact contamination.Source: A Feature Ranking, Artifact Controls,… · p. 12
  • Eligible units must meet artifact-risk, activation-frequency, support, dataset-coverage, and coefficient-consistency requirements before the top-scoring units are retained.The stated thresholds include maximum artifact risk at most 0.80, maximum activation frequency below 0.95, activity on at least 100 instances in one setting, evidence from at least two datasets, and mean coefficient consistency at least 0.60.Source: A Feature Ranking, Artifact Controls,… · p. 12

B DLA-Based Candidate Selection

  • DLA selects correct- or top-wrong-directed features using alignment magnitude, mean direction, cross-setting consistency, and shortlist-rank criteria; large features meeting neither rule are retained as mixed-direction diagnostics.Directional features also receive an artifact-control label under specified risk, description, or density conditions.Source: B DLA-Based Candidate Selection · p. 13

C Forced-Letter Behavioral Results

The forced-letter behavioral results report test accuracy and wrong high-confidence rate over rendered prompt instances. The wrong high-confidence rate captures instances that are both incorrect and strongly favor a wrong letter.

C Forced-Letter Behavioral Results

  • Table 6 reports forced-letter test accuracy and wrong high-confidence rate over rendered prompt instances.Its N counts unique original examples, not independent rendered prompts.Source: Table 6 · p. 13, C Forced-Letter Behavioral Results · p. 13
  • Wrong high-confidence rate is the fraction of all instances that are wrong and assign at least .70 probability to the strongest wrong letter.Metrics are averaged over prompt-template and answer-permutation instances.Source: C Forced-Letter Behavioral Results · p. 13

D Statistical Audit Details

The statistical audit evaluates prediction of four binary behavioral targets and two continuous outcomes across activation positions and datasets. Its results vary by position and outcome, while the appendix specifies the audit, prompt construction, and HalluSAE comparison scope.

D Statistical Audit Details

  • The audit fits six targets on discovery instances and evaluates on test partitions, using binary classification and regression probes alongside correlation, bootstrap, and mutual-information analyses.The targets comprise four binary outcomes and continuous margin and entropy; Table 7 defines them.Source: D Statistical Audit Details · p. 13, Table 7 · p. 14
  • Across positions, question-span models are weakest overall, while GR and oracle spans are stronger for several binary targets; regression performance is heterogeneous across positions.Median R2 is negative at Q for both regression targets and positive at FP, O, and GR; run coverage is 192 each at FP, Q, O, C, and TW, and 128 at GR.Source: D Statistical Audit Details · p. 14, Table 8 · p. 14, Table 9 · p. 14

E Statistical Audit by Dataset and Position

F Prompt Templates

  • Answer handling differs by response mode: forced-letter reads next-token answer-letter distributions, generated modes decode deterministically, and answer-order permutations preserve strings while reassigning answer texts to letters.Source: F Prompt Templates · p. 14

G HalluSAE Predictive and Artifact Audit

  • Table 13 audits adapted HalluSAE predictive performance, artifact-risk associations, and overlap with the paper’s shortlists and intervention candidates.The supplied passage identifies the table’s scope but does not report its results.Source: G HalluSAE Predictive and Artifact Audit · p. 15

H Detailed Causal Intervention Results

The intervention analysis compares selected and random-active features across intervention variants and measures margin shifts under steering. Selected question/option features show the clearest aggregate strength response, but that response is not consistently monotonic.

H Detailed Causal Intervention Results

  • Intervention outcomes distinguish events from unique feature–instance pairs and compare HalluSAE, selected, and random-active features, with oracle C/TW outcomes also reported.Evaluated variants include zero and mean replacement and steering at one, two, and four feature standard deviations; C and TW have no random-active counterpart.Source: H Detailed Causal Intervention Results · p. 15, Table 14 · p. 17, Table 15 · p. 17
  • Only 51.8% of selected Q/O pairs increase monotonically across steering strengths, compared with 46.6% for random Q/O and 48.6% for oracle C/TW.Figure 3 reports mean absolute margin shifts for matched pairs at +1σ, +2σ, and +4σ, with 95% feature-unit bootstrap intervals.Source: H Detailed Causal Intervention Results · p. 15, Figure 3 · p. 15
  • The size-adjusted selected–random difference in mean |∆m| is estimated by feature-equal regression controlling for measured intervention displacement and other event characteristics.The adjustment includes span length, intervention variant, baseline margin and correctness, activation frequency, token position, dataset, and model configuration.Source: H Detailed Causal Intervention Results · p. 15, Table 17 · p. 17

I Model Size and Budget

  • The appendix reports a compact prompt-construction table and states that experiments and analysis used approximately 500 GPU hours on 20 Nvidia V100 SXM2 GPUs.The table describes templates and response modes, with option lines and label sets shortened for datasets with fewer choices.Source: I Model Size and Budget · p. 15, Table 12 · p. 16

Related papers

Loading 2610.38067v1…