PaperlayerView exact revision on arXiv

Source-linked AI summary

StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions

Chao Gao, Haijiang Liu, Qiyuan Li, Caicai Guo, Frank van Harmelen, Jinguang Gu

arXiv:2609.01081v1cs.CLcs.AI

TL;DR

The paper asks whether inconsistent answers under support- and elimination-oriented prompts reflect different internal representations. It compares paired framings through an untrained [STATE] interface and activation substitution, finding separable intermediate-layer states whose exchange changes predictions and improves cross-framing agreement, while mean-difference steering is more bounded than matched contrastive addition under the tested protocol.

  • Problem

    Support- and elimination-oriented prompts can yield inconsistent answers despite encoding the same decision objective, raising whether they induce distinguishable internal representations.

  • Method

    StateSwap uses paired minimally varied prompts, an untrained [STATE] residual-stream interface, and cross-framing activation substitution to probe representation and behavioral relevance.

  • Results

    Across two evaluated models, [STATE] activations are separable, substitution changes predictions and is associated with higher accuracy and cross-framing agreement, and mean-difference directions have lower layer-wise variability than matched contrastive directions.

  • Takeaways & Limitations

    The findings provide intervention-based evidence that framing-conditioned intermediate activations are behaviorally relevant within the tested multiple-choice protocol.

  • Takeaways & Limitations

    The evaluation uses four-option multiple-choice questions with deterministic greedy decoding and does not test sampling-based decoding, longer multi-step generation, or open-ended tasks.

Abstract

from arXiv · show

Large language models often answer the same multiple-choice question inconsistently when it is posed under support-oriented and elimination-oriented framings. We investigate whether these discrepancies arise from different internal representations induced by the two framings. We introduce a dual-framing protocol with minimally varied prompts that use either support- or elimination-oriented framing while keeping the evaluation target fixed. To probe the internal computation, we append an untrained special token, [STATE], and treat its residual-stream activation as an intervention interface. Across both models, the two framings induce separable [STATE] activations concentrated in intermediate layers. Swapping these activations between paired prompts systematically changes predictions and improves cross-framing agreement, providing intervention-based evidence that the activations are behaviorally relevant. Beyond instance-level substitution, mean-difference steering directions derived from the dual-framing contrast exhibit more bounded layer-wise responses than matched contrastive activation addition directions under the evaluated protocol.

1 Introduction

StateSwap asks whether logically equivalent support- and elimination-oriented framings induce distinguishable internal representations and whether exchanging those representations affects predictions. It identifies separable [STATE] activations and reports prediction changes associated with higher accuracy and cross-framing agreement, alongside more bounded responses from mean-difference steering directions.

  • Process-of-elimination prompting can improve or impair LLM accuracy, motivating tests of whether equivalent SUP and ELIM framings induce different internal representations.
  • The study uses controlled, deterministic paired prompts to separate framing-related activation effects from decoding variability.
  • An untrained [STATE] token serves as an interface for analyzing framing-sensitive hidden states and testing activation substitution while textual inputs remain fixed.
  • SUP and ELIM prompts induce distinguishable [STATE] representations despite encoding the same underlying decision problem.
  • Activation substitution changes predictions and is associated with higher accuracy and cross-framing agreement in the evaluated settings.
  • Mean-difference steering directions show lower layer-wise variability than matched contrastive activation addition directions under the evaluated protocol.

2 Related Work

The paper builds on mechanistic interpretability, activation intervention, steering, and localized representation-interface research. It adapts these perspectives to test whether semantically equivalent framings produce distinguishable decision-related states that can be aligned by intervention.

  • Prior work studies LLM reasoning through prompting, fine-tuning, symbolic tools, collaboration, verification, sparse autoencoders, causal tracing, and circuit localization.
  • Activation patching and inference-time intervention motivate testing whether localized computations and activation-space directions influence model behavior.
  • Prompt tuning, prefix tuning, and special-token interfaces provide precedents for controlling or reading sequence-level representations without changing base parameters.
  • StateSwap uses a fixed, untrained token as a residual-stream read–write interface for comparing and aligning framing-dependent representations.

3 Methodology

The methodology constructs aligned SUP and ELIM prompts ending in an untrained [STATE] token, localizes framing-sensitive layers, and substitutes donor states across the paired prompts. The intervention changes only selected full-dimensional residual states before normal generation continues.

  • 3.1 Input Construction and State Interface: Each prompt appends [STATE] at the final position, whose residual-stream activation is tracked across Transformer layers.
  • 3.1 Input Construction and State Interface: Because [STATE] lacks pretrained lexical semantics, its contextualized representation functions as a single-token read–write interface for inference-time intervention.
  • 3.1 Input Construction and State Interface: SUP and ELIM prompt pairs differ only in whether they support correct options or eliminate incorrect options, with matched [STATE] positions ensured by padding.
  • 3.2 State Substitution and Prediction Sensitivity: The selected contiguous layer region W receives cached [STATE] states from the complementary framing for each question and intervention direction.
  • 3.2 State Substitution and Prediction Sensitivity: Substitution overwrites only the post-block [STATE] state, while other vectors remain unchanged at the intervention point and downstream computation proceeds normally.
  • 3.3 Layer Localization: Candidate intervention regions are localized by scanning feature windows, computing paired Cohen’s d statistics, and merging adjacent high-intensity layers.

4 Experimental Setup and Evaluation Protocol

Experiments evaluate StateSwap on two reasoning-focused multiple-choice benchmarks and two open-weight instruction-tuned models under deterministic, zero-shot conditions. The protocol compares baseline and substituted states using accuracy, cross-framing overlap, correctness structure, and response-level similarity.

  • The evaluation uses MMLU-17 and MedQA-CH, with 50 training examples used solely to localize candidate intervention layers.
  • Qwen-2.5-7B-Instruct and GLM-4-9B are evaluated zero-shot with greedy decoding and fixed templates, context length, and decoding hyperparameters.
  • Random [STATE] initialization is tested across seeds, producing negligible variation in generated responses under otherwise fixed settings.
  • Baseline and intervention conditions compare SUP and ELIM outputs after substituting [STATE] representations over a selected layer region.
  • Reported ensemble results use paired deterministic realizations and strict-majority voting within separate SUP and ELIM groups.
  • Evaluation reports framing-specific accuracy, Jaccard cross-framing decision overlap, symmetric and asymmetric correctness, LAC, and BERTScore-F1.

5 Results

The evaluation finds that SUP and ELIM framings become most separable at intermediate layers, and exchanging their [STATE] activations changes predictions while generally improving accuracy and cross-framing agreement. Controls further distinguish structured substitution from unstructured perturbations and show that placement matters.

  • EQ1: Separable Decision-Related Representations: SUP and ELIM [STATE] representations are mixed early, clearly separated at intermediate layers, and less separated later.This pattern localizes framing-dependent structure to a subset of intermediate layers.
  • EQ1: Separable Decision-Related Representations: 10 random-label permutations remain consistently low across layers and window sizes, unlike the structured intermediate-layer peak under true labels.The persistent gap indicates that localization is not explained by windowing, random projection, or label imbalance.
  • EQ2: Effects of State Substitution: [STATE] substitution consistently changes prediction outcomes across both models, benchmarks, and framings while increasing framing-specific accuracy and cross-framing Jaccard overlap.Greedy decoding removes sampling randomness from the BASE→SUB comparison.
  • EQ2: Effects of State Substitution: Substitution creates substantial Correct/Wrong transition mass in both models, with more previously wrong predictions becoming correct than correct predictions becoming wrong.The imbalance indicates an asymmetric behavioral effect across framing directions.
  • EQ3: Interface, Content, and Position Specificity: Final-content-token substitution stays near baseline similarity, whereas structured cross-framing substitution preserves high similarity and alters behavior more effectively than the control.Unstructured interventions reduce response similarity, while cross-framing substitution preserves high-similarity regimes.
  • EQ3: Interface, Content, and Position Specificity: More than 95% of generations under tested misaligned placements become incoherent or fail to produce a valid task-relevant response.This establishes a strong position-sensitivity boundary for the intervention.
  • EQ3: Interface, Content, and Position Specificity: The dual-framing steering direction produces systematic layer-dependent effects in both models without the [STATE] token.The extension compares this direction with the original CAA baseline across intermediate layers.

6 Discussion

The discussion argues that SUP–ELIM is a controlled contrast for studying framing-sensitive representations while acknowledging that [STATE] may encode information beyond framing. It also identifies finer-grained within-class factors as an important scope boundary.

  • Controlled framing contrast: Language and reasoning-first prompt pairs can expose broader lexical, syntactic, response-strategy, or generation-plan factors rather than only the decision distinction of interest.This motivates using the more tightly controlled SUP–ELIM contrast.
  • Controlled framing contrast: SUP–ELIM prompts preserve the question, options, language, decoding procedure, and answer key while changing whether the model supports the correct option or eliminates incorrect ones.This design reduces lexical and syntactic confounds relative to broader prompt contrasts.
  • Interpretation: The controlled contrast supports the conclusion that logically equivalent decision objectives can induce distinguishable intermediate representations whose exchange affects downstream predictions.The authors explicitly state that framing is not guaranteed to be the only information encoded at [STATE].
  • Scope boundary: SUP and ELIM are broad observational labels that may contain finer-grained states associated with domain, uncertainty, response strategy, or required deliberation.Hierarchical or multi-factor analyses could separate these factors and test which remain independently substitutable.

7 Conclusion

StateSwap uses a dedicated residual-stream interface to study framing-sensitive representations. Across two evaluated LLMs, cross-framing activation exchange changes downstream predictions, while controls distinguish structured substitution from several alternative interventions.

  • Conclusion: StateSwap is a training-free framework using an untrained [STATE] token as a residual-stream interface for framing-sensitive representations.The framework compares support- and elimination-oriented prompts across two evaluated LLMs.
  • Conclusion: Across two evaluated LLMs, SUP and ELIM prompts induce separable intermediate-layer [STATE] activations, and exchanging them systematically changes downstream predictions.Controls distinguish structured substitution from token insertion, final-content-token replacement, unstructured perturbations, and prompt ensembling.

Limitations

The study’s conclusions are bounded by controlled inference settings, the multiple-choice task structure, the [STATE] interface design, and white-box access requirements.

  • Scope of evaluation: Experiments use four-option multiple-choice questions and deterministic greedy decoding, excluding sampling-based decoding, longer generation, and open-ended tasks.These choices isolate activation-level effects from sampling variance but limit the scope of the claims.
  • Task dependence: The formulation is tied to multiple-choice questions, where support and elimination prompts define complementary option sets and outputs are parsed into correct and incorrect options.Extension beyond MCQs would require paired prompts with a shared evaluation target and task-specific agreement metrics.
  • Interface dependence: The method relies on an untrained [STATE] token at a fixed position, so observed effects may depend on the interface design or intervention site.Different random initializations showed negligible variation, but independence from the interface design is not claimed.
  • Access requirements: StateSwap assumes white-box access to model activations and parameters, leaving applicability to black-box models untested.Majority voting over paraphrased prompts reduces prompt-realization noise during evaluation but is not part of the intervention method.

Ethical Statements

The paper reports no human subjects, user studies, or personal-data collection, and conducts experiments offline on public benchmarks using openly released models.

  • Ethical statements: The work involves no human subjects, user studies, or collection of personal data.Generated responses are synthetic benchmark outputs not associated with real individuals.
  • Research setting: Experiments are conducted offline on publicly available multiple-choice benchmarks, including MMLU and MedQA-CH, with openly released language models.The paper identifies the benchmark outputs as synthetic.
  • Use boundaries: StateSwap is presented as a diagnostic method for studying framing-sensitive internal representations under controlled inference settings, not as a deployed alignment or decision-support system.The evaluated setting is limited to semantically equivalent multiple-choice prompts and deterministic decoding.
  • Risk considerations: The authors note that activation-level interventions could be adapted for broader model steering, including biased or undesired behavioral changes.Claims are restricted to offline analysis, with ablations used to distinguish structured state substitution from generic perturbations.

B.1 Datasets and Data Splits

The evaluation uses two four-option multiple-choice benchmarks on official test splits, with English and Chinese settings kept separate. Prompts append [STATE] and use paired support- and elimination-oriented instructions under deterministic decoding.

  • Data splits: All reported task evaluations use the benchmarks’ official test splits, while 50 training examples are reserved only for localizing the intervention window.The calibration examples do not enter reported test evaluation.
  • Datasets: The evaluation covers MedQA-CH and a reasoning-focused MMLU subset across 5,622 multiple-choice questions.MedQA-CH contains 1,015 Chinese clinical questions, while the MMLU subset spans 17 reasoning-oriented subject categories.
  • Data splits: English and Chinese benchmarks are evaluated separately rather than mixed within a single evaluation setting.MMLU is evaluated in English and MedQA-CH in Chinese.
  • Prompt protocol: Each prompt contains an instruction, question, options, and an appended [STATE] token marking the model’s output position.The instruction templates separately target identifying correct or incorrect options while preserving fixed formats across datasets and models.
  • Inference protocol: Each framing produces two deterministic greedy predictions, aggregated by strict-majority voting; a 1–1 tie counts as incorrect.This separates prompt-template variation from decoding randomness.
  • Evaluation: Outputs are normalized into Correct options and Wrong options fields, from which uppercase option letters are parsed into sets for accuracy and agreement calculations.Accuracy checks whether the ground-truth option belongs to the predicted correct-option set.

C.1 Ensemble and Per-Task Results

StateSwap-based ensembles outperform direct SUP–ELIM aggregation across both evaluated models, while substitution generally improves accuracy and error elimination. The steering extension compares dual-framing and CAA directions under matched evaluation settings.

  • Ensemble Results: Dual StateSwap achieves the best ensemble accuracy for both Qwen-2.5-7B and GLM-4-9B.This suggests transferred decision-state information contributes beyond ordinary variance reduction from ensembling.
  • Ensemble Results: Optimized weighted voting still leaves dual StateSwap stronger than direct SUP–ELIM aggregation.The comparison searches over α for 30 trials, so the gain is not explained by the tested tie-breaking heuristic alone.
  • Per-Task Results: Substitution produces consistent or improved accuracy across most MMLU categories under both framings.Table C.3 reports Base and Sub accuracy for Elim and Sup across individual tasks.
  • Decision Determinacy: Right shifts occur more frequently than left shifts in eliminated-incorrect mass across models and datasets.The reported asymmetry indicates substitution more often strengthens than relaxes incorrect-option elimination, aligning with ACC Elim and Jaccard gains.
  • Steering Results: The dual-framing direction retains higher accuracy than CAA at a = −2, −1, and 2 for both models.For GLM, the respective means are 59.03 vs. 37.16, 69.67 vs. 48.85, and 68.15 vs. 60.35; for Qwen, they are 71.21 vs. 52.26, 81.61 vs. 75.97, and 74.77 vs. 62.90.
  • Steering Results: Relative to CAA, the layer-wise accuracy range narrows from 50.54 to 19.90 points for GLM and from 60.89 to 28.08 points for Qwen.The evidence supports a more bounded layer-wise response under this matched protocol, while strong coefficients can still cause sharp failures.

D.2 Analysis of the Intervention Window

The intervention window is localized to intermediate layers, where framing separability is strongest and output perturbations offer the most favorable stability–effect profile. This localization is supported across models and bootstrap resamples, while interface placement remains task-critical.

  • Window Comparison: Shallow and late windows yield larger cosine distances of 0.130–0.131 and the largest KL values of 1.634–1.636.These ranges also have heavier KL tails, with p90=6.832, indicating more extreme probability reallocations among affected examples.
  • Window Comparison: The 11–20 window has the most favorable stability–effect profile: cosine IQR=0.066, p90=0.257, and KL mean=1.487.It maintains substantial distributional shifts while reducing geometric dispersion and the heaviest KL tails relative to extreme windows.
  • Layer Localization: Intermediate layers show the strongest framing separability in both Qwen-2.5-7B and GLM-4-9B.Separability is weak early, peaks in the intermediate region, and attenuates later; the same pattern appears in PCA and heatmap diagnostics.
  • Layer Localization: The selected intervention windows are layers 11–20 for Qwen-2.5-7B and layers 21–40 for GLM-4-9B.These windows are chosen from separability patterns rather than downstream task performance.
  • Layer Localization: Bootstrap resampling places most highest-scoring Qwen layers between 11 and 20 across 500 resamples of 1,000 examples.Exact layer rankings differ between diagnostics, but localization to the intermediate region remains consistent.
  • Interface Position: More than 95% of generations fail in the two misaligned interface configurations, whereas the canonical configuration preserves coherent task-relevant generation for most inputs.The result indicates substitution effectiveness depends on placing [STATE] after the complete question and option context.
Loading 2609.01081v1…