PaperlayerView exact revision on arXiv

Source-linked AI summary

How Language Models Choose Sides: Internal Representations of Instruction Hierarchy

Enrique Balp-Straffon, Chih-Hao Hsu, Rushiraj Gadhvi, Sunishchal Dev, Callum Stuart McDougall, Anusha Mujumdar

arXiv:2608.28648v1cs.AIcs.CLcs.LG

TL;DR

Instruction-tuned LLMs often resolve direct system-versus-user conflicts inconsistently, raising the question of whether user-preferring behavior reflects a missing internal arbitration signal. The paper benchmarks eight models and probes residual activations, finding readable conflict representations and effective steering only when intervention directions have the right geometry.

  • Problem

    The paper asks how instruction-tuned LLMs arbitrate direct system-versus-user conflicts and whether user-preferring behavior reflects an absent internal conflict-resolution signal.

  • Method

    The authors evaluate eight models on 41 paired constraints with deterministic verifiers under baseline, conflict, and same-channel controls, then probe and steer residual activations.

  • Results

    Across models, conflict behavior forms hierarchy, anti-hierarchy, and no-effect regimes; on Llama-3.1-8B, probing reaches 0.97 balanced accuracy and steering raises genuine system compliance from 0.132 to 0.530.

  • Takeaways & Limitations

    User-preferring conflict resolution can coexist with a readable internal arbitration signal, while successful intervention depends on readout geometry rather than probe accuracy alone.

  • Takeaways & Limitations

    Causal evidence is limited to one model and four conflicts, and transfer to semantic or safety-critical conflicts remains open.

Abstract

from arXiv · show

We study how instruction-tuned LLMs arbitrate direct conflicts between system and user instructions. We introduce a benchmark of 41 paired constraints with deterministic verifiers and evaluate eight models under matched baseline, conflict, and same-channel control conditions. Behaviourally, the models split into three regimes by System Authority Delta: hierarchy-respecting models use the system channel as an authority signal, anti-hierarchy models follow the system less often than their same-channel baseline predicts, and no-effect models show little channel sensitivity. Llama-3.1-8B is the strongest anti-hierarchy case in our suite, following the system in only 0.10 of conflict trials. We use this behavioural failure case to ask whether user-preferring arbitration reflects the absence of an internal conflictresolution signal. It does not: on Llama-3.1-8B, the conflict outcome is linearly decodable from residual-stream activations at 0.97 balanced accuracy, 17 percentage points above a metadata-only baseline, with analogous signals on Qwen2.5-7B and gpt-oss-20b. Steering with a layer-12 mean of four per-conflict logistic-regression directions raises genuine system compliance from 0.132 to 0.530, while directions selected mainly for pooled separability steer poorly. User-preferring conflict resolution can therefore coexist with a readable internal arbitration signal, and successful intervention depends on the geometry of the readout rather than probe accuracy alone

1. Introduction

The paper examines how instruction-tuned LLMs resolve system-versus-user conflicts, motivated by failures of instruction hierarchy. It asks whether internal conflict representations are readable and steerable, finding that probe geometry matters for successful intervention.

  • Motivation: Instruction hierarchy failures arise when user inputs override developer-specified system constraints, creating risks including prompt injection and jailbreaks.Prior approaches train or encode instruction priority, but may learn superficial shortcuts rather than genuine role separation.
  • Research questions: The study asks whether conflict outcomes are linearly readable from residual activations and whether that readout can causally change which instruction wins.Across three models, conflict outcomes are decodable at 0.93–0.97 balanced accuracy.
  • Findings: Averaging four per-constraint logistic-regression directions at layer 12 raises system compliance from 0.132 to 0.530.The pooled direction is nearly inert despite higher linear separation, showing that probe separability alone does not identify effective steering directions.
  • Findings: 0.97 balanced accuracy separates system-following from user-following activations in Llama-3.1-8B at layer 12.The figure depicts distinguishable activation clusters and a steering intervention that crosses the decision boundary.

2. Benchmark and Behavioural Regimes

The benchmark tests paired system-versus-user constraints under matched baseline, conflict, and same-channel conditions. Across eight models, system-channel authority varies from hierarchy-respecting to anti-hierarchy and no-effect regimes.

  • Benchmark: 41 mutually exclusive constraint pairs span language, formatting, lexical choice, style, syntax, and content, each evaluated with deterministic verifiers.The benchmark includes system and user baselines, opposing-channel conflicts, and same-channel controls in both assignment directions.
  • Benchmark: 114,800 prompts per model combine a 5×5 system-style × user-style grid with 50 semantic tasks.The conflict condition is compared against same-channel resolution on identical instruction pairs.
  • Behavioural regimes: Baseline compliance exceeds 0.94 on every model, while conflict system compliance ranges from 0.10 to 0.96.This spread reflects channel-level conflict resolution rather than inherent constraint difficulty.
  • Behavioural regimes: System Authority Delta partitions the models into hierarchy, anti-hierarchy, and no-channel-effect regimes.Hierarchy models show positive system-channel effects, anti-hierarchy models follow the system less than same-channel baselines predict, and no-effect models show little channel sensitivity.
  • Behavioural regimes: Llama-3.1-8B is the strongest anti-hierarchy case, with system compliance of 0.10 and System Authority Delta of −0.29.Its unusually low system-following rate motivates the later steering analysis.

3. Probing

The probing analysis tests whether conflict outcomes are encoded in residual-stream activations beyond metadata features. Linear probes decode the outcome accurately, but different probes identify geometrically distinct directions.

  • Probe setup: Three probe methods classify FOLLOWED_SYSTEM versus FOLLOWED_USER from balanced residual activations at the final input position.DiM uses class-mean differences, IID-MM applies covariance correction, and LR fits a regularized classifier.
  • Readability: 0.97 peak balanced accuracy is achieved by IID-MM, compared with 0.81 for DiM and 0.92 for LR.These results establish that conflict outcomes are linearly readable, but do not by themselves establish causal control.
  • Geometry: The three probes produce classification-equivalent separating normals, although their residual variance aligns differently with the probe axis.DiM shows partial alignment, whereas IID-MM and LR have residual axes orthogonal in distribution.
  • Readability: 0.800 ± 0.008 balanced accuracy is achieved by the metadata baseline, making IID-MM’s 0.97 a 17-point improvement.The baseline encodes constraint type, style identities, and conflict direction; permuted labels collapse performance to 0.494–0.498.
  • Geometry: 0.022, 0.069, and 0.296 are the mean pairwise absolute cosine similarities for DiM–IID-MM, DiM–LR, and IID-MM–LR directions.The probes separate the same classes while recovering geometrically distinct directions.

4. Steering the Precedence Readout

The paper tests whether residual-stream directions encoding system-versus-user conflict outcomes can causally increase system compliance. A layer-12 mean of per-conflict probes succeeds, whereas pooled-separability directions steer poorly and can damage response quality.

  • Setup: The steering study evaluates Llama-3.1-8B-Instruct on four Condition-C conflicts across eight directional cells with 96 generations per cell.The conflicts are JSON versus plain text, bulleted versus numbered lists, past versus present tense, and Hello versus Greetings starts.
  • Direction families: The intervention compares pooled and per-conflict logistic-regression directions using additive or projection steering applied to the residual stream.Additive steering replaces h with h + αv, while projection steering uses a matched scalar projection.
  • Quality control: 98.3% of generations are repetition loops in one overall-probe setting, so raw verifier scores require a rule-based genuine-quality filter.That setting reports raw SCR 0.276, illustrating why surface labels on broken text are insufficient.
  • Mean-of-probes intervention: 0.132 to 0.530 genuine SCR: the layer-12 mean of per-conflict probes produces a fourfold increase at additive α = +7.Genuine SCR rises through α = +5 and +6, then plateaus before repetition becomes severe at α = +10; six of eight cells exceed 0.65 at the peak.
  • Direction geometry: 0.11–0.26 genuine SCR: individual per-conflict probes at α = +5 are weaker than their equal-weight mean, which reaches 0.395.The individual probes include 0.113 for JSON and 0.248 for lists; the overall pooled probe is also weaker than the mean.
  • Dose response: 0.530 genuine SCR: additive steering peaks at α = +7, while matched projection steering reaches 0.522 and mainly preserves coherence at extreme magnitudes.The figure reports overall genuine SCR as the black curve and selected high-responding cells as coloured curves.
  • Separability versus control: 0.156 genuine SCR: the strongest-separating IID mass-mean direction at L12 steers poorly and introduces refusals and repetition.This shows that linear separability alone does not identify the best causal steering direction.

5. Conclusion

The paper finds that instruction hierarchy varies across models and that user-preferring behavior can coexist with a readable internal arbitration signal. Causal evidence supports steering in one model and four conflicts, leaving transfer to broader settings unresolved.

  • Conclusion: Across eight models, instruction hierarchy separates into hierarchy, anti-hierarchy, and no-channel-effect regimes.Within the Gemma-4 family, the system-channel effect strengthens with scale.
  • Conclusion: Llama-3.1-8B usually follows the user, yet its residual stream linearly encodes which side will win.The paper therefore distinguishes readable internal arbitration from the behavior observed at the output layer.
  • Conclusion: 0.132 to 0.530 genuine system compliance: averaging per-conflict logistic-regression directions provides causal control, unlike directions chosen mainly for pooled separability.The conclusion frames the result as evidence that steering depends on readout geometry rather than probe separability alone.
  • Limitations and next steps: The causal evidence is limited to one model and four conflicts, motivating cross-model replication and evaluation on the full 41-conflict benchmark.The benchmark’s formatting, lexical, and stylistic constraints enable deterministic verification, but transfer to semantic or safety-critical conflicts remains open.

A.1. Per-model 5×5 style grid

The 5×5 Condition C grid shows that jailbreak-style user framing tracks the model’s hierarchy regime: it nearly eliminates system compliance in anti-hierarchy Llamas but not in hierarchy-regime models.

  • The jailbreak user style collapses SCR almost to zero on the two anti-hierarchy Llamas but not on Gemma-4-31B, gpt-oss-20b, or Gemma-4-E4B.
  • ≤0.02 SCR marks the anti-hierarchy Llamas’ collapse along the jailbreak-style user column.
  • +0.27 SCR is the authority-framed system prompt’s lift for gpt-oss-20b relative to the bare/plain cell.
  • Near-zero system-side framing effects characterize Llama-3.1-8B, whereas hierarchy-regime models respond with greater system-framed compliance.

A.2. Metacommentary and refusals

Condition C responses are partitioned into four disjoint response types, revealing both refusal tails and model-specific differences in conflict acknowledgement.

  • The four Condition C response tags are clean, meta_content, refusal_content, and bare_refusal, and they sum to 100%.
  • 6.9% is the highest combined refusal rate, observed for Llama-3.1-8B, versus 0.1% for Qwen2.5-7B and Gemma-4-E2B.
  • 34.3% of Gemma-3-27B’s Condition C responses contain metacommentary, compared with 15.2% for Gemma-4-E4B and 0.3–6.3% for every other model.
  • Gemma-3-27B often acknowledges conflicts without reliably resolving toward the system, while Gemma-4-E4B routes acknowledgements toward system compliance.

B. Cross-Model Probing Replication

Cross-model replication preserves the central probing result: conflict outcomes are linearly readable, but the learned directions depend on position, category, and probing method.

  • The replicated cross-model conclusion is that conflict outcomes remain linearly readable across anti-hierarchy and hierarchy-regime models.
  • 0.951 and 0.930 are the peak IID-MM balanced accuracies for Qwen2.5-7B and gpt-oss-20b, respectively.
  • Near-chance cross-position transfer shows that probes trained at last_system or elsewhere do not generalize across positions.
  • Off-diagonal transfer is weaker than within-category transfer, indicating category-structured probing directions.
  • The outcome classes separate along the IID-MM probe axis in both replicated models, matching the Llama-3.1-8B pattern.

C. Qualitative Steering Examples

The qualitative examples show that steering can flip surface constraint compliance, but stronger interventions may distort content, exploit verifier weaknesses, or reinforce the user side.

  • Verifier artifacts and over-steering: At α = +5, steering produced verifier-passing future-tense responses even though the system required present tense.The absence-based verifier treated missing past markers as present tense, miscrediting semantically future-tense output.
  • Verifier artifacts and over-steering: Verifier design and manual inspection are necessary because correlated side effects can inflate apparent system compliance.The paper motivates genuine-SCR filtering after observing verifier artifacts and semantic distortion under high steering magnitudes.
  • Directional steering: A negative α push strengthened user-side behavior, yielding nested JSON when the user requested JSON and the system requested plain English.The example illustrates bidirectional movement along the same steering axis, with positive α defined as the system direction.
  • Partial system flips: The intervention changed the opening word from “Greetings” to “Hello,” but the resulting response began with a refusal-flavoured hedge before providing content.This partial flip demonstrates that surface compliance and content quality can diverge.
  • Verifier artifacts and over-steering: Over-steering can satisfy a surface constraint while shortening content or adding refusal-flavoured metacommentary.The Hello/Greetings example satisfies the system’s opening-word constraint but produces shorter, lower-quality content.

D.1. Threshold calibration

Threshold calibration turns verifier scores into stable binary labels while using response-structure tags and matched prompt templates to support behavioural analysis and verifier auditing.

  • Threshold calibration: Float-valued verifiers use per-conflict, per-model thresholds calibrated for baseline accuracy and a low-density Condition-C decision boundary.The thresholds are selected using the procedure in Table 3 and stored in the released repository.
  • Response auditing: Response-structure tags supplement constraint labels for behavioural analysis, verifier audits, and genuine-SCR quality filtering rather than serving as outcome labels.The tags remain independent of the system/user verifier label and are summarized in Table 4.
  • Response auditing: An audit loop validates Condition-C labels because refusals, metacommentary, partial compliance, and off-task text make baseline balanced accuracy insufficient for mechanistic analysis.The paper reports the validated label set rather than the full audit transcript.
  • Prompt conditions: The benchmark uses five system and five user wrapping templates, with Condition C varying both channels and other conditions providing matched controls.Condition A has only the system constraint, Condition B only the user constraint, and Condition D places both opposing constraints in one user message.
  • Conflict inventory: The inventory contains 41 conflicts spanning boolean and float constraints, with deterministic verifier logic specified for each pair.Table 7 lists the two constraint sentences, conflict type, and scoring rule for every conflict.
Loading 2608.28648v1…