PaperlayerView exact revision on arXiv

Source-linked AI summary

When Does Correction Become Repair? Mechanistic Auditing of Internal Interventions in Tool-Using LLMs

Jiayi Li, Ruizhe Li

arXiv:2609.36138v1cs.CLcs.SE

TL;DR

Pre-execution tool-decision errors are difficult to assess from aggregate metrics because a changed action may still be wrong or may damage correct decisions. SAKIKO audits interventions through destination-resolved outcomes and preregistered statistical licensing; its evaluations show direction-specific gains, but not every favorable estimate supports a repair claim.

  • Problem

    In K-way action spaces, leaving an erroneous action may lead to another invalid action, while aggregate metrics obscure destinations and collateral damage.

    Source: §1 · p. 1, §5.4 · p. 9

  • Method

    SAKIKO discovers directional error channels, routes inputs to channel-keyed interventions, audits destinations and preservation, and applies preregistered statistical licensing.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    Channel-keyed interventions produced direction-specific gains, but +55 net gain on Phi-3.5 accompanied 52 of 93 exposed correct decisions broken, and two sealed models were declined on inconclusive evidence.

    Source: §1 · p. 2, §5.3 · p. 9

  • Takeaways & Limitations

    Behavioral movement, destination correctness, preservation of correct decisions, and evidential sufficiency do not inherently coincide.

    Source: §1 · p. 2, §6 · p. 10

    The frozen license tests preservation at the population level, not conditional on Router exposure.

    Source: §5.2 · p. 9

Abstract

from arXiv · show

Before invoking external tools, an agentic LLM must select among a K-way action space: executing a call, seeking clarification, answering directly, or declining. While internal activation steering can alter these pre-execution decisions, conventional aggregate metrics obscure where altered states land and what collateral damage they inflict. We present SAKIKO, an auditing framework that formalizes representation repair via directional error discovery, router-conditioned intervention, destination-resolved verification, and prospectively frozen statistical licensing. Across seven LLMs on When2Call and MetaTool, channel-keyed interventions induce direction-specific net gains in five models; across three sealed evaluations, none of 59 budget-matched random directions matches calibrated target gain. Crucially, destination auditing shows that behavioral movement does not equal repair: an intervention achieving +55 net gain corrupts over half of the baseline-correct decisions it touches, and promising point estimates on Qwen3-4B and Gemma-2-9B are formally declined due to finite-sample uncertainty. SAKIKO establishes the necessity of outcome-resolved adjudication before claiming internal repair. Code: https://github.com/ruizheliUOA/mechanistic-tool-use-llm.

1 INTRODUCTION

Pre-execution tool-decision errors occur before tools are invoked, and in K-way action spaces leaving an error does not guarantee reaching the correct action. SAKIKO evaluates whether internal interventions constitute repair by checking destinations, collateral, and evidential sufficiency.

Problem

  • In K-way action spaces, departing an erroneous action can simply shift probability to another invalid action, so behavioral movement alone is not repair.The errors occur upstream of generated content, before tool invocation, retrieved evidence, or execution feedback.Source: §1 · p. 1

Framework

  • SAKIKO combines directional error discovery, per-input detection, channel-keyed intervention, destination-resolved verification, and statistical licensing.It treats representation-level repair as an evidence process rather than an isolated steering operation.Source: §1 · p. 2

Empirical findings

  • +55 net gain on Phi-3.5 came with 52 of 93 exposed baseline-correct decisions broken, while promising estimates on Qwen3-4B and Gemma-2-9B were not licensed.The paper also reports near-identical net gains but different target-hit rates for activation steering and an output-score baseline on Qwen3-8B.Source: §1 · p. 2

Mechanistic decoupling

  • Linear decodability does not imply steerability, and the best injection site, dosage, and intervention success can diverge from probe accuracy.The authors argue that verification cannot be reduced to internal detection alone.Source: §1 · p. 3

2 RELATED WORK

Related work addresses downstream tool execution and emerging approaches to detecting or steering pre-execution decisions. SAKIKO distinguishes its focus by adjudicating where redirected decisions land and whether repair is supported.

Tool Learning and Epistemic Selection

  • Prior tool-learning work focuses on API adherence, parameter synthesis, and environment generalization, while the upstream decision of whether external action is needed remains distinct.The cited work on failure taxonomies treats agents as black boxes, omitting latent geometry and representation-level intervention.Source: §2 · p. 3

Agent Observability and Activation Steering

  • Mechanistic studies decode pre-execution states and use activation steering or routers, but prior approaches often evaluate binary exits or tool-use outcomes.Wu et al. (2026a) score task performance rather than where redirected decisions land; Shi et al. (2026) use a call/no-call counter-bias shift.Source: §2 · p. 3

SAKIKO Distinction: Audited Repair in K-Way Action Spaces

  • SAKIKO extends prior steering and observational multiclass work with destination, preservation, and uncertainty audits for K-way action spaces.The cited prior approaches include binary or pairwise steering (Wu et al., 2026b; Shi et al., 2026) and observational multiclass diagnosis (Zhao et al., 2026).Source: §2 · p. 3

3 THE SAKIKO FRAMEWORK

SAKIKO frames repair as a sequence of tests rather than a single steering operation. It tracks directional errors through interventions, destination and collateral audits, and statistical licensing.

3.1 PROBLEM FORMULATION AND OUTCOME TOPOLOGY

  • A directional error channel is an ordered pair from the gold action to the baseline error, and intervention outcomes partition into source retained, gold arrival, or other wrong.Baseline-correct decisions touched by the Router are separately divided into correct retained and broken.Source: §3.1 · p. 3
  • Aggregate net gain cannot determine target-hit rate or collateral because multiple outcome classes collapse to the same net quantity.The paper distinguishes whole-population gain from channel-specific gain; the latter can exceed the former when correction reaches an error outside the keyed channel.Source: §3.1 · pp. 3–4

3.2 OVERVIEW OF THE SAKIKO PIPELINE

  • The pipeline discovers directional channels, trains a Router on frozen representations, applies channel-specific steering, audits full K-way destinations and clean decisions, then licenses against a preregistered hierarchy.The Router gates intervention on per-input channel detection; inputs it does not route remain unperturbed.Source: §3.2 · p. 4
  • Intervention adds a channel-specific direction at a selected layer with a frozen dose, while structural controls include random, reversed, layer-misaligned, and protocol-permitted cross-channel perturbations.The sealed inventory excludes cross-channel substitution by construction.Source: §3.2 · p. 5
  • Verification separates gold arrivals from lateral errors and reports collateral on both population and Router-exposed denominators; licensing tests a preregistered ten-condition gate.An ADMIT certifies population-level preservation under the frozen rule, not preservation conditional on Router exposure.Source: §3.2 · p. 5

4 EMPIRICAL EVALUATION: CHANNEL-KEYED CORRECTION

Across When2Call and MetaTool, channel-keyed interventions produce target-directed gains, and structural controls support direction specificity. The evaluation also shows that gains can be limited by task support, cross-model failures, and preregistered screening gates.

4.1 EXPERIMENTAL SETUP

  • The evaluation used three benchmarks separately, but ACEBench was retired before intervention because channel support and paraphrase agreement failed preregistered criteria.ACEBench readout reached 0.756 accuracy and 0.687 macro-F1, while paraphrase agreement was 0.56 against a 0.80 threshold.Source: §4.1 · p. 5

4.3 EMPIRICAL CONFIRMATION OF DIRECTIONAL SPECIFICITY

  • +55 calibrated gain on Phi-3.5-mini exceeded random and sign-reversed gains of +14 and layer-misaligned +3; all 59 sealed random vectors trailed calibrated gain (p = 0.017).Calibrated vectors also produced more gold arrivals than random baselines on all three sealed models.Source: Figure 3 · p. 6, §4.3 · pp. 6–7

4.2 INTERNAL INTERVENTIONS INDUCE TARGETED ACTION TRANSITIONS

  • +79 net decisions on Qwen2.5-7B and +55 on Phi-3.5-mini; sealed target gains were positive for Qwen3-8B, Qwen3-4B, and Gemma-2-9B, with MetaTool gains in both directions.On Qwen3-8B, the Router fired on 82.8% of errors, yielding 59.8% source exits and 43.7% target arrivals.Source: §4.2 · p. 6

4.3 EMPIRICAL CONFIRMATION OF DIRECTIONAL SPECIFICITY

  • Steering performance depends on channel and layer alignment, while probe accuracy alone does not guarantee steering efficacy.The authors characterize the evidence as locating the intervention site, not identifying fine-grained circuit components.Source: §4.3 · p. 7

4.4 CROSS-MODEL ANALYSIS AND PRE-INTERVENTION ATTRITION

  • Directional specificity held in five of seven architectures; Llama-3.1-8B and Mistral-7B failed structural specificity, and Qwen3.5-9B was screened out for severe class skew and only 29 baseline-correct references.The 29 instances fell below the preregistered minimum support of 30.Source: §4.4 · p. 7

5 ANALYSIS: FROM DIRECTIONAL CORRECTION TO LICENSED REPAIR

The analysis distinguishes directional correction from genuine repair by auditing destination correctness, preservation of already-correct decisions, and evidential sufficiency. Identical action shifts can produce repair, lateral error, or collateral damage.

5.1 DECOUPLING CORRECTION FROM DESTINATION CORRECTNESS

  • On Phi-3.5-mini, 46 of 153 error-state exits miss the gold target; on Qwen3-8B, similar net gains (+38 vs. +35) conceal different lateral-error counts (14 vs. 21).The Qwen3-8B THc values are 0.731 for activation steering and 0.638 for logit shifting; the descriptive difference does not survive multiplicity correction.Source: §5.1 · p. 8, Figure 5 · p. 8

5.2 DECOUPLING DESTINATION CORRECTNESS FROM PRESERVATION

  • Phi-3.5-mini steering breaks 52 of 93 exposed correct decisions (E2 = 55.9%), while router confidence fails to separate broken from intact cases.The break rate remains above 50% even at τc = 0.9; the passage also warns that population loss can conceal localized degradation.Source: §5.2 · p. 8
  • The preregistered preservation rule licenses population-level preservation, not preservation conditional on router exposure.Exposure-conditional metrics were added in subsequent runs rather than retrofitted into the frozen rules.Source: §5.2 · p. 9

5.3 PROSPECTIVE LICENSING AND EVIDENTIAL SUFFICIENCY

  • Qwen3-8B clears all preregistered criteria with THc = 0.731 (CI [0.604, 0.846]), TGc = 0.276 (CI [0.115, 0.425]), and zero breaks; Qwen3-4B and Gemma-2-9B are declined.Their target-gain intervals span zero, and finite-sample intervals also cross sub-threshold bounds.Source: §5.3 · p. 9
  • A 0.58 hit rate against a 0.50 null requires 245 exits to resolve, versus approximately 30 for a 0.73 hit rate; relaxing hit-rate gates does not reverse both declines.The declined models' target-gain intervals still cross zero, and exposed collateral damage remains uncontained.Source: §5.3 · p. 9

5.4 ONE TRANSITION, THREE VERDICTS

  • The same tool-call-to-information-request shift yields a target repair, a lateral error, or a broken correct decision, which aggregate accuracy collapses into a net gain of +55.The figure contrasts transition, accuracy, and destination readouts; its counts are frozen outcome-class totals, not specific transition frequencies.Source: §5.4 · pp. 9–10, Figure 6 · p. 9

6 CONCLUSION

SAKIKO evaluates whether internal interventions genuinely repair pre-execution action decisions rather than merely changing behavior. The conclusion is limited to categorical action selection and calls for auditing destinations, collateral damage, and uncertainty.

6 CONCLUSION

  • Channel-keyed steering induces direction-specific corrections across five architectures, but conventional net gains can conceal lateral errors, collateral damage, and failures to clear confidence bounds.Two of three sealed models are declined on destination intervals despite favorable point estimates.Source: §6 · p. 10
  • SAKIKO pairs outcome-resolved tracking with preregistered statistical gating to distinguish certified interventions from cosmetic behavioral drift.Source: §6 · p. 10
  • The license covers categorical action-mode selection, not downstream execution validity or safety among router-exposed decisions.Source: §6 · p. 10

A LIMITATIONS

The paper frames tool-decision repair as a multiclass, directional problem: leaving an error state does not guarantee reaching the target, and aggregate gains do not reveal destinations or collateral.

A LIMITATIONS

  • Transfer beyond When2Call is unestablished, and the only admitted repair claim is Qwen3-8B on cannot answer → tool call under gradient activation steering.The evaluation also notes that When2Call labels are synthetic and may contain unmodeled systematic bias.Source: A LIMITATIONS · p. 16

B.1 ACTIONS, CHANNELS AND POPULATIONS

  • The four pre-execution actions are tool call, clarification, direct answer, and decline; each directional error channel pairs a required action with the source action produced.Channels vary across models and tasks, so the framework discovers them rather than assuming a fixed set.Source: §B.1 · p. 16

B.2 POST-INTERVENTION OUTCOME CLASSES

  • Intervention outcomes partition channel errors into retained source, target arrival, or another wrong action, while exposed baseline-correct decisions are either retained or broken.Unrouted inputs remain unchanged and are not counted in the exposed-correct cohort.Source: §B.2 · p. 17

B.3 ENDPOINTS

  • A given net gain cannot determine target-hit or collateral, and target-hit is uninformative in a binary action space because every source exit reaches the sole alternative.The population-level collateral rate can understate risk among router-exposed correct decisions.Source: §B.3 · p. 18

C.1 TOOL-AUGMENTED AGENTS AND THE EPISTEMICS OF ACTION SELECTION

  • Tool-learning work has focused on downstream execution, while the upstream choice of whether external action is warranted remains a distinct decision problem.The paper contrasts post-hoc intervention on frozen activations with approaches that train the policy.Source: §C.1 · p. 19

C.2 INTERNAL OBSERVABILITY AND DETECTION OF TOOL-USE FAILURES

  • Prior internal-state approaches diagnose or classify tool-use failures, but multiclass diagnosis alone neither identifies a directional channel nor steers toward its target.The paper distinguishes its intervention question from a binary-margin study where leaving the source leaves only one destination.Source: §C.2 · pp. 19–20

C.3 REPRESENTATION ENGINEERING AND ACTIVATION STEERING IN AGENTS

  • Binary or pairwise steering treats leaving an erroneous state as reaching the alternative, an assumption that does not hold in the evaluated multiclass settings.In a K-way action space, movement can redirect a decision to another incorrect action.Source: §B.1 · p. 16, §C.3 · p. 20

C.4 CONTRASTING SAKIKO: FROM BEHAVIOURAL MOVEMENT TO ADJUDICATED REPAIR

SAKIKO distinguishes steering movement from repair by auditing destinations, collateral, and uncertainty. Its protocol prospectively fixes the evidence required to license a repair claim.

C.4 CONTRASTING SAKIKO: FROM BEHAVIOURAL MOVEMENT TO ADJUDICATED REPAIR

  • SAKIKO tracks multiclass destinations, baseline preservation, and uncertainty-qualified licensing because leaving an erroneous action can lead to another wrong action.Aggregate net gain can conceal collateral breaks on baseline-correct decisions the intervention touches.Source: §C.4 · pp. 20–21
  • The empirical boundary includes synthetic When2Call labels and English text prompts; transfer to other benchmarks, languages, or speech is not established.ACEBench was retired before steering experiments, and MetaTool supplies historical evidence in a binary setting.Source: §C.4 · p. 21, §D.2 · p. 22

D.1 DISCOVER: CHANNEL CONSTRUCTION

  • Discovery retains model- and task-specific directional channels, while support gates require enough errors and baseline-correct rows before intervention.A channel failing the support gate is unadjudicable on the available sample, not shown to be uncorrectable.Source: §D.1 · p. 22, §D.2 · p. 22

D.3 DETECT: OBSERVATION SITE AND ROUTER

  • A validation-thresholded Router detects channel-associated states, but high discrimination establishes readability only, not that intervention at that site will help.The Router is not treated as a calibrated probability and leaves the forward pass untouched when it does not fire.Source: §D.3 · pp. 22–23

D.4 ESTIMATE: CORRECTION DIRECTIONS

  • The frozen direction estimates a correction vector from training activations; when the Router fires, an additive perturbation modifies the injection-site state, with its absolute budget matched across directional arms.The online system uses frozen weights and a Router, direction bank, thresholds, and injection hook; no reference label is consulted at inference.Source: §D.4 · p. 23, §D.5 · p. 23, §D.6 · p. 24

E.1 THE PROPERTY HIERARCHY

  • A repair claim requires correctability, population-level preservation, and licensability; exposure-conditional preservation is stronger and is not certified in this study.The frozen gate tests population-level collateral, while exposure-conditional collateral is reported separately.Source: §E.1 · pp. 24–25, Table 6 · p. 25

E.2 THE FROZEN TEN-CONDITION GATE

  • The frozen gate tests scientific quantities at point estimates and interval bounds, so favorable point estimates alone do not suffice for admission.The gate replay is not monotone: a rule that drops specificity can readmit a setting whose movement is not established as direction-specific.Source: §E.2 · pp. 25–26, §E.3 · p. 26

E.3 WHAT THE LICENCE ADDS OVER SIMPLER RULES

  • The full licensing conjunction accepts one unit, while aggregate gain alone accepts all nine; three units clear point estimates and non-vacuous bounds, but two are still declined at the interval stage.The replay is retrospective across units gathered under different protocols, not a set of independent confirmatory experiments.Source: §E.3 · p. 26

E.4 WHAT A PRESERVATION-CERTIFYING GATE WOULD REQUIRE

A preservation-certifying gate would require bounded exposure-conditional breakage and enough routed correct decisions to make that bound attainable; none of the three sealed settings meets the exposure minimum. The paper distinguishes setting-specific verdicts from benchmark roles and reports ACEBench’s channel-discovery failure separately from readout certification.

E.4 WHAT A PRESERVATION-CERTIFYING GATE WOULD REQUIRE

  • Figure 7 contrasts retrospective reporting rules: aggregate gain alone accepts all nine units, while the full frozen conjunction accepts one.These are diagnostic readings, not formal verdicts; the rules are not nested.Source: Figure 7 · p. 27
  • A certifying gate would require a one-sided 95% upper bound on E2 below 0.05 and minimum exposure; Qwen3-8B, Gemma-2-9B, and Qwen3-4B fall short of the required 59, 59, and 59 exposed decisions.Their exposures are 6, 11, and 50, respectively; closing the gap requires more exposed decisions or a larger evaluation population.Source: §E.4 · p. 27
  • A successor gate should also require channel coverage and a prospectively fixed effect-size threshold, beyond the frozen gate’s positive-gain test.The sealed-setting gold-arrival shares over the channel are 0.437, 0.298, and 0.177.Source: §E.4 · pp. 27–28

E.5 WHAT EACH VERDICT ASSERTS

  • An ADMIT licenses a repair claim only for that setting, protocol, and dose; population-level preservation does not establish deployment safety.A DECLINE means evidence is insufficient under the evaluated protocol and sample support, not that correction is impossible or that models can be ranked.Source: §E.5 · p. 28

F DATASETS AND ACTION SPACES

F.1 WHEN2CALL

  • When2Call uses four candidate responses per input, teacher-forced scoring, and a frozen deterministic readout across 3,652 items split 2,556/548/548.Three action modes occur as reference labels; direct answering appears only as a distractor, and labels were not re-annotated.Source: §F.1 · p. 28

F.2 METATOOL

  • MetaTool is binary, so leaving the source mode necessarily reaches the target and cannot test destination correctness.It is used to assess whether opposite channels on disjoint populations both improve, rather than a single global shift in tool-call propensity.Source: §F.2 · p. 28

F.3 ACEBENCH

  • ACEBench’s readout passed certification, but channel discovery failed its preregistered support and paraphrase-agreement criteria, so it yields no intervention or licence outcome.The paraphrase check scored 0.56 against 0.80; the authors caution that the reformulation used a weaker format directive, so this is not clean evidence of wording fragility.Source: §F.3 · pp. 28–29

G MODELS AND IMPLEMENTATION

The implementation freezes model sites, dosage, and execution procedures before formal evaluation. In the Qwen3-8B adjudication, the real direction passed the frozen gate and outperformed all 59 budget-matched random directions, while the preservation claim remained limited by sparse router exposure.

G MODELS AND IMPLEMENTATION

  • Sealed layer sites were mechanically derived from a normalized-depth rule, and dosage came from a frozen six-point grid rather than per-model searches.The sealed settings used relative doses 1.0, 0.25, and 0.125; Phi-3.5 used a distinct threshold-and-scale parameterization.Source: G MODELS AND IMPLEMENTATION · p. 29

H.1 PROCEDURE IN TEMPORAL ORDER

  • The intervention procedure ordered discovery, fitting, dose calibration, freeze, gate-only verification, a single formal run, and endpoint computation after a zero-arm exactness check.Source: §H.1 · p. 30

H.2 DESTINATION-RESOLVED OUTCOME

  • The Qwen3-8B formal run resolved all decisions: of 87 channel errors, 38 reached the required action and 14 landed on a third action; the router exposed six correct decisions and broke none.Fifteen channel errors were unrouted, while 20 routed errors retained their source.Source: §H.2 · p. 30, Table 10 · p. 30

H.3 ADJUDICATION

  • None of the 59 budget-matched random directions matched the real target gain of 0.2759 (add-one p = 0.0167), and all ten frozen conditions passed.The zero arm reproduced baseline predictions; the reversed direction moved at most one channel error, and a wrong-layer injection reached target gain 0.0575.Source: §H.3 · pp. 30–31
  • The preservation pass was population-level only: zero breaks among 211 baseline-correct decisions gave an upper bound of 0.0141, but only six were exposed and the exposure-conditional bound was 0.3930.The result does not establish safety on decisions the router touches.Source: §H.3 · p. 31

I FORMAL EVALUATION: QWEN3-4B

The frozen protocol formally declined Qwen3-4B despite promising point estimates, because destination evidence was too uncertain. Historical settings and MetaTool show positive movement in some configurations, but their evidence supports narrower conclusions and does not carry the sealed protocol’s formal verdict.

I FORMAL EVALUATION: QWEN3-4B

  • Qwen3-4B produced 37 required-action arrivals from 64 exits, with 27 third-action errors; its net gain was +39 overall and +36 at channel level.The router exposed 50 correct decisions and broke one; none of 59 random directions matched the target gain.Source: §I.1 · p. 31

I.2 WHY THE PROTOCOL DECLINED

  • Qwen3-4B received a FORMAL DECLINE because its target-hit interval extended below one half and its target-gain interval included zero, despite passing the other frozen conditions.The two sealed settings had aggregate gains of +40 and +39, but destination-interval width separated their verdicts.Source: §I.2 · p. 31, Table 11 · p. 32

J HISTORICAL EXPERIMENTS

  • The four historical settings receive no formal verdict because the sealed gate was developed later and never run on them.Their recorded counts can be assessed only retrospectively.Source: J HISTORICAL EXPERIMENTS · p. 32

J.1 PHI-3.5-MINI

  • Phi-3.5-mini’s +55 net run fixed 107 errors but broke 52 of 93 exposed correct decisions; exposure-conditional collateral was 55.9%.Across five seeds, net gains were 55, 54, 53, 60, and 67, while breaks ranged from 16 to 52; those runs varied in configuration.Source: Figure 8 · p. 32, §J.1 · p. 33
  • Phi-3.5 controls separated from the real +55 arm: random and reversed directions each scored +14, a different layer +3, and cross-channel substitution −17.Four of five control arms survive only as aggregate counts, so the battery establishes an ordering rather than a decomposition.Source: §J.1 · p. 33

J.2 QWEN2.5-7B

  • Qwen2.5-7B gained +79 on the locked test, fixing 92 baseline errors while breaking 13 correct decisions; its destination composition cannot be adjudicated from surviving aggregate records.Its real direction scored +79 versus +16 for reversal and a +25.2 mean across ten budget-matched random directions.Source: Figure 8 · p. 32, §J.2 · p. 33

J.3 METATOOL

  • On MetaTool, the unnecessary-call and missed-call channels gained 8.6 and 13.0 on average, respectively, and 21.6 together.Because the opposite channels act on disjoint populations, the result cannot be explained by a single global shift in tool-call propensity; binary actions do not test destination correctness.Source: §J.3 · p. 33

J.4 MISTRAL-7B AND LLAMA-3.1-8B

Neither Llama-3.1-8B nor Mistral-7B passed the specificity comparison. Their secondary audit records support insufficient evidence that the recovered directions caused the observed correction, not a conclusion that no correctable direction exists.

J.4 MISTRAL-7B AND LLAMA-3.1-8B

  • Mistral-7B’s real net gain of +12 fell below the random mean of +29; 17 of 20 random directions reached it, and reversal scored +19.Llama-3.1-8B also failed specificity, with as many as 5 of 20 random directions reaching the real effect.Source: §J.4 · pp. 33–34
  • Both findings rely on estimator-suspect secondary audit records and 20 random directions, not the sealed protocol’s 59; they do not establish that either model lacks a correctable direction.Llama’s 19 required-action arrivals versus 44 other destinations are descriptive, not evidence that a specific correction failed to arrive, because specificity failed first.Source: §J.4 · p. 34

K CONTROL EXPERIMENTS

SAKIKO’s controls and destination-resolved evaluation distinguish targeted repair from aggregate behavioral movement. Across the evaluated settings, only Qwen3-8B clears the full five-stage pipeline, while steering effects and collateral vary by configuration.

K CONTROL EXPERIMENTS

  • The 59 random directions were unit-norm, near-orthogonal to each other and to the calibrated direction, and frozen before evaluation was opened.The control battery excluded alternative arms before sealing to prevent it from becoming a search.Source: K CONTROL EXPERIMENTS · pp. 34–35
  • Across five settings, gating raises Net in three and lowers it in two, so the authors make no claim that gating is necessary for Net gains.In three directly comparable settings, gating increases arrivals per broken decision; on Qwen3-8B, the gated arm breaks nothing.Source: K CONTROL EXPERIMENTS · p. 35
  • The score-space comparator and activation intervention have no stable ordering across three sealed settings, and no paired test survives multiplicity correction.Activation leads on Qwen3-8B and Gemma-2-9B target gain, while the comparator leads on Qwen3-4B.Source: K CONTROL EXPERIMENTS · p. 35

L.1 TWO INTERVENTIONS, EQUAL AGGREGATE, DIFFERENT DESTINATIONS

  • +40 against +39 aggregate gains mask seven decisions redistributed to a third wrong action on Qwen3-8B; paired tests support no superiority claim.The contrast is descriptive, and the comparison was not prespecified.Source: §L.1 · p. 36, Table 13 · p. 36

M MECHANISTIC INSIGHTS AND STAGE DECOUPLING

  • Mechanistic follow-ups find that probe decodability, steering efficacy, extraction geometry, gating, and dosage do not move together.The analyses are exploratory and outside the frozen conjunction; dosage effects on Qwen3-4B are non-monotonic, with target-hit rates from 0.525 to 0.911.Source: M MECHANISTIC INSIGHTS AND STAGE… · pp. 36–37

N SYNTHESIS: THE SPECTRUM FROM MOVEMENT TO REPAIR

  • Only Qwen3-8B clears all five stages and earns an ADMIT verdict; other configurations stop at distinct criteria.Historical stops carry no formal verdict because the gate was written later and was never run on them.Source: N SYNTHESIS: THE SPECTRUM FROM… · p. 37
  • SAKIKO’s staged adjudication shows that behavioral movement, destination correction, preservation, and statistical certainty do not inherently coincide.Destination-resolved checks expose lateral error redistribution, collateral damage, and finite-sample uncertainty hidden by aggregate gains.Source: N SYNTHESIS: THE SPECTRUM FROM… · p. 37

O STATISTICAL PROCEDURES

The statistical procedures test target gains against a frozen random-direction null and quantify uncertainty in destination and collateral outcomes. They also constrain interpretation through prospective selection and licensing rules, while identifying where the available samples cannot certify preservation or target-hit rates.

O STATISTICAL PROCEDURES

  • When no break is observed, the bootstrap collateral bound is exactly zero and cannot provide finite-sample information; the Clopper–Pearson bound for Qwen3-8B’s 0/211 is 0.0141.Substituting that bound into the gate does not change the verdict.Source: O STATISTICAL PROCEDURES · p. 37
  • With K = 59 and no random direction reaching the real target gain, the add-one p-value is 1/60 = 0.0167 in each sealed setting.This is the design’s minimum reportable p-value, not a measure of how far the real direction exceeds the null.Source: O STATISTICAL PROCEDURES · p. 38
  • Qwen3-8B’s 52 exits exceed the approximate requirement of 30 for an 80%-powered test, while Gemma-2-9B’s 27 and Qwen3-4B’s 64 fall several times short of theirs.The table approximates the gate’s bootstrap criterion rather than replacing it.Source: O STATISTICAL PROCEDURES · p. 38, Table 14 · p. 38
  • Certifying a 5% collateral bound at α = 0.05 requires at least 59 exposed decisions with zero breaks, and no setting attains the required exposure.Target-hit is undefined at zero source exits, and collateral on zero exposed rows is vacuous rather than preservation.Source: O STATISTICAL PROCEDURES · p. 38

P.1 SEPARATION OF DEVELOPMENT AND CONFIRMATION

  • The preregistration fixes the primary statistic, ten-condition conjunction, execution and interpretation rules before evaluation access; results condition on channel, estimator, and dose selected on development data.The confirmatory claim is limited to the adjudicated channel, not a general ranking against a frozen score shift.Source: §P.1 · p. 39

P.2 CORRECTIONS APPLIED AFTER THE FACT

  • The corrected Qwen2.5-7B layer sweep records norms 2.12, 5.88, 10.41, and 14.26 over layers 16 to 22; its endpoint validation gains are +13 and +46.The tabular rendering is a formatting error; the indices are authoritative, and no verdict changed.Source: §P.2 · p. 39, Table 15 · p. 40

Q DECLINED AND FAILED SETTINGS

Several settings are declined or fail at different evidence gates rather than being treated as repaired based on point estimates alone. The section also distinguishes which recorded results can be resolved by destination and which remain aggregate-only.

Q DECLINED AND FAILED SETTINGS

  • Qwen3.5-9B stops before intervention because its refusal-channel reference population has 29 rows against a frozen minimum of 30.The sealed population was never opened.Source: Q DECLINED AND FAILED SETTINGS · p. 40
  • ACEBench is retired because its channel structure misses the preregistered support minimum and its paraphrase gate returns 0.56 against 0.80.Its readout passed the validity criteria.Source: Q DECLINED AND FAILED SETTINGS · p. 40
  • Llama-3.1-8B and Mistral-7B are readable but not direction-specific; both fail the specificity comparison, rely on secondary audit records, and used K = 20 rather than K = 59.Source: Q DECLINED AND FAILED SETTINGS · p. 40

R REPRODUCIBILITY

  • Table 16 distinguishes results with row-level records from historical settings retained only as aggregate counts that cannot be re-resolved by destination.The formal Qwen3-8B run has a reproducibility path, while artifact files require Git LFS retrieval for usable contents.Source: Table 16 · p. 41, R REPRODUCIBILITY · p. 41

Q DECLINED AND FAILED SETTINGS

  • Qwen3-4B and Gemma-2-9B meet point thresholds and frozen-denominator preservation, but fail both destination-interval conditions.No setting, including the admitted one, attains exposure-conditional preservation certification.Source: Q DECLINED AND FAILED SETTINGS · p. 41
Loading 2609.36138v1…