PaperlayerView exact revision on arXiv

Source-linked AI summary

SCIT: Testing Causal Cache Carriers in Latent Chain-of-Thought Models

Yi Ding, Lijun Huang, Menglin Yang

arXiv:2608.27265v1cs.CL

TL;DR

Latent-CoT analyses can test whether latent steps affect answers, but do not identify which transformer object carries the computation. SCIT tests counterfactual cache-segment interventions to identify that carrier. In the main CODI-GPT2 arithmetic checkpoint, transfer runs through late value-cache suffixes, with matched-corruption necessity evidence.

  • Problem

    Existing latent-CoT analyses can test whether a latent step affects an answer, but leave unresolved which transformer object carries that effect.

    Source: §1 · p. 1

  • Method

    SCIT tests exact source–recipient counterfactuals by patching declared cache segments and comparing components and controls to produce a carrier map.

    Source: §1 · p. 1

  • Results

    The main CODI-GPT2 arithmetic checkpoint shows late value-cache suffix transfer with sufficiency and matched-corruption necessity, while the 8B regime shifts to prompt-prefix readout.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The supported generalization is a competence-gated carrier-regime map, not a universal carrier claim or a prediction for new tasks or models before intervention.

    Source: §6 · p. 9

    Evidence centers on variants of synthetic two-hop arithmetic and does not establish broad task generality or latent-tail reasoning outside that regime.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Latent chain-of-thought models move intermediate reasoning from emitted text into continuous states, improving compactness but hiding the causal object. We introduce SCIT, the Suffix Cache Interchange Test, a causal protocol that constructs exact source-recipient counterfactuals, patches declared cache segments, and identifies which transformer object carries the counterfactual computation. SCIT combines sufficiency tests with K/V component splits, hidden-state controls, semantic source controls, decoded validation, and matched corruption. On CODI-GPT2 and a Sim-CoT-style GPT-2 reproduction, counterfactual arithmetic transfers primarily through value-cache suffix trajectories rather than hidden states, keys, reusable answer slots, or single-token triggers. Complete sufficiency-and-necessity evidence for the late-value-suffix mechanism holds for the main CODI-GPT2 checkpoint; the Sim-CoT-style checkpoint shows the same sufficiency and decoded-control pattern but insufficient matched-corruption evidence for a necessity call. Beyond these local arithmetic cells, SCIT reveals carrier-regime shifts: arithmetic-like GPT-2/1B cells preserve latent-tail value/KV transfer, whereas competent 8B and repaired non-arithmetic cells route through prompt-prefix or full-cache K/V; boundary cells receive no mechanism call. SCIT therefore contributes a cache-level diagnostic, a checkpoint-specific GPT-2 arithmetic mechanism, and a competence-gated carrier map rather than a universal latent-tail claim.

1 Introduction

Latent chain-of-thought moves reasoning into continuous states, making internal causal carriers—not readable rationales—the interpretability target. SCIT tests which cache objects carry counterfactual computation and reports checkpoint-specific mechanisms alongside competence-gated carrier regimes.

1 Introduction

  • Moving reasoning from text into continuous states makes rationale-reading inadequate and shifts interpretability toward locating computation inside the transformer.Prior latent-step interventions can show that latent computation matters without distinguishing which transformer object carries the effect (Li et al., 2026).Source: §1 · p. 1
  • SCIT constructs exact source–recipient counterfactuals, patches declared cache segments during latent rollouts, and tests sufficiency alongside component, hidden-state, semantic, decoded, and corruption controls.Its output is a carrier map rather than a single positive patch score.Source: §1 · pp. 1–2
  • SCIT uses controlled arithmetic counterfactuals with known recipient, counterfactual, semantic-control, and random answers as a causal-identification device, not a benchmark-breadth claim.Open-ended datasets rarely provide these exact counterfactuals, partial-variable controls, and known negative labels without extra assumptions.Source: §1 · p. 1
  • SCIT identifies a checkpoint-specific late-value-suffix carrier in CODI-GPT2 arithmetic, sufficiency-only corroboration in Sim-CoT-style GPT-2, and competence-gated carrier shifts or no-calls elsewhere.CODI-GPT2 passes matched-corruption necessity; the Sim-CoT-style checkpoint does not. The broader map distinguishes carrier shifts from replication of the CODI-GPT2 mechanism.Source: §1 · p. 2
  • An active carrier is the smallest gated cache segment sufficient under oracle interchange and unmatched by the main disjoint competitor; carrier shifts and no-call rows mark different limits on interpretation.A carrier shift has transfer but differs from the GPT-2 arithmetic latent-tail carrier; a no-call row fails competence, decoded-behavior, support, or source-control checks.Source: §1 · p. 2

2 Related Work

SCIT builds on work on explicit and latent reasoning, representation inspection, and causal interventions, but focuses on identifying which transformer object carries a counterfactual computation.

Explicit reasoning traces and faithfulness

Latent and implicit chain-of-thought

  • Unlike latent-CoT analyses that treat latent steps as causal variables, SCIT asks which specific transformer object—such as a cache component, segment, or hidden state—implements the causal effect.Latent-CoT methods move reasoning into continuous states (Hao et al., 2024; Wei et al., 2025); recent causal analyses examine latent-step necessity, influence propagation, and representational commitment (Li et al., 2026).Source: §2 · p. 2

Reading hidden states versus causal localization

  • Probing and decoding can show information is recoverable from hidden states, but not that the model used it; SCIT treats decoding as support and relies on causal controls.SCIT uses interchange, component controls, semantic source controls, and matched corruption for its main causal claim.Source: §2 · pp. 2–3

Causal interventions and circuit discovery

  • Although interchange intervention accuracy already tests interventions on prespecified causal variables, SCIT varies intervention targets and controls to discover the transformer-object carrier map.Its intervention grid varies cache segments, K/V components, hidden-state coupling, source semantics, decoded behavior, and matched corruption rather than targeting only a single circuit or benchmark score.Source: §2 · p. 3

3 Method

SCIT defines a cache-patching contract, validates counterfactual readouts against controls, and assigns carrier status only after simpler explanations are tested and task competence is established.

3.1 SCIT protocol

  • SCIT transplants a selected source cache slice into a recipient rollout, testing latent-tail, prompt-prefix, or full-cache segments with K/V and hidden-state variants.The current hidden state is separate from the cache unless explicitly copied; variable-length rollouts align latent positions by reverse distance from their respective stops.Source: §3.1 · p. 3, Figure 1 · p. 4
  • Exact counterfactual sources must transfer while same-answer, partial-variable, one-token, hidden-only, and key-only controls fail to support a context-bound arithmetic trajectory.Synthetic balls, books, and coins prompts vary surface wording within one two-hop arithmetic program; key-only and value-only splits are diagnostic interventions, not natural activation samples.Source: §3.1 · p. 4
  • SCIT scores target win and counterfactual-versus-recipient margin, checks decoded answers, and uses matched random corruption to assess whether a sufficient segment is required.Matched corruption replaces recipient cache entries with same-template, same-range random-source entries and tracks both target win and recipient-answer log-probability drop.Source: §3.1 · pp. 4–5

4 Experimental Setup

The setup tests two GPT-2 latent-reasoning checkpoints using held-out source–recipient pairs, controlled cache interventions, decoded checks, and competence-gated carrier cells. A staged replay reproduces large-N carrier calls with fewer localization configurations, while cached-state staging reduces runtime in one matched 8B cell.

4 Experimental Setup

  • Both checkpoints were tested on balls, books, and coins using held-out pairs, fixed teacher-forced scoring, and decoded checks; Sim-CoT-GPT2 is a CODI-compatible reproduction, not the original released checkpoint.Main localization used 240 held-out CODI-GPT2 balls pairs and two 60-example seeds for other GPT-2 rows; matched-corruption tables aggregate 360 examples, while carrier-regime rows usually use N = 128.Source: §4 · p. 5
  • The evidence battery proceeds through sufficiency, component and source controls, matched corruption, decoded validation, and carrier-regime calibration; targeted repair is used only after task competence is established.Appendix Table 6 details evidential roles, and Appendix Table 8 classifies broader cells as latent-tail positives, carrier shifts, no-call boundaries, or reporting audits.Source: §4 · p. 5
  • Each main GPT-2 diagnostic run used a single GPU, and the same intervention code extended to matched LLaMA3-1B and LLaMA3-8B cells to test carrier persistence, shifts, or failures at scale.Resources are logged per job rather than normalized to a single GPU-hour total across optional repair and calibration runs.Source: §4 · pp. 5–6
  • Competence floors .70/.80/.90 and all 16 audited carrier-threshold settings preserved 12/12 large-N calls; exact and near ties were rare (0.063%/0.072%).The main text reports point estimates because oracle-versus-control gaps were large; appendix captions provide row-level SEMs or bootstrap intervals where relevant.Source: §4 · pp. 5–6
  • The predeclared staged replay recovered all 12 large-N carrier calls (5 latent-tail, 7 prompt-prefix) while cutting localization configurations from 108 to 60 (44.4%); cached-state staging reduced one matched 8B cell’s wall time by 34.6–40.0%.The replay tests oracle K/V over latent suffix, prompt prefix, and whole cache before component checks on the selected segment. Naive staging changed runtime by at most 1.8%; cached-state staging raised peak CUDA allocation from 16.5 to 17.3 GiB, and the timing result is not an end-to-end claim.Source: §4 · pp. 6–7

5 Results

Results identify a late value-cache suffix as the local arithmetic carrier in the main CODI-GPT2 checkpoint, while carrier calls shift across scales and tasks. Training-path variation and competence boundaries limit generalization; reporting audits leave the displayed carrier map unchanged.

5.1 Local arithmetic carrier

  • CODI-GPT2’s layers 8–9 value block reaches 0.875–0.908 target win versus 0.017–0.029 in layers 10–11; Sim-CoT-GPT2 shows 0.958–0.992 versus 0.133–0.192.CODI-GPT2 requires a multi-token suffix: one token reaches 0.203, while k ≥3 reaches 0.984–1.000. Matched corruption supports sufficiency and necessity only for CODI-GPT2 (recipient target win 0.211; log-probability drop 13.655); Sim-CoT-GPT2’s drop to 0.606 is directional necessity support only.Source: §5.1 · p. 7

5.2 Carrier-regime boundaries

  • Competent 8B arithmetic, entity, relation, and expression/list cells shift to prompt-prefix or full-cache K/V, with prompt-prefix K/V at 1.000 and latent-tail K/V at 0.000 in main rows.Clean three-hop and public/matched 1B arithmetic-like cells retain value/KV transfer. Follow-ups indicate visible prompt fields are sufficient before rollout and remain readout-active after latent encoding; answer-completion timing remains ambiguous.Source: §5.2 · p. 7, Figure 3 · p. 8
  • On the relation-chain task, latent/prompt/whole-cache K/V target win is 0/1/1 at both 1B and 8B, selecting prompt/source binding across three equivalent renderings.At 8B, decoded transfer is 0/1 for latent/prompt patching, while recipient preservation after matched corruption is 1/0.Source: §5.2 · p. 8
  • No-call cells remain outside the mechanism claims, including Qwen3-4B probes, graph-path and broader finite-state cells, ultra-OOD variants, fixed-rule holdouts, and high-random-baseline wide-OOD rows.These are reported as boundary or secondary evidence rather than treated as failed replications.Source: §5.2 · p. 8
  • In a held-out N = 48 variable-stop pilot, reverse-aligned latent K/V+h transfer succeeded on 47/48 pairs, including 26/27 unequal-stop pairs, while prompt K/V alone succeeded on 0/48.k = 2 was the shortest tested window supported in both stop groups; this supports a joint carrier in the pilot, not value-specific necessity or a stopping-policy-invariant mechanism.Source: §5.2 · p. 8

5.3 Path sensitivity and reporting audits

  • Three newly trained CODI-style seeds had weak late-value sufficiency (0.049–0.138) and corruption drops (0.026–0.089), with no localized-carrier call despite whole-cache transfer.Their latent/prompt/whole-cache target-win rates were .370/.203/.870, .208/.427/.865, and .427/.141/.906; neither disjoint segment reached the .80 gate. Range-augmented T = 16 also produced two competent seeds with different carrier basins.Source: §5.3 · pp. 8–9
  • Reporting-rule sweeps preserve the same 5 latent-tail and 7 prompt-prefix calls, with exact top-score ties in only .063% of retained cache-interchange score rows.The audits vary competence gates over .70/.80/.90 and sufficiency floors .75–.90 with competitor ceilings .10–.30; they address threshold and tie-breaking explanations, not universality.Source: §5.3 · p. 9

6 Discussion

SCIT identifies a checkpoint- and competence-dependent carrier rather than a universal latent-tail mechanism. The discussion distinguishes a controlled GPT-2 arithmetic result from an 8B prompt-prefix regime and outlines future evaluation and training extensions.

6 Discussion

  • In the studied GPT-2 arithmetic checkpoints, the best operational description is a context-bound value-cache trajectory, not hidden-state-only, key-only, or reusable-slot transfer.Value-only success establishes sufficiency under recipient routing and downstream readout; it does not imply natural inference uses isolated values or inert keys, nor identify a symbolic trace or single-head circuit.Source: §6 · p. 9
  • In 8B cells, prompt-prefix state is sufficient before latent rollout, and post-prefix ablations support visible-field readout rather than latent-suffix storage.Zeroing prompt-prefix K/V or task-relevant visible spans collapses answer readout, while zeroing the latent suffix does not; early completion remains possible because the pre-rollout prompt state is already sufficient.Source: §6 · p. 9
  • SCIT supports a focused GPT-2 arithmetic mechanism and broader diagnostic framework, but not a universal carrier map or prediction for new tasks and models.It complements Coconut, CODI, and Sim-CoT training papers (Hao et al., 2024; Wei et al., 2025) by testing which transformer object carries a known counterfactual computation after training; seed/path audits rule out a single latent-tail map as a deployment guarantee.Source: §6 · p. 9
  • A proposed GSM8K extension would use executable solution graphs to verify counterfactual answers, cross graph edits with rendering variants, and retain competence and transfer controls.This is an evaluation pathway, not evidence of GSM8K robustness in the present paper.Source: §6 · p. 9
  • For loop or recurrent models, SCIT could remap carriers by iteration or reverse distance from stopping, while a training extension could optimize clean and interchange losses.Evaluation would retain unseen depths, loop counts, seeds, and matched-corruption necessity as criteria.Source: §6 · p. 9

Limitations

SCIT’s evidence comes from controlled synthetic tasks and checkpoint-specific mechanisms, so it does not establish broad task or recipe generality. The experiments also leave open how carrier patterns vary across architectures, scales, stopping policies, and decoding conditions.

Limitations

  • SCIT’s controlled cells are variants of a synthetic two-hop arithmetic diagnostic, not a broad benchmark, and do not establish generality to natural multi-step word problems.Expanded cells expose carrier shifts and competence boundaries, but do not establish latent-tail reasoning beyond the arithmetic regime.Source: Limitations · pp. 9–10
  • The variable-stop pilot freezes clean stopping decisions and lacks matched-corruption necessity evidence, so its joint K/V+h finding is specific to one stopping policy.It tests one synthetic checkpoint with realized stops 3–5 and does not show learned, native, or open-ended halting support.Source: Limitations · p. 10
  • Cross-checkpoint transfer is not universal: LLaMA3-1B supports all-layer value-cache transfer, LLaMA3-8B supports full-cache/prompt-prefix transfer, and Sim-CoT depends more on source hidden state at decode time.The OOD numeric-range check reduces small-integer concerns, but its high random-source baseline makes it secondary.Source: Limitations · p. 10
  • The late-8–9 value-carrier result is evidence for the realized original CODI-GPT2 checkpoint, not a stable training-recipe property.Sim-CoT-GPT2 has strong sufficiency and decoded-control support but only directional matched-corruption damage; newly trained high-accuracy CODI-style seeds do not reproduce the carrier.Source: Limitations · p. 10
  • The Qwen3-4B result is a calibration boundary, not mechanism evidence or an explanation of failure, and the apparent 1B/4B/8B pattern is not a scale law.Architecture, tokenizer, checkpoint provenance, and repair recipe differ across the LLaMA-family rows and Qwen3-4B; resolving the pattern requires matched factors and competence gates.Source: Limitations · p. 10
  • Scale generalizability requires competence-gated diagnostics across architectures and sizes with architecture-relative layer partitions; broader unconstrained decoding and downstream behavior also remain untested.Main controls are validated under greedy decoding and only a limited temperature-sampling diagnostic.Source: Limitations · p. 10

Ethics Statement … A.4 Core Arithmetic Contract

SCIT’s appendix documents a checkpoint- and competence-dependent carrier map, while limiting causal claims to tested settings. It lays out the intervention, scoring, control, and audit evidence used to assess those claims.

Ethics Statement; A.1 How to Read the Appendix

  • The appendix limits claims to tested checkpoints, templates, and interventions, warns against treating controlled localization as a guarantee of faithful deployed reasoning, and classifies no-calls by distinct failed prerequisites.Its appendix map distinguishes calibration cells and boundary rows from broad benchmark-generalization claims.Source: Ethics Statement · p. 10, §A.1 · p. 12

A.2.1 Intervention and Prompt Definitions; Appendix block Tables / figures Purpose

  • The intervention contract crosses cache components with cache-only or cache-plus-hidden-state handling and applies them to latent-tail, prompt-prefix, or whole-cache segments.Arithmetic templates preserve the same count computation across books and coins, while varying surface nouns and transfer wording.Source: §A.2.1 · p. 12, Figure 4 · p. 13, Table 4 · p. 13

A.2.2 Scoring and Semantic Source Construction

  • Localization scores fixed candidate answers using summed teacher-forced log-probabilities, with semantic controls separating oracle sources from sources matching only answers or partial variables.Matched random sources are aligned by template family, numeric range, latent step, and cache-segment shape; the two GPT-2 checkpoints are local reproductions, not off-the-shelf artifacts.Source: §A.2.2 · p. 12

A.3.2 Concern-to-Evidence Map; A.3.3 Carrier Ledger; A.3.6 Method Positioning

  • The appendix positions SCIT as a combined causal contract and provides a concern map and carrier ledger that place gates, sufficiency, necessity, decoded validation, failed-gate controls, and reporting audits together.The ledger is intended to apply the same evidential contract to large-model rows and GPT-2 arithmetic rows.Source: §A.3.2 · p. 18, §A.3.3 · p. 18, §A.3.6 · p. 18

A.3.4 8B Field Routing; A.3.5 Metric Audit

  • The 8B field-routing and metric audits test whether prefix-carrier calls reflect visible prompt fields and whether the large-N split depends on competence gates, carrier thresholds, or tie order.Visible lookup, fact, rule, or query fields can recover transfer while latent-suffix K/V remains inactive; the reported threshold changes do not alter the 1B/8B split calls.Source: §A.3.4 · p. 18, §A.3.5 · p. 18, §A.3.7 · p. 19

A.3.1 Carrier-Status Mini Tables

  • Carrier calls are competence-gated: arithmetic-like 1B cells preserve value/KV transfer, while competent 8B and repaired task cells shift toward prompt-prefix or full-cache carriers; failed-gate cells receive no call.The 8B follow-up finds prompt-prefix K/V sufficient from the first scanned latent step while latent-suffix K/V remains inactive; no-latent scoring favors visible-field readout but does not rule out early completion.Source: Table 9 · p. 16, Table 10 · p. 16, Table 11 · p. 16

A.3.7 Evidence Map

  • The evidence map notes unresolved scope limits: localization varies by architecture, high-baseline OOD cases use continuous margins, and decoded checks support exact-context arithmetic rather than broader deployment claims.It also notes mixed 1B readout diagnostics and that prompt-prefix timing evidence favors visible-field readout without ruling out early completion.Source: §A.3.7 · p. 19

A.4 Core Arithmetic Contract; A.3.7 Evidence Map

  • The evidence map reports weak hidden-only and key-only patches, arithmetic transfer through value/cache replacement, and multi-token suffix evidence, with matched corruption as the primary necessity test.The core arithmetic contract is supported by transfer, semantic-source controls, and matched-corruption results; enlarged candidate sets and sampled decoding check that the semantic-control separation is not a small-candidate artifact.Source: §A.3.7 · p. 19, §A.4 · p. 22

A.5 Scope, Generalization, and Repair Boundaries · A.6 Trajectory Content and Auxiliary Carrier Checks

SCIT’s scope tests distinguish controlled causal diagnosis from broad task generalization, and show that competent expanded tasks can use different cache carriers. Auxiliary trajectory analyses refine the arithmetic carrier account but do not replace causal interchange, corruption, or decoded validation.

A.5 Scope, Generalization, and Repair Boundaries

  • SCIT’s controlled task suite prioritizes exact source semantics over broad reasoning-benchmark coverage.The roadmap distinguishes evidence for broad mechanism generalization from evidence for a reliable carrier-regime diagnostic.Source: §A.5 · p. 22, Table 22 · p. 24
  • Arithmetic transfer survives free decoding, a clean three-hop task, and the public 1B checkpoint, while localization varies by architecture and non-arithmetic calls require competence.Augmentation and targeted repair use the same task families, so they test whether competence restores transfer rather than provide independent generalization evidence.Source: §A.5 · p. 22
  • Relation-chain can use prompt-prefix source-answer carriers at 1B/8B, whereas minimal finite-state and high-update fixed-rule cells localize to visible prompt/source-cache regimes.In repaired 8B expression/list cells, successful prompt-prefix patches transfer changed problem fields; latent-suffix and answer-prefix spans remain ineffective.Source: §A.5 · pp. 23–24
  • Competent 8B expression/list repairs shift transfer to prompt-prefix/full-cache K/V, while widened support helps far-OOD deployment but leaves ultra-OOD cells failing or source-control limited.Existing checkpoints show no zero-shot task breadth and targeted 1B repair is insufficient; ultra-OOD expression-tree becomes a no-call, and list-filter/sum remains source-control limited.Source: §A.5 · p. 23
  • Fixed-rule sequence holdouts fail the free-generation gate despite competence on seen supports, making support generalization a boundary rather than a positive mechanism claim.The same-horizon experiment removes a small part of the support to test whether the repair executes the rule beyond seen examples.Source: §A.5 · p. 24

A.6 Trajectory Content and Auxiliary Carrier Checks

  • 48/48 donor-label selections in each passing relation-chain rendering show competence-conditional stability, not unrestricted paraphrase or strategy robustness.Five competence-passing checkpoint–form cells preserve prompt-prefix rather than latent-suffix transfer across original, compact, and brief renderings.Source: Table 24 · p. 25
  • Arithmetic transfer requires a short cache window in CODI and multiple latent positions in Sim-CoT, consistent with a suffix trajectory rather than a single-token trigger.Reverse-aligned interchange used independently realized stops under a stopping rule fixed before the held-out intervention slice and independent of labels or correctness.Source: §A.6 · p. 25
  • Ranked heads recover transfer faster than random same-size sets, while artifact checks address lexical, positional, readout, and generic cache-fragility alternatives without serving as standalone causal proof.Training-path tests find that stronger and range-augmented recipes can recover arithmetic transfer, but adjacent recipes, competent seed replicates, and farther-OOD cells expose carrier-basin and deployment boundaries.Source: §A.6 · p. 25

A.7 Carrier-Regime and Scale Details

The carrier regime varies by task, checkpoint, and scale: arithmetic-like cells retain latent-tail value/KV transfer, while competent non-arithmetic and matched 8B cells use prompt-prefix or full-cache K/V. Matched corruption supports this shift, while weak-competence cells remain outside mechanism claims.

A.7 Carrier-Regime and Scale Details

  • Weak-competence graph cells and original entity/graph checkpoints are boundaries, not evidence for a carrier mechanism; the public 1B three-hop row does not establish semantic-source separation.The calibration summary specifies that the public 1B three-hop row supports deeper arithmetic transfer, but its semantic controls are not below the oracle; original entity and graph rows fail the competence screen.Source: Table 25 · p. 26, §A.7 · p. 26, Table 27 · p. 27
  • 1B arithmetic-like and conditional cells transfer through latent-tail value/KV, while entity lookup, repaired relation-chain, and competent 8B cells use prompt-prefix or full-cache K/V.For repaired relation-chain, prompt-prefix, query-only, and whole-cache patches reach 1.000 target win at both scales, while latent-suffix K/V reaches 0.000; matched corruption damages the corresponding carrier segments.Source: §A.7 · p. 26
  • Layers 8–9 are the strongest restricted two-layer block for the main GPT-2 late split; the final 10–11 block is weak, and four-layer windows containing 8–9 nearly recover the full effect.The late-value pattern also appears at larger OOD numeric values, strongest for CODI-GPT2; Sim-CoT-GPT2 has the same direction but a larger nonlate baseline.Source: §A.7 · pp. 26–27
  • At 1B scale, replacing the current hidden state alone is insufficient, whereas replacing the full latent-cache trajectory is sufficient or near-sufficient.All-layer value-cache transfer persists at much larger numeric values on public and matched 1B checkpoints, but high random-source wins make this robustness evidence rather than semantic-source separation.Source: §A.7 · p. 27
  • For repaired 8B expression-tree and list-filter/sum tasks, transfer follows visible prompt fields rather than latent-suffix or answer-prefix spans.Expression-tree follows source operand/fact bindings; list-filter/sum follows rule/fact and threshold/filter fields, while ultra-OOD remains a competence or source-control boundary.Source: Table 28 · p. 27, Table 30 · p. 28

A.8 Component, Source, and Decoded-Behavior Details

The appendix finds that final-step counterfactual transfer is carried primarily by late value-cache suffixes, not by the current hidden state or keys. Exact-context sources preserve the effect across lexical domains and decoded-behavior checks, while Sim-CoT’s matched-corruption evidence remains too weak for a full necessity call.

Cache and component diagnostics

  • At the final latent step, late-value cache-only patching reaches 0.469/0.998, versus 0.002/0.006 for nonlate value cache-only patching.The source hidden state is mainly needed one step earlier.Source: §A.8 · p. 28, Table 48 · p. 35
  • At the final step, CODI-GPT2 hidden-state-only replacement reaches 0.246/0.000, versus 0.704/1.000 for latent-cache-only replacement; Sim-CoT shows the same qualitative pattern.This indicates that replacing the current hidden state alone does not explain final-step counterfactual transfer.Source: §A.8 · p. 28, Table 49 · p. 35, Table 50 · p. 35
  • At the final latent step, value-only suffix patching is sufficient, whereas key-only patching fails even when the current hidden state is also replaced.The diagnostic splits the late cache into key and value components.Source: §A.8 · p. 28, Table 51 · p. 36

Cross-template source controls

  • Across CODI-GPT2 and Sim-CoT-GPT2, exact-context cross-template sources preserve the oracle effect, while partial-variable controls remain much weaker.The controls test transfer across lexical domains while varying whether the source preserves the exact counterfactual arithmetic context.Source: §A.8 · p. 28, Table 52 · p. 36, Table 53 · p. 36

Decoded-behavior validation

  • Under greedy free decoding, exact-context patches decode the counterfactual answer at near-oracle rates, while partial-variable controls almost never do.This pattern is reported for both CODI-GPT2 and Sim-CoT-GPT2.Source: §A.8 · p. 28, Table 54 · p. 37, Table 55 · p. 37
  • With sampled decoding, oracle and cross-template contexts produce the counterfactual answer, while partial controls mostly produce answers outside the candidate set.Sampling used five decodes per patched state, with temperature 0.7, top-p 0.95, and top-k 40.Source: §A.8 · pp. 28–29, Table 56 · p. 38

Matched-corruption replication

  • Sim-CoT’s matched-corruption result has the same sign as CODI-GPT2 but is weaker, so it supports directional necessity evidence rather than a full sufficiency-and-necessity call.The replication selectively damages the late-value suffix.Source: §A.8 · p. 29, Table 58 · p. 38

Decoded-behavior validation

  • Cache-only late-value patches preserve the decoded counterfactual effect in CODI-GPT2 and a substantial effect in Sim-CoT-GPT2, where source hidden-state replacement improves readout.The check tests whether decoded behavior follows the transplanted cache rather than only the supplied source hidden state.Source: §A.8 · p. 29, Table 57 · p. 38

B Reproducibility Notes

The SCIT repository provides the code, configurations, generators, command paths, and artifact builder for reproducing the paper’s analyses and tables. It does not include local model checkpoints or large generated JSONL outputs.

B Reproducibility Notes

  • The public reproduction workflow generates SCIT pairs, runs analysis or carrier-split runners, and rebuilds markdown and CSV tables from JSONL outputs.The repository includes the code, configs, generators, and artifact builder; listed evidence spans GPT-2 arithmetic, necessity tests, cache diagnostics, free-generation checks, carrier splits, repair diagnostics, and SEM summaries.Source: B Reproducibility Notes · p. 39
  • The repository omits local model checkpoints and large generated JSONL outputs.Code and exact command paths are publicly available at the SCIT repository.Source: B Reproducibility Notes · p. 39

C Compute

SCIT’s staged-plus-cached workflow reduces wall time in the tested 8B cell, while a held-out GPT-2 template run requires roughly tens of minutes to under one GPU-hour on one GPU. The broader calibration package additionally includes 1B/8B training and repair jobs.

C Compute

  • A two-seed GPT-2 template run takes roughly tens of minutes to under one GPU-hour on one modern academic GPU.Each held-out localization or semantic diagnostic run uses one GPU; the broader calibration package includes additional 1B/8B training and repair jobs.Source: C Compute · p. 39
  • On one H100, staged+cached execution reduced full-grid wall time by 40.0% and 34.6% across two execution orders.The paired times were 907.1 →544.6 s and 822.9 →538.1 s; all Stage-1 and Stage-2 records matched. Caching 192 clean states raised peak CUDA allocation from 16.462 to 17.291 GiB, while staging without reuse changed wall time by only −1.5% to +1.8%, attributing the speedup to staging plus caching.Source: C Compute · p. 39

Related papers

Loading 2608.27265v1…