Source-linked AI summary

Reading Is Not Using: Retrieval, Judgment, and the Design of AI Financial Research Workflows

Miao Liu, Zhizhe Liu

arXiv:2608.24842v1cs.CLcs.AI

TL;DR

Financial AI systems may retrieve information without using it in investment judgments. The paper examines how information moves through an AI analyst’s processing stages and finds that judgment influence can fall to the empirical noise floor despite retrieval, revealing that reading is not using.

  • Problem

    It remains unclear whether information an AI analyst can retrieve actually influences its investment judgment.

    Source: §1 · p. 2
  • Method

    The paper studies how an AI analyst carries information across internal processing stages from document reading to final recommendation.

    Source: §2.1 · p. 9
  • Results

    Information integration does not keep pace with retrieval, reaching the empirical noise floor through 128,000 tokens and showing essentially no influence in a complete filing despite retrieval.

    Source: §1 · pp. 3–4
  • Takeaways & Limitations

    Reading and downstream decision influence separate sharply, so an AI analyst’s decision context need not equal its advertised context window.

    Source: §1 · p. 4

    The paper bounds its mechanism claims to AI processors rather than human analysts.

    Source: §1 · p. 7

Abstract

from arXiv · show

Large language models (LLMs) are increasingly deployed as AI analysts to process financial disclosures and support AI-assisted investment decisions. Yet such systems are usually evaluated by what they can retrieve, not whether retrieved information affects their judgments. We identify a retrieval-integration gap in long-context financial analysis. Holding focal-firm information fixed and varying only unrelated context from 2,000 to 128,000 tokens, we find that a risk disclosure's influence on investment judgments falls to the experimental noise floor even as direct retrieval remains accurate. The pattern replicates across model families and judgment tasks and in experiments removing real disclosures from actual 10-K filings. More capable models postpone but do not eliminate the gap. Causal memory interventions show that compressed summaries and source-text lookup jointly transmit disclosures into judgments. Workflow architecture determines whether this transmission succeeds: chunk-and-summarize pipelines evict relevant information, whereas a targeted, structured restatement adjacent to the decision restores its influence. AI analyst performance is therefore jointly determined by model capability and workflow architecture. Retrieval-based evaluations can certify systems whose investment judgments ignore information they demonstrably retrieved.

1 Introduction

The paper examines a retrieval–integration gap in AI financial analysis, where systems retrieve decision-relevant disclosures but fail to incorporate them into investment judgments. It focuses on how model capability and workflow architecture determine whether retrieved information becomes decision-useful.

Core finding

  • Disclosure influence falls to the noise floor as context expands, despite accurate retrieval, showing that reading and using information are distinct stages.For the primary model, the disclosure raises sell probability by 3.2 percentage points at 2,000 tokens but becomes indistinguishable from neutral text between 8,000 and 32,000 tokens through 128,000 tokens.Source: §1 · p. 3

Judgment behavior

  • Long context compresses judgment sensitivity without destroying risk ordering: response range shrinks 5.6-fold while severity ranking degrades much less.At full length, the model can still rank risks but no longer scales judgments to their severity.Source: §1 · pp. 3–4

Robustness and capability

  • The gap replicates across model families and real 10-K disclosures, while higher capability postpones failure rather than eliminating it.The largest open-weight model retains a 3.4-percentage-point effect at 128,000 tokens, whereas real disclosures affect judgments in short excerpts but essentially not complete filings despite perfect retrieval.Source: §1 · p. 4

Mechanism

  • Representation analyses locate the failure at integration: disclosure content remains encoded and retrievable, but decision-position legibility and transmission weaken as context grows.Causal interventions implicate both compressed recurrent state and source-text lookup as transmission channels, with statistically indistinguishable effects.Source: §1 · pp. 4–5
  • Causal memory interventions show that removing disclosure content from compressed state removes roughly two-thirds of its influence, while implanting it restores nearly half.Attention-blackout experiments identify a disclosure-specific lookup effect of comparable magnitude.Source: §1 · pp. 4–5

Workflow design

  • Workflow architecture determines whether retrieved information reaches judgment: chunk-and-summarize pipelines erase influence, whereas targeted structured restatement restores it.At 128,000 tokens, placing decision-relevant facts immediately before judgment raises influence to 8.5 percentage points, with all twelve firms responding in the predicted direction.Source: §1 · p. 5
  • AI analyst performance is jointly determined by model capability and workflow architecture: capability moves the failure frontier, while architecture determines whether decision-relevant information reaches judgment.The information artifact, model, and surrounding workflow jointly determine whether available information becomes consequential action.Source: §1 · pp. 5, 7
  • The retrieval–integration gap is a post-delegation failure in which agents retrieve decision-relevant information but fail to incorporate it into delegated judgments.Successful retrieval therefore does not imply completion of the delegated objective, especially in multistage tasks with limited human visibility.Source: §1 · p. 6
  • Retrieval accuracy is an incomplete evaluation proxy because retrieved information need not affect the supported decision; marginal decision influence measures that effect counterfactually.The measure holds other decision-relevant information fixed and can reveal judgments effectively invariant to retrieved facts.Source: §1 · p. 6
  • Two transmission channels—a compressed running summary and attention-based lookup—show that context load can disrupt how disclosures reach judgments.A decision-proximal representation principle follows: extract critical information into a targeted, structured representation placed at the point of judgment.Source: §1 · pp. 6–7
  • Generic chunk-and-summarize workflows and additional reasoning fail to restore disclosure influence, whereas targeted, structured restatement succeeds, including in complete filings with real disclosures.Experimenter-written restatement works as well as the model’s own extraction, and the same model, information, and task produce different judgments solely because routing differs.Source: §1 · pp. 5, 7
  • The contribution distinguishes machine readability from machine decision usefulness: retrieval becomes inexpensive, but integration remains sensitive to document length, model capability, and workflow architecture.AI analysts should be validated by whether economically meaningful information changes judgments, not merely by factual-answer accuracy.Source: §1 · p. 7

2 Background and Hypotheses

This section frames delegated financial analysis as a transmission problem, distinguishing retrieval from the influence of retrieved disclosures on judgments. It motivates evaluating disclosure influence under fixed-information conditions and hypothesizes that unrelated context, position, and workflow organization shape whether retrieved information is integrated into decisions.

2.1 Delegation to an Agentic IS Artifact

  • Delegation transfers information acquisition and analysis to an agentic artifact whose internal transmission stages are hidden from the user.The artifact reads documents, forms an internal representation, and produces a judgment; retrieval can succeed while the delegated objective fails.Source: §2.1 · p. 9

tial

  • Financial disclosure effectiveness requires both acquiring information and integrating it into a judgment, so machine readability need not imply machine decision usefulness.The processing-cost framework treats acquisition and integration as separate costs, while conditional retrieval measures whether a named item can be located rather than whether it changes the decision.Source: tial · pp. 10–11

2.3 Evaluating a Delegated Decision Task

  • Retrieval accuracy is an incomplete evaluation proxy because it certifies one processing stage without showing that retrieved content reached the delegated judgment.The proposed outcome measure is marginal decision influence: the change in judgment attributable to a disclosure while holding other decision-relevant information fixed.Source: §2.3 · p. 12
  • Delegated decision evaluation should measure whether retrieved information changes judgment against an explicit absent-information counterfactual while holding focal-firm information fixed and varying surrounding load.The framework distinguishes “the model read it” from “the model used it,” because internal decoding or retrieval does not establish causal influence on output.Source: §2.3 · pp. 12–13

2.4 Hypotheses

  • H1 predicts that, with the focal firm’s information fixed, a risk disclosure’s marginal influence on investment judgment declines as unrelated context grows, even when the full filing remains visible.The hypothesis treats additional unrelated material as competing for finite representational capacity, while drawing only a predictive analogy to limited attention.Source: §2.4 · p. 13
  • H2 predicts that disclosure influence can fall while accurate retrieval persists, identifying a post-delegation integration failure rather than a reading deficit.This distinction exposes a construct gap in retrieval-based evaluation: systems may pass retrieval tests yet produce judgments invariant to retrieved facts.Source: §2.4 · p. 14
  • H3 predicts that filler arriving after a disclosure reduces its influence more than identical filler arriving before it, localizing degradation to within-context retention.H3 is a pre-specified secondary comparison, and a null behavioral difference would not exclude retention failures detectable through internal measurements.Source: §2.4 · p. 14
  • H4 predicts that reorganizing workflow architecture restores disclosure influence at long context, whereas adding computation does not.The proposed routing intervention is to require a targeted statement of what the filing says about the risk immediately before deciding; generic pre-decision summarization may worsen integration.Source: §2.4 · pp. 14–15

3 Experimental Design

This section presents an experimental design for testing how a known risk disclosure affects AI analysts’ investment judgments as unrelated context varies, while separating retrieval from decision influence. It details the controlled materials and manipulations, leakage controls, inference procedures, model selection, and instrumentation used to assess these effects across context lengths and model families.

3 Experimental Design

  • The design measures a disclosure’s marginal influence on investment judgment separately from retrieval, making the retrieval–integration gap directly observable.Retrieval and decision readouts come from distinct calls, so stating a disclosure and using it in judgment are independent measurements.Source: §3 · p. 15, §3.1 · p. 17

3.1 The Measurand

  • Marginal decision influence is the judgment difference between a target filing containing one risk disclosure and the average of five otherwise identical neutral replacements.The measure is defined on the delegated decision task rather than retrieval accuracy and remains in that task’s own units, regardless of whether the model can articulate the disclosure.Source: §3.1 · pp. 15–16

3.2 Materials

  • Materials use annual reports from twelve U.S. registrants with authored, quantified risk paragraphs that pass whole-filing coherence audits and a single-edit rule.The disclosures cover locally describable exposures such as covenant headroom, litigation settlements, indemnification caps, and similar risks.Source: §3.2 · p. 17

3.3 The Length Manipulation

  • The main length manipulation holds the firm’s information set fixed, retaining the target or neutral paragraph, local section context, and decision prompt while adding genre-matched, economically unrelated text at 2,000, 8,000, 32,000, or 128,000 tokens.Source: §3.3 · p. 18
  • Redacted-document directed retrieval tests whether answering without the disclosure reflects leakage or memorization rather than document-based retrieval.The redacted version serves as a probe for non-document sources of answers.Source: §3.3 · p. 19

3.4 Inference

  • Inference combines firm-level bootstrap intervals for mean Use(ℓ) with randomization inference against pseudo-insertions that establish a length-specific noise floor.Because insertion itself perturbs the deterministic reader, a null at 128,000 tokens is interpreted as lost influence only when shorter lengths resolve nonzero effects; unresolved effects are reported as such, not as informative zeros.Source: §3.4 · pp. 19–20

3.5 Models

  • The model set centers on Qwen3.5, with 9B and 27B variants forming a scale ladder, while behavioral results report both 9B variants side by side.Each model passes a pre-specified capability gate.Source: §3.5 · p. 20

4 The Retrieval–Integration Gap Under a Fixed Infor-

This section examines how increasing unrelated context affects the integration of retrieved disclosures into investment judgments across constructed and real filings, models, and context manipulations. It distinguishes retrieval–integration gaps from retrieval failure while assessing the roles of model capability, context load, repetition, proximity, and semantic interference.

mation Set

  • The retrieval–integration gap persists for disclosures drawn from firms’ actual filings, while the fixed-information design rules out rational updating on newly added content.The section also shows that the decline is not specific to one judgment phrasing or model family.Source: mation Set · p. 21, §4.1 · pp. 22–23

4.1 Declining Decision Influence as Context Grows

  • At 2,000 tokens, the 9B model’s disclosure effect is +0.032 sell-probability units, falling to +0.005, +0.001, and +0.004 at 8,000, 32,000, and 128,000 tokens.Against irrelevant-insertion noise, the effect is indistinguishable from the null at 32,000 and 128,000 tokens; the decline replicates across model families.Source: §4.1 · pp. 21–22

4.2 What the Disclosure Still Does: Ordinal Without Cardinal

  • At 128,000 tokens, the response range collapses from 0.23 to 0.04 sell-probability units, while pairwise severity ordering falls only from 0.93 to 0.79.The surviving signal is therefore ordinal: the model still ranks more severe disclosures as worse but transmits little of their magnitude.Source: §4.2 · p. 24

4.3 The Decline Is Not Retrieval Failure

  • At 128,000 tokens, the 9B model retrieves the disclosure for twelve of twelve firms, yet its behavioral use is statistically indistinguishable from an irrelevant insertion.The divergence between accurate retrieval and absent judgmental influence defines the retrieval–integration gap.Source: §4.3 · pp. 24–25
  • Directed retrieval remains essentially perfect across lengths, but integration into revised judgments fails, so retrieval accuracy does not establish decision usefulness.An artifact audited only on what it can state would pass Table 3 while its investment judgments remained invariant to retrieved facts.Source: §4.3 · p. 25

4.4 Real Disclosures, Surgically Redacted

  • Removing a real disclosure shifts investment judgments by +0.037 on average in roughly 2,000-token excerpts, versus +0.000 for complete filings; directed retrieval passes for twenty filings.Among sixteen unambiguously adverse disclosures, the excerpt effect is +0.042 with an interval excluding zero.Source: §4.4 · p. 26

4.5 Capability Moves the Horizon Outward

  • The integration horizon moves outward with capability: Gemini 3.1 Flash-Lite’s disclosure-use effect falls from +0.165 at 2,000 tokens to +0.029 at 128,000 tokens.The 27B model uses the disclosure far more at every length, reaching +0.034 at 128,000 tokens, roughly equal to the 9B model.Source: §4.5 · pp. 27–28

4.6 Secondary and Additional Analyses

  • At lengths past the failure horizon, bringing the disclosure within 1,000 tokens of the decision recovers nothing; below the horizon, contrasts from 1,000 to 5,500 tokens change use by less than 0.0001.The contrast has 95% CI [−0.007, +0.009] and exact p = 0.99.Source: §4.6 · p. 29
  • Increasing total context from 8,000 to 32,000 lowers use by 0.008 (CI [+0.002, +0.015], p = 0.03), whereas a moderate distance change does not.This supports total context load as a driver, though it does not fully distinguish the proposed explanations.Source: §4.6 · p. 29
  • Repetition partially refreshes disclosure influence: four occurrences help in pooled length cells, but at 128,000 tokens they provide less than a third of the influence of one occurrence at 2,000 tokens.Two occurrences do not improve every cell, so repetition is neither uniform nor restorative.Source: §4.6 · p. 29
  • Semantically similar risk disclosures intensify interference: use is +3.7 versus +5.5 percentage points at 2,000 tokens and turns negative at 8,000 tokens (−4.3).The local 9B model shows no such separation, suggesting susceptibility to semantic interference varies with capability.Source: §4.6 · p. 29
  • Point estimates turn negative with confidence intervals excluding zero for Gemma at 32,000 tokens (−0.012) and Llama at 128,000 tokens (−0.006), making models marginally less inclined to sell.Because roughly forty cells were tested, the authors treat these sign inversions as suggestive rather than definitive.Source: §4.6 · p. 30

5 Mechanism: Where the Disclosure Is Lost

This section locates the retrieval–integration gap by distinguishing reading, transmission, and judgment failures, then measures internal states with two independent instruments. The instruments separately assess readable words and present concepts at chosen positions.

5 Mechanism: Where the Disclosure Is Lost

  • Three mechanisms could explain the gap: the disclosure may never be represented, fail to reach the judgment, or be carried forward but underweighted.These correspond to reading, transmission, and judgment failures, which imply different remedies.Source: §5 · p. 30

5.1 Two Instruments

  • The analysis uses two independent instruments to inspect internal states directly: a vocabulary-space probe and an interpretable feature dictionary.The approach exploits access to intermediate states in an AI analyst artifact, unlike archival studies of human information processing.Source: §5 · p. 30, §5.1 · pp. 30–31
  • The vocabulary-space probe asks which words can be read from an internal representation, at any document position and model depth (Gurnee et al., 2026).The paper identifies this as the first application of the method in information systems research.Source: §5.1 · pp. 30–31
  • The feature dictionary measures whether selected risk concepts are present by tracking sparse-autoencoder directions activated by covenant, default, and litigation language (Deng et al., 2026).The instruments are estimated independently and measure different properties: readable words versus present concepts.Source: §5.1 · p. 31

Appendix C.

This section examines how disclosures are transmitted from local representations into decisions, identifying source-text lookup and recurrent summaries as key channels. It shows how both channels weaken with longer contexts, motivating workflow designs that place decision-relevant information nearer to judgment.

Appendix C.

  • The probes are applied at both the disclosure position and the decision position, separating where the target paragraph is read from where the recommendation is produced.The decision position is defined as the final token before the model produces its recommendation.Source: Appendix C · p. 31

5.2 The Disclosure Stays Encoded Where It Was Read

  • At the disclosure position, risk information remains encoded across document lengths, with stable probe ranks and flat, target-specific feature activation from 2k to 128k tokens.The median rank of the best risk word is approximately 80, 52, and 67 at 2k, 32k, and 128k tokens; feature activation is roughly sixty times stronger at 2k and thirty-fold stronger at 32k and 128k than in neutral controls.Source: §5.2 · p. 31

with Length

  • At the decision position, disclosure-specific signal is absent at every length, while pending-answer legibility worsens roughly eightfold from 2k to 128k tokens.The median rank of the best decision token rises from 68 at 2k to 454 at 32k and 533 at 128k tokens; target-to-neutral legibility remains approximately 1.0.Source: with Length · p. 32

5.4 Which Channel Carries the Disclosure to the Judgment

  • Both transmission channels causally carry disclosure influence: attention blackouts remove 44–48% of the disclosure contrast, while summary transplants remove 65% and recreate 44%.The two estimated channel effects are statistically indistinguishable, identifying dual carriers rather than ranking one channel above the other.Source: with Length · p. 33, §5.4 · p. 35
  • A neutral sham transplant shifts the readout by 0.046, but real disclosure removal exceeds it by +0.116 within firm across eleven of twelve firms.The sham effect is 37 percent of the removal effect and 54 percent of the smaller recreation effect; the exact sign-flip test gives p = 0.002.Source: §5.4 · p. 35
  • Running summary and attention-based lookup each carry a substantial share of disclosure influence, with no resolvable ranking between them.Their paired removal difference is +0.025 with exact sign-flip p = 0.54, and the channels need not sum to one because they feed each other.Source: §5.4 · p. 35
  • Disclosure influence remains flat with distance below the tested horizon but declines with total context at fixed distance, supporting capacity pressure on memory formed during reading.Directed retrieval survives because naming the disclosure can redirect the intact lookup route.Source: §5.4 · pp. 35–36
  • The retrieval–integration gap is a failure of integration rather than comprehension: disclosures remain encoded and retrievable, while competing context weakens both transmission channels.The operative state is described as distributed and compressed rather than localized, and both recurrent summary and attention-based lookup are causally implicated.Source: §5.4 · p. 36
  • The design implication is to route decision-relevant information to the point of judgment, rather than merely increase computation after the information arrives.Section 6 tests this implication against alternative workflow architectures.Source: §5.4 · p. 36

6 Workflow Architecture: Routing, Not Computation

This section examines how workflow architecture routes retrieved disclosures into judgments, comparing targeted, structured restatement or extraction near the decision with generic chunked summaries and passive source-text restatement. It also considers whether the deciding model must author the restatement and whether these routing effects hold in complete real 10-K filings.

6.1 Four Workflow Architectures

  • The four architectures tested direct restatement, chunked note aggregation, verbatim re-presentation, and extended reasoning while holding the filing and decision outcome fixed.Extract-then-decide restated a directed retrieval answer before deciding; chunk-then-aggregate summarized segments; re-present placed the target paragraph before the question; reasoning varied computation effort.Source: §6.1 · pp. 37–38

6.2 A Sharply Ordered Gradient Across Workflows

  • At 128k tokens, extract-then-decide retained 67% of its short-context influence (+0.126 → +0.085), while chunk-then-aggregate eliminated it (−0.001).The targeted workflow increased retention fivefold, whereas the generic retrieve-summarize-decide implementation reproduced none of the disclosure’s decision influence at any tested length.Source: §6.2 · pp. 38–39
  • Chunked summarization failed because per-segment token budgets evicted risk disclosures from consolidated notes before the decision stage, unlike targeted extraction.The disclosure was absent from 24 of 24 firm-arrangement cells at 2k and 21 of 24 at 128k; the failure was extraction-stage eviction, not decision-stage neglect.Source: §6.2 · p. 39
  • Neither verbatim re-presentation nor extended reasoning restored influence at 128k; reasoning instead reduced short-context use from +5.5 to +0.7–+1.9 percentage points.Verbatim re-presentation yielded +0.013 at 128k, with seven of twelve firms positive and within the empirical-null band.Source: §6.2 · pp. 39–40

6.3 Decomposing the Successful Workflow

  • A targeted structured restatement adjacent to the decision was the operative package: model-authored extraction produced +0.113 at 2k and +0.055 at 128k.Experimenter-written extraction with the same facts, structure, and token budget produced +0.151 and +0.056, statistically indistinguishable from the model’s own extraction; no single ingredient was isolated.Source: §6.3 · p. 40
  • The successful workflow need not depend on model introspection because focal extraction can be supplied externally, although the tested answer key was experimenter-written rather than a field workflow.This decomposition isolates externally supplied focal extraction as a possible workflow component without claiming that it represents deployed analyst practice.Source: §6.3 · p. 40

6.4 Complete Real Filings

  • +0.043 influence versus −0.004 for direct reading, with eleven of twelve firms positive, when extract-then-decide is applied to complete real 10-K filings.The 95 percent interval was [+0.026, +0.059], and the disclosure survived into consolidated notes in zero of twelve cases.Source: §6.4 · p. 41

sentation Principle

  • Influence returns when a targeted, structured answer sits adjacent to the decision while the source remains present, but not when raw text or diluted notes replace that representation.The same model, filing, and decision question produce materially different judgments based on where and in what form decision-relevant information is placed; added reasoning effort did not substitute for this reorganization.Source: sentation Principle · pp. 41–42
  • Retrieval audits are insufficient: a system can pass filing question-answering tests while assigning an audited disclosure no weight, so evaluation must reach the judgment stage.The retrieval–integration gap is a property of the deployed workflow as well as the model.Source: sentation Principle · p. 43

7 Conclusion … A.3 Ecological Arm

This section synthesizes the paper’s retrieval–integration gap, emphasizing how reader capabilities, context length, memory transmission, and decision-proximal representations shape whether retrieved disclosures affect judgments. It also situates these findings in coherent, investor-visible 10-K filings through the ecological arm.

7 Conclusion

  • Decision influence falls from 3.2 percentage points to the noise floor as unrelated context grows from 2,000 to 128,000 tokens, while retrieval remains flat.Across a seven-step severity ladder, response range compresses by a factor of 5.6, although risk ordering degrades far less.Source: §7 · p. 43
  • Memory interventions show that compressed summaries and source-text lookup both transmit disclosure influence, with neither channel exclusively carrying the effect.Forgetting in recurrent memory removes roughly two-thirds of influence, retaining it recreates nearly half, and attention blackouts remove 44 to 48 percent.Source: §7 · p. 44
  • A targeted structured restatement adjacent to the decision raises full-length retained influence from 12 to 67 percent, whereas chunk-based summarization destroys it.Generic summaries evict the disclosure in twenty-four of twenty-four synthetic cases and all twelve complete real filings; an experimenter-written extraction works as well as the model’s own.Source: §7 · pp. 44–45
  • The findings define a post-delegation failure: an agent can demonstrably retrieve a fact yet fail to use it in the judgment it was delegated to form.Retrieval-based evaluations can therefore certify systems whose judgments are invariant to successfully retrieved facts; marginal decision influence provides a counterfactual, continuous, manipulable outcome measure.Source: §7 · p. 44
  • Model capability and workflow architecture play complementary roles: capability extends the integration horizon, while architecture determines whether decision-relevant information reaches judgment.Generative AI relocates disclosure-processing costs from finding information to integrating it, with integration degrading faster than retrieval as filings grow.Source: §7 · p. 46
  • The causal channel evidence comes from one hybrid model family, while behavioral dissociation replicates across other families and real-filing replication remains capability-conditional.More capable models sustain influence deeper into context, whereas a low-cost commercial system may fail to respond even to short real disclosures.Source: §7 · pp. 46–47
  • Disclosure usefulness depends on which machine reads the filing, because degradation begins at a capability floor beneath the retrieval–integration gap.The evaluation instrument is designed to determine whether future systems close or merely move this gap across disclosure lengths.Source: §7 · p. 47

A.1 Target Disclosures

  • Each of twelve firms receives one coherent, researcher-authored risk paragraph describing verifiable quantities, with candidates discarded when coherence requires edits in multiple locations.Audits check consistency with statements, notes, management discussion, risks, legal proceedings, audit opinions, timelines, and GAAP or SEC requirements.Source: §A.1 · p. 68

A.2 Controls, Severity Ladder, and Filler

  • Five neutral replacements, a seven-step severity ladder, and genre-matched filler isolate marginal decision influence across exact 2k–128k-token contexts.Target and neutral conditions share the focal mini-filing byte-for-byte, while filler-induced optimism shifts are absorbed by differencing.Source: §A.2 · p. 68

A.3 Ecological Arm

  • The ecological arm inserts target paragraphs into complete 10-K filings and removes contradictory passages through audited, machine-verified deletions.Across twelve filings, the registry contains 33 deletion entries, including 30 nonzero logical deletions covering 33 removed occurrences; all filings pass assembly checks.Source: §A.3 · pp. 68–69

A.4 Real-Disclosure Redaction Sample · B Measurement, Readouts, and Inference

The real-disclosure redaction experiment removes focal passages from post-training-cutoff 10-K filings while eliminating echoed amounts, isolating whether disclosures influence judgments beyond retrieval. It also uses placebo and excerpt variants, memorization probes, and preregistered directional labels to interpret firm-level effects.

A.4 Real-Disclosure Redaction Sample

  • Twenty of twenty-five screened disclosure candidates passed criteria requiring self-contained passages with at least two verifiable quantities and no filing-wide repetition.Candidates covered six categories, including liquidity constraints, litigation exposures, customer concentration, and regulatory liabilities.Source: §A.4 · p. 69
  • The experiment replaces each surviving disclosure with a token-length-matched neutral passage and removes repeated distinctive amounts from all document versions.The excision registry reviewed 102 sentence-level records; 99 applicable records were applied as deletions in current builds.Source: §A.4 · p. 69
  • A placebo replaces a distant sentence-aligned block, while excerpt versions retain roughly 2,000 tokens centered on the disclosure slot.Both controls preserve sentence alignment and approximate token length to distinguish focal-disclosure effects from generic document changes.Source: §A.4 · p. 69
  • Redacted full-filing retrieval serves as a memorization probe because correct answers can only arise from residual echoes or pretraining.Three firms triggered this probe, causing their measured influence to be understated.Source: §A.4 · p. 69
  • Directional labels were assigned before the 27B extension, with four ambiguous cases including post-year-end financing and settlements resolving litigation overhangs.The predicted sign heterogeneity appears in the firm-level results.Source: §A.4 · p. 69
  • The redaction contrast isolates the focal passage because, without echo excision, retrieval could recover the disclosure from repeated filing amounts.This design addresses the specific failure mode in which retrieval remains successful despite removal of the original passage.Source: §A.4 · p. 69

B.1 Decision Readouts

Decision readouts use model-appropriate numeric measures: open-weight models yield sell probabilities from three action-code logits, while API models produce twelve-month default probabilities. Logit recomputation and parsing rules address measurement precision and malformed responses.

Decision Readouts

  • Open-weight models report sell probability from a softmax over buy, hold, and sell next-token logits at the final prompt position.The fixed decision prompt requests exactly one action code, making the readout deterministic.Source: §B.1 · p. 70
  • Because bf16 quantization is coarser than measured contrasts, the three action-code logits are recomputed in float32 from the final-layer hidden state.The bf16 output head quantizes logits at approximately 0.125.Source: §B.1 · p. 70
  • Commercial API models use twelve-month default probability in percentage points, while seven malformed numeric responses—all from the production model—are excluded when firm-cells lack four parsed neutral controls.Six malformed responses occur in before-arrangement cells reported in the main table; cells are retained when at least four of five neutral controls parse.Source: §B.1 · p. 70

B.2 Retrieval Readout · B.3 Inference · B.4 Computational Environment

Retrieval is scored against frozen firm-level fact keys, while inference uses permutation tests and bootstrap intervals to distinguish information-use failure from instrument failure. Computational conditions are held fixed and affected runs are regenerated under verified-correct hardware configurations.

B.2 Retrieval Readout

  • Directed retrieval answers are matched to frozen per-firm fact keys with alias groups, allowing paraphrased amounts or statuses to score without judgment calls.Section-label normalization and code-fence stripping were required for cross-family rows; neither changed the primary family’s scores across more than 3,700 rescored results.Source: §B.2 · p. 70
  • The primary family produced zero false-positive retrievals at every context length, whereas cross-family rows produced 5–25%, which the differenced design absorbs.Neutral-condition filings provide the false-positive check.Source: §B.2 · p. 70

B.3 Inference

  • Inference operates at the firm-filing level using pooled randomization tests with 2,000 permutation draws and ten pseudo-target insertions per cell.The identical insertion machinery resolves clearly non-zero effects at short lengths, supporting interpretation of long-context nulls as loss of use rather than instrument failure; results are reported in Appendix Table 11.Source: §B.3 · p. 70
  • Firm-level bootstrap confidence intervals use 10,000 draws for every model–length–arrangement cell reported in Table 2.The bootstrap procedure complements the pooled randomization test by quantifying uncertainty across the reported cells.Source: §B.3 · p. 70

B.4 Computational Environment

  • Runtime conditions were pinned across paired conditions using one library version, kernel path, numeric precision, and device policy.Hardware-induced numerical variation was treated as an environmental property to hold constant rather than average over.Source: §B.4 · p. 70
  • Results produced under a discovered multi-GPU communication defect were discarded and regenerated under a verified-correct configuration.The correction addressed a compute-platform defect rather than averaging over hardware-induced variation.Source: §B.4 · pp. 70–71

C Mechanism Methods … D.2 Memory-Channel Transplant

The mechanism analysis combines vocabulary and sparse-autoencoder readouts with causal interventions to test how disclosures reach decisions. Results indicate that risk content is encoded but not retained as conceptual content at decision time, while attention access and recurrent memory provide distinct, architecture-dependent transmission routes.

C.1 Vocabulary-Space Probe

  • The vocabulary-space probe reads frozen risk words at the disclosure and action codes before the answer, tracing retention trajectories between fact and decision positions.Probe ranks are measured at both positions and interpolated across intermediate positions, with medians summarized across firms.Source: §C.1 · p. 72
  • Probe validation shows readability tracks depth rather than attention type, supporting the interpretation that decision-readout findings are not artifacts of the analyzed layer’s architecture.Median ranks agree within adjacent linear/full layer pairs, including approximately 122,000 versus 75,000 at layers 7/8 and 26 versus 116 at layers 27/28.Source: §C.1 · p. 72

C.2 Sparse-Autoencoder Instrument

  • Risk content is strongly encoded at the disclosure but absent from decision-position representations, which instead appear to carry a pointer for retrieving information at readout time.Sparse-autoencoder activation is approximately 60 times neutral at 2k and roughly 30 times neutral at 32k and 128k at the fact position, while target and neutral documents are indistinguishable at the decision position.Source: §C.2 · p. 72

D Causal Interventions

  • The causal program tests decision-position patching, fact-position ablation, and short-to-long transplantation with matched random, unrelated-concept, wrong-layer, and neutral-document controls.The interventions target risk subspaces or a short-context decision-position difference transplanted into the long-context decision position.Source: D Causal Interventions · p. 73
  • All three initial subspace interventions are null under float32 readout, ruling out a single localized carrier while motivating tests of distributed, compressed memory and attention lookup.Ablation effects do not exceed matched random ablations (t = −1.38), transplants show no dose response or target/neutral asymmetry, and a wrong-layer control reaches t = +2.41 in the wrong direction.Source: D Causal Interventions · p. 73

D.1 Attention-Access Interventions

  • Attention-access experiments identify disclosure-slot content selectivity through target–neutral difference-in-differences, using matched all-heads blackouts and causal masking rather than full attention-matrix materialization.The neutral-document slot blackout is cleanly null, while an unstable content-free-span sham is reported as a failed control rather than used for identification.Source: §D.1 · pp. 73–74

D.2 Memory-Channel Transplant

  • Memory-channel transplantation provides the strongest causal evidence, implicating the recurrent running summary as a major carrier while showing architecture-specific localization and important limits on carrier-share interpretation.In the hybrid model, recurrent states are transplanted across target and neutral cache runs; one of twelve firms responds to neither direction, effects need not be additive, and the pure-attention comparison supports localization replication rather than a carrier-share estimate.Source: D Causal Interventions · p. 73, §D.2 · p. 74

E Robustness and Additional Results · E.1 Capability Gate · E.2 Arrangement Contrast

Robustness checks show that length-decay interpretation requires baseline severity-ordering ability, while the behavioral arrangement contrast is a calibrated null. The retention interpretation therefore relies on internal-representation evidence and the workflow gradient.

E.1 Capability Gate

  • The full-attention control fails the capability gate, with only a two-percentage-point severity range and rank correlation near zero.It is excluded from decay interpretation.Source: §E.1 · p. 75
  • The post-trained 9B and base 9B models pass the capability gate with wide severity ranges.The gate tests whether models can order disclosure severities in a short context.Source: §E.1 · p. 75
  • The 2B model passes marginally, discriminating within a compressed severity band around a strong sell prior.This marginal capability limits its interpretive range for length decay.Source: §E.1 · p. 75
  • Post-training approximately doubles severity discrimination on identical stimuli.This comparison establishes a capability difference between post-trained and base models under the same inputs.Source: §E.1 · p. 75

E.2 Arrangement Contrast

  • The arrangement contrast is a calibrated null: before-minus-after gaps are statistically indistinguishable from pseudo-target insertion gaps at every length.The retention interpretation in the main text instead rests on internal-representation evidence and the workflow gradient.Source: §E.2 · p. 75
  • Filler placement changes the distance between the focal disclosure and decision, but does not produce a behavioral gap beyond pseudo-target variation.Appendix Table 14 reports this null at every tested length.Source: §E.2 · p. 75

E.3 Sign Inversions

Sign inversions were rare and marginal, appearing in only a few cells across model families and context lengths. Because roughly two false-positive stars were expected by chance, the authors treat them as suggestive, with adjacent-length cross-family echoes motivating their report.

E.3 Sign Inversions

  • Sign inversions appeared in only a few cells: one mid-length cell, one long-context cell, and an arrangement-dependent flip in the post-trained 9B.These cells came from different model families or arrangements.Source: §E.3 · p. 75
  • Roughly two false-positive stars were expected by chance across the forty tested cells, so the reported inversions are suggestive rather than definitive.The chance expectation limits how strongly these marginal effects should be interpreted.Source: §E.3 · p. 75
  • The inversions were reported because adjacent context lengths produced a cross-family echo.This cross-family pattern was the reason to retain otherwise marginal sign flips in the appendix.Source: §E.3 · p. 75

E.4 Repetition and Semantic Interference

Repetition only partially and unevenly improves disclosure use, failing to restore long-context influence. Semantic interference is real but capability-dependent and does not primarily drive the main effect.

Repetition

  • Fourfold repetition exceeds single-occurrence influence in pooled length cells, whereas twofold repetition does not improve every cell.Repetition was controlled with fixed total length and final position.Source: §E.4 · p. 75
  • At 128k, fourfold repetition recovers less than one third of a single mention’s 2k influence.Source: §E.4 · p. 75

Semantic Interference

  • Same-risk filler reduces disclosure use at 2k and makes it negative at 8k only for the production API model, not the local workhorse.Semantic interference is capability-dependent and is not the primary driver because the main effect also occurs with economically unrelated filler.Source: §E.4 · p. 75

E.5 Out-of-Sample API Arm and Workflow Details

Out-of-sample and ecological tests preserve the retrieval–use dissociation, while workflow audits show that chunked extraction can evict disclosures before judgment. Targeted arrangements restore influence, but extended reasoning and proximity changes do not reliably overcome long-context failure.

Out-of-sample API arm

  • The dated out-of-sample API snapshot reproduces flat directed-retrieval performance and zero neutral false positives, but its noisy numeric readout cannot support firm-level behavioral claims.It is retained as an out-of-sample retrieval witness only.Source: §E.5 · p. 75

Capability gate

  • Post-training approximately doubles severity discrimination at 2k, while the full-attention control fails the severity-ordering gate and is excluded from the main analysis.The gate is necessary because a model unable to order severities at 2k cannot support length-decay interpretation.Source: §E.5 · p. 77, Table 10 · p. 77

Reasoning and remedies

  • Extended reasoning does not restore use at 128k and reduces measured use at 2k, while the remedy retains residual length erosion with a 128k/2k ratio of approximately 0.5–0.7.These results indicate that successful retrieval and additional reasoning effort do not guarantee decision-stage use.Source: §E.5 · pp. 76–77, Table 13 · p. 77, Table 12 · p. 77

Long-context anomalies

  • Sign inversions at long context are suggestive rather than confirmed because approximately two false-positive stars are expected across forty tested cells.The cross-family echo and arrangement-dependent flip motivate cautious reporting rather than firm behavioral conclusions.Source: §E.5 · p. 78, Table 15 · p. 78

Ecological validation

  • In complete investor-visible 10-K filings, models retrieve surgically inserted risks almost perfectly but their investment judgments remain unchanged.This replicates the retrieval–use dissociation outside synthetic contexts.Source: §E.5 · p. 79

Workflow architecture

  • Chunk-then-aggregate notes routinely truncate segment facts and evict the single-point risk disclosure, attributing zero influence to extraction-stage loss.A bounded generic chunked pipeline eliminates disclosure influence even at 2k tokens, where the document fits in one chunk.Source: §E.5 · pp. 76, 79, Table 18 · p. 79

Context and distance

  • At 32k tokens and beyond, all disclosure-use cells reach the noise floor, and moving the disclosure next to the decision does not rescue influence.Below the failure horizon, increasing total context from 8k to 32k lowers use by 0.008 (p = 0.03), whereas moving it from 1,000 to 5,500 tokens changes use by less than 0.0001 (p = 0.99).Source: §E.5 · p. 80, Table 21 · p. 80

Memory channels

  • Decision-time attention to the disclosure exists at short length but collapses at full length, and severing access shifts roughly one-quarter to one-third of the 2k contrast.Matched neutral-slot shams identify a disclosure-specific component equal to 44–48% of the disclosure contrast; recurrent-transplant and directional-attention effects are statistically indistinguishable.Source: §E.5 · pp. 81–82, Table 23 · p. 81, Table 24 · p. 82
Loading 2608.24842v1…