PaperlayerView exact revision on arXiv

Source-linked AI summary

Fresh at Decision, Stale at Action: Routing New Information to LLM Decisions Already Made

Ayoub Merimi, Souhail Bakkali, Abdeljalil Sakat

arXiv:2610.18126v1cs.AIcs.GTcs.IRcs.MA

TL;DR

A decision can become wrong between selection and execution when decisive information appears afterward. The paper models this gap with a timestamp theory and decision ledger; controlled streams confirm its predictions across five LLMs.

  • Problem

    Fresh evidence can support the right choice when made yet fail to ensure it remains right when acted on, because decisive information may appear in between.

    Source: §1 · pp. 1–2

  • Method

    The paper defines decisions by timestamps, factorizes remaining accuracy into routing and temporal judgment, and uses a ledger to index decisions for later news.

    Source: §1 · p. 2, §8 · p. 8

  • Results

    Controlled streams confirm all four predictions across five LLMs; models switch in 91–100% of cases after decisive news arrives, and changing-decision accuracy tracks routing recall.

    Source: §1 · p. 2

  • Takeaways & Limitations

    LLM agents should record decision times and dependencies, route news through an index until the revision deadline, and check when each change takes effect.

    Source: §8 · p. 8

    Results come from synthetic streams; the theory assumes a constant hazard rate, one revision deadline, and binary outcomes, unlike staged, graded real-world commitments.

    Source: Limitations · p. 8

Abstract

from arXiv · show

Freshness shapes an LLM decision at the moment it is made; the action it commits to is latent, carried out later, in a changing world. We therefore introduce a timestamp theory of decision validity and a decision ledger that realises it. Six timestamps define a decision; a freshness ceiling bounds what up-to-date evidence guarantees at action time, placing the remaining risk in information published afterwards. Accuracy then factorises into two capabilities: routing, which delivers that information to affected decisions before their revision deadlines, and temporal judgment, which reads it for the time of action. The ledger is persistent state holding each decision's times, evidence and background knowledge, indexed by watch conditions written at decision time. Controlled decision streams with exact labels confirm four predictions across five LLMs, from 8B to frontier scale: freshness alone yields exactly the accuracy the ceiling predicts; models act on decisive news once it arrives, so accuracy follows routing; the ledger's index sets what routing reaches, and watch conditions reach indirect dependencies at low exposure; and the remaining errors concern when a change takes effect. Freshness makes a decision right when it is made; routing and temporal judgment keep it right when it is carried out.

1 Introduction

The introduction distinguishes evidence freshness at decision time from whether a decision remains valid when acted on. It presents a timestamp theory and decision ledger, then reports controlled tests of the theory’s predictions across five LLMs.

1 Introduction

  • A dinner choice becomes invalid when a later notice about the supplying substation affects the district, illustrating how decisive information can refer to an option indirectly.Source: §1 · p. 1, Figure 1 · p. 1
  • A freshness ceiling bounds what evidence at decision time can guarantee at action time, leaving later information to reduce risk before the revision deadline.Accuracy factorises into routing that delivers the information and judgment that reads it for the action time.Source: §1 · p. 2
  • The decision ledger stores each decision’s times, evidence, and background knowledge so a router can match new information to open decisions.Unlike a query that retrieves what a user asks about today, the ledger retrieves what earlier decisions still need.Source: §1 · p. 2
  • With perfect decision-time evidence and no later information, accuracy equals the share of decisions that should stand, as the freshness ceiling predicts.Source: §1 · p. 2
  • 91–100% of decisive-news cases trigger a switch under every router, and all four predictions hold across five LLMs.The reported tests span Ministral 8B to GPT-6.1 Sol.Source: §1 · p. 2
  • When decisive news arrives, accuracy tracks routing recall, which depends on whether the router’s index contains the entity named by the news.The decision text reaches 37.5% of news using another name for a dependency, while ledger background knowledge reaches all of it.Source: §1 · p. 2

2 Related Work

Related work addresses time-sensitive answers, updating knowledge, agent actions, and persistent memory. This paper focuses on delivering new information to open decisions and judging those decisions at action time.

2 Related Work

  • Earlier freshness and temporal-knowledge work evaluates whether answers are current when produced, whereas this paper judges decisions when carried out.Source: §2 · p. 2
  • Knowledge-editing, ripple-effect, and revision studies concern updates reaching dependent facts; here, new facts must reach every open decision that depends on them.Source: §2 · pp. 2–3
  • Agent benchmarks study plans and actions over time, while this work studies concurrent decisions, shared world news, revision deadlines, and news-to-decision routing.Source: §2 · p. 3
  • Agent memory work includes temporal validity and time-topic indexing; the decision ledger applies persistent, evidence-grounded state to commitments, with watch conditions written at decision time.Watch conditions draw on document expansion and generated queries; the passage also notes long-context reasoning difficulties when relevant content lacks literal overlap.Source: §2 · p. 3

3 A Theory of Decisions Over Time

A decision’s validity is evaluated when its action occurs, not merely when it is chosen. The theory bounds what fresh evidence can guarantee and separates later accuracy into routing and judgment.

3 A Theory of Decisions Over Time

  • Six timestamps distinguish evidence writing and verification, decision, revision deadline, action, and outcome observation; the revision deadline marks when commitment prevents further change.The ordering is td < tr ≤ ta ≤ te, and the activity examples include an order shipping, a table being taken, or a train departing.Source: §3 · p. 3
  • The freshness ceiling bounds the probability that a fact remains true at action time by e^-λf∆act, regardless of evidence freshness.Evidence age at action decomposes into the freshness gap and execution gap; retrieval can shrink the former, while the task sets the latter.Source: §3 · p. 3
  • Only information arriving after the decision can reduce the ceiling’s residual risk, provided it reaches the decision before its revision deadline.The proposition also extends to decisions with several dependencies.Source: §3 · p. 3
  • Decision accuracy factorises into routing decisive information before revision and judging correctly whether it warrants a change for the action time.Routing determines which items a decision sees; judgment determines whether to switch or keep, including when a change’s active interval overlaps the activity.Source: §3 · pp. 3–4
  • The ledger makes routing depend on indexed decision information, while the model is called again only for decisions that receive news.The router sends each published item to matching open decisions, and the model reads routed items at the revision deadline.Source: §3 · p. 3, Figure 2 · p. 4
  • Four predictions cover the no-routing freshness baseline, delivery-driven changes, misleading-item exposure, and temporal judgment under oracle routing.With oracle routing, residual errors should concentrate on judging whether the change interval overlaps the activity.Source: §3 · p. 4

4 The Decision Ledger

The decision ledger preserves open decisions beyond their original session and routes later information to them. Its indexed contents determine which decisions can receive news.

4 The Decision Ledger

  • Each persistent ledger entry stores the chosen option and alternatives, action time, revision deadline, evidence, and background facts about dependencies and upstream entities.The dossier may include facts irrelevant to the user’s activity, including information the assistant looked up while deciding.Source: §4 · p. 4
  • When an item is published, the router scores open entries and sends it to the K highest-scoring decisions; entries receiving no items require no model call.At the revision deadline, the model reads routed items in publication order and keeps or switches.Source: §4 · p. 4
  • Decision-text routing reaches news naming the option or its stated dependency, while whole-dossier routing also reaches other names and upstream entities—with unrelated dossier facts as additional exposure.The decision-text index can use BM25 or dense embeddings; whole-dossier routing expands the indexed text with every dossier fact.Source: §4 · p. 4
  • Watch conditions written at decision time name entities whose changes would stop the activity, including other names and upstream entities, while omitting what the user does not need.Three to eight conditions are indexed with the decision text or scored by their best sentence-encoder similarity; this aims to match dossier reach with less exposure.Source: §4 · p. 4
  • The time filter drops an item when its stated start time is after the activity ends, implementing the temporal check required by P4.It performs that check by rule rather than relying on the model’s judgment.Source: §4 · p. 4

5 Controlled Decision Streams

The paper uses fictional decision and news streams so labels follow from timestamps and stated facts. The conditions vary how news refers to a decision and how difficult its negatives are.

5 Controlled Decision Streams

  • Fictional streams provide control over decisiveness, reference, and timing because real logs lack that control and real entities can invite answers from model memory.The generator and data are released with the paper.Source: §5 · pp. 4–5
  • Each decision has two options, and option 1 is initially better; label option 2 when a revision-window item targets it, its dependency, or an upstream entity and blocks it during activity; otherwise, option 1.The generator excludes decisions invalidated before they were made.Source: §5 · p. 5
  • DIRECT tests decisive news naming the option or stated dependency, STRESS adds hard negatives, and INDIRECT moves references into background knowledge and upstream entities.In INDIRECT, balanced decisions receive two related notices and one decoy, so neither news amount nor wording reveals the label.Source: §5 · p. 5
  • Each decision’s revision window contains dozens to hundreds of items, while only a handful concern it.INDIRECT dossiers also include unneeded facilities and traps involving entities from other decisions in irrelevant roles.Source: Table 1 · p. 5, §5 · p. 5

6 Experimental Setup

The experiments compare routers under controlled conditions and evaluate multiple models using accuracy, routing, judgment, exposure, and token measures. The supplied table captions describe STRESS and INDIRECT results, but do not provide cell values.

6 Experimental Setup

  • Routers include no delivery, broadcast, name match, the section 4 methods, and gold-labeled oracle variants; each condition uses the same re-decision prompt.The router methods send each item to the K=3 best-scoring open decisions, with dense routing using BGE-base.Source: §6 · p. 5
  • STRESS evaluates Claude Sonnet 4.6, GPT-6.1 Sol, and Qwen3-235B-A22B, while INDIRECT evaluates Claude Sonnet 4.6, Qwen3-235B-A22B, and Ministral 8B.Mistral 7B is reported in Appendix E; hosted models use temperature 0 and structured output.Source: §6 · pp. 5–6
  • The STRESS table reports final-decision accuracy for three models on the same 75 decisions, alongside delivered items, decisive-item recall, and Claude input tokens per decision.Its caption identifies the bold entry as the best router without gold labels.Source: Table 2 · p. 6
  • The INDIRECT table reports accuracy on 128 balanced decisions, decisive-notice recall, and irrelevant-role traps out of 96, with watch conditions written by Claude for every model.Its caption also identifies the bold entry as the best router without gold labels.Source: Table 3 · p. 6
  • Evaluation measures accuracy, routing recall, switch and keep rates, irrelevant-role traps, and input-token cost; exact McNemar tests compare per-decision correctness.Comparisons are made per model and pooled over the models of a condition.Source: §6 · p. 6

7 Testing the Predictions

The tests support the freshness ceiling and show that decision accuracy depends on whether decisive news reaches a decision and whether the model judges its timing correctly. Across routing conditions, the ledger’s index shapes which notices arrive, while broader delivery can expose decisions that should stand to misleading news.

7 Testing the Predictions

  • Figure 4b compares decisions kept against items delivered; Claude kept at least 87.5% of decisions that should stand, while weaker models kept fewer than half under BM25 and dense routing.Watch conditions delivered 92.2% of decisive notices and 20 traps; a delivered trap increased needless switching for every model under retrieval and broadcast routing.Source: Figure 4 · p. 6, §7 · p. 7
  • 45.5% on DIRECT, 36.0% on STRESS, and 50.0% on INDIRECT matched the freshness ceiling; name matching raised Qwen3-8B on DIRECT to 70.0%.With perfect evidence at decision time and no later information, accuracy equalled the share of decisions that should stand.Source: §7 · p. 6
  • 90.6–100% of decisions with delivered decisive notices switched, and Figure 4a relates needed switches to routing recall.Claude’s points were within 4.7 points of recall under every router; weaker models also sometimes switched without receiving the decisive notice.Source: §7 · pp. 6–7, Figure 4 · p. 6
  • On INDIRECT, the decision text reached 37.5% of notices using another dependency name and 12.5% naming an upstream entity; the dossier reached all decisive notices.Claude’s accuracy was 78.9% with the decision-text index, 93.0% with the dossier, and 91.4% with watch conditions.Source: Figure 3 · p. 6, §7 · p. 7
  • On STRESS, BM25 delivered 100.0% of decisive items and raised model accuracy from 72.0–73.3% with name matching to 85.3–98.7%.The decision text was a sufficient index when news named the stated dependency; results were as accurate as oracle routing for Claude and GPT.Source: §7 · p. 7
  • With oracle delivery of exactly related, non-changing news, Table 4 isolates judgment; errors clustered around when a change takes effect.Ministral 8B and Qwen3-235B abandoned decisions when changes had ended before activity, while Qwen3-8B abandoned 94% of DIRECT decisions with late changes; Claude kept all such decisions. A rule-based time filter raised Qwen3-235B on STRESS from 85.3% to 92.0%.Source: Table 4 · p. 7, §7 · p. 7
  • Broadcasting used 4.0 times BM25’s tokens on STRESS and 3.1 times on INDIRECT; Qwen3-8B fell from 71.8% to 55.5% on DIRECT.Broadcasting also reduced Qwen3-235B on STRESS from 85.3% to 78.7%, while watch conditions outperformed it on INDIRECT.Source: §7 · p. 7

8 Conclusion

The conclusion argues that freshness alone cannot keep a decision valid until action: later information must reach the decision, and changes must be judged for when they take effect. It recommends recording decision times and dependencies, routing news until revision deadlines, and revalidating the timing of changes.

8 Conclusion

  • Freshness is necessary but insufficient for LLM decisions; the freshness ceiling bounds what evidence at decision time guarantees, while remaining accuracy depends on routing and judgment.The controlled tests confirm the predictions that freshness alone leaves changing decisions stale, delivery is sufficient for change, misleading deliveries cost correct decisions, and residual errors are mostly temporal.Source: §8 · pp. 7–8
  • The paper’s design principles are to record each decision’s times and dependencies, route news through an index written at decision time, and revalidate when changes take effect.News should be routed to a decision until its revision deadline.Source: §8 · p. 8

Limitations

The theory uses a single revision deadline, binary outcomes, and a constant hazard rate; real commitments and changing facts can be more complex. Its ledger also depends on recorded dependencies and treats routed news as trustworthy.

Controlled streams

  • Reported result magnitudes apply to synthetic controlled streams, where timing, references, and labels can be isolated.Real news is longer and noisier, dependencies may be implicit, and relevance may be uncertain; testing logged agent decisions and other languages remains future work.Source: Limitations · p. 8

Scope of the theory

  • The theory assumes one revision deadline, binary outcomes, and a constant hazard rate, unlike staged or graded commitments and facts that change in bursts.The factorisation carries over to these settings, but its terms then depend on time and revision cost.Source: Limitations · p. 8

Reach of the ledger

  • Routing recall is bounded by dependencies recorded at decision time, and the ledger treats routed news as trustworthy.Discovering dependencies after a decision and assessing source credibility extend beyond this paper.Source: Limitations · p. 8

Ethics Statement

The study uses fictional decision streams without personal data, and hosted models were queried only on this synthetic data. Systems revising decisions for users may act on false or adversarial news, motivating source verification and user confirmation for consequential decisions.

Ethics Statement

  • The decision streams use fictional entities and no personal data, and hosted models were queried only on synthetic data.Systems revising decisions for users may act on false or adversarial news; the paper recommends source verification and user confirmation for consequential decisions.Source: Ethics Statement · p. 8

A Formal Results

The formal results bound decision validity by the time between decision and action, then factor accuracy into delivery and judgment. The appendix also specifies controlled streams, prompts, and implementation details used to test these mechanisms.

A Formal Results

  • With a constant change hazard, evidence verified before the decision bounds the probability a fact still holds at action time by e^-λf∆act.The bound is attained when evidence is verified at decision time; fresher evidence approaches it, while only post-decision information changes the decision itself.Source: A Formal Results · pp. 10–11
  • When a choice depends on multiple independent facts, its freshness ceiling tightens with each dependency, including implicit ones such as a district’s supplying substation.This motivates recording dependencies alongside the choice in the ledger.Source: A Formal Results · p. 11
  • Decision accuracy decomposes by whether the decision should stand or change and, for changes, whether decisive news was delivered; router gains depend on recall, judgment, and misleading exposure.With the same model and unchanged keep rates, the router comparison gives A′ − A = π(ρ′ − ρ)(S − L) + (1 − π)(K′ − K).Source: A Formal Results · p. 11

B Decision Streams

  • The released synthetic streams vary reference style and negative difficulty, with labels checked against timestamps and stated facts.DIRECT and STRESS include option or dependency notices; INDIRECT routes through other names and upstream entities, with controlled timing and decoys.Source: B Decision Streams · p. 11

B.1 A Decision from INDIRECT

  • In the running INDIRECT example, decision-text routing and two dense routers miss the substation notice, while BM25 over the dossier or watch conditions reaches it and models switch.The dossier includes the district’s other name and upstream substation, alongside irrelevant facilities and traps.Source: §B.1 · p. 12, Table 5 · p. 12

C Prompts

  • The prompts ask models to write specific watch conditions from decision-time evidence and dependencies, then reconsider using new notices and the action interval.The re-decision prompt distinguishes notices by when they take effect, not merely when they were published.Source: C Prompts · p. 13

D Implementation

  • Implementation specifies model decoding and retries, BM25 and dense routing configurations with a time filter, and scripts and logs for every reported number.The scripts and per-decision logs are released with the paper.Source: D Implementation · p. 13

E Additional Results

Additional results examine how router reach, model behavior, and notice timing shape performance. They report router comparisons across DIRECT, INDIRECT, and STRESS settings, including a second STRESS sample and Mistral 7B results.

Where the errors are

  • On INDIRECT, routing determines which needed switches are possible, while temporal judgment accounts for remaining decisions, especially when changes ended before or start after the activity.The breakdown is by reference type and related news received, with means over three models and 16 decisions per row and model.Source: E Additional Results · pp. 13–14, Table 6 · p. 14

Every router on DIRECT

  • 97.1% of decisive pairs are delivered by BM25 over decision text on DIRECT; with oracle routing, Qwen3-8B makes every needed switch and keeps 62.0% of decisions that should stand.The table defines recall as the share of decision-changing items delivered and reports precision, switch, keep, tokens, and McNemar tests.Source: E Additional Results · p. 14, Table 8 · p. 15

Routing with other K

  • With K=1 on INDIRECT, the whole dossier delivers 85.9% of decisive notices and watch conditions deliver 71.9%.Table 10 compares decisive-pair delivery and lexical traps as the number of decisions routed to each notice varies.Source: E Additional Results · p. 14, Table 10 · p. 15

A second STRESS sample

  • On a second STRESS sample of 73 decisions, Claude Sonnet 4.6 accuracy is 61.6% without routing, 89.0% with BM25, and 94.5% with the related oracle.The table reports keep accuracy and needless switches for decisions that should stand.Source: E Additional Results · p. 14, Table 11 · p. 15

Mistral 7B on INDIRECT

  • Mistral 7B returns valid answers for 74–95% of prompts depending on router and switches on 48.9% of answered decisions given only decisive notices.It mostly keeps its original choice on INDIRECT.Source: E Additional Results · p. 14

Related papers

Loading 2610.18126v1…