PaperlayerView exact revision on arXiv

Source-linked AI summary

Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops

Yubo Li, Ramayya Krishnan, Rema Padman

arXiv:2610.18579v1cs.AI

TL;DR

When a System One model chooses wrongly, its output alone cannot reveal whether it misread the state or read correctly and acted wrongly. The paper pairs independent factual probes with action questions on the same states and finds that most errors by strong systems follow correct reading. On constructed states, composing the model’s fact answers in code can repair decisions, while real states often call for a missing sub-goal.

  • Problem

    A wrong action from a System One model does not reveal whether it misread the agent’s state or read it correctly and still chose badly.

    Source: §1 · p. 1

  • Method

    The paper behaviorally pairs each model’s action choice with independent questions about what the agent has seen, and evaluates when probes can support attributing errors to decisions.

    Source: §1 · p. 2, §11 · p. 20

  • Results

    Most wrong actions follow correct reading; for the six most accurate systems, 73–85% of decision errors occur with every measured fact answered correctly, while code composition reaches 0.96–1.00 on constructed states where probes determine the action.

    Source: §1 · p. 2

  • Takeaways & Limitations

    On constructed states with decisive facts, composing the model’s own fact answers in code repairs decisions; on open tasks, the missing element can be the next sub-goal rather than another fact.

    Source: §11 · pp. 20–21

    The paired outcomes are behavioral evidence rather than evidence of internal computation, and natural-state probes may omit facts needed for the decision.

    Source: §10 · p. 20

Abstract

from arXiv · show

System One models choose an agent's next action by scoring a supplied set of options in a single pass. This makes them fast and cheap, but when they choose wrongly there is nothing to inspect: did the model misread what the agent has seen, or read it correctly and still act wrongly? We answer behaviorally. On a frozen benchmark of agent decision states from simulated environments, customer-service policies, tool use, web navigation and coding, we pair every next-action question with factual probes about the same state, each answered independently. Across seventeen fast systems and a reasoning model, the answer is consistent: most wrong actions follow correct reading, and the more accurate a system, the more of its errors take this form. These errors are not noise; they gather on the same states across systems of every kind. Where the probed facts provably determine the action, models that read every fact still fail to combine them, and their own answers, composed by code, repair the failure. On real states, what is missing is often not a fact but the next sub-goal.

1 Introduction

System One models choose from supplied options without producing reasoning text, leaving wrong actions hard to diagnose. Paired factual probes show that strong systems usually read correctly before acting wrongly, with distinct patterns on constructed and real states.

1 Introduction

  • The paper pairs each next-action question with independent factual probes on the same state, using a frozen benchmark spanning simulated environments, policies, tool use, web navigation, and coding.The evaluation covers seventeen fast systems and a reasoning model; the measured choice interface returns probabilities over supplied options without text.Source: §1 · pp. 1–2
  • For the six most accurate systems, 73–85% of decision errors occur when every measured fact is answered correctly, and this share grows with accuracy.The errors also recur on the same 141 states across all 14 competent fast systems, where chance predicts none.Source: §1 · p. 2
  • On generated states where probes determine the action, systems lose 20–45 points when deciding from the trace rather than from facts they read perfectly.Their own fact answers recover 15–28 points, while composing the action in code reaches 0.96–1.00.Source: §1 · p. 2
  • On real states, correct probe answers add at most 7 points for all but two systems, while stating the next sub-goal adds 16 points for JEV on ALFWorld.Any block of facts adds at most 5 points there.Source: §1 · p. 2
  • Removing history hurts where earlier observations matter; policy edits are acted on in only 22–72% of cases despite being read correctly, and calibration error ranges from 2.6% to 26.6%.These controls show that accuracy alone omits important differences in how systems fail.Source: §1 · p. 2
  • The paired outcomes are behavioral evidence: they do not identify internal computation, and the probes are not claimed to exhaust every relevant fact.The paper also contributes a model-free criterion for when probes support attributing an error to the decision.Source: §1 · p. 2

2 Measuring Comprehension and Decision on the Same State

The paper separates what an agent knows from its next-action choice by asking independent questions about the same decision state. It also specifies when probe answers can support a decision-failure diagnosis and uses controlled variations to examine outcomes.

2.1 Decision states

  • A decision state contains the goal, action-observation history, applicable rules, and admissible next actions; correctness means choosing an action in the gold set computed from agent-available information.In simulators, gold actions include first actions of shortest plans based on what the trace entails.Source: §2.1 · pp. 2–3

2.2 Factual probes

  • Closed factual probes ask about state-determined facts, and each probe and decision is answered independently using the identical state as input.“Paired” denotes identical inputs, not observed stages of the model’s computation.Source: §2.2 · p. 3

2.3 Four observable outcomes

  • Crossing whether all applicable probes are correct with whether the action is correct yields four observable outcomes, but none alone proves complete understanding or a purely decisional failure.Questions unsupported by an input limit count as wrong and are reported as coverage.Source: §2.3 · p. 3

2.4 When can a probe set support the diagnosis?

  • Probe sufficiency requires a common correct action across all states sharing the same probe answers; otherwise correct probes cannot determine the action.The check considers the whole state group, and passing it does not establish that the model used the facts.Source: §2.4 · p. 3

2.5 Controlled variations

  • Controlled variations remove history, edit relevant or irrelevant observations, change stated rules, and vary trace length and distance from decisive observations.Encoding controls are not analyzed in this section.Source: §2.5 · p. 3

3 Benchmark

The frozen benchmark combines agent decision records from exact-ground-truth simulators and several real-world task types. Its split and request details were fixed before analysis, with later studies using freshly generated states.

3 Benchmark

  • The frozen benchmark contains 65,228 records, including 59,234 test and 5,994 development records, with most paired states drawn from three simulators with exact ground truth.The simulator trajectories include expert, noisy, and invalid-command variants; gold sets were checked against breadth-first search in the real environments.Source: §3 · p. 3
  • The benchmark also covers customer-service policy states, tool selection, web navigation, AgentRM trees, and 360 coding records built from real repository histories.A further 32,700 text-understanding primitive records belong to the frozen population but are not analyzed.Source: §3 · p. 3
  • Records are split by source and frozen in a hash-sealed manifest; an independent audit flagged 98 test records, retained for sensitivity analysis.Follow-up studies use freshly generated states, including states whose probes are sufficient by construction.Source: §3 · p. 4

4 Systems and Protocol

The evaluation compares deployed choice interfaces, including native one-pass System One models, likelihood readouts, and a reasoning reference. Requests and options are held constant, while latency and cost are measured in separate panels.

4 Systems and Protocol

  • Fourteen System One models provide a native distribution over supplied options in one pass: two hosted APIs, ten open-weight models, and two recipes; GPT-6.1 serves as a deliberative reference.Three open-weight models are also evaluated through complete-option likelihood readouts with thinking disabled.Source: §4 · p. 4
  • Every system receives the same frozen requests without truncation, option removal, or record reselection; unsupported questions count as wrong and are reported as coverage.Comparisons concern deployed choice interfaces and do not isolate architecture, size, or training data.Source: §4 · p. 4
  • Latency is measured separately on 163 test items answered sequentially after warm-up, while GPT-6.1 questions are sent in parallel within each item.Some system timings are estimated from sequential accuracy runs.Source: §4 · p. 4
  • The cost frontier includes Eikos-27B at 0.795 for about 1.4 times JEV’s cost and Gemma-4-31B at 0.811 for about five times; GPT-6.1 reaches 0.884 for $642 under batch pricing.Hosted model versions can change; a rerun of JEV’s decisions five days later matched 96.3% of 2,783 states.Source: §4 · p. 4

5 Reading Correctly Is Not Enough

Across the benchmark, correct factual reading often coexists with a wrong action, and these errors cluster on shared states across systems. Audits indicate that label issues do not explain the pattern and make the headline estimate conservative.

5.1 Most decision errors follow correct reading

  • For the six most accurate systems, 73–85% of decision errors occur with every measured fact correct, and the action remains wrong in 10–21% of such states.The converse also holds: when at least one fact is wrong, the action is still correct in 62–70% of states.Source: §5.1 · p. 4, Table 2 · p. 6, Figure 3 · p. 7
  • The pattern spans domains, but system rankings vary: error shares after correct reading range from 68–82% in ALFWorld to 80–94% in TextWorld for the six strongest systems.On τ 2-bench, GPT-6.1's caution in replying or re-verifying accounts for 80% of its errors, partly because the benchmark scores successful-trajectory actions as gold.Source: §5.1 · pp. 7–8, Figure 4 · p. 8

5.2 Errors after correct reading are cheaper, but not free

  • Most errors cost at most one extra step in ALFWorld (86–90%) and TextWorld (77–84%), while in BabyAI the share is 42–56%.Misread-fact errors more often cost at least two steps in BabyAI (58% vs. 44%) and are unrecoverable in TextWorld (20% vs. 6%); ALFWorld slightly reverses the pattern (14% vs. 10% cost at least two steps).Source: §5.2 · p. 8, Figure 5 · p. 9
  • Exact lookahead finds that 5–42% of decisions scored wrong are also optimal, and that this share grows as decision accuracy rises.For GPT-6.1, the chosen action is optimal for 34% of facts-right errors in ALFWorld, 64% in BabyAI, and 96% in TextWorld.Source: §5.2 · pp. 8, 10

5.3 The same states defeat many systems

  • Facts-right, action-wrong errors concentrate on shared states: 56% fall where at least seven of 14 competent fast systems fail, versus 16% under the permutation null.All 14 systems make such errors on the same 141 states, where the null expects none.Source: §5.3 · p. 10

5.4 How much of this is label error?

  • Audits found 42 conflicting test labels and 56 inputs also present in development, but excluding those 98 records changed no system group score by more than 3.5 points.A separate ALFWorld label audit found 398 affected records; excluding them raised the after-correct-reading error share for every audited system, making the headline estimate conservative.Source: §5.4 · p. 10

6 Why Do Correct Readers Decide Wrongly?

Section 6 tests whether added factual information repairs wrong decisions, first where probes determine the action and then on real states. The results distinguish failures to combine facts from failures to choose a sub-goal.

6.1 When the probes are sufficient by construction

  • The generated procedures make the action uniquely determined by four probed rule facts, allowing failures to combine correctly read facts to be isolated.The probes ask those four rule facts, so correct answers determine the action.Source: §6.1 · pp. 10, 12
  • JEV’s errors reflect scattered-evidence consolidation, while Clef’s mainly reflect capture by irrelevant cues; targeted rewrites distinguish these mechanisms.A summary of facts in one place restores JEV’s facts-only level, while removing distractors raises Clef’s accuracy by 33 points.Source: §6.1 · p. 12
  • On generated states, JEV rises from 0.791 on the raw trace to 0.968 with its own fact answers and 0.996 when code composes the action.Code composition also reaches 0.987 for Qwen3.8-27B, 1.000 for Clef, and 0.965 for Qwen3.5-9B.Source: §6.1 · p. 13

6.2 Back to real states: probe-then-decide

  • On real benchmark states, correct probe answers add only 2.9–7.1 points for most fast systems, and the probes leave most errors in place.SimpleJev-27B and Torchcast-27B are exceptions; the benchmark probes measure reading rather than specifying the decision.Source: §6.2 · pp. 13–14

6.3 Decision-oriented probes

  • For JEV on ALFWorld, stating the next sub-goal adds 16.3 points, compared with 5.3 from decision-oriented facts, and facts plus sub-goal reach 0.929.JEV answers the sub-goal question correctly in only 66.8% of states, and many decision errors occur despite separately correct facts and sub-goal answers.Source: §6.3 · pp. 14–15
  • The most useful information varies by system: sub-goal blocks help six other System One models by 6.7–13.8 points, while incomplete fact summaries can lower accuracy.For example, Gemma falls from 0.659 to 0.341 on mixed seen/unseen-object states when given decision-oriented facts.Source: §6.3 · pp. 15–16
  • The paired diagnosis separates consolidation, cue capture, sub-goal selection, and grounding a sub-goal in the right object as distinct failure sources.Each points to a different remedy.Source: §6.3 · p. 16

7 Controls, Boundaries and Calibration

The controls show that models use history when the task requires it, but exploration becomes harder as traces lengthen. Relevant edits and calibration tests expose additional differences across systems.

7 Controls, Boundaries and Calibration

  • Removing history leaves BabyAI nearly unchanged but collapses accuracy on ALFWorld states requiring remembered observations and on TextWorld.For example, JEV falls from 87.2% to 13.7% on the ALFWorld distance ladder; the other listed strong systems also fall sharply.Source: §7 · pp. 16, 18, Figure 10 · p. 17
  • On ALFWorld, long traces hurt exploration more than states where the needed object has already been seen; distance from decisive evidence has little effect over the tested range.For JEV, exploration accuracy falls from 0.72 to 0.48 beyond seven steps, while no competent system loses more than three points across zero to 15 irrelevant intervening steps.Source: §7 · p. 18
  • In customer-service policy edits, systems can answer every probe correctly yet change the required tool call after relevant edits in only 22–57% of matched cases.Eikos-27B is an exception, changing the call after 72% of relevant edits and never after an irrelevant one.Source: §7 · p. 18
  • Stated TextWorld rule changes are followed by strong systems in 90–99% of states, including a later cohort generated from new seeds.Kev-9B reaches 83.0% without ever answering all rule probes correctly.Source: §7 · p. 18
  • Calibration varies more across systems than accuracy: Cygnet has 2.6% expected calibration error, while Gemma’s likelihood readout has 19.1%.JEV’s expected calibration error is 6.8%; the figure also compares reliability diagrams and accuracy against mean confidence.Source: §7 · p. 18, Figure 11 · p. 19

8 Related Work

The paper connects next-action evaluation to work on decision interfaces, knowing versus doing, composition, and behavioral diagnosis. Its approach pairs decisions with factual probes on the same state.

8 Related Work

  • Prior agent benchmarks measure task success, progress, or per-step correctness; this work pairs each decision with probes on the same state to explain what aggregate accuracy leaves unresolved.The passage situates supplied-option scoring as an agent interface offered by hosted models and open alternatives.Source: §8 · p. 19
  • Related work studies distraction, external planning, and failures to compose correct sub-answers; here, probes are answered in isolation and the calling program does the composition.The paper also tests whether the probe set can support the diagnosis.Source: §8 · p. 19
  • The paper infers competence through controlled questions and input variations rather than relying on chains of thought, which need not faithfully reflect computation.It distinguishes this behavioral approach from probing classifiers.Source: §8 · pp. 19–20

9 Discussion

The discussion recommends diagnosing errors on target states before choosing a remedy. It argues for diagnostic evaluation over a single score and frames trust in fast decisions as a deployment choice.

9 Discussion

  • Practitioners should measure paired outcomes, then use the diagnosis to choose among improving state representation, composing explicit rules in code, or supplying a sub-goal.For open tasks such as ALFWorld, the paper identifies planning—not reading or consolidation—as the relevant problem when the sub-goal is missing.Source: §9 · p. 20
  • Diagnostic panels and fresh states with probes sufficient by construction are more informative for a workflow than a single aggregate score.Similar paired accuracy can mask different failure sources, such as consolidation for JEV and cue capture for Clef.Source: §9 · p. 20
  • GPT-6.1 gains seven to twelve decision-accuracy points over JEV at roughly seventy times its cost, while the remaining errors stay the same kind.The discussion frames deployment as deciding which states need a composed decision or deliberation.Source: §9 · p. 20

10 Limitations

The evaluation supports behavioral attribution, but probe correctness does not reveal internal computation or prove that a model used a fact. Its conclusions are bounded by probe coverage, label quality, deployment variation, and single-decision testing.

Behavioral evidence and probe coverage

  • The paired outcomes are behavioral observations, not evidence of which internal computation produced an action or whether a correct probe answer was used.A facts-right, action-wrong result on natural states may also reflect unasked facts; probes address this only in covered domains.Source: §10 · p. 20

Labels

  • Gold actions can be stricter than necessary, and co-located receptacles caused wrong location labels in 303 of 8,197 paired ALFWorld records.The authors retain frozen labels and report sensitivity analyses.Source: §10 · p. 20

Deployments, not architectures

  • Systems differ in serving stack, readout, and input limits; author runtimes shorten some long states, while hosted models may change between versions.A JEV rerun agreed 96.3% after five days, and OpenAI Decisions was a public beta.Source: §10 · p. 20

Scope

  • The evaluation covers single decisions on frozen traces rather than closed-loop execution, and follow-up studies include only subsets of systems.The composition ladder and probe-then-decide studies each include sixteen systems; decision-oriented probes include twelve, counting the reasoning reference.Source: §10 · p. 20
  • The reasoning reference answered each question in a separate call with a verbalized distribution, so its calibration is not comparable.Source: §10 · p. 20

11 Conclusion

Across seventeen fast systems and a reasoning model, most wrong actions follow correct reading, and errors recur on states that challenge systems of different kinds. The findings distinguish failures to combine decisive facts from failures to identify the next sub-goal, motivating separate reading and decision roles.

Conclusion

  • Most wrong actions follow correct reading, and errors gather on states that defeat systems of every kind.When decisive facts are known, models can still fail to combine them; in open tasks, additional facts do not supply the missing next sub-goal.Source: §11 · pp. 20–21
  • When probes determine the action, models can fail to combine decisive facts, while composing their answers in code repairs the decision.Source: §11 · p. 21
  • The proposed division of labor lets fast models read through narrow questions and delegates decision composition to code, a planner, or deliberation where needed.Separately measuring reading and acting indicates which states require that additional decision support.Source: §11 · p. 21

A Extended Related Work

This section positions the study’s behavioral diagnosis among work on agent decisions, failure attribution, and knowing–doing gaps, then specifies its decision states, probes, outcome measures, sufficiency checks, and controlled variations.

A Extended Related Work

  • The study’s choice-only interface differs from aggregate action evaluation by pairing decisions with independent probes on the same state and testing whether probes suffice for diagnosis.Its behavioral approach also avoids relying on generated rationales or access to hidden representations.Source: A Extended Related Work · pp. 25–26

B.1 Decision states

  • A decision state contains a goal, executed history and observations, applicable rules, and admissible next actions; the model selects from fully contextualized options.Gold is a set of acceptable actions derived only from agent-visible information, and correctness means the chosen action belongs to that set.Source: §B.1 · pp. 26–27

B.2 Factual probes

  • Factual probes ask closed questions about state, progress, constraints, and evidence, with each probe and decision answered independently on the same input state.Pairing means identical state input, not observed stages of model computation.Source: §B.2 · p. 27

B.3 Four observable outcomes

  • The four outcomes cross all-probes-correct status with decision correctness, alongside conditional rates, mean probe accuracy, and applicable-state counts.Input-limit failures count as wrong and are separately reported as coverage; pooled counts summarize prevalence rather than rank systems.Source: Table 5 · p. 27, §B.3 · p. 27

B.4 When can a probe set support the diagnosis?

  • A probe set supports attributing a facts-right, action-wrong result only when states sharing correct probe answers have at least one action correct across the whole group.Finite environments can be checked exhaustively; finding no counterexample in a natural-data sample does not prove sufficiency, nor does sufficiency prove the model used the facts.Source: §B.4 · p. 28

B.5 Controlled variations

  • Controlled variations include trace ablations, relevant and irrelevant observation edits, rule changes, encoding controls, and difficulty ladders.The paper reports trace ablations, minimal pairs, rule changes, and trace-length and distance ladders; it does not analyze the encoding controls or remaining ladders.Source: §B.5 · p. 28

C Benchmark Construction

The frozen benchmark combines agent decision states with a supporting text-understanding track, using source-separated splits and audits. Its agent data span simulated environments and applied decision settings, while text-understanding primitives are not analyzed as next-action skill.

C.1 Simulated decision states

  • Most simulated paired states come from ALFWorld, BabyAI-Text, and TextWorld, with expert, noisy, and invalid-command trajectories creating failures and detours.Gold actions use only trace-visible information; determinable states have stable gold sets under alternative assumptions about unseen cells, and sampled labels were checked against breadth-first search.Source: §C.1 · p. 28, Table 6 · p. 29

C.2 Policy, tools, web and coding

  • Applied decision data cover customer-service policies, offline tool and web traces, and coding traces built from real repository images; completed-trajectory judgments remain a separate family, not next-action decisions.The coding panel asks which edit caused a named test regression, whether evidence is current or stale, and which limited next action follows.Source: §C.2 · pp. 29–30

C.3 Text-understanding primitives

  • A separate set of 24 task groups and 32,700 test records measures text-understanding primitives, but has no paired probes and is not analyzed as next-action skill.Source: §C.3 · p. 30

C.4 Splits, freezing and audits

  • Splits separate source groups between development and test, and the population is frozen before analysis; an audit found 98 records with label conflicts or dev overlap.The authors retain the frozen population, report a sensitivity analysis excluding those records, and add freshly generated confirmation cohorts because public training games may not be unseen and fixed items may be overfit.Source: §C.4 · p. 30

D Systems, Protocol, Cost and Latency

This section describes the evaluated choice interfaces and protocol, reports cost and latency comparisons, and details label-audit and probe-construction choices.

D Systems, Protocol, Cost and Latency

  • The evaluation includes native one-pass option-distribution systems, likelihood-scored open-weight models, published recipes, and a deliberating GPT-6.1 Sol reference.The reference answers one question per request; its median item latency is 2.3 seconds, about eleven times JEV’s.Source: D Systems, Protocol, Cost and Latency · pp. 30–31
  • Identical frozen inputs are used without truncation; unsupported requests count as wrong, and closed-loop execution is outside the study’s scope.Readouts differ by design, so comparisons are between deployed interfaces rather than isolated architecture, model size, or training-data effects.Source: D Systems, Protocol, Cost and Latency · p. 31
  • 0.795 is Eikos-27B’s accuracy at estimated $0.19 per 1,000 states; JEV-to-Gemma paired decision accuracy rises 0.763 to 0.811 at about five times the cost.The reasoning reference reaches paired decision accuracy 0.884 at $9.84 per 1,000 states, about 73 times JEV’s cost.Source: D Systems, Protocol, Cost and Latency · pp. 31–32

E Label audit: co-located receptacles

  • The ALFWorld audit found trace-label disagreements from co-located receptacles, and excluding 398 flagged records raises every strong system’s share of errors after correct reading.For 303 paired test records, four audited systems selected the receptacle named in the text in 297–301 cases.Source: E Label audit: co-located receptacles · p. 32, Table 8 · p. 32

F Decision-oriented probes: definitions

  • Decision-oriented ALFWorld probes cover location, inventory, object sightings and processing, placed-object counts, visited receptacles, and the next sub-goal.After exclusions, 644 of 705 sampled states remain, with 26 probe questions per state on average.Source: F Decision-oriented probes: definitions · pp. 32–33

G Generated procedures

  • Generated household and refund-policy procedures use four rule facts plus four distractors, with sufficiency verified by enumerating assignments of the rule facts.The 1,200 test states use seeds disjoint from those used for development template checks.Source: G Generated procedures · p. 33

Related papers

Loading 2610.18579v1…