PaperlayerView exact revision on arXiv

Source-linked AI summary

Evidence-Inference Reconstruction: When The Evidence Is Recalled But The Reasoning Goes Wrong

Megan Diehl, Ser-Nam Lim

arXiv:2609.33778v1cs.AI

TL;DR

Multi-hop agents often retry reasoning after detecting intermediate errors, although those retries can be costly when the needed evidence has already been found. EIR collects evidence along one retrieval trajectory and answers in a separate final call; across three benchmarks, it improves Answer F1 over the baseline and iterative methods while averaging fewer model calls.

  • Problem

    Corrective multi-hop agents retry intermediate reasoning or trajectories after errors, incurring substantial computation even when relevant evidence may already have been found.

    Source: §1 · p. 2

  • Method

    EIR uses requirements and conclusions to guide one retrieval trajectory, accumulates source evidence, and answers in a separate final call using the collected evidence.

    Source: §1 · p. 2, §3 · p. 3, §3.3 · p. 5

  • Results

    Across matched subsets of three benchmarks, EIR improves Answer F1 by 8.3–32.8 points over the baseline, 10.6–29.1 over Agentic SSR, and 1.1–15.9 over Reflexion.

    Source: §1 · p. 2

  • Takeaways & Limitations

    When relevant evidence has been collected, EIR can produce an evidence-backed answer without retrying intermediate reasoning, even if irrelevant information is also present.

    Source: §1 · p. 2

    The evaluations cover two language models, three multi-hop benchmarks, and one Wikipedia search setting, and the compared methods use different procedures for selecting evidence and producing answers.

    Source: §6 · p. 9

Abstract

from arXiv · show

Modern multi-hop LLM agents are equipped with built-in mechanisms to detect errors in intermediate reasoning steps. Such errors trigger corrective actions from these agents, which mostly follow the paradigm of retrying the steps or the reasoning trajectories. Not only are these retries expensive, we present in this paper that they are also potentially unnecessary. To this end, we introduce Evidence-Inference Reconstruction (EIR), which uses structured state to guide one retrieval trajectory, accumulating source evidence in the process. We show that as long as the relevant evidence has been collected, EIR is capable of generating the correct answer in a single final model call even if erroneous evidence has been mixed in due to incorrect intermediate reasoning steps. In one evaluation, using Haiku 4.5 and GPT-4.1 Mini, we evaluate EIR on matched 1,000-question subsets of HotpotQA, 2WikiMultiHopQA, and MuSiQue, showing that EIR improves Answer F1, the overlap between the model's and the correct answer, over the baseline by 8.3--32.8 points, Agentic SSR by 10.6--29.1 points, and Reflexion by 1.1--15.9 points. Additionally, we show that EIR averages 4.85 total model calls per question, compared with 35.29 for Agentic SSR and 12.41 for Reflexion. Together, these results corroborate EIR's central premise: separating evidence retrieval from the final answer model call can improve answer accuracy while utilizing substantially less computation.

1 INTRODUCTION

EIR separates evidence retrieval from final answer generation, aiming to avoid costly retries when the needed evidence is already collected. Across three benchmarks, it improves Answer F1 over the baseline, Agentic SSR, and Reflexion while using fewer model calls.

1 INTRODUCTION

  • Corrective methods retry reasoning after detected errors, but retries cost computation and can revisit steps even when correct evidence is already present.Intermediate errors may also lead to erroneous evidence being encountered in later steps.Source: §1 · p. 2
  • EIR guides one retrieval trajectory with requirements and conclusions, stores retrieved source text separately, and generates the answer in a single final model call.The final call uses accumulated evidence rather than carrying forward intermediate reasoning, allowing reconsideration without repeating evidence-acquisition actions.Source: §1 · p. 2
  • In the Humphrey example, Agentic SSR and Reflexion retrieve the needed family relationships but still return incorrect answers after further searches or repeated trials.EIR answers Mary de Bohun after acquiring the evidence; Figure 2 is an instrumented reproduction outside the confirmatory protocol.Source: §1 · p. 2, Figure 2 · p. 3
  • EIR improves Answer F1 by 8.3–32.8 points over the baseline, 10.6–29.1 over Agentic SSR, and 1.1–15.9 over Reflexion.On matched 1,000-question subsets of HotpotQA, 2WikiMultiHopQA, and MuSiQue, it averages 4.85 calls per question versus 35.29 for Agentic SSR and 12.41 for Reflexion.Source: §1 · pp. 2–3

2 RELATED WORK

Related agentic retrieval and corrective-reasoning approaches can remain computationally costly, and intermediate errors can affect later answers. EIR instead bases its final answer on accumulated evidence alone.

2 RELATED WORK

  • Agentic RAG integrates real-time external retrieval, but can share corrective reasoning’s risk that an intermediate error cascades into a wrong final answer.The passage contrasts this with EIR’s single final answer call using accumulated evidence.Source: §2 · p. 3
  • Some approaches store evidence alongside reasoning or track evidence gaps, but still do not fully separate evidence retrieval from final answering.The passage notes that potentially incorrect intermediate reasoning can therefore still affect the answer.Source: §2 · p. 3
  • EIR can answer correctly despite irrelevant collected information when the relevant information has been retrieved, because its final call uses only accumulated evidence.Source: §2 · p. 3

3 METHOD

EIR uses structured retrieval state to collect evidence along one trajectory, then answers in a separate call using the question and accumulated source text. This boundary keeps retrieval reasoning out of the final answer, while limiting recovery to evidence the agent actually found.

3 METHOD

  • EIR guides one retrieval trajectory with requirements for missing information and conclusions recording supported claims, then generates the answer in a separate call from collected evidence.The final call receives the original question and retrieved source text, not the agent’s retrieval reasoning.Source: §3 · p. 3, §3.3 · p. 5

3.1 TASK AND INTERACTION MODEL

  • An episode begins with a question, a pool of paragraph titles, and a turn budget; the agent initially sees only the question and titles.At each turn, it may read an unread paragraph or submit an answer.Source: §3.1 · p. 4

3.2 STATE-GUIDED EVIDENCE ACQUISITION

  • EIR retains every read sentence in an evidence store and updates requirement and conclusion lists to guide subsequent reading and submission decisions.Requirements describe information still needed; conclusions pair claims considered supported with the evidence records cited for them. The update check enforces that citations refer to read records, but does not verify that they support the claims, so errors can persist.Source: §3.1 · p. 4, §3.2 · p. 4

3.3 FINAL ANSWER RECONSTRUCTION

  • After retrieval ends, a separate call answers from only the original question and accumulated evidence, so retrieval mistakes do not directly carry into the answer.The final call cannot recover information the agent never discovered.Source: §3.3 · p. 5

4 EXPERIMENT DESIGN

The experiments compare EIR with baselines on matched benchmark questions, add a corpus-search evaluation, and specify answer, evidence, computation, and statistical measures.

4.1 EVALUATION SETUP

  • The evaluation uses 1,000 questions each from HotpotQA, 2WikiMultiHopQA, and answerable MuSiQue with Haiku 4.5 and GPT-4.1 Mini.The question sets use ten paragraphs per question for HotpotQA and 2WikiMultiHopQA, and twenty for MuSiQue.Source: §4.1 · p. 5

4.2 BASELINES

  • Direct interleaves reasoning and paragraph selection, while Agentic SSR verifies proposed actions and Reflexion retries retrieval; EIR tracks requirements and conclusions while selecting paragraphs.Full Pool reads every paragraph before answering as an idealized comparison, though the paper describes it as impractical for larger document collections.Source: §4.2 · pp. 5–6

4.3 EXTENDED EVALUATION WITH CORPUS SEARCH

  • The corpus-search extension tests the same HotpotQA questions and both models against 5.23 million Wikipedia introductory paragraphs using BM25-ranked search results and paragraph reads.Trajectories allow up to 12 action turns and six read attempts per turn.Source: §4.3 · p. 6

4.4 METRICS AND STATISTICAL PROCEDURE

  • The study reports Answer F1, Exact Match, Support F1, and model-call counts, with paired-question resampling for Answer F1 and McNemar tests for Exact Match.Holm correction is applied separately across comparison groups; metrics average over all 1,000 questions, including failed outputs.Source: §4.4 · p. 6

5 RESULTS

EIR improves answer quality over direct answering and iterative correction while using fewer calls; results also show limits in evidence retrieval and in comparisons with full-pool answering.

Overall Performance

  • 8.27–32.79 points: EIR improves Answer F1 over Direct across three benchmarks with both models, with every gain significant.The evaluation compares matched 1,000-question subsets and includes only questions with readable outputs from both methods.Source: §5 · p. 6, Table 1 · p. 7

Accuracy and Computation against Iterative Correction

  • 10.56–29.12 Answer F1 points: EIR exceeds Agentic SSR, and it retains significant gains over oracle-stopped Reflexion in five of six comparisons.EIR averages 4.85 calls per question, versus 35.29 for Agentic SSR and 12.41 for oracle-stopped Reflexion.Source: §5 · p. 7, Table 1 · p. 7

Answering from All Provided Paragraphs

  • 5.60 points with Haiku 4.5 and 6.23 with GPT-4.1 Mini: EIR improves Answer F1 over Full Pool on 2WikiMultiHopQA.Full Pool reads all provided paragraphs and answers in one call; its higher observed GPT-4.1 Mini score on HotpotQA is 3.08 points above EIR, while three other comparisons are inconclusive.Source: §5 · p. 7, Table 2 · p. 8

Requirements and Conclusions during Retrieval

  • Requirement and conclusion lists improve Answer F1 on 2WikiMultiHopQA and MuSiQue, but show no significant benefit on HotpotQA with either model.The significant gains are 2.45 points for GPT-4.1 Mini on 2WikiMultiHopQA and 4.83 points for Haiku 4.5 on MuSiQue; revising conclusions has mixed effects.Source: §5 · p. 8

Transfer to Corpus Search

  • In the 5.23-million-paragraph search setting, EIR significantly improves Answer F1 over Direct, Agentic SSR, and Reflexion’s first trial with both models.Against oracle-stopped Reflexion, GPT-4.1 Mini favors EIR by 4.68 points; Haiku 4.5 favors Reflexion by 0.86 points overall.Source: §5 · p. 8, Table 3 · p. 8

Final Answer Call

  • The separate final answer call improves mean Answer F1 over the answer submitted during retrieval by 21.20 points with GPT-4.1 Mini and 6.88 with Haiku 4.5.The result supports the possibility of improving answers from already collected evidence.Source: §5 · p. 8

Evidence Availability and Remaining Errors

  • EIR sometimes answers incorrectly despite reading all necessary evidence, while it misses at least one supporting paragraph in 3.0%–37.5% of answers across the reported model-benchmark combinations.Adding missing supporting paragraphs and repeating the final answer call yields exact matches in 26.84%, versus 1.36% with only the originally retrieved paragraphs; a separate retrieval ablation raises overall EM by 2.13 points but uses about 37% more calls and reverses some exact answers.Source: §5 · p. 8, Figure 3 · p. 9

6 LIMITATIONS

The evaluation covers two language models, three supplied-paragraph benchmarks, and one additional Wikipedia search setting, with each model-benchmark evaluated once. Comparisons use different evidence-selection and answer procedures, though the paper reports targeted ablations on retrieval lists.

Task Scope and Evidence Dependence

  • The evaluation covers only two language models, three multi-hop benchmarks with supplied questions and paragraphs, and one additional Wikipedia search setting.Each model and benchmark is evaluated once; the authors identify expansion to more models and benchmarks as a possible extension.Source: §6 · p. 9

Attribution and Interfaces

  • Because methods use different evidence-selection and answer procedures, the comparisons characterize complete methods rather than isolating each component.The authors note that more uniform procedures could potentially be applied across methods; retrieval-list ablations provide targeted evidence about those lists.Source: §6 · p. 9

7 CONCLUSION

The conclusion argues that repeated reasoning or retrieval can be unnecessary once an agent has gathered the evidence needed to answer. EIR separates evidence acquisition from final inference, but its accuracy depends on the evidence being available and correctly interpreted.

7 CONCLUSION

  • Across three benchmarks and both models, EIR significantly improves Answer F1 over Agentic SSR in all six comparisons and oracle-stopped Reflexion in five of six, while using substantially fewer calls.The full-corpus evaluation also shows accuracy and computation advantages over Agentic SSR using both models and oracle-stopped Reflexion using GPT-4.1 Mini.Source: §7 · p. 9
  • EIR uses requirements and conclusions to guide one evidence-retrieval trajectory, then generates the final answer from accumulated facts without carrying forward earlier reasoning or answers.The authors conclude that separating evidence acquisition from final inference supports avoiding unnecessary retries once the needed evidence has been collected.Source: §7 · p. 9
  • EIR can still produce incorrect outputs when relevant evidence is missing, misleading, or incorrectly interpreted, so the work does not establish reliability for high-stakes decisions.The experiments assess answer accuracy and computational behavior, not mitigation of biases inherited from pretrained models and existing text corpora.Source: §7 · p. 10

A RECORDED SSR AND REFLEXION TRACES

In the Humphrey example, Agentic SSR and Reflexion retrieve relationships that could identify his aunt but nevertheless return incorrect answers. The appendix also specifies how evidence is recorded and how questions and benchmark inputs are handled.

A RECORDED SSR AND REFLEXION TRACES

  • Both Agentic SSR and Reflexion read that Eleanor de Bohun was Humphrey’s mother and Mary de Bohun was Eleanor’s sister, yet return incorrect answers.The example’s EIR trace is illustrative and separate from the scored evaluation; its six reads, submission call, and final answer call total eight calls.Source: A RECORDED SSR AND REFLEXION TRACES · p. 12
  • Agentic SSR’s first two reads establish the aunt’s relationships, but later actions pursue unrelated family branches and it submits Lady Anne Neville after rejected reasoning fails to meet the refinement agreement rule.The diagram shows all verification calls for the first turn and condenses later turns.Source: Figure 4 · p. 12
  • Reflexion’s five trials all read the supporting relationships but identify Humphrey’s mother as his aunt; none matches the reference, so all trials run, using 18 action calls and four reflection calls.Source: Figure 5 · p. 13

B.1 EVIDENCE RECORDS, QUESTIONS, AND BASELINES

  • EIR records requirements and conclusions with statuses, and conclusions cite previously read sentences; it validates that citations exist, not that they support the claim.Source text is stored in separate records and cannot be rewritten; the main evaluation uses 1,000 questions per benchmark and preserves benchmark paragraph text, sentence boundaries, and order.Source: §B.1 · p. 13
  • Agents initially see paragraph titles, and the main evaluation supplies ten paragraphs per HotpotQA and 2WikiMultiHopQA question and twenty for answerable MuSiQue questions.Answering and reflection prompts do not include reference answers or labels identifying needed evidence.Source: §B.1 · p. 13

B.2 MODEL AND ANALYSIS SETTINGS

These sections describe prompt and comparison setups for EIR, Direct, Agentic SSR, and Reflexion, then report selected accuracy, parsing, and supporting-citation findings. The supplied passages also define the scope of the computation comparison and retrieval-list analyses.

B.3 PROMPT INPUTS AND ANSWER GENERATION

  • The EIR final-answer prompt contains only the question and collected, position-labeled sentences, excluding earlier answers, lists, and reasoning; Full Pool instead supplies every paragraph, including distractors.During retrieval, EIR uses its lists to guide searches and reads, then answers from all collected text.Source: §B.3 · p. 14

C.1 ANSWER ACCURACY AND SUPPORTING CITATIONS

  • The accuracy table reports Exact Match and Support F1 alongside the main Answer F1 results for all 1,000 questions per benchmark and model.The table caption specifies paired bootstrap intervals for EIR-minus-Direct Support F1, each excluding zero.Source: §C.1 · p. 15, Table 5 · p. 15

C.2 OUTPUTS THAT COULD NOT BE PARSED

  • Unparseable outputs range from 2.8%–20.0% for Direct with Haiku 4.5 and 1.5%–5.3% with GPT-4.1 Mini, versus 0.0%–0.1% and 1.0%–1.4% for EIR.All questions remain in the main scores; among questions with valid outputs from both methods, the Haiku MuSiQue Answer F1 gain falls from 11.16 to 3.76 points over 800 questions.Source: §C.2 · p. 15

D.1 ADAPTATION TO ACTION SELECTION

  • Agentic SSR uses three verifier calls per action-selection assessment and permits at most one refinement when rejection and correction agreement criteria are met; Reflexion retries retrieval with written feedback.Oracle-stopped Reflexion ends at the first exact match or returns trial five, and additional verification, refinement, and reflection calls count toward computation.Source: §D.1 · p. 15

D.2 ANSWER F1

  • The Answer F1 comparison table reports EIR-minus-baseline differences in percentage points on matched questions, with paired bootstrap intervals and Holm adjustment.Its accompanying text defines positive differences as favoring EIR.Source: Table 6 · p. 16, §D.2 · p. 16

D.3 EXACT MATCH

  • On MuSiQue with Haiku 4.5, oracle-stopped Reflexion has higher Exact Match even though EIR’s Answer F1 point estimate is higher, because Answer F1 gives partial credit for word overlap.Source: §D.3 · p. 16

D.4 SUPPORTING CITATIONS

  • Against oracle-stopped Reflexion, EIR has significantly higher Support F1 on all three benchmarks with Haiku 4.5; with GPT-4.1 Mini, Reflexion leads on all three, significantly only on 2WikiMultiHopQA.Support F1 is reported on a 0–100 scale, with the table comparing EIR against oracle-stopped Reflexion.Source: §D.4 · p. 17, Table 8 · p. 17

D.5 COMPUTATION

  • The average-accuracy and computation table compares methods averaged over three benchmarks and two models, including ratios to EIR and costs at fixed prices.The Reflexion cost range reflects missing token breakdowns rather than statistical uncertainty.Source: Table 9 · p. 17

E.1 TURN BUDGET SENSITIVITY ANALYSIS

The sensitivity analyses test EIR’s read budget, the contribution of retrieval-list feedback, and dependency repair. They find that budget increases affect benchmarks differently, while the lists and repair procedure show distinct accuracy patterns.

E.1 TURN BUDGET SENSITIVITY ANALYSIS

  • On HotpotQA and 2WikiMultiHopQA, read budgets other than H = 6 change Answer F1 by at most 1.7 points without significant differences; on MuSiQue, H = 10 improves Haiku 4.5 Answer F1 by 3.37 points and costs 15.4% more calls.For GPT-4.1 Mini on MuSiQue, H = 10 improves Answer F1 by 2.11 points, but the difference is not significant after adjustment; H = 4 significantly lowers Haiku 4.5 Answer F1 by 3.18 points.Source: Table 11 · p. 18, Figure 6 · p. 18, §E.1 · pp. 18–19

E.2 LIST-FEEDBACK CONTROL

  • Including EIR’s requirement and conclusion lists increases Answer F1 on 2WikiMultiHopQA and MuSiQue with both models, but shows no positive effect on HotpotQA.Gold-document recall does not consistently increase with the lists; the control reads more paragraphs per question in every comparison.Source: §E.2 · p. 19

E.3 REVISING DEPENDENT CONCLUSIONS

  • Dependency repair yields lower Answer F1 than default EIR in five of six comparisons, significantly on HotpotQA and 2WikiMultiHopQA for both models.Applying proposed changes significantly improves Answer F1 only on MuSiQue with Haiku 4.5.Source: §E.3 · p. 19, Table 13 · p. 20, Table 14 · p. 20

F MISSING EVIDENCE AND ADDITIONAL RETRIEVAL

  • Reading every benchmark-labeled evidence paragraph does not guarantee a correct answer, and Exact Match can count differently worded answers as errors when normalization does not equate them.The missing-evidence analyses supply omitted paragraphs directly and then test model-selected additional retrieval for known errors or all questions.Source: F MISSING EVIDENCE AND ADDITIONAL… · p. 20

F.1 ERRORS AFTER READING ALL OR PART OF THE REQUIRED EVIDENCE

The error analyses distinguish answers that lacked required evidence from those that failed despite reading it. Adding evidence improved answers in targeted and all-question tests, while additional retrieval also increased computation and changed some previously correct answers.

F.1 ERRORS AFTER READING ALL OR PART OF THE REQUIRED EVIDENCE

  • Of 2,758 Exact Match failures, 1,510 had all required paragraphs read, while 1,248 were missing at least one.Among the 1,248 with omitted evidence, 1,047 lacked one required paragraph, 194 lacked two, and seven lacked three.Source: §F.1 · p. 20, Table 15 · p. 20

F.2 SUPPLYING THE MISSING SUPPORTING PARAGRAPHS

  • Adding missing paragraphs yielded 335 exact matches (26.84%) and a 28.66-point Answer F1 gain, versus 17 exact matches (1.36%) and a 1.22-point gain from repeating the answer call unchanged.Benchmark labels selected the missing paragraphs, so this tests receiving them rather than the ability to find them.Source: §F.2 · p. 21

F.3 SELECTING MORE EVIDENCE AFTER A KNOWN FAILURE

  • For the 1,248 known failures, model-selected paragraphs yielded 174 exact matches (13.94%) and a 14.78-point Answer F1 gain; this test does not establish whether the model recognizes its own errors.Of those matches, 173 followed a successful additional read.Source: §F.3 · p. 21

F.4 ADDITIONAL RETRIEVAL WITHOUT KNOWING WHICH ANSWERS FAILED

  • Across all questions, further reading raised Answer F1 from 66.89 to 69.27 and Exact Match from 54.03% to 56.17%, while average calls rose from 4.847 to 6.654.211 nonexact answers became exact, while 83 exact answers became nonexact.Source: §F.4 · pp. 21–22

G FULL POOL EVALUATION

The appendix compares EIR with answering from the full paragraph pool and describes the search, reading, and baseline settings. Results report Answer F1 comparisons, observed Support F1, and computation costs, with qualifications on statistical testing and evaluation scope.

G FULL POOL EVALUATION

  • Full Pool had higher observed Support F1 in all six comparisons, while EIR’s two 2WikiMultiHopQA Answer F1 gains also held for Exact Match; Full Pool’s GPT-4.1 Mini HotpotQA advantage was not significant for Exact Match.Full Pool used 6,000 calls and cost $10.01, compared with EIR’s 29,082 calls and $44.28.Source: Table 18 · p. 22, G FULL POOL EVALUATION · p. 22

H.1 QUESTIONS, SEARCH, AND READING

  • The search extension used 5,233,329 HotpotQA Wikipedia introductory paragraphs but reused the same 1,000 scored questions, testing transfer to a larger collection rather than new-question performance.Search used BM25 ranking; the extension’s turn and read limits differed from the main evaluation, so cross-evaluation differences are descriptive.Source: §H.1 · p. 23

H.2 MODEL AND BASELINE SETTINGS

  • EIR retained its lists and separate final answer call, while Reflexion carried forward written reflections across up to five trials; Agentic SSR’s verifier temperature differed from the cited rules, and that change was not separately tested.Calls used temperature 0.0 and a 2,048-token output limit in this evaluation.Source: §H.2 · p. 23

H.3 SCORES AND STATISTICAL COMPARISONS

  • Oracle-stopped Reflexion’s call total includes earlier trials and reflections, and excluding ten interrupted Haiku 4.5 questions did not change the conclusion against it.Tables 19 and 20 report scores and within-model comparisons.Source: §H.3 · p. 23, Table 19 · p. 24, Table 20 · p. 24

H.4 WHAT THE RECORDED RUNS SHOW

Recorded-run analyses show that EIR’s separate final answer call can substantially improve answers based on retrieved text, while model comparisons vary. The analyses also qualify what can be inferred from saved runs, post-evidence calls, and rejected actions.

H.4 WHAT THE RECORDED RUNS SHOW

  • Reading every benchmark-labeled paragraph establishes text availability, not model understanding; the final-call analysis also excludes different numbers of forced or interrupted runs across models.The final prompts contained only the question and saved source sentences, and score changes may reflect answer wording as well as reasoning.Source: §H.4 · pp. 24–25
  • Against oracle-stopped Reflexion, EIR’s Haiku 4.5 difference was −0.86 Answer F1 points and statistically inconclusive, while its GPT-4.1 Mini difference was a significant +4.68 points.For Haiku 4.5, EIR’s mean loss of 66.42 points exceeded its mean gain of 52.43 despite more frequent wins.Source: §H.4 · p. 24
  • Reflexion’s later trials added 11.83 Answer F1 points with Haiku 4.5 and 5.48 with GPT-4.1 Mini, but reflection, retrieval, and exact-match stopping changed together; post-support call counts do not show those calls were unnecessary.The post-support counts exclude the read that completes the evidence and do not cover Agentic SSR.Source: §H.4 · p. 24, Table 21 · p. 25
  • For accepted retrieval answers, the final call raised Answer F1 by 21.20 points for GPT-4.1 Mini and 6.88 points for Haiku 4.5.Exact Match improved on 254 questions and was lost on 33 with GPT-4.1 Mini; the corresponding Haiku 4.5 counts were 55 and 20.Source: §H.4 · p. 25
  • Of Haiku 4.5 EIR’s 590 forced endings, 323 followed invalid citations, 234 followed typed-position or missing-identifier errors, and 33 followed attempted reading or searching on the final turn; all still received a final answer call.The comparison includes differences in how methods accept responses.Source: §H.4 · p. 25

Related papers

Loading 2609.33778v1…