PaperlayerView exact revision on arXiv

Source-linked AI summary

PRIME: Towards Verifiable Evidence-Grounded Reasoning in LLM Agents

Hamed Khosravi, Xiaoming Huo

arXiv:2610.22426v1cs.AIcs.CL

TL;DR

Agents checking records against organizational requirements can make unsupported conclusions or miss stronger conclusions by failing to test an alternative route. PRIME precommits evidence routes, gates reports on recorded checks, and guides or completes further checking. On SUFFICE, it eliminated false authorization in the reported comparison, while its completion guarantee depends on correct rules, exact checks, known locations, and sufficient budget.

  • Problem

    Record-review requirements can allow alternative check combinations, yet agents may report without support or stop at a weaker conclusion without checking another viable route.

    Source: §1 · p. 1

  • Method

    PRIME precommits evidence routes, uses a separate runtime to gate conclusions against recorded checks, and guides or completes checks that could support a stronger conclusion.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    False authorization fell from 26.7% for the fully informed agent to 0.0% with the gate on SUFFICE.

    Source: §7.1 · p. 6

  • Takeaways & Limitations

    With correct rules, exact checks at known locations, and sufficient remaining budget, completion reaches the strongest supported conclusion.

    Source: §4 · p. 4, §8 · p. 8

    PRIME’s guarantees concern authorization, not the correctness of the requirements or observations.

    Source: §5 · p. 5, Limitations · p. 8

Abstract

from arXiv · show

LLM agents are used to check records against an organization's requirements, for example whether a vulnerability report is ready for investigation. Requirements often allow several ways to be met, such as a severity rating with a curator-reviewed advisory or a published advisory link. Agents fail by accepting a report after a required check has failed, or by settling for a weaker conclusion without trying another way that could still succeed. We introduce PRIME (Precommitted Rules for Inference from Minimal Evidence). Before checking, it fixes evidence routes specifying how each conclusion may be justified. A separate program records each check result and refuses any conclusion the results do not justify. It also points the agent to checks that could justify a stronger conclusion, or runs them itself. PRIME reports only conclusions authorized by its rules and ledger. With correct requirements, exact checks at known locations, and sufficient budget at stopping, completion reaches the strongest supported conclusion. On 480 public-record tasks, an agent shown the requirements reported unjustified conclusions on 26.7% of tasks, while PRIME reported none. Its directions and added checks raised the average score for justified conclusions from 30.1, with refusal alone, to the maximum of 59.2 out of 100.

1 Introduction

Agents checking records against organizational requirements can make unsupported conclusions or stop at a weaker conclusion without checking an alternative route. PRIME precommits evidence routes, gates conclusions on recorded checks, and guides or completes further checking.

1 Introduction

  • Requirements may allow alternative routes, so a failed curator review need not rule out acceptance if a published-advisory link remains unchecked.Agents can instead accept without support or settle for a weaker conclusion without testing the alternative.Source: §1 · p. 1
  • Prior approaches address verification, evidence linking, risk calibration, rule enforcement, or check selection, but do not combine fixed alternative routes for authorization and guidance.The paper identifies this combination as the gap PRIME addresses.Source: §1 · p. 1
  • PRIME fixes sufficient check combinations before review, then uses a separate runtime to record results and refuse conclusions lacking a fully checked route.The runtime also identifies checks that could complete a route to a stronger conclusion, and can run them if the agent stops early.Source: §1 · pp. 1–2
  • On 480 heldout tasks across four models, PRIME reported no unsupported conclusions; for Qwen2.5-32B, supported-conclusion score rose from 30.1 with refusal alone to 59.2 after remaining checks ran.PRIME’s directions raised the score to 45.5 before those additional checks.Source: §1 · p. 2

2 Problem Formulation

The paper formulates review as selecting checks under a call budget while requiring every reported conclusion to be justified by the recorded evidence. A review program encodes the evidence routes that determine which conclusions are authorized.

2 Problem Formulation

  • A review case consists of a specification naming conclusions and checks, plus an initially unchecked evidence view; checks and emissions consume a call budget.Conclusions are ordered by strength, and evidence atoms represent yes-or-no conditions settled by registered procedures.Source: §2 · p. 2
  • A review program assigns each conclusion routes, each a conjunction of evidence literals, and any completed route justifies that conclusion.The program is correct when its routes match the intended standard for every evidence assignment and conclusion.Source: §2 · p. 2
  • The ledger records each evidence atom as unknown, true, or false; a conclusion is authorized when a route to it or a stronger conclusion is established.Authorization compares the ledger with the program, not directly with the world.Source: §2 · p. 2
  • The decision problem is to check, report, or stop under a budget, maximizing the utility of the strongest true conclusion while requiring reported conclusions to be justified by checks actually run.The constraint is verifiable from the fixed program and ledger and must hold in every episode, not merely on average.Source: §2 · p. 2

3 PRIME

PRIME commits a review program before checking, gates reports against the evidence ledger, and uses route residuals to guide further checks. When enabled, completion runs remaining registered checks after the agent stops, subject to the remaining budget.

3 PRIME

  • PRIME commits the program from the specification and randomness alone before checks, validates and hashes it, then gates reports to conclusions authorized by the ledger.If no accepted request is made, it returns the strongest authorized conclusion or abstains; a fixed-template renderer lists supporting checks.Source: §3 · p. 3
  • To avoid stopping at an unsupported weaker conclusion, PRIME guides checking using route residuals: the literals not yet established.A route is live only if none of its literals is false or certified unavailable.Source: §3 · p. 3
  • With remaining budget, PRIME targets the strongest conclusion having a live route within budget and selects that conclusion’s cheapest live route.The agent sees the target, the selected route’s residual, and currently authorized conclusions, but chooses the next check.Source: §3 · p. 3
  • With completion enabled, the runtime runs open checks on the selected route after the agent stops, recomputing the target until authorization, no reachable stronger conclusion, or budget exhaustion.Completion uses registered resolvers and cannot supply evidence requiring a model or person’s judgment.Source: §3 · p. 3

4 Guarantees and Why They Matter

PRIME’s gate authorizes only conclusions supported by the committed program and ledger, regardless of agent policy; completion reaches the strongest supported conclusion under stated correctness and budget conditions. The guarantees concern authorization, while errors can still arise from incorrect rules or observations.

4 Guarantees and Why They Matter

  • For every agent policy, budget, and stopping point, PRIME’s output lies in the authorized set; a sound renderer keeps user-visible conclusions within that set.The committed program is independent of evidence, so requirements cannot be tailored to observed evidence.Source: §4 · p. 3
  • PRIME’s end-to-end error can still come from a program authorizing an unsupported conclusion or from an incorrect admitted evidence value.The gate and renderer terms are zero under the stated conditions, but authorization alone does not establish truth.Source: §4 · p. 3
  • With correct routes, exact checks at known locations, no withheld relevant evidence, and sufficient remaining budget, completion reports the strongest supported conclusion.The guarantee covers a chain of conclusions, an empty Φx, and completion enabled; early stopping without completion and checks requiring field discovery are outside its scope.Source: §4 · pp. 4–5
  • The sufficient budget can exceed the cheapest true route: in the example, five calls reach c2, while three or four calls leave PRIME at c1.The target first pursues c3 and spends checks learning that e2 is false before turning to the alternative route.Source: §4 · p. 4

5 Positioning, Advantages, and Costs

PRIME uses one precommitted set of alternative evidence routes both to authorize graded conclusions and guide further checking. Its guarantees concern authorization, and depend on the review program and evidence checks meeting stated conditions.

5 Positioning, Advantages, and Costs

  • PRIME is presented as the first method to use one committed set of alternative routes both to authorize graded conclusions and guide checks toward stronger ones within budget.The ingredients—minimal routes and budgeted checking—are connected to prime implicants and sequential testing.Source: §5 · p. 5
  • The same fixed routes guide checking and authorization, making every reported conclusion auditable against the program and evidence ledger.Source: §5 · p. 5
  • PRIME requires a supplied or generated review program; its guarantees cover authorization, not whether rules or observations are correct.Verifying model judgments adds calls; known fields with deterministic checks can instead be scripted, while agent-directed checking is relevant when evidence must first be located.Source: §5 · p. 5

6 SUFFICE

SUFFICE evaluates whether agents collect evidence under a budget and report graded conclusions authorized by completed alternative routes. It comprises public-record tasks with executable requirements and scores both support and the strength of reported conclusions.

6 SUFFICE

  • SUFFICE addresses the joint evaluation gap of budgeted evidence collection and graded conclusions authorized by completed alternative routes.The accompanying comparison states that only PRIME uses alternative routes both to authorize a conclusion and choose the next check.Source: §6 · p. 5, Table 1 · p. 5
  • 938 tasks use public records from OSV, ClinicalTrials.gov, openFDA, and SEC EDGAR, organized into 120 task families with requests, three strength-ordered conclusions, routes, and field predicates.An LLM coding assistant drafted candidate fields and conclusion descriptions, but checks are executable.Source: §6 · p. 5
  • The 938 tasks comprise 480 balanced, 218 natural-prevalence, 192 development, and 48 validation tasks; development and validation use 60 families disjoint from held-out ones.The natural-prevalence split is not analyzed; held-out variants include locator uncertainty and text-judgment checks.Source: §6 · p. 5
  • False authorization is measured against source records, premature conclusions against recorded checks, and supported utility scores the strongest source-supported conclusion reported.The direct-answer Qwen2.5-32B-Instruct baseline scores 30.3 utility and 42.1% false authorization on the balanced split; executable requirements determine scores.Source: §6 · p. 6
  • The information and guidance comparisons on the balanced split were preregistered; other analyses, including premature-conclusion rates added afterward, are secondary.Inference uses task families, paired sign-flip tests, 95% bootstrap confidence intervals, and Holm correction within registered groups.Source: §6 · p. 6

7 Experiments

The experiments test PRIME’s authorization gate, guidance, and completion across public-record tasks, text judgments, adapted systems, and generated regulatory requirements. They compare unsupported reporting and supported utility under different evidence and budget conditions.

7.1 Information versus enforcement

  • 26.7% false authorization for the fully informed agent fell to 0.0% with PRIME’s gate, with or without guidance, showing that access to requirements and ledger alone did not prevent unsupported reporting.Across other models, the gate removed 18.0 and 19.5 points of false authorization for gpt-oss-20b and Mistral-Small-24B; guidance raised utility by 1.4 to 31.6 points across models.Source: §7.1 · p. 6, Table 2 · p. 6

7.2 Guidance and evidence location

  • At 20 tool requests, guidance raised utility from 30.1 to 45.5, and completion reached 59.2, the split’s maximum, by 2bmin.With uncertain evidence locations at bmin, PRIME scored 51.5 versus 40.3 for blind enumeration; at 2bmin, replayed completion restored 59.2 after PRIME’s early stopping left it at 48.8.Source: §7.2 · pp. 6–7, Figure 2 · p. 7

7.3 Judging textual evidence

  • 4.6% false authorization for model-judged evidence fell to 0.8% with verification, while supported utility decreased from 57.5 to 55.1 and verification added 1.2 model calls per episode.Screening requires a second model judgment before an attestation enters the ledger; guidance increased false authorization by prompting more judgments.Source: §7.3 · p. 7, Table 4 · p. 8

7.4 Closest control systems

  • 0% false authorization for PRIME and every adapted system except GLEAN (14.4%); PRIME with completion scored 59.2 utility against 38.7–52.0 for AgentSpec, ResearchLoop, and Proof-or-Stop.HESP also scored 59.2 but was premature on 11.7% of tasks; when only conclusions supported by recorded checks received credit, HESP scored 49.0 versus 59.2 for PRIME with completion.Source: §7.4 · p. 7, Table 5 · p. 8

7.5 Requirements generated from a regulation

  • With PRIME using completion and the protocolized program, conclusions were stated for 92.5% of registrations, with 3.2% unsupported, matching the scripted ceiling-targeting policy.Without completion, utility was 32.3 against 47.7. The program omitted deadline extensions for filed certifications; 55 of the 70 unsupported conclusions involved such a certification.Source: §7.5 · p. 8

8 Conclusion

PRIME treats review as budgeted evidence acquisition under a hard authorization constraint. Its evidence routes govern both which conclusions may be reported and which checks can support stronger ones.

8 Conclusion

  • With correct rules, exact checks at known locations, and sufficient budget, completion reaches the strongest supported conclusion.The authorization gate holds for every agent policy; remaining errors arise only from the rules or recorded observations.Source: §8 · p. 8
  • One committed set of evidence routes both authorizes reports and directs checks that could justify stronger conclusions.Empirically, guidance and completion recovered conclusions missed by the gate alone, while model-judged checks showed the importance of accurate admitted observations.Source: §8 · p. 8
  • PRIME eliminated false authorization with supplied programs and deterministic checks, while completion reached the maximum justified-conclusion score in the benchmark.The paper provides code and episode logs for PRIME.Source: §8 · p. 8

Limitations

PRIME’s guarantees concern authorization and depend on trustworthy execution and evidence; the evaluation also has limits in its policies, models, and settings.

Limitations

  • The guarantees assume faithful evidence recording and a trusted runtime and renderer, and do not establish that requirements or observations are correct.The guarantees cover authorization, not correctness of the requirements or observations.Source: Limitations · pp. 8–9
  • SUFFICE tests executable requirements across four public-record domains rather than independently validated organizational policies.Confirmatory results use one primary model, with one greedy run per task; only that model ran the text-judgment and regulation experiments.Source: Limitations · p. 9
  • The regulation study compares generated programs with an independent registry tracker, not legal adjudication; agent-directed search for model-judged evidence locations remains untested.Theorem 4.3 assumes correct programs, exact checks, and known fields, while free cached checks preclude a general termination guarantee.Source: Limitations · p. 9

Ethical Considerations

PRIME prevents conclusions unsupported by recorded checks, but its guarantees depend on correct requirements and faithful observations. Its formal guarantees also rely on a frozen program, a sound renderer, and a faithfully maintained evidence ledger.

Ethical Considerations

  • PRIME's guarantees concern authorization, so incorrect requirements or observations can still produce incorrect conclusions.Consequential deployments should validate programs and keep a person responsible for decisions.Source: Ethical Considerations · p. 9

A Formal Details and Implementation Checks

  • For partially ordered conclusions, PRIME reports every maximal authorized conclusion separately, and Theorems 4.1 and 4.2 still apply.Theorem 4.3 assumes conclusions form a chain.Source: A Formal Details and Implementation… · p. 13
  • The runtime classifies route literals as satisfied, repairable, open, or impossible, and raises an error rather than guessing when none applies.If settled atoms contradict the constraints, it authorizes nothing and exposes the conflicting attestations for review.Source: A Formal Details and Implementation… · p. 13
  • The guarantees assume a finite ordered set of conclusions, an externally supplied frozen constraint formula, a sound renderer, and a ledger that faithfully records admitted values.Exactly resolved atoms receive the full guarantee; person- or model-attested atoms are covered only as faithful records of attestations.Source: A Formal Details and Implementation… · p. 13
  • Before commitment, validation rejects malformed, unsatisfiable, redundant, or order-incoherent routes and atoms without registered resolvers.A check with an unevaluable field path or predicate establishes nothing.Source: A Formal Details and Implementation… · p. 13

B Proofs of the Results in Section 4

The proofs establish that PRIME's output is restricted to authorized conclusions and that, under exact checks and sufficient budget, completion reaches the strongest supported conclusion. They also specify the assumptions and stopping conditions behind these guarantees.

Proofs of the Results in Section 4

  • The proofs' authorization guarantees apply to every episode that produces an output, although free cached repeats mean some policies need not terminate.The runtime charges every emission and recomputes whether a stronger conclusion remains reachable before accepting it.Source: B Proofs of the Results in Section 4 · p. 14
  • Every conclusion shown to the user lies in the program's authorized set for the final evidence ledger.When no conclusion is authorized at stopping, the runtime abstains.Source: B Proofs of the Results in Section 4 · p. 15
  • The user-facing false-conclusion event is bounded by program and resolver errors, because masking and rendering contribute no errors under the theorem's premises.The bound follows from the union of error events and the guarantees that remove the mask and renderer terms.Source: B Proofs of the Results in Section 4 · p. 15
  • With correct rules, a chain of conclusions, an empty constraint formula, exact resolvers, and known fields, completion reaches the strongest supported conclusion when its remaining budget covers the unknown required call classes.Completion checks open atoms on the target route while preserving the budget-versus-remaining-calls invariant.Source: B Proofs of the Results in Section 4 · p. 16
  • Under those conditions, each completion check lowers the number of remaining required call classes by one, so completion takes at most that many checks.Cached repeats change neither budget nor that count.Source: B Proofs of the Results in Section 4 · p. 16
  • Completion stops with the strongest supported conclusion authorized, either because an equally strong conclusion is already authorized or because the budget is exhausted after all atoms on its live route are checked.The final output in a chain reports that conclusion.Source: B Proofs of the Results in Section 4 · p. 16

C Experimental Setup

The experiments evaluate PRIME on public-record tasks using controlled models, policies, budgets, and a preregistered statistical protocol. Across tested models and decoding settings, enforcement removed false authorization; added guidance and checks improved utility over refusal alone.

C Experimental Setup

  • The balanced split contains 480 tasks, including 199 (41.5%) with at least one unsatisfied route requirement and 129 with unavailable evidence, so its rates describe this constructed mix rather than real-world prevalence.The split also includes 233 unsatisfied atoms, some requiring values from another record, impossible thresholds or dates, or synthetic field values.Source: C Experimental Setup · p. 17
  • Confirmatory runs primarily used Qwen2.5-32B-Instruct at temperature zero, with one run per task; additional experiments used other models and decoding settings.The setup describes public English-language records and the models and policies used across the studies.Source: C Experimental Setup · pp. 16, 18
  • Budgets count tool calls, checks use deterministic field resolvers or model attestations, and cached repeats with matching arguments and read status cost no calls.Reporting requests are always charged; the study also compares policies that choose checks by different rules.Source: C Experimental Setup · p. 19

D Statistical Protocol

  • The statistical protocol uses family-level paired differences, sign-flip randomization tests, 95% bootstrap intervals, and Holm correction within registered groups.Tasks within a family share their request, conclusions, routes, and atoms.Source: D Statistical Protocol · pp. 19–20
  • The confirmatory analysis covers the information ladder and guidance contrasts on the balanced split, with Qwen2.5-32B-Instruct primary and Qwen2.5-7B secondary.Other models, alternate splits, and the regulatory evaluation are secondary analyses without multiplicity-corrected tests.Source: D Statistical Protocol · p. 20

E Additional Results

  • 4.8 points (95% CI [1.2, 8.6]) was the significant false-authorization change from displaying the ledger, while the mask removed 28.0 points (95% CI [22.7, 33.5]).These are the reported information-ladder findings.Source: §E.1 · p. 20

E.1 The full information ladder and other models

  • 19.9% to 26.7% was the false-authorization range from +verify to the fully informed agent, while utility rose from 10.0 to 38.0; the mask alone reduced false authorization to zero.The passage reports this pattern for Qwen2.5-32B-Instruct; the mask removed false authorization for every tested model and decoding setting.Source: §E.1 · p. 20
  • 2.5 (95% CI [0.9, 4.2]) and 5.6 (95% CI [3.1, 8.1]) were the family-mean utility costs of the mask for gpt-oss-20b and Mistral-Small-24B, respectively.On the security domain alone, the mask removed 31.8 points of false authorization for Qwen2.5-32B.Source: §E.1 · p. 20

E.2 Registered guidance cells and alternatives to guidance

Across four experiments, PRIME’s guidance, completion, and verification are tested against alternatives on public-record tasks. Results show how these mechanisms affect utility, unsupported conclusions, and the strength of reported conclusions.

E.2 Registered guidance cells and alternatives to guidance

  • The ideal after-the-fact filter scored 43.5, below PRIME’s 45.5, while the flat gate scored 8.1 and stated conclusions on 9.8% of answerable tasks.The filter cannot change which evidence was gathered; the flat gate lacks graded conclusions, guidance, and completion.Source: §E.2 · pp. 21–22
  • Guidance exceeded the 6.16-point minimum in all eight Qwen2.5-32B-Instruct budget-and-route cells, with larger gains on branching tasks and smaller gains as budgets grew.For Qwen2.5-7B, all eight gains were 0.6–3.1 points and every interval ended below 6.16; the random-order policy reached the 59.2 ceiling from 2bmin.Source: §E.2 · p. 21, Figure 4 · p. 22
  • At 20 calls, PRIME reached utility 45.5 versus 30.1 without guidance (gain 15.6, 95% CI [12.5, 18.6]); runtime completion added 0.5 to 0.9 calls per episode.The runtime refused 364 emission attempts; on tasks with a supported strongest conclusion, the gate alone reached 25.7, compared with 65.8 for PRIME.Source: §E.2 · pp. 21–22, Figure 5 · p. 22

E.3 Field choice and other models

  • Guidance raised utility at bmin by 28.8 points for gpt-oss-20b and 10.3 for Mistral-Small-24B; field-choice benefits varied by model and completion compensated for early stopping and poor choices.On the locator-uncertain copy, Mistral-Small-24B’s completion finished 2.8 points above blind enumeration; Table 11 reports the three-model results.Source: §E.3 · p. 23, Table 11 · p. 24

E.4 Text judgments and verification

  • Under PRIME, admitted wrong attestations led to authorized conclusions through routes containing wrongly attested atoms; the text-evidence score tests agreement with structured references for specified conditions.The two remaining unsupported conclusions involved product mentions that differed from structured registry identifiers.Source: §E.4 · pp. 23–24
  • Screening attestations reduced false authorization from 4.6% to 0.8%, while supported utility fell from 57.5 to 55.1.The verifier refused 65 attestations in 59 episodes, including 46 whose values matched the benchmark reference; it added 1.2 model calls and 3.6% more tokens per episode.Source: §E.4 · p. 24

E.5 The closest systems

  • PRIME with completion scored 59.2, 10.3 family-mean points above ResearchLoop (47.7) and 5.6 above Proof-or-Stop, whose utility was 7.7 points above PRIME without completion.None of the three runs the remaining checks itself.Source: §E.5 · p. 26
  • GLEAN reduced false authorization from 26.7% to 14.4% but retained unsupported conclusions and lowered utility from 38.0 to 35.0; HESP reached 59.2 at 20 calls but made premature conclusions on 11.7% of tasks.GLEAN verifies logged trajectories but cannot direct the agent to checks that would establish a stronger conclusion.Source: §E.5 · pp. 26–27

F Externally Derived Regulatory Requirements

This appendix evaluates two programs encoding U.S. clinical-trial reporting requirements against tracker records. The results distinguish program agreement and coverage from what the runtime can establish, and show how completion affects the conclusions reached.

F Externally Derived Regulatory Requirements

  • On 1,999 held-out registrations, the protocolized program covered 92.4% with 3.5% unsupported conclusions, while the assistant-built program covered 26.3% with 1.8% unsupported.The tracker is not a legal adjudication, and the assistant-built program’s deadline reading was selected by agreement with the tracker on a development pool.Source: F Externally Derived Regulatory… · pp. 27–28
  • The assistant-built program agreed with the tracker on 518 of 526 held-out registrations it classified (98.5%), while its 1.8% unsupported rate used a different definition.Twenty-three records had fields updated between the tracker export and record fetch; all 23 agreed with the tracker, and agreement on unchanged records was 495 of 503 (98.4%).Source: F Externally Derived Regulatory… · pp. 28–29, Table 15 · p. 29
  • The protocolized program omitted certification extensions, phase-1 and feasibility exclusions, and a U.S.-connection condition; these omissions and the assistant-built program’s applicability limits shaped coverage and disagreements.The held-out sample included 526 registrations classified by the assistant-built program; 353 others lacked an applicability field its deployed predicates could read.Source: F Externally Derived Regulatory… · pp. 28–29
  • The runtime stated c3 for 73 of 86 late filers and for none of 80 trials that never filed, because no submission event established c3 for the latter.For 13 other late filers, the submission date appeared only in an estimated field the program did not read, so the runtime stated c2.Source: F Externally Derived Regulatory… · p. 29
  • On 319 development registrations, PRIME with completion reached utility 74.3 and matched the scripted policies; without completion, it reached 36.5 after the agent stopped at c1.On the held-out subset of 600, completion matched the ceiling-targeting policy at utilities 47.7 and 19.0 for the two programs, while PRIME alone reached 32.3 and 9.9.Source: F Externally Derived Regulatory… · pp. 29–30, Table 16 · p. 30

G Exploratory Pilot on Hand-Built Tasks

In a separate hand-built pilot, enforcement prevented unsupported conclusions, while guidance could improve utility. The related-work discussion distinguishes PRIME’s route-based authorization from evidence checking, action control, and diagnostic probing.

G Exploratory Pilot on Hand-Built Tasks

  • The exploratory pilot covered 21 software-review tasks, 8 research-review tasks, and 31 record tasks over 27 records, with one route per conclusion.Runs used gpt-4.1 and gpt-4o-mini at 20 tool calls, except where noted; results were kept separate from confirmatory statistics.Source: G Exploratory Pilot on Hand-Built Tasks · p. 30
  • ReAct stated unsupported conclusions on 25.0% to 100.0% of tasks in each pilot domain, while the static checklist and PRIME without guidance stated none.With gpt-4o-mini on software tasks, PRIME reached 0.0% (utility 38.9) against ReAct’s 71.4% (34.1); Table 17 reports the pilot comparison.Source: G Exploratory Pilot on Hand-Built Tasks · p. 30, Table 17 · p. 31

H Extended Related Work

  • Prior reliability methods check answers, link claims to sources, calibrate factuality, or identify minimal evidence, while PRIME targets explicit requirements across ordered conclusions and alternative routes.The discussion also situates evidence collection and stopping methods as related but distinct approaches.Source: H Extended Related Work · p. 30
  • External controllers primarily govern actions or transitions, whereas PRIME governs which evidence-dependent conclusion may be released; action guards can complement its evidence resolvers.The discussion names runtime shielding, AgentSpec, Progent, Doshi et al., and AGATE as related controller approaches.Source: H Extended Related Work · pp. 30–31
  • PRIME keeps the evidence ledger as runtime state, using committed requirements both to guide remaining checks and to reject conclusions unsupported by recorded checks.Its target is narrower than verifying all intermediate reasoning or the factual correctness of observations.Source: H Extended Related Work · p. 31
  • PRIME’s contribution is a shared precommitted program for route-aware evidence acquisition and graded answer authorization, not the invention of evidence gating or classical formal ingredients.Unlike diagnostic-triage approaches, it tracks logically sufficient routes, including disjunctive alternatives and weaker authorized conclusions; incorrect programs or fallible resolvers can still cause errors.Source: H Extended Related Work · p. 31
Loading 2610.22426v1…