PaperlayerView exact revision on arXiv

Source-linked AI summary

YouRA: A Persistent-State Architecture for Evidence-Traceable Autonomous Research Agents

Yoonkyu Woo, Woojin Lee, Jin-Xia Huang

arXiv:2610.01097v1cs.AI

TL;DR

Autonomous research agents can produce papers, but persistent, verifiable state and claim–evidence alignment remain missing across long pipelines. YouRA preserves research state, execution evidence, and failure history through VSA, an Independent Controller, and Stateful Reflection. On MLR-Bench’s predefined ten-task subset, it outperforms MLR-Agent and AI Scientist V2 on scalar Overall across all three matched backbones.

  • Problem

    Research agents do not maintain the research trajectory as explicit, persistent, verifiable state, limiting long-horizon coherence and claim–evidence alignment.

    Source: §1 · pp. 1–2

  • Method

    YouRA combines VSA state records, an independent controller that separates control from execution, and Stateful Reflection for structured failure memory and recovery.

    Source: §1 · p. 2, §3.2 · p. 5, §3.3 · p. 5

  • Results

    On MLR-Bench’s predefined ten-task subset, YouRA achieves the highest 4-judge end-to-end Overall on every matched backbone, outperforming MLR-Agent and AI Scientist V2.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A scalar end-to-end score cannot by itself establish that gains reflect better research rather than persuasive fabrication, so it should be paired with evidence-oriented diagnostics.

    Source: §6 · p. 9

    Evaluation is limited to machine-learning and deep-learning tasks; alignment with human reviewers remains unvalidated for this judge configuration.

    Source: Limitations · p. 10

Abstract

from arXiv · show

End-to-end research agents can now produce complete scientific papers, yet manuscript claims often diverge from executed experiments. This gap is structural: research state, failure histories, and claim-evidence alignment are not maintained as persistent, verifiable state across long-horizon pipelines. We present YouRA (Your Research Agent), an architecture for stateful, evidence-traceable autonomous research. YouRA preserves research state, execution evidence, and failure history across the research trajectory by integrating three components: a Verification State Architecture (VSA) that tracks hypotheses, gates, and evidence pointers; an Independent Controller that turns state and reflection records into lifecycle, recovery, and debate/review control while separating control from execution; and Stateful Reflection that logs failures as structured lessons and routes recovery through bounded repair, redesign, or reset. On MLR-Bench's predefined ten-task end-to-end subset, YouRA improves over both MLR-Agent and AI Scientist V2 on scalar Overall across all three matched backbones. An automated diagnostic using MLR-Bench's hallucination taxonomy reports intersection/union counts for four fact-based failure types, and data-provenance diagnostic shows more real-data-based outputs. Ablating each of the four components (the VSA, the Independent Controller, MCP tool access, and reflection-guided recovery) supports their separable contributions. Removing either core-state component drops YouRA below the full system. Code: https://github.com/PrayPrey/Your-Research-Agent.

1 Introduction

Autonomous research agents can produce complete papers, but persistent, verifiable research state and alignment between manuscript claims and executed experiments remain unresolved across long pipelines. YouRA makes research state and evidence durable, and on MLR-Bench’s predefined ten-task subset it outperforms both baselines on Overall across all three matched backbones.

Research gap

  • Prior end-to-end systems automate research pipelines, but none externalizes the research trajectory as explicit, persistent, verifiable state spanning the full pipeline.The paper identifies this gap as limiting reliability over long horizons.Source: §1 · p. 1
  • Research agents lack persistent, verifiable state across the pipeline, leaving long-horizon coherence and alignment between manuscript claims and executed evidence unresolved.Output-only judging may miss claims based on hallucinated results or failed runs, while workflow control can drift as execution dialogues grow.Source: §1 · pp. 1–2

Contributions

  • YouRA combines VSA-managed hypotheses, gates, and evidence pointers with an independent controller and Stateful Reflection for failure-based recovery.The controller separates lifecycle control from task execution, while reflection routes recovery through bounded repair, redesign, or reset.Source: §1 · p. 2

Evaluation

  • YouRA achieves the highest 4-judge end-to-end Overall among systems on every matched backbone, outperforming both MLR-Agent and AI Scientist V2 on MLR-Bench’s ten-task subset.Order-controlled pairwise preferences also favor YouRA over both baselines on every reported backbone; diagnostics examine fact-based failures and data provenance.Source: §1 · p. 2
  • The paper argues that final claims should be judged by their traceability to executed evidence and the research trajectory, not by manuscript persuasiveness alone.Its evaluation also reports automated failure-taxonomy and data-provenance diagnostics, alongside component ablations.Source: §1 · p. 2

2 Related Work

Prior research agents vary in pipeline design, collaboration, and evaluation, but the paper identifies persistent, verifiable state across stages as a remaining architectural gap. YouRA targets hypothesis-testing research pipelines and supplements output-based evaluation with evidence-oriented diagnostics.

End-to-End Autonomous Research

  • Single-pipeline agents chain ideation through paper writing, while AI Scientist V2 adds tree search and role-separated agents specialize by stage (Lu et al., 2024; Yamada et al., 2025).Human-feedback-integrated designs accept researcher input at predefined stage checkpoints.Source: §2 · pp. 2–3

Collaborative Systems and Persistent State

  • Across prior approaches, research trajectories are not carried between stages as durable, verifiable state, and failures do not become structured constraints on later work.YouRA links debate outputs to VSA state and Reflective Memory so they remain tied to downstream verification.Source: §2 · p. 3
  • VIRSCI and AI Co-scientist focus on ideation, while Enterprise Deep Research targets information synthesis; YouRA instead addresses hypothesis-testing pipelines across research stages (Gottweis et al., 2025; Prabhakar et al., 2025).YouRA extends Reflexion’s task-scoped failure memory to lessons persisting across heterogeneous stages.Source: §2 · p. 3

Evaluation and Reliability

  • Output-only evaluation can miss gaps between manuscript claims and executed experiments, so YouRA pairs benchmark rankings with hallucination-taxonomy and data-provenance diagnostics.The paper notes limits in LLM judging, paper–code discrepancy detection, and long-horizon implementation fidelity (Yan et al., 2026).Source: §2 · p. 3

3 Methodology

YouRA maintains durable research state and uses it to coordinate lifecycle control and failure recovery. Its workflow links hypothesis formation and gated experiments to evidence checking, manuscript review, and refinement.

3 Methodology

  • YouRA addresses context fragmentation, unreliable workflow control, and missing failure memory with VSA, an Independent Controller, and Stateful Reflection.State externalization stores hypotheses, gates, failures, and evidence pointers in structured files rather than conversation history.Source: §3 · pp. 3–4

3.1 Collaborative Research Lifecycle Overview

  • Ten VSA-controlled stages move from problem scoping and literature grounding through hypothesis debate, gated experiments, evidence checking, manuscript drafting, fact review, and refinement.The hypothesis loop iterates through experiment design, implementation planning, and coding and validation.Source: §3.1 · pp. 4–5

3.2 Verification State Architecture

  • The YAML-based VSA records hypotheses, gate results, routing decisions, evidence pointers, and failed hypotheses; later stages use these records as constraints on revisions and manuscript claims.Its invariants include gate-driven progression, timestamped and attributed mutations, and researcher steering through VSA updates.Source: §3.2 · p. 5

3.3 Independent Controller

  • An independent controller uses VSA and reflection records to select lifecycle continuation, recovery boundaries, or debate and review roles, separate from task execution.For hypothesis debate, six personas contribute to candidate hypotheses, and refuted approaches become constraints while validated elements become reusable evidence.Source: §3.3 · p. 5

3.4 Stateful Reflection

  • Stateful Reflection records reasons and evidence for outcomes, turning refuted approaches into constraints and validated or partially validated elements into reusable evidence.Recovery escalates from local repair to hypothesis redesign when recoverable, or problem-scoping reset when the hypothesis contradicts executed evidence.Source: §3.4 · p. 5

4 Experiments

On MLR-Bench’s predefined ten-task subset, YouRA ranks first in Overall across all three matched backbones, and each tested component contributes to scalar performance. The comparisons use released system configurations, while within-YouRA ablations help assess component contributions.

4.1 Setup

  • End-to-end and pairwise evaluation use MLR-Bench’s predefined ten-task subset and three matched execution backbones, comparing systems in their released configurations rather than with tool-normalized access.YouRA’s controller is fixed as GPT-5.2 while its execution backbone varies.Source: §4.1 · p. 6

4.2 Main Results

  • YouRA ranks first in Overall against both MLR-Agent and AI Scientist V2 across all three matched backbones, with higher Soundness than AI Scientist V2 in each setting.Both Overall gaps are significant under a paired sign-flip permutation test over 30 matched backbone-task cells.Source: Table 1 · p. 6, §4.2 · p. 7
  • 72 wins, 33 ties, and 15 losses against MLR-Agent, and 55 wins, 38 ties, and 27 losses against AI Scientist V2, across 120 order-controlled comparisons per baseline.YouRA comes out ahead on every backbone.Source: Table 2 · p. 6, §4.2 · p. 7

4.3 Ablation Studies

  • All four component ablations significantly lower Overall; removing VSA or the Independent Controller also puts Overall below both baselines on Sonnet 4.5 and Opus 4.5.Drops are largest on Soundness and Significance, and pairwise comparisons do not reliably rank components by drop magnitude.Source: §4.3 · pp. 7–8, Table 3 · p. 8

5 Discussion

Human validation indicates that most automated hallucination flags are genuine, while corrected counts distinguish MLR-Agent from YouRA and AI Scientist V2. The separate provenance analysis reports more real-data-based outputs for YouRA than for MLR-Agent.

5.1 Human validation

  • Roughly three quarters of 270 automatically flagged instances were confirmed as genuine hallucinations, with comparable precision across systems.The system differences in precision were not statistically significant.Source: §5.1 · p. 8
  • Precision-corrected hallucination counts place MLR-Agent significantly above both other systems, while YouRA and AI Scientist V2 are not statistically distinguishable.The latter result is absence of evidence for a difference, not evidence of equivalence.Source: §5.1 · p. 8, Table 5 · p. 9

5.2 Data provenance

  • 27/30 YouRA papers were classified as real-data-based by both analyzers, compared with 11/30 for MLR-Agent.The provenance diagnostic covers Sonnet 4.5, Sonnet 4.6, and Opus 4.5 tasks.Source: §5.2 · p. 8
  • The hallucination and data-provenance diagnostics add failure-type and provenance views to the score and pairwise-preference results.Table 4 describes category and per-system precision, plus annotator-auditor agreement; Table 5 reports precision-corrected counts and pairwise differences.Source: §5.2 · p. 9, Table 4 · p. 9, Table 5 · p. 9

6 Conclusion

YouRA targets drift between manuscript claims and executed experiments by maintaining research state, evidence, and failure history as durable structures. On MLR-Bench, its scalar score separates it from both baselines, while the diagnostics indicate that the gain does not rest on fabricated content.

6 Conclusion

  • YouRA makes research state, execution evidence, and trajectory-level failure history explicit and durable across the pipeline.Its VSA-based state externalization, Independent Controller, and Stateful Reflection record hypotheses, failures, evidence links, and recovery decisions.Source: §6 · p. 9
  • Figure 2 classifies 30 generated papers per system as Real, Synthetic, or Fabricated under each of two independent analysis pipelines.Exact counts are in Appendix Table 15.Source: Figure 2 · p. 9
  • YouRA improves scalar end-to-end scores over both baselines across matched backbones, with significantly fewer confirmed hallucinations than MLR-Agent and no distinguishable count from AI Scientist V2.The passage says YouRA's runs rely on real data more often than either baseline under a conservative reading.Source: §6 · p. 9

Limitations

The evaluation is bounded by its machine-learning focus, judge configuration, input-length constraints, and differing system resources.

Limitations

  • All experiments focus on machine-learning and deep-learning tasks, leaving applicability to other scientific domains for future investigation.Source: Limitations · p. 10
  • Human-reviewer alignment requires separate validation for the MLR-Judge configuration, and context-length truncation may disadvantage papers with extensive artifacts.When limits are exceeded, code, result files, or logs can be truncated by the judge input constraints.Source: Limitations · p. 10
  • Systems use different tool configurations and token budgets, so scores reflect performance under their respective resource conditions.The paper proposes standardized tool access and token budgets as a future benchmark direction.Source: Limitations · p. 10

Ethical Considerations

The paper frames YouRA as assistance for researchers rather than a replacement and calls for verification, disclosure, and policy-compliant use.

Ethical Considerations

  • Researchers should independently verify outputs and disclose AI involvement in manuscript preparation.Source: Ethical Considerations · p. 10
  • Commercial LLM APIs and external tools may expose private code, data, or unpublished ideas, so deployments should follow institutional and data-use policies.Source: Ethical Considerations · p. 10
  • External-service dependencies may limit accessibility for under-resourced groups, motivating future open-weight or local alternatives.Source: Ethical Considerations · p. 10
  • The hallucination and provenance diagnostics are audit tools for claim-evidence alignment, not methods for polishing unsupported manuscripts.Responsible use includes preserving artifact provenance and making claim-to-evidence links inspectable.Source: Ethical Considerations · p. 10

A System Implementation Details

YouRA’s deployed workflow combines external tools and durable shared state with independent lifecycle control, reflection-guided recovery, and stage-specific agents. Executed evidence is checked and carried into manuscript drafting and review.

State externalization and tools

  • The workflow separates transient MCP calls from durable artifacts, using structured stage templates, Archon knowledge storage, and on-demand procedural skills around the VSA.Claude Code hosts execution and hooks; the external MCP services provide knowledge management, web and academic search, code navigation, and structured reasoning.Source: A System Implementation Details · pp. 11–12

VSA and controller

  • Every agent reads and writes a centralized YAML VSA, while the independent controller uses VSA records to choose stage transitions, recovery, and debate or review roles.The controller reads a compact state summary and the latest artifact pointer, separately from the task-execution agent; its Stop hook continues, terminates, or escalates for human intervention.Source: A System Implementation Details · pp. 12–13, Figure 3 · p. 12

Reflection-guided failure handling

  • Reflection routes failed Coding & Validation gates through bounded local repair, same-hypothesis retry, Hypothesis Debate redesign, or Problem Scoping reset.Routing-relevant failures and escalated outcomes are recorded in Reflective Memory and VSA recovery history for downstream stages.Source: A System Implementation Details · pp. 13, 15

Hypothesis Debate

  • Hypothesis Debate uses six scientific personas and a controller-selected sequence, then structures the outcome for downstream stages and verification planning.Re-entered debates consult Reflective Memory to avoid repeating refuted directions; verification planning decomposes the hypothesis into a 3–7-node sub-hypothesis DAG with MUST_WORK/SHOULD_WORK gates.Source: A System Implementation Details · pp. 13–14

Hypothesis loop

  • Each sub-hypothesis moves from experiment specification through implementation planning and tested coding to validation, refutation, or escalation.An independent validator checks implementations, and a detector checks whether outputs come from actual data processing rather than fabricated placeholders.Source: A System Implementation Details · p. 14

Evidence transfer and manuscript review

  • Evidence Check records whether executed results support each prediction and passes that summary to manuscript drafting and adversarial fact review.Drafting uses a three-group sequence; refinement checks claims against the VSA, evidence summary, artifacts, results, logs, and code until no major findings remain.Source: A System Implementation Details · p. 15

B Reproducibility Details

The appendix documents reproducibility settings and operational routing telemetry, then reports supplementary context and paired end-to-end significance tests. The tests find positive Overall differences for YouRA against both baselines across all three backbones.

B Reproducibility Details

  • The released repository includes source code, schemas, configurations, prompts, generated artifacts, raw outputs, and scripts for regenerating tables and figures.Benchmark and upstream materials remain subject to their original licenses and are not redistributed; generated artifacts inherit upstream licensing obligations.Source: B Reproducibility Details · pp. 15–16
  • The controller and evaluation layer use commercial API paths without a uniform deterministic replay guarantee, and experiments use provider-default sampling parameters.The appendix also specifies the single-workstation hardware and distinguishes API-based controller/evaluation from local experiment-code execution.Source: B Reproducibility Details · p. 16

C Operational Telemetry

  • The reported 86 archive-producing routing events comprise 28 for Sonnet 4.5, 40 for Opus 4.5, and 18 for Sonnet 4.6.These counts cover redesign or reset departures from Coding & Validation in final working directories; bounded in-loop repairs create no archive snapshots.Source: C Operational Telemetry · p. 17, Table 9 · p. 17

D Supplementary Interpretation Details

  • Supplementary interpretation covers tool-access differences, recovery-routing distributions, and a 201-task upstream Idea/Proposal analysis; order-controlled pairwise preferences remain part of the main results.The appendix distinguishes these contextual analyses from the separately reported hallucination diagnostic.Source: D Supplementary Interpretation Details · p. 17

D.1 Statistical Significance of the End-to-End Comparison

  • YouRA leads MLR-Agent by +0.85 (p=0.006) and AI Scientist V2 by +0.64 (p=0.044); both remain significant after Holm correction (adjusted p=0.012 and 0.044).Per-backbone mean differences are positive across all three backbones for both comparisons; the permutation runs at the cell level, not on individual judge verdicts.Source: §D.1 · p. 18

D.2 201-Task Upstream Context

On the upstream Idea/Proposal comparison, results vary by backbone and rubric dimension rather than showing a uniform YouRA advantage. The automated hallucination diagnostic reports separate judge-agreement bounds and a favorable YouRA profile on execution-evidence categories.

D.2 201-Task Upstream Context

  • On Sonnet 4.5, MLR-Agent scores higher on Idea-stage Overall and Proposal Clarity and Soundness; on Opus 4.5, Overall does not differ significantly.For Opus 4.5, YouRA scores higher on Novelty and Feasibility, while MLR-Agent scores higher on Soundness and Significance; AI Scientist V2 is absent from this setting.Source: §D.2 · p. 18, Table 13 · p. 19, Table 14 · p. 19

E Automated Hallucination Diagnostic

  • The diagnostic adapts four MLR-Bench fact-based failure categories and reports automated four-judge intersection and union counts rather than a single score.Intersection requires all four judges to flag a case; union requires at least one, and the original human-verification step is not part of the automated diagnostic.Source: E Automated Hallucination Diagnostic · p. 18
  • YouRA has a favorable all-judge-agreed profile for hallucinated methodology and faked experimental results, while citation flags vary by backbone.The automated flags have 75.2% overall precision in human validation, and the diagnostic is not used as a standalone ranking metric.Source: E Automated Hallucination Diagnostic · p. 19

F Data Provenance Analysis

The provenance analysis classifies experiment data use at the paper level, using two independent analyzers and file-level evidence. Both analyzers separately classify more YouRA runs as real-data-based than runs from either baseline.

Protocol

  • Two analyzers independently inspect experiment code and result files, agree on 85 of 90 system-backbone-task cells, and count a run as real-data-based only when both label it Real.A Synthetic or Fabricated label from either analyzer counts against a run under this conservative rule.Source: F Data Provenance Analysis · p. 19

Counts by analyzer

  • 27/30 YouRA runs are labeled real-data-based by the Opus 4.6 analyzer and 28/30 by GPT-5.4, compared with 12/30 and 12/30 for MLR-Agent.AI Scientist V2 has 18/30 real-data runs under each analyzer; non-real cases are synthetic except for one MLR-Agent case labeled fabricated.Source: F Data Provenance Analysis · p. 20, Table 15 · p. 20

G Component Ablations

Each component removal significantly lowers Overall, but the tests do not establish a relative ranking. Ablations also show backbone-dependent drops and distinct failure patterns for the core-state components.

G Component Ablations

  • All four component removals significantly lower Overall across 30 matched cells under Holm-corrected paired sign-flip tests.Each cell averages four judges’ Overall scores.Source: G Component Ablations · p. 20, Table 16 · p. 21
  • Every pairwise contrast has Holm-adjusted permutation p=1.000, so the sample does not support ranking components by drop size.The contrasts test drop-size differences across the same 30 matched cells.Source: Table 17 · p. 21, G Component Ablations · p. 21
  • All ablations fall below both baselines on Sonnet 4.5 and Opus 4.5; on Sonnet 4.6, all remain above AI Scientist V2, while only w/o VSA falls below MLR-Agent.Source: G Component Ablations · p. 21
  • Ablation drops are largest on Sonnet 4.5 and Opus 4.5, smallest on Sonnet 4.6; stronger execution partially compensates, but full YouRA remains highest on every backbone.Source: G Component Ablations · p. 21
  • Across configurations, Overall drops track Soundness and Significance more than Clarity; core-removal reviews show evidence inconsistencies without VSA and recovery failures without the Independent Controller.Source: G Component Ablations · p. 21

Related papers

Loading 2610.01097v1…