PaperlayerView exact revision on arXiv

Source-linked AI summary

StateGuard: Analytical-State Management with Validity-Aware Intervention for Long-Horizon Data Agents

Wenle Liao, Zhao Wang, Jingchao Zhang, Jiajie Jin, Yimeng Xu, Zhicheng Dou

arXiv:2609.34134v1cs.AI

TL;DR

Long-horizon data agents can silently inherit stale artifacts as requirements and dependencies evolve. StateGuard externalizes analytical states and manages their validity through evidence-based verification and repair, with training based on manager-oriented counterfactual supervision and validity-guided optimization. Across three benchmarks, it consistently improves agent performance while reducing downstream error propagation.

  • Problem

    As analytical processes evolve over long horizons, agents can lose track of whether implicitly stored constraints, variables, and conclusions remain valid, allowing stale artifacts to propagate errors downstream.

    Source: §1 · pp. 1–2

  • Method

    StateGuard externalizes analytical states and manages their validity through runtime evidence verification and hierarchical intervention, training with counterfactual manager trajectories and validity-guided policy optimization.

    Source: §1 · pp. 2–3, §3.4 · p. 6

  • Results

    Across three long-horizon data-analysis benchmarks, StateGuard consistently improves data-agent performance and reduces downstream contamination from incorrect upstream results.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    Explicit analytical-state management provides a way to keep analytical progress verifiable and traceable during long-horizon data analysis.

    Source: §6 · p. 10

    Persistent state maintenance adds inference cost, particularly with dense review points or long trajectories.

    Source: E LIMITATIONS AND FUTURE DIRECTIONS · p. 35

Abstract

from arXiv · show

LLM-based agents have shown strong capabilities in automated data analysis and are increasingly moving toward long-horizon, multi-stage analytical workflows. However, as the analytical process evolves, constraints, variables, and conclusions remain implicitly embedded in interaction histories, making it difficult for agents to track which analytical artifacts remain valid over increasingly long horizons and changing dependencies. Consequently, stale artifacts may be silently inherited, propagating errors to downstream stages. To address this challenge, we propose StateGuard, an analytical-state validity management framework for long-horizon data agents. StateGuard externalizes evolving analytical progress into a state graph containing constraints, versioned variables, intermediate conclusions, and cross-state relations, treating each state as an executable, verifiable, and traceable object rather than textual memory alone. StateGuard maintains state validity through evidence-grounded verification and hierarchical intervention. To equip StateGuard with these capabilities, we first introduce Manager-Oriented Counterfactual Supervision, which constructs 3K state-centric trajectories through counterfactual runtime synthesis to fine-tune StateGuard for state maintenance, verification, and repair. We then apply Validity-Guided Policy Optimization, using runtime validity evidence to provide fine-grained learning signals for protocol correctness, state grounding, and intervention quality. Experiments on three diverse long-horizon data-analysis benchmarks show that StateGuard consistently improves data-agent performance while reducing dependency-induced downstream error propagation, demonstrating the advantages of explicit analytical-state management for reliable long-horizon data analysis.

1 INTRODUCTION

Long-horizon agents can silently carry stale analytical artifacts across changing dependencies. StateGuard externalizes and verifies analytical states, repairs inconsistencies, and is reported to improve performance while reducing downstream error propagation.

1 INTRODUCTION

  • Long-horizon analysis requires agents to revise intermediate results and synthesize conclusions across dependent stages, where early inconsistencies can propagate into incorrect results.Prior evaluations report performance degradation as analytical processes and dependencies extend over longer horizons.Source: §1 · p. 1
  • Generic memory and execution-progress methods do not explicitly track whether derived artifacts remain valid as dependencies evolve.A prior rounding-to-nearest-1k constraint may remain executable but conflict with later revisions, allowing inconsistencies to propagate silently.Source: §1 · p. 2
  • StateGuard externalizes analytical progress as explicit states and checks validity using runtime evidence, tracing how dependencies affect downstream artifacts.States capture constraints, versioned variables, intermediate conclusions, and relations to prior states; detected inconsistencies trigger evidence-grounded hierarchical repair.Source: §1 · p. 2
  • Manager-Oriented Counterfactual Supervision and Validity-Guided Policy Optimization train state maintenance, verification, and intervention using synthesized trajectories and runtime validity evidence.The training pipeline constructs 3K manager-oriented trajectories and uses fine-grained signals for protocol correctness, state grounding, and intervention quality.Source: §1 · pp. 2–3
  • StateGuard consistently improves performance across three long-horizon data-analysis benchmarks and reduces downstream contamination from incorrect upstream results.The evaluated benchmarks are LongDS-Bench, DABstep, and DAComp-DE.Source: §1 · p. 3

2 RELATED WORK

Prior data-science agents organize workflows and memory, while process-verification methods inspect reasoning or repair failures. StateGuard instead represents analytical states and their dependencies explicitly, focusing on whether derived artifacts remain valid as conditions change.

2 RELATED WORK

  • Data-science systems improve task organization, but evolving analytical states and artifacts remain largely implicit in interaction trajectories.StateGuard explicitly maintains those states and their cross-stage dependencies during execution.Source: §2 · p. 3
  • Memory methods store, retrieve, or summarize information, whereas StateGuard checks whether previously derived analytical artifacts remain valid as dependencies change.It treats analytical state as executable, verifiable, and dependency-traceable rather than textual memory alone.Source: §2 · p. 3
  • Process verification, failure attribution, auditing, and trajectory repair generally target reasoning steps or generic trajectory positions; StateGuard models inconsistencies as invalid states or variables.Its framework traces propagation through dependencies and uses evidence-grounded intervention and hierarchical repair.Source: §2 · p. 3, Figure 2 · p. 4

3 METHODOLOGY

StateGuard models evolving analysis as explicit states whose validity and dependencies can be checked, then manages those states through runtime verification and repair. Its training combines synthesized manager trajectories with validity-guided policy optimization.

3.1 PROBLEM FORMULATION

  • Each analytical state records an issue, constraints, versioned variables, intermediate conclusions, and relations to prior states, forming a graph for dependency tracking.StateGuard aims to verify newly formed states under their constraints and cross-state dependencies before inconsistencies spread downstream.Source: §3.1 · p. 4

3.2 OVERVIEW OF STATEGUARD

  • StateGuard is an external manager: the data agent alone executes tasks, while control actions govern state lifecycle and tools gather verification evidence.StateGuard provides hints through a restricted state interface rather than planning or generating answers.Source: §3.2 · p. 5

3.3 ANALYTICAL STATE VALIDITY MANAGEMENT AND ERROR REPAIR

  • Draft states are checked against trajectory, related-state, and executable evidence; repair escalates from hints to artifact removal, rollback, or abandonment.The repair cycle returns corrected trajectories to repair, update, and verification.Source: §3.3 · pp. 5–6

3.4 MANAGER-ORIENTED COUNTERFACTUAL SUPERVISION

  • Counterfactual runtime synthesis yields 3K manager-oriented trajectories, retaining execution-verified clean, intervention, and repair triplets while using benign interventions as hard negatives.The synthesized tasks cover progression, branching, invalidation, and composition across analytical stages.Source: §3.4 · p. 6

3.5 VALIDITY-GUIDED POLICY OPTIMIZATION

  • Policy optimization rewards legal runtime actions, evidence-grounded state contents, and warranted repairs, penalizing false repairs more heavily than missed interventions.The signals cover protocol correctness, state grounding, and intervention quality.Source: §3.5 · pp. 7–8

4 EXPERIMENTS

Across heterogeneous long-horizon analysis settings, StateGuard improves agent performance and reduces dependency contamination. Ablations show that both structured validity management and training contribute to its results.

4.1 EXPERIMENTAL SETUP

  • Evaluation spans LongDS multi-turn analysis, DABstep multi-step reasoning, and DAComp-DE repository engineering, against general-agent and specialized DataMind baselines.LongDS reports turn-level accuracy; DAComp-DE reports component and cascading-failure scores for implementation, and cascading-failure scores for evolution.Source: §4.1 · p. 8

4.2 MAIN RESULTS

  • StateGuard improves performance across heterogeneous settings; DeepSeek-V4-Pro’s LongDS turn-level accuracy rises from 44.53 to 54.05.The evaluations cover multi-turn analysis, repository engineering, and multi-step reasoning.Source: §4.1 · p. 8, §4.2 · p. 9

4.3 CASCADING ERROR ANALYSIS

  • On DeepSeek-V4-Pro, average DCR drops from 60.25 to 55.90; DCR can be low or undefined for weaker backbones with few correct upstream units.DCR measures excess downstream error associated with incorrect upstream units.Source: §4.3 · p. 9

4.4 ABLATION STUDIES

  • Removing SFT drops performance to 31.65, versus 54.05 for the full model; removing RL retains 51.78, while prompt-only StateGuard reaches 47.50.The authors report that SFT is essential for acquiring the structured state lifecycle, while validity-guided RL further stabilizes verification and intervention.Source: §4.4 · p. 9
  • Free-form memory scores 43.16 versus 44.53 for ReAct; removing validity maintenance yields 40.77 and raises DCR to 68.45.The ablation reports that free-form memory contributes little, while the full model reaches 54.05.Source: §4.4 · p. 9

5 CASE ANALYSIS

A revised COGS value makes earlier COGS-dependent results stale in the equity-IRR case. StateGuard records the revision as a new state, identifies affected artifacts, and targets recomputation.

5 CASE ANALYSIS

  • After COGS changes to 0.62, StateGuard links the new state to the prior COGS state with INVALIDATE and targets recomputation of affected artifacts.The linear trajectory instead reuses earlier COGS-dependent results under the original rounding-to-1K constraint, propagating errors to final IRR.Source: §5 · pp. 9–10

6 CONCLUSION

This section summarizes StateGuard’s explicit analytical-state management and validity maintenance, including its training approach and benchmark effects. It also covers how StateGuard communicates state and repair guidance, enforces manager decisions, and adapts its verification protocol to different workflow lifecycles.

6 CONCLUSION

  • StateGuard improves data-agent performance while reducing downstream error propagation across benchmarks through explicit analytical-state management.The framework externalizes analytical progress into explicit states and maintains their validity through evidence verification and hierarchical intervention.Source: §6 · p. 10

AI USE STATEMENT

  • Generative AI assisted with language editing, software development and debugging, and synthetic training-trajectory generation; outputs were reviewed, tested, and filtered, and the authors take responsibility for the final work.The statement reports no other research components requiring disclosure as involving generative AI assistance.Source: AI USE STATEMENT · p. 10

A.1 OVERALL STATEGUARD RUNTIME ROLLOUT

  • As an external runtime manager, StateGuard reviews turn-based tasks at turn boundaries and fixed-step tasks after configurable windows or upon termination.This adapter-defined schedule isolates state-management policy from benchmark-specific execution schedules.Source: §A.1 · p. 13

A.2 LIFECYCLE AND STATE RELATIONS

  • StateGuard supports turn-aligned and segment-induced state lifecycles, with relations representing initialization, progress, branching, invalidation, or combination of upstream states.Turn-aligned states open before execution and are verified afterward; segment-induced states are formed after a meaningful analytical unit is observed.Source: §A.2 · pp. 13–14

A.3 CONTROL AND TOOL ACTIONS

  • At manager activations, StateGuard can gather targeted verification evidence, propose lifecycle actions, and invoke tools that test execution, code structure, syntax, probes, or dependencies.The harness enforces lifecycle legality and transactional updates; repair uses localized evidence and may escalate to removing identified invalid variables.Source: §A.3 · pp. 15–16

A.4 RUNTIME OBSERVATION AND STATE REPRESENTATION

  • StateGuard gives the manager a compact projection of pending work rather than raw worker history and represents states with constraints, used variables, conclusions, relations, and runtime provenance.Executable constraints can be tested against task data, variable versions establish lineage, and committed states are immutable.Source: §A.4 · pp. 16–17

A.5 STATE HINTS AND REPAIR HINTS

  • State guidance selectively transfers relevant committed analysis, while repair guidance localizes affected quantities and faulty reasoning without supplying executable corrections or full answers.Repair hints are non-solution-bearing, and the same error localization grounds runtime intervention.Source: §A.5 · pp. 17–18

A.6 HARNESS AND ADAPTER

  • The manager proposes analytical semantics while the harness enforces protocol and lifecycle constraints, deterministic bookkeeping, and component-scoped transactional execution.The harness also provides bounded recovery, while adapters isolate benchmark-specific execution semantics from the shared runtime.Source: §A.6 · pp. 18–19

A.5 STATE HINTS AND REPAIR HINTS

  • Localized, concrete evidence gates repair, and stronger intervention removes affected invalid variables from the live workspace before recomputation.The harness controls the repair schedule and attempt budget.Source: §A.5 · p. 18, §A.7 · p. 20

A.7 STATEGUARD PROMPTING AND RUNTIME TEMPLATES

  • Repair requires a localized violation supported by concrete evidence; ambiguous evidence does not justify intervention, and the harness controls repair mode and budget.The manager checks execution evidence, intent-code alignment, code validity, query requirements, and concrete conflicts with related states.Source: §A.7 · p. 20
  • Turn-aligned workflows create one state per turn, while segment-induced workflows create states only when accumulated traces form meaningful analytical units.The shared controller preserves the manager-worker authority boundary: state and repair hints are reference-only, not ground truth or replacement solutions.Source: §A.7 · pp. 20–21

B.1 SOURCE DATA

StateGuard’s training pipeline turns executable analytical dependencies and plausible counterfactual changes into state-centric supervision, then optimizes the manager using runtime validity evidence. Its training data combines diverse source tasks with replayed, filtered intervention trajectories.

B.1 SOURCE DATA

  • The SFT source pool combines DSBench, BIRD-INTERACT, and IDA-Bench trajectories, and is disjoint from the evaluation benchmarks.The sources cover data-science and modeling workflows, interactive database tasks, and multi-round notebook analysis.Source: §B.1 · pp. 21–22

B.2 TASK CONSTRUCTION AND COUNTERFACTUAL INTERVENTION SYNTHESIS

  • Dependencies are reconstructed from executable variable or program consumption and categorized as INIT, PROGRESS, BRANCH, or COMBINE.One realization parses SQL FROM/JOIN references to recover dataflow.Source: §B.2 · p. 22
  • Counterfactual branches alter a localized analytical assumption while preserving the original committed state, creating parallel hypotheses rather than overwriting valid state.Perturbations include numerical parameters, formula semantics, comparison directions, category scopes, and filtering conditions.Source: §B.2 · p. 22

B.3 TRAJECTORY GENERATION AND FILTERING

  • Trajectory generation replays the successful parent exactly before a predefined intervention, then generates subsequent behavior live to isolate the intervention from unrelated sampling variation.Retained trajectories must preserve prefix fidelity, realize the intervention, follow valid lifecycle behavior, and complete the task.Source: §B.3 · p. 23

B.4 MANAGER-ORIENTED SFT AND VALIDITY-GUIDED RL

  • Manager-oriented SFT mixes local runtime decisions with complete trajectories, while validity-guided RL uses runtime evidence to reward protocol correctness, state support, and intervention quality.State assertions count as supported when grounded in worker results, verification-tool outputs, or executable constraint checks; unverified natural-language reasoning alone does not count.Source: §B.4 · pp. 23–24
  • Grouped DAPO rollouts share the same worker prefix and runtime context, so reward differences primarily reflect alternative manager decisions rather than separate worker executions.Each training instance uses eight manager rollouts.Source: §B.4 · p. 25

C.1 BENCHMARK DETAILS

The evaluation spans long-horizon analysis, repository-level data engineering, and multi-step data analysis. Baseline comparisons retain the benchmark-specific worker implementation and execution interface, with the manager disabled for baseline runs.

C.1 BENCHMARK DETAILS

  • LongDS-Bench contains 68 tasks and 2,225 turns across six domains, testing persistent reasoning over state updates, counterfactual revisions, rollback, and multi-stage composition.The reported metric is turn-level accuracy.Source: §C.1 · pp. 25–26
  • DAComp-DE covers repository-level Implementation and Evolution tasks with long dependency chains, and reports Component Score and Cascading Failure Score for Implementation and Cascading Failure Score for Evolution.Tasks may span more than 4,000 lines of code across over 30 files.Source: §C.1 · p. 26
  • DABstep contains 450 multi-step analysis tasks, with 72 easy and 378 hard instances, and reports accuracy separately for the two subsets.Its tasks combine structured data with contextual documentation and require code-based processing and multi-source reasoning.Source: §C.1 · p. 26

C.2 BASELINE DETAILS

  • General-agent baselines use the same benchmark-specific worker implementation and ReAct-style interface as StateGuard, while disabling the manager removes state maintenance, validity checks, and repair hints.Manager actions do not consume worker steps; additional worker budget is reserved for manager-triggered re-analysis or correction in DABstep and DAComp-DE.Source: §C.2 · pp. 26–27, Table 3 · p. 27

C.3 EVALUATION DETAILS

The evaluation follows each benchmark’s native scoring protocol and separately measures dependency-linked error contamination where explicit dependencies are available. A matched LongDS evaluation also assesses uncertainty in the observed accuracy gain.

C.3 EVALUATION DETAILS

  • LongDS scores individual turns with the official LLM judge, DABstep scores tasks with its official programmatic scorer, and DAComp-DE uses weighted execution-based table scores.The benchmark metrics have different scoring units and should not be treated as numerically interchangeable.Source: §C.3 · pp. 27–28
  • 9.44 points: StateGuard improves DeepSeek-V4-Pro LongDS micro-accuracy, with a 95% confidence interval of [1.69, 14.13] over 10,000 bootstrap resamples.The task-macro gain is 9.52 points; only 0.59% of resamples yield a non-positive gain.Source: §C.3 · p. 28

C.4 DCR DETAILS

  • Dependency Contamination Rate compares downstream error rates for units with incorrect upstream dependencies against units whose observed upstream dependencies are correct.For DAComp-DE, CFS evaluates tables inside the model-produced pipeline and preserves the realized contamination path; isolated Component Score evaluation does not.Source: §C.4 · pp. 28, 29, 31
  • DCR is not computed for DABstep because its tasks lack an explicit inter-unit dependency structure comparable to LongDS turns or DAComp-DE derived tables.Units without declared dependencies are excluded, and DCR is undefined if either comparison group is empty.Source: §C.4 · pp. 29, 31

C.5 ABLATION CONFIGURATIONS

The ablations distinguish local observation, unrestricted persistent state management, and prescribed analytical-state validity management. Runtime overhead is estimated from manager activity, with review cadence identified as its principal driver.

C.5 ABLATION CONFIGURATIONS

  • The state ablations contrast a fresh local observer without persistent state against Free-Form Memory, which retains the state and repair machinery but removes prescribed content and verification policies.The Free-Form Memory harness schema and lifecycle remain unchanged.Source: §C.5 · pp. 31–32

D.1 RUNTIME OVERHEAD

  • Manager overhead is estimated from review-boundary timing after subtracting ordinary worker-step time and manager-free harness bookkeeping.Intervals longer than 900 seconds are excluded from the timing decomposition.Source: §D.1 · pp. 32–33
  • Review frequency, rather than repair frequency, is the principal determinant of total overhead, making the accuracy–latency trade-off adjustable through review cadence.Average REPAIR actions range from 0.20–0.52 per task, and manager inference runs locally with Qwen3-8B.Source: §D.1 · p. 33

D.2 STATE AND INTERVENTION STATISTICS

StateGuard maintains substantive analytical states and workflow-specific relations, while its runtime emphasizes state propagation and evidence checks over frequent repairs. Construction, relation selection, and intervention patterns vary with benchmark workflow structure.

State Construction

  • LongDS averages 26.2 committed states per task, with states covering 80.2% of evaluated turns; DAComp-DE coverage reaches all Implementation and 98% of Evolution tasks.Figure 4a presents state-construction statistics.Source: §D.2 · p. 33, Figure 4 · p. 34
  • Committed states typically contain 3–4 constraints, 3–5 variables, and 2–3 intermediate conclusions, with most variables carrying concrete values.These records preserve executable analytical artifacts rather than only textual summaries.Source: §D.2 · p. 33

State Relations

  • LongDS relations are chiefly COMBINE (44.0%) and PROGRESS (34.3%), whereas DAComp-DE relations are predominantly PROGRESS (56.8% and 64.5% across its two settings).LongDS has 441 states with multiple upstream relations among 2,399 committed relations.Source: §D.2 · p. 34
  • LongDS proposes relations when states open and finalizes them later, while fixed-cadence workflows select relations post hoc at finalization.Only 25 LongDS cases require reselection.Source: §D.2 · p. 34

Hints, Verification, and Repair

  • LongDS delivers 1,827 state hints versus 36 error hints, showing that runtime communication primarily propagates context rather than repeatedly correcting errors.The same state-hint predominance holds on DAComp-DE.Source: §D.2 · pp. 33–34
  • Evidence tools run on 96% of LongDS tasks and all DAComp-DE tasks, while repairs remain limited to 0.20–0.52 per task across four settings.Only 74 repairs are issued, nearly all on a first repair attempt.Source: §D.2 · p. 34, Figure 4 · p. 34

D.3 ADDITIONAL CASE ANALYSIS

Paired analysis finds more LongDS improvements than regressions and mostly stable DAComp-DE tasks. Case studies show both localized recovery of a consistent analytical path and downstream propagation from an incorrect committed artifact.

Overall Paired Behavior

  • 350 LongDS turns improve and 140 regress, for a net gain of 210 correct turns; only 73 turns meet the broader manager-attributable criterion and 23 meet the stricter reuse criterion.At task level, 35 of 68 LongDS tasks improve, 23 decrease, and 10 are unchanged.Source: §D.3 · p. 34
  • Most DAComp-DE tasks remain stable: Implementation has 9 improvements, 4 regressions, and 17 unchanged tasks, while Evolution has 13, 5, and 32, respectively.Fine-grained pairing covers LongDS at turn level and DAComp-DE at task level; DABstep is excluded because its full test-set ground truth is withheld.Source: §D.3 · pp. 34–35

Positive Case: Recovering a Consistent Analytical Path

  • In a LongDS case, StateGuard retrieves the relevant upstream state after an ambiguous Turn 3 and prompts recomputation on the intended cleaned table, preserving a consistent downstream ranking.Figure 5 contrasts this path with the baseline's inconsistent analytical base; the intervention restores context rather than supplying a replacement solution.Source: §D.3 · p. 35, Figure 5 · p. 35

Failure Case: Propagation from an Incorrect Committed State

  • An incorrect committed job-listing count in a Kaggle Survey 2019 task is later reused as a denominator, propagating numerical errors downstream.The case motivates stronger semantic verification because internally consistent analysis can still be semantically incorrect.Source: §D.3 · p. 35

E LIMITATIONS AND FUTURE DIRECTIONS

The discussion identifies external orchestration, inference cost, and incomplete graph-wide validity guarantees as boundaries of the current approach. It proposes internalized state management, more adaptive review, and stronger verification as future directions.

Internalizing State Management

  • State management remains an external manager around the worker, rather than an intrinsic part of the worker policy.The proposed longer-term direction is to internalize state awareness, revision tracking, dependency-sensitive reasoning, and selective validation.Source: E LIMITATIONS AND FUTURE DIRECTIONS · p. 35

Lightweight and Adaptive Management

  • Persistent state maintenance adds inference cost, particularly with dense review points or long trajectories.Proposed efficiency directions include smaller distilled managers, adaptive review cadence, and event-triggered activation.Source: E LIMITATIONS AND FUTURE DIRECTIONS · pp. 35–36

Stronger Validity Guarantees

  • Current verification is evidence-driven and localized, so it does not provide formal guarantees over the full analytical dependency graph.Future work could add executable invariants, typed states, provenance-aware dependency checking, and global cross-state consistency constraints.Source: E LIMITATIONS AND FUTURE DIRECTIONS · p. 36
Loading 2609.34134v1…