Source-linked AI summary

GitHarness: Git Init Your Harness Working Memory for Perpetual User Requirements

Zhibang Yang, Xinke Jiang, Yuxuan Liu, Mingyu Zhang, Zhixin Zhang, Zhengxing Song, Yue Fang, Guohong Qiu, Ruiqing Li, Xu Chu, Junfeng Zhao, Yasha Wang

arXiv:2609.36789v1cs.MAcs.SE

TL;DR

Long-horizon agents face changing user requirements that can make accumulated work stale, while prior approaches do not link requirement changes to reuse decisions. GITHARNESS versions requirements with harness work states and selects compatible historical states for local updates. Across five task domains, its training-free configuration outperforms native execution, and the adapted 8B Git Agent outperforms native cross-turn execution in 29 of 30 settings.

  • Problem

    Prior methods clarify current intent or reuse execution histories under fixed objectives, but do not connect requirement changes to decisions about which prior work to preserve, discard, or recompute.

    Source: §1 · p. 2
  • Method

    GITHARNESS couples requirement and work states in a branchable history, with a trainable Git Agent selecting a compatible state for harness restoration and local updating.

    Source: §1 · p. 2
  • Results

    Across five task domains, the adapted 8B Git Agent outperforms native cross-turn execution in 29 of 30 settings.

    Source: §6 · pp. 9–10
  • Takeaways & Limitations

    The framework is designed to exclude obsolete states, inherit compatible work, and focus further execution on affected parts.

    Source: §1 · p. 2

    Accurate requirement interpretation and historical-state selection can be challenging with ambiguous or conflicting feedback, and restoring a compatible state does not guarantee preservation of unaffected work.

    Source: §6 · p. 10

Abstract

from arXiv · show

LLM-based agents increasingly collaborate with users on long-horizon tasks, accumulating evidence, code, and drafts through extensive search, reasoning, and execution. As users inspect these results, they may supply missing information requirement completion, introduce new requirements requirement elicitation, or revise existing ones requirement shift. These changes often affect only part of the accumulated work, yet agents may carry forward obsolete information or turn local revisions into global rewrites. Existing approaches clarify current intent without determining how prior work should change, or reuse execution histories under a fixed objective. We address this gap by formulating dynamic-requirement collaboration as joint requirement tracking and local update. We introduce GitHarness, a pluggable Git-style framework that organizes requirement states and their corresponding harness work states into a branchable version history. A trainable Git Agent resolves requirement changes and selects a semantically compatible historical state. A unified version interface then restores that state and creates a new branch, enabling the underlying harness to exclude obsolete information, inherit compatible work, and focus execution on affected parts. The Git Agent is trained through interface-level black-box reinforcement learning, with downstream harnesses and task-execution models kept fixed. We also construct MTAgentBench, a verifier-preserving benchmark covering mathematical reasoning, text-to-SQL, agentic search, software engineering, and research synthesis. Experiments demonstrate strong task performance alongside effective requirement tracking, preservation of valid work, and efficient execution.

1 INTRODUCTION

Long-horizon agents must adapt as users complete, elicit, or shift requirements without carrying obsolete work forward or rewriting unaffected parts. GITHARNESS connects requirement changes to version selection and local work updates.

1 INTRODUCTION

  • User feedback can complete, elicit, or shift requirements after agents have accumulated evidence, code, and drafts through long-horizon work.Such tasks include software engineering, open-world search, and deep research.Source: §1 · p. 1
  • Retaining all history risks stale requirements and results, while restarting discards reusable work; existing approaches do not connect requirement changes to decisions about what to preserve, discard, or recompute.Intent-reconstruction methods clarify current requests, while trajectory-management methods typically assume a fixed objective.Source: §1 · p. 2
  • GITHARNESS couples effective requirement states with corresponding work states in a branchable history, then restores a compatible state so execution can focus on affected parts.A unified version interface maps these operations onto different harnesses without redesigning their working memory or execution procedures.Source: §1 · p. 2
  • The paper formulates dynamic-requirement collaboration as joint requirement tracking and selective work reuse, and introduces MTAGENTBENCH across five task domains with structured state and transition annotations.The benchmark covers mathematical reasoning, text-to-SQL, agentic search, software engineering, and research synthesis.Source: §1 · p. 3

2 RELATED WORK

Prior work addresses evolving intent and accumulated execution history separately; GITHARNESS links requirement transitions to decisions about which historical work remains compatible.

2 RELATED WORK

  • Dialogue-history and intent methods clarify evolving requests, but do not determine how requirement changes propagate to previously produced evidence, code, drafts, and other work.These approaches include retrieval, summarization, query rewriting, and intent-aware tool-log construction.Source: §2 · p. 3
  • Memory and recovery methods organize or restore execution history, but generally reuse it according to reasoning state, failure, or exploration under a given task rather than changed user requirements.Related version-control systems organize context or reasoning trees, but lack requirement-driven compatibility decisions.Source: §2 · p. 3
  • GITHARNESS uses requirement transitions to locate compatible historical states and decide what work to preserve, discard, or recompute.Source: §2 · p. 3

3 PROBLEM FORMULATION

The formulation represents each turn with a requirement state and a work state, then treats new feedback as a change that may invalidate prior work. The system must resolve the current requirement, select a compatible historical base, and update locally.

3 PROBLEM FORMULATION

  • At each turn, the system maintains a requirement state and a work state; new feedback can change the goal and invalidate prior work.The agent must retain valid work and update affected parts rather than re-executing everything from scratch.Source: §3 · p. 3
  • Requirement changes are modeled as edits relative to a historical anchor: completion adds consistent missing details, elicitation adds dimensions, and shift revises or withdraws prior requirements.Source: §3 · p. 3
  • Given accepted history and new feedback, the system must recover the full current requirement, select a compatible historical base, and update the work state locally.The anchor may be an earlier version rather than the most recent state, and change-type identification is not required by the method.Source: §3 · p. 4
  • Locality comes from base selection and branch-local execution, which confines edits to what the new requirement demands rather than predicting affected artifacts directly.Each instance has a final-task verifier and may also have turn-level verifiers.Source: §3 · p. 4

4 METHOD

GITHARNESS versions requirement and work states together, resolves each update by selecting a historical base, and executes from that branch. Its training targets the Git Agent while the downstream harness and executor remain fixed.

4 METHOD

  • Each successful turn is saved as a requirement-and-work snapshot in a version graph, supporting continuation, rollback, and branching from any past state.The framework resolves the requirement and base version before restoring that base for targeted modifications.Source: §4 · p. 4

4.1 REQUIREMENT RESOLUTION

  • The Git Agent uses a short ReAct loop because resolving the full requirement and selecting a historical base require reasoning over earlier versions.INSPECT reads a past requirement; COMMIT declares the requirement and base; REUSE selects a version that already satisfies the requirement.Source: §4.1 · p. 5
  • A successful COMMIT or REUSE returns the resolved requirement, selected base, target version, and a non-authoritative update hint; COMMIT branches when the selected base is not the current head.Source: §4.1 · p. 5

4.2 BRANCH EXECUTION

  • Execution uses EXACTREUSE when a past version already satisfies the requirement, FRESHSOLVE when no usable base exists, and REUSEANDPATCH for targeted changes from a partially compatible base.A lightweight workspace manifest guides the choice between fresh solving and reuse-and-patch for a partial match.Source: §4.2 · p. 5
  • For execution modes that need work, the update agent initializes and updates a candidate state; normal termination commits it, while failure leaves the version graph unchanged.Historical checkpoints are immutable, so failed executions do not corrupt prior states.Source: §4.2 · p. 5

4.3 TRAINING WITH HARNESS RL

  • Training focuses on Git Agent decisions while the update agent and domain harness remain a fixed environment; only Git Agent calls receive training updates.Rollouts interleave Git Agent calls with update-agent execution and observations assembled into later context.Source: §4.3 · pp. 5–6
  • The reward combines final-artifact correctness with turn-level query tracking, whose judger assesses requirement fidelity, base optimality, and exclusion of withdrawn requirements.The inner normalization by the number of Git Agent positions prevents rollouts with richer version-resolution histories from dominating the gradient update.Source: §4.3 · p. 6

5 EXPERIMENTS

Experiments evaluate GITHARNESS across five task domains, two backbones, and three harnesses. It outperforms Native across all settings, while ablations and analyses examine efficiency, adaptation, guidance, and long-horizon performance.

5.1 EXPERIMENTAL SETUP

  • The evaluation covers GSM8K, BIRD, BrowseComp-Plus, SWE-bench Verified, and DeepResearch Bench across two backbones and three harnesses.The compared baselines are Native, Restart, U-Fold, and GCC; absolute scores are compared only within fixed backbone–harness settings.Source: §5.1 · pp. 6–7, §5 · p. 6, §5.2 · p. 7

5.2 MAIN RESULTS

  • Across two backbones, three agent harnesses, and five task domains, GITHARNESS outperforms Native in all 30 settings and matches or exceeds the strongest competing baseline in 28, with 27 strict wins and one tie.Source: §5.2 · pp. 7–8
  • Token usage falls versus Native by 35.8% on SQL, 54.5% on Search, 73.6% on Code, and 36.7% on Research while performance improves in all four domains.GITHARNESS has both the best performance and lowest token usage among compared methods on Search, Code, and Research; Figure 3 plots performance against token use, with bubble area indicating estimated API cost.Source: Figure 3 · p. 7, §5.2 · p. 8

5.3 ABLATION AND ANALYSIS

  • Historical version selection and execution-mode choice contribute to performance and efficiency; removing version selection drops Code resolved rate from 74.0 to 44.0 and raises token use from 1.29M to 4.51M.Without INSPECT, Code resolved rate is 42.0 with 4.22M tokens; disabling REUSEANDPATCH or FRESHSOLVE also degrades performance, with FRESHSOLVE removal lowering SQL by 10.0 points and Search by 11.0.Source: §5.3 · p. 8
  • With downstream execution fixed, Harness RL raises Qwen3-8B Git Agent scores from 51.0 to 81.0 on Math and 55.0 to 73.0 on SQL.The RL-adapted configuration also outperforms the Qwen3-32B configuration on four of five StackPlanner domains.Source: §5.3 · pp. 8–9, Figure 4 · p. 8
  • On the 200-task validation set, λ = 1 raises Query Eq. to 53.0 but lowers Task Score to 65.0, whereas λ = 0 yields 47.0 Query Eq. and 70.0 Task Score.The reported comparison motivates the balanced reward used in the final policy; Figure 4 also shows per-update training reward and its five-update moving average over 40 updates.Source: Figure 4 · p. 8, §5.3 · p. 9
  • Non-parametric memory guidance improves 13 of 15 settings, including Research gains across all three harnesses ranging from 2.07 to 4.27 points.The fixed memory is distilled from separate source datasets and contains no evaluation instances, reference answers, or task-specific evidence.Source: §5.3 · p. 9
  • Across 10–30-turn interleaved sessions, GITHARNESS ranks first at every horizon on Math and Research; at H = 30, it leads the strongest baseline by 6.0 Math points and 12.79 RACE points.From H = 10 to H = 30, its Research score decreases by 2.21 points, versus at least 3.37 points for the baselines.Source: §5.3 · p. 9, Table 4 · p. 9

6 CONCLUSION, LIMITATIONS AND FUTURE WORK

GITHARNESS organizes requirement states and work states in a branchable history, restoring compatible work so agents can preserve valid results and update affected content. Across five domains, two backbones, and three harnesses, it performs strongly against Native and competing baselines; its accuracy depends on correctly interpreting requirements and selecting historical states.

6 CONCLUSION, LIMITATIONS AND FUTURE WORK

  • GITHARNESS couples effective requirements with corresponding harness states in a branchable history, restoring compatible work and updating affected content.The conclusion describes preserving valid work and discarding obsolete states when processing new feedback.Source: §6 · p. 9
  • After interface-level RL adaptation, the 8B Git Agent outperforms native cross-turn execution in 29 of 30 settings.The training-free configuration outperforms Native in all 30 settings and matches or exceeds the strongest competing baseline in 28 of 30, including 14 of 15 under DeepSeek-V4-Flash.Source: §6 · pp. 9–10
  • Accurate requirement interpretation and historical-state selection can be challenging under ambiguous or conflicting feedback.Restoring a compatible state also does not guarantee that the underlying agent preserves all unaffected work.Source: §6 · p. 10
  • The evaluated interactions are generated from public benchmark tasks rather than collected from real users.The study uses public benchmarks and reports that no private conversations or personally identifiable information are used.Source: ETHICS STATEMENT · p. 10
  • The authors report that they will release GITHARNESS, constructed trajectories, and reproduction metadata and materials.The reported materials include prompts, model configurations, training settings, baseline implementations, evaluation procedures, and reproduction commands.Source: REPRODUCIBILITY STATEMENT · p. 10

A DATASET CONSTRUCTION

The dataset construction process builds multi-turn requirement trajectories backward from a source task and its verifier. It varies requirements across turns while retaining only trajectories whose final requirement is semantically equivalent to the source task.

A DATASET CONSTRUCTION

  • The source task and its verifier serve as the final anchor, and trajectories are retained only when their terminal requirement is semantically equivalent to that anchor.This lets single-turn and multi-turn evaluation share the same final objective and native verifier without new answer annotations.Source: A DATASET CONSTRUCTION · p. 15
  • Hidden construction variables are distinct from the Git Agent’s INSPECT, COMMIT, and REUSE actions and the update agent’s domain tools.Construction labels, anchors, and reference states are hidden from evaluated agents and used only for validation and reward computation.Source: A DATASET CONSTRUCTION · p. 15
  • Preceding states are constructed by applying requirement-transition operators to historical anchors and expressing the changes as natural-language feedback.The method instantiates three requirement-change directions with nine operators.Source: A DATASET CONSTRUCTION · p. 15
  • Intermediate states can depart from the source task through a transient edit and later recover through a semantic compensating edit.Examples include withdrawing a temporary output-control requirement or restoring parameters or premises.Source: A DATASET CONSTRUCTION · p. 15

B ADDITIONAL METHOD DETAILS

The inference procedure resolves feedback against a version graph, restores or forks a compatible work state, and commits successful work as a checkpoint. Training updates only Git Agent call tokens, while other interaction components provide context without receiving gradient.

B ADDITIONAL METHOD DETAILS

  • At inference, GITHARNESS resolves feedback and either restores a bound checkpoint or executes from a selected base state before persisting a successful result.When a compatible state is available, execution can fork from it; otherwise the procedure can use FRESHSOLVE, then commit the finished work state.Source: B ADDITIONAL METHOD DETAILS · pp. 15–16
  • The update agent’s action space includes inspecting recovered state, invoking domain tools, editing candidate artifacts, and requesting normal termination.Concrete tools, artifact representations, and stopping protocols remain domain specific; REUSEANDPATCH means continued execution from a fork, not a deterministic file patch.Source: B ADDITIONAL METHOD DETAILS · p. 16
  • Only Git Agent call-token positions contribute to policy updates; user, system, runtime, tool-observation, and update-agent tokens receive no gradient.Episode rewards are assigned to corresponding Git-Agent segments before group normalization and the GRPO update.Source: B ADDITIONAL METHOD DETAILS · pp. 16–17

C EXPERIMENT DATASETS

The evaluation dataset spans five benchmark domains, with Math and SQL supplying RL training trajectories and Search, Code, and Research held out for transfer evaluation. The construction also tests requirement changes in multi-turn interactions, including conversations that interleave multiple tasks; the harnesses retain their native reasoning and tool-use procedures.

C EXPERIMENT DATASETS

  • The evaluation covers GSM8K, BIRD, BrowseComp-Plus, SWE-bench Verified, and DeepResearch Bench across Math, SQL, Search, Code, and Research.Only Math and SQL provide RL training trajectories; Search, Code, and Research are held out entirely from RL training, and the Math and SQL candidate pools contain 400 tasks for training and validation.Source: C EXPERIMENT DATASETS · p. 17
  • GSM8K, BIRD, BrowseComp-Plus, SWE-bench Verified, and DeepResearch Bench use normalized exact match, SQL execution accuracy, normalized exact match, resolved outcome, and RACE, respectively.The stated evaluation sets include 100-task test sets for GSM8K and BIRD, 100 BrowseComp-Plus queries, a fixed 100-task SWE-bench subset, and all 100 DeepResearch Bench tasks.Source: C EXPERIMENT DATASETS · p. 17
  • Multi-turn trajectories use PRESERVE, EXTEND, and REVISE transitions, with the final effective query semantically equivalent to the original source task.Interleaved conversations contain 10–30 user turns and two to four active tasks; final outputs are verified separately by benchmark-specific evaluators and macroaveraged.Source: C EXPERIMENT DATASETS · pp. 17–18

D HARNESS AND ADAPTER DETAILS

  • The three evaluated harnesses are TC-RAG, StackPlanner, and OpenHands, each retaining its native reasoning and tool-use procedure through a lightweight adapter.StackPlanner is described as a centralized hierarchical multi-agent framework that separates high-level coordination from subtask execution and uses task-level and experience memory.Source: D HARNESS AND ADAPTER DETAILS · p. 18

E BASELINE IMPLEMENTATION DETAILS

The baselines represent four approaches to managing context across turns, ranging from relying on native harness state to explicitly versioning persistent context.

E BASELINE IMPLEMENTATION DETAILS

  • Native relies on the task harness’s conversational state and workspace, with no added cross-turn context-management module.Prior-turn information is retained only through the harness’s native state mechanism.Source: E BASELINE IMPLEMENTATION DETAILS · p. 18
  • Restart reconstructs accumulated requirements into a self-contained query each turn, then solves in a fresh session without prior traces or artifacts.Each turn is solved independently from a clean workspace.Source: E BASELINE IMPLEMENTATION DETAILS · p. 18
  • U-Fold retains the full dialogue and tool-call history while building a compact working context from an evolving intent-aware dialogue summary and task-relevant tool log.The compact context supports the current interaction.Source: E BASELINE IMPLEMENTATION DETAILS · pp. 18–19
  • GCC manages context through a version-controlled persistent file system with COMMIT, BRANCH, MERGE, and CONTEXT operations.The operations checkpoint progress, explore alternatives, integrate branch states, and retrieve historical context at different granularities.Source: E BASELINE IMPLEMENTATION DETAILS · p. 19

F IMPLEMENTATION DETAILS

Implementation keeps downstream execution controlled across compared methods, trains only the Git Agent, and handles multi-task finalization through independent task-local invocations.

F IMPLEMENTATION DETAILS

  • Within each backbone–harness block, compared methods share the downstream model, inputs, tools, execution budget, and verifier; gold resources and feedback remain evaluator-only.Method-specific persistent state remains part of the evaluated system.Source: F IMPLEMENTATION DETAILS · p. 19
  • Only the Qwen3-8B Git Agent is trained with LoRA GRPO; the downstream executor, query-equivalence judge, and other components remain fixed.The selected checkpoint uses 40 updates, 160 prompt groups, and 1,280 sampled episodes; reward weight λ = 0.2 is validation-selected.Source: F IMPLEMENTATION DETAILS · p. 19
  • For a final message spanning multiple active tasks, the scheduler invokes GitHarness independently per task history and workspace, then aggregates the outputs.Each task-local invocation ends in one COMMIT or REUSE, preserving single-handoff semantics.Source: F IMPLEMENTATION DETAILS · p. 19

G ADDITIONAL EXPERIMENTS

The additional experiments report token and cost trade-offs, domain-dependent version and execution choices, and skill-injection results; later sections document computational settings and the prompts used for resolution, routing, execution, and evaluation.

G ADDITIONAL EXPERIMENTS

  • COMMIT is most frequent in Math, SQL, Search, and Research, while branch-execution strategies vary by domain; TCRAG skill injection improves four of five domains.INSPECT is particularly common in Code and Research; skill injection gains 3.0 points on Math and 4.27 points on Research, with figures showing action and strategy patterns.Source: G ADDITIONAL EXPERIMENTS · pp. 20–21, Figure 7 · p. 22, Figure 8 · p. 22
  • GITHARNESS achieves the lowest token usage and estimated cost on Search, Code, and Research, while Math and SQL show task-dependent trade-offs.Figures 5–6 show token–performance trade-offs for Math and Code and cost–performance across five domains; Table 10 gives numerical details.Source: G ADDITIONAL EXPERIMENTS · pp. 19–20, Figure 5 · p. 20, Figure 6 · p. 21

H COMPUTATIONAL RESOURCES AND SOFTWARE ENVIRONMENT

  • Inference experiments mostly used an Ubuntu 22.04.5 server with two Xeon Platinum 8358 CPUs, 1007 GiB RAM, and eight A800 GPUs; RL training used one H800.The section also specifies model-serving configurations, task environments, and token-cost pricing.Source: H COMPUTATIONAL RESOURCES AND SOFTWARE… · p. 21

I PROMPT TEMPLATES

  • The prompt appendix presents core GITHARNESS prompts in abridged form, with the released code providing the complete experimental prompts.The Git Agent prompt converts each turn into a complete resolved requirement and chooses INSPECT, COMMIT, or REUSE.Source: I PROMPT TEMPLATES · p. 22, §I.1 · p. 22

I.1 REQUIREMENT RESOLUTION

  • Requirement resolution treats updates as edits to the active requirement, preserves unmentioned clauses for local changes, and rebuilds around a new goal when the user explicitly changes goals.Before a terminal COMMIT or REUSE, the prompt requires checking the requested outcome, current slots and dependencies, active requirements, and withdrawn content.Source: §I.1 · pp. 22–23

I.2 DOWNSTREAM EXECUTION ROUTING

  • The downstream router chooses fresh solving, reuse-and-patch, or inspection based on recomputation and validation costs, stale-work risk, and applicable verified work.Inspection must be followed by a final strategy choice; uncertainty defaults to fresh solving.Source: §I.2 · p. 24

I.3 WORKSPACE EXECUTION

  • Workspace instructions favor solving small self-contained tasks directly, while reading or patching prior files when accumulated work is valuable or a change is localized.If existing work already satisfies the task, the agent finishes without editing.Source: §I.3 · pp. 24–25

I.4 EVALUATION PROMPT

  • The frozen equivalence judge compares prediction and reference as complete current tasks, treating missing, stale, withdrawn, contradictory, or goal-changing content as non-equivalent.Equivalent paraphrases are accepted, while decorative wrappers are ignored and explicit output constraints are enforced.Source: §I.4 · pp. 25–26
Loading 2609.36789v1…