PaperlayerView exact revision on arXiv

Source-linked AI summary

HarnessBridge: Learnable Bidirectional Controller for LLM Agent Harness

Xiaoxuan Wang, Haixin Wang, Alexander Taylor, Jason Cong, Yizhou Sun, Wei Wang

arXiv:2606.12882v1cs.AI

TL;DR

Long-horizon LLM agents are constrained by manually engineered harnesses that struggle with growing trajectories and complex interactions. HarnessBridge learns a bidirectional harness controller that compresses observations and filters actions, matching or surpassing specialized harnesses while reducing token usage and trajectory length on Terminal-Bench 2.0 and SWE-bench Verified.

  • Problem

    Manually engineered harnesses make long-horizon agent–environment interaction difficult to scale as trajectories grow and interactions become more complex.

  • Method

    HarnessBridge learns bidirectional observation and action projections that compress interaction histories and map proposed actions to executable transitions or trajectory-grounded rejections.

  • Results

    HarnessBridge matches or surpasses strong specialized harnesses while substantially reducing token usage and trajectory length across Terminal-Bench 2.0 and SWE-bench Verified.

  • Takeaways & Limitations

    A learnable harness can provide competitive or stronger task performance with lower token cost and shorter trajectories while transferring from smaller generators to larger commercial models.

  • Takeaways & Limitations

    Evaluation is confined to coding settings, and single-run results emphasize relative trends over absolute values.

Abstract

from arXiv · show

Large language models are increasingly deployed as agents for long-horizon tasks, yet their performance is shaped not only by model capability and environment design, but also by the harness that mediates agent--environment interaction. Existing harnesses are largely manually engineered, making them difficult to scale as trajectories grow longer and interactions become more complex. In this work, we ask whether harness can be generated by a learnable plug-in module that can be trained in an end-to-end fashion. We introduce HarnessBridge, a lightweight learnable harness controller that parameterizes the agent--environment interface as a bidirectional projection. HarnessBridge learns two bidirectional projections: observation projection, which distills raw trajectories into compact, decision-relevant states, and action projection, which converts proposed actions into executable transitions or trajectory-grounded rejections. We train HarnessBridge on a harness supervision dataset via unified instruction tuning. On Terminal-Bench~2.0 and SWE-bench Verified, HarnessBridge matches or surpasses strong specialized harnesses while substantially reducing token usage and trajectory length, and generalizes from smaller generators to larger commercial models.

1 Introduction

HarnessBridge treats the agent–environment interface as an end-to-end learnable bidirectional projection policy, addressing inefficiencies and failures in manually engineered harnesses. It is trained through unified instruction tuning and achieves competitive task performance with improved efficiency and generalization.

  • Motivation: Harnesses format observations, manage context, invoke tools, parse outputs, validate actions, and handle environmental feedback, making harness engineering important to agent performance.Reported gains include better context construction, retry logic, summarization, and action validation.
  • Motivation: Long-horizon interaction accumulates redundant or stale context and ineffective, invalid, looping, or malformed actions, increasing token costs and obscuring decision-critical state.These problems affect both environment-to-agent context and agent-to-environment behavior.
  • HarnessBridge: HarnessBridge parameterizes the agent–environment interface as bidirectional projections that compress trajectories into decision-relevant states and map proposed actions into executable transitions or grounded rejections.The interface should preserve task-relevant state, suppress stale context, reject unproductive actions, and avoid unsupported hallucinated information.
  • Training and evaluation: HarnessBridge learns both interface directions from a harness supervision dataset by framing their behaviors as instruction-following tasks and applying unified instruction tuning to a lightweight LLM.The dataset covers both environment-to-agent and agent-to-environment mappings.
  • Training and evaluation: On Terminal-Bench 2.0 and SWE-bench Verified, HarnessBridge matches or surpasses strong specialized harnesses while substantially reducing token usage and trajectory length.The evaluation spans open-sourced and commercial models, and the paper reports good generalization.

2 Related Work

Prior work establishes tool-using LLM agents for sequential decision making, but long-horizon stability remains difficult because accumulated context and execution errors degrade scalability. Existing harnesses and auto-harness methods mainly rely on manual heuristics or optimize external scaffolds, rather than learning a runtime policy that mediates observations and actions.

  • LLM Agents: LLM agents interleave reasoning and acting, call external APIs, and acquire reusable skills through interaction, enabling sequential decision making.These capabilities move agents beyond single-turn generation toward interaction with external environments.
  • Long-Horizon Challenges: Long-horizon interaction accumulates stale or redundant observations, propagates errors, repeats tool calls, and increases execution cost.These factors make stable and scalable tool-using agents an open problem.
  • Manual Harnesses: Existing harnesses use manually designed heuristics, including trajectory summarization, retrieval-based memory, context compaction, retry rules, and tool-call validation.Extending the context window alone does not prevent stale, redundant, or low-signal trajectory information from degrading decision quality.
  • Auto-Harness: Auto-harness methods optimize workflows, prompts, scaffolds, or harness code, but typically target the external scaffold rather than learning a runtime policy mediating observations and actions.Meta-Harness is presented as an important step beyond manual harness engineering.

3 Method

HarnessBridge makes the agent–environment interface a learnable bidirectional policy while keeping the generator fixed. Its observation and action projections selectively control exposed state and executed transitions through unified, trajectory-grounded transformations.

  • Learnable Interface: HarnessBridge parameterizes harness engineering as an end-to-end learnable interaction policy, optimizing only the harness while keeping the generator policy fixed.The harness mediates both what the agent observes and what the environment executes.
  • Observation Projection: Observation projection preserves decision-critical history, compresses verbose information, drops irrelevant units, and exposes a learned active-state index before the projected chronology.The active state includes unresolved errors, open constraints, established facts, pending goals, and remaining decision variables.
  • Action Projection: Action projection passes productive proposals to the environment or rejects them without an environment step, returning trajectory-grounded feedback for revision.If trajectory-grounded evidence cannot be provided, the policy defaults to passing the action.
  • Training Formulation: A shared policy Pθ is trained through unified conditional generation rather than separate observation- and action-projection modules.Training examples are curated from prompted instruction-tuned models, retaining successful trajectories and using an LLM judge to assess projection quality.
  • Raw Trajectory Preservation: HarnessBridge retains the raw trajectory as the authoritative record while selectively exposing compressed projected views, avoiding irreversible updates after every interaction step.This design mitigates hallucinated summaries and accidental removal of useful history.

4 Experiments

Experiments evaluate HarnessBridge on coding-focused, long-horizon agent benchmarks for token efficiency, task performance, cross-model and cross-environment generalization, and component contributions. Results show competitive or improved success with lower token use, transfer beyond the supervision setting, and complementary value from both projections.

  • Experimental setup: Experiments measure success rate and average input-token consumption on Terminal-Bench 2.0 and SWE-bench Verified across multiple generator models and harness baselines.The evaluation uses seven frozen generator models, while HarnessBridge is initialized from Qwen3.5-0.8B and instruction-tuned for harness-control decisions.
  • Main results: HarnessBridge jointly improves success rate and reduces token consumption compared to other harness designs.On SWE-bench Verified with Qwen3.5-35B-A3B, it remains competitive while using the lowest token budget among reported harnesses.
  • Main results: 33.7% with Qwen3.5-35B-A3B and 20.7% with GLM-4.7-Flash are the highest Terminal-Bench 2.0 success rates achieved by HarnessBridge under both generators.Terminal-Bench 2.0 is out of domain because supervision uses only SWE-bench trajectories, yet the learned harness remains strong.
  • Generalization: 22.5% versus 15.7% is the largest reported success-rate improvement on GPT-5.4-Nano among five unseen generators.Across the unseen generators, HarnessBridge preserves or improves success rate while reducing token consumption; gains are larger when baselines incur higher interaction costs.
  • Generalization: HarnessBridge-D improves task success over Terminus 2 while substantially reducing token usage when DeepSeek-V4 supplies supervision instead of Qwen3.5-35B-A3B.The improvement holds across heterogeneous generators, including the GPT-5.4 family, which shares no lineage with the sampling model.
  • Ablation: Removing either action projection or observation projection reduces success rate on both tested backbones, showing that both modules contribute to task performance.Action projection steers outputs toward executable, task-relevant operations, while observation projection organizes environment feedback.

5 Analysis

HarnessBridge’s projections compress stale trajectory context and redirect redundant actions toward meaningful tests. Across outcome categories, this produces shorter, more token-efficient trajectories, with the largest gains when HarnessBridge succeeds and the baseline fails.

  • Projection behavior: Observation projection preserves task context and recent turns while reducing stale intermediate trajectory history.In the django-13964 SWE-bench trajectory, the raw context is dominated by repository exploration, code inspection, and test attempts no longer directly relevant to the next action.
  • Projection behavior: Action projection rejects redundant checks and redirects the agent to test the modified codebase.In xarray-4356, the agent follows the redirect and obtains a meaningful signal about the patch state.
  • Projection behavior: Figure 4 measures category-level compression rate and active-state rate as means of per-turn rates over all turns.Compression rate captures downstream invocations where a turn was compressed; active-state rate captures invocations lifting content into the persistent active-state block.
  • Efficiency analysis: HarnessBridge reduces turns and token consumption across all outcome categories, with token reductions exceeding turn reductions.The efficiency gain occurs regardless of whether tasks ultimately succeed and indicates more compact per-turn context in addition to shorter trajectories.
  • Efficiency analysis: 18 turns versus 52 turns (−65%) and 11% of baseline tokens (−89%) characterize the Gained category.These means compare HarnessBridge with the baseline on tasks where HarnessBridge succeeds and the baseline fails.

6 Conclusion

HarnessBridge makes the agent–environment interface a learnable harness policy through jointly trained bidirectional projections. It delivers competitive or stronger benchmark performance with lower token cost, shorter trajectories, and transfer from smaller to larger commercial models.

  • Learnable harness policy: HarnessBridge recasts the agent–environment interface as a learnable harness policy.Its harness behavior is learned rather than defined solely through manual engineering.
  • Bidirectional projections: Jointly trained bidirectional projections compress raw trajectories into decision-critical context and map proposed actions into executable transitions or trajectory-grounded rejections.The two projections mediate both observation processing and action execution.
  • Benchmark results: HarnessBridge achieves competitive or stronger performance with lower token cost and shorter trajectories on the benchmarks.The conclusion reports these efficiency and performance gains across the evaluated benchmarks.
  • Model transfer: HarnessBridge transfers from smaller generators to larger commercial models.This demonstrates generalization across generator scale.

A Preliminary Study

The preliminary study shows that HarnessBridge can distill harness-control behavior into a small model while balancing observation-history compression and action-rejection design. A moderate reserved history window offers a better performance–efficiency trade-off, whereas rule-only rejection has limited impact on semantically unproductive actions.

  • Harness Backbone Comparison: Fine-tuned Qwen3.5-0.8B HarnessBridge matches the performance and efficiency of the much larger Qwen3.5-35B-A3B harness at substantially lower inference cost.This suggests harness-control behavior can be effectively distilled into a small model.
  • Effect of Reserved History Window on Observation Projection: A moderate reserved history window achieves a better trade-off, while overly aggressive or overly conservative observation compression can hurt performance or efficiency.The window preserves full recent history before activation and enables compression and dropping beyond it.
  • Effect of rejection mode on Action Projection: Rule-only action rejection has limited impact because many inefficient actions are syntactically valid but semantically unproductive.This mode rejects simple invalid actions, including malformed, empty, duplicated, or repeated commands, and returns predefined revision feedback.

B Action Category Analysis

The analysis partitions agent turns into ten mutually exclusive, syntactically determinable action categories. These categories organize behavior into information gathering, state-changing action, execution/verification, and a residual reasoning-only class.

  • Category design: Ten mutually exclusive action categories are defined by deterministic patterns over each action’s command field.The design targets dominant behaviors on SWE-bench Verified and Terminal Bench without semantic inspection of observations.
  • Behavioral modes: Information gathering comprises file_read, search, navigation, and git_operation.These categories represent one of three broad behavioral modes identified in the analysis.
  • Behavioral modes: State-changing actions comprise file_edit, dependency_setup, and interrupt_abort.This mode captures actions that modify state or interrupt execution.
  • Behavioral modes: Execution and verification comprise script_execution and test_execution, while reasoning_only captures turns with no parseable command.reasoning_only is characteristic of larger reasoning-tuned generators.

C Details of Data Curation · D Experiment Implementation

HarnessBridge supervision is curated separately for observation and action projection from solved SWE-bench Verified trajectories, while the generator remains frozen. Automated prompting and strict filtering produce trajectory-grounded, efficiency-improving examples for unified instruction tuning.

  • C Details of Data Curation: Supervision is constructed from solved SWE-bench Verified trajectories for harness-level projection modules, while the Qwen3.5-35B-A3B generator remains frozen.The shared policy learns control in both directions, but data construction is component-specific.
  • C Details of Data Curation: Approximately 40K raw supervision candidates are generated by extensively prompting existing models and reduced through strict quality-oriented filtering.The curation goal is useful harness-level decisions rather than large collections of raw intervention traces.
  • C Details of Data Curation: Action-projection examples label whether proposed actions should reach the environment or be rejected with feedback, emphasizing sparse rejected-action cases.Rejection records include the category, concern, grounding evidence, and suggested revision.
  • C Details of Data Curation: Rejected actions are retained only when trajectory-grounded, followed by successful correction, and beneficial for reducing environment steps or token consumption.This criterion prevents arbitrary or unverifiable rejections from becoming supervision.
  • C Details of Data Curation: Observation-projection examples identify trajectory turns to preserve, summarize, or omit and include an active-state index of decision-relevant facts.Examples use multiple reserved-history windows to learn appropriate rather than overly aggressive compression.
  • C Details of Data Curation: 5,405 high-quality examples remain after GLM-4.7-Flash judging, deduplication, trajectory capping, rejection validation, and distribution balancing.Both components are converted to a unified instruction-following format for supervised fine-tuning.
  • C Details of Data Curation: The unified dataset fine-tunes a Qwen3.5-0.8B model with supervised fine-tuning for both harness projection components.The retained examples combine observation- and action-projection supervision.

D.1 More Experiment Setup · D.2 Baseline Tuning and Evaluation Fairness

The experiments use standardized Harbor-based evaluation and calibrated HarnessBridge training and deployment procedures. Fairness controls hold generators, tasks, decoding, budgets, and execution constraints constant across harnesses.

  • D.1 More Experiment Setup: All experiments use the Harbor framework with open-source models served on NVIDIA H200 GPUs.
  • D.1 More Experiment Setup: Meta-Harness is tuned on 100 SWE-bench Verified tasks for 3 iterations using Qwen-3.5-35B-A3B, then evaluated on full SWE-bench Verified and Terminal-Bench 2.0.
  • D.1 More Experiment Setup: HarnessBridge sets each backbone’s observation-window size near the median Terminus 2 trajectory length for that benchmark, independent of its own performance.This calibration accounts for differing interaction profiles, including shorter Claude-Opus-4.7 and longer DeepSeek-V4-Pro trajectories.
  • D.1 More Experiment Setup: SFT fine-tunes Qwen-3.5-0.8B with Adam, learning rate 1e−5, batch size 64, bf16 precision, and 3 epochs on NVIDIA H200 GPUs.
  • D.1 More Experiment Setup: The lightweight Qwen3.5-0.8B controller processes approximately 3× as many tokens as the generator, but has much lower per-token inference cost.It can run with low memory and high throughput on commodity accelerators, while local deployment further amortizes costs with commercial API generators.
  • D.1 More Experiment Setup: After controller input/output tokens are included, HarnessBridge still reduces total compute-weighted inference cost across evaluated settings.The result indicates end-to-end interaction-cost reduction rather than computation merely shifting from generator to harness.
  • D.2 Baseline Tuning and Evaluation Fairness: All harnesses share the same generator backbone, benchmark split, decoding configuration, serving backend, temperature, task budget, timeouts, and success criteria.Harnesses are not tuned on evaluation trajectories; methods retain their standard construction mechanisms under frozen generators, task suites, and execution constraints.

D.3 Data Curation for HARNESSBRIDGE-D

HarnessBridge-D curates supervision around the unusually long, reasoning-heavy DeepSeek-V4 trajectories, emphasizing valid solutions, efficient action projections, and compact observation targets. The resulting efficiency-focused recipe is reported as HarnessBridge-D and yields 5287 retained examples for fine-tuning.

  • Curation principles: DeepSeek-V4 trajectories are substantially longer and more reasoning-heavy than Qwen3.5-35B-A3B, making turn verbosity and history length dominant sources of low-quality supervision.Supervision is retained only from solved trajectories.
  • Action projection: Action-projection supervision retains only HarnessBridge-D-solving trajectories, requiring both-pass cases to reduce token usage and interaction turns relative to Terminus 2.The criterion is conditioned on whether the baseline already solves the task.
  • Action projection: The turn-agnostic recipe lowers success from 57.3 to 48.3 on DeepSeek-V4-Pro and from 49.4 to 43.8 on DeepSeek-V4-Flash, while incurring approximately 37% validator rejection.It also causes frequent timeouts; the turn-saving recipe roughly halves rejection and cuts timeouts.
  • Action projection: The turn-saving recipe retains only genuinely efficiency-improving trajectories and is therefore reported as HarnessBridge-D.A validator trained on turn-increasing supervision over-rejects enough to make the harness worse than the Terminus 2 baseline.
  • Observation projection: Observation targets exceeding 6K tokens are discarded, and structured targets must parse successfully to provide a useful compression signal.Each example marks trajectory turns to preserve, summarize, or drop.
  • Final dataset: 5287 high-quality examples remain after judge filtering with DeepSeek-V4-Pro, deduplication, trajectory-level capping, and length- and efficiency-based criteria.These examples are retained for fine-tuning the HarnessBridge-D controller.

D.4 Experiment Result for HARNESSBRIDGE-D

HarnessBridge-D, trained only on DeepSeek-sampled supervision, substantially reduces input-token usage across all generators on Terminal-Bench 2.0 while maintaining comparable success rates. The largest savings occur for the token-heavy GPT-5.4 baselines, reaching roughly 90%.

  • Overall results: HarnessBridge-D substantially reduces input-token consumption across all generators while maintaining comparable success rates on Terminal-Bench 2.0.The model is trained only on DeepSeek-sampled supervision.
  • GPT-5.4 results: 9.80M and 9.41M input tokens are used by the unconstrained GPT-5.4 family baselines, making them the most token-heavy.These baseline usages correspond to GPT-5.4-Nano and GPT-5.4, respectively.
  • GPT-5.4 results: Roughly 90% lower usage on GPT-5.4-Nano reduces input tokens to 1.00M.HarnessBridge-D achieves a similar reduction on GPT-5.4.

E Limitation … Overall Score

The paper evaluates HarnessBridge in coding settings while presenting generic tool-use projections, and specifies detailed policies and judging rubrics for observation compression and action validation. Training-example retention depends on weighted quality scores, decision thresholds, and safeguards against information loss, unsupported blocking, and trivial records.

  • E Limitation: Evaluation is confined to coding settings, although the generic projections are expected to extend to web navigation, computer use, and multi-step research workflows.Observation and action projection operate over generic tool-use trajectories and long interaction histories rather than code-specific structure.
  • PROMPT (Observation Projection) / Your Job / Principles: Observation projection summarizes or drops redundant history while preserving decision-relevant state, exact values, unresolved issues, and protected recent turns.The active-state index retains errors, tests, constraints, files, hypotheses, decisions, failed approaches, and partial progress; recent turns are kept automatically.
  • PROMPT (Action Projection) / Decide: PASS or REJECT / Mitigating Factors: Action projection rejects shallow edits, premature submissions, wasteful exploration, and contradicted paths, but defaults to PASS when evidence is insufficient or repeated rejection has not helped.Mitigating factors include legitimate exploration, traced mechanisms, first attempts, and 3+ rejections of the same category.
  • LLM Judge PROMPT (Action Projection) / 1. INPUT WORTHINESS (1-5) – gates the record / 3. DECISION_CORRECTNESS (1–5) / 4. EVIDENCE_GROUNDEDNESS (1–5) — most important quality axis on BLOCKs / 5. CONCERN_SPECIFICITY (1–5) / 6. SUGGESTION_ACTIONABILITY (1–5): Action-projection examples are judged on input worthiness, schema, decision correctness, evidence groundedness, concern specificity, suggestion actionability, category fit, and bias safety.The rubric requires evidence on BLOCK outputs to include a direct quote and concrete concern and suggestion fields.
  • LLM Judge PROMPT (Observation Projection) / 1. INPUT_WORTHINESS (1–5) — serves Concern 4: Observation-projection judges assess per-turn decisions, unresolved-issue coverage, summary quality, cross-reference integrity, and budget compliance.Every turn is labeled KEEP, SUMMARIZE, or DROP; summaries must preserve paths, line numbers, error codes, function names, variables, and numeric test outputs.
Loading 2606.12882v1…