Source-linked AI summary
Harness Continual Learning: Continual Adaptation Beyond Model Parameters
Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li, Lei Wang, Yang Gao
TL;DR
Agents can adapt through evolving harnesses beyond model parameters, but retaining earlier reliable behavior remains an open continual-learning problem. This paper formalizes Harness Continual Learning with guarded harness evolution and finds capability accumulation and failure recovery across tasks alongside measurable forgetting and an adjustable stability–plasticity trade-off.
Problem
Agent adaptation increasingly occurs through evolving prompts, memories, tools, skills, and routing policies, creating a need to study continual learning beyond model parameters while retaining earlier behavior.
Method
HCL jointly versions four harness components and uses an optimizer–evaluator process to propose candidate updates and commit them only after current, historical, and validity checks.
Results
Across textual reasoning, multimodal perception, and open-world interaction, harness evolution accumulates capabilities and supports failure recovery while exhibiting measurable harness-level forgetting.
Takeaways & Limitations
Historical-retention control makes the stability–plasticity trade-off adjustable, supporting continual harness adaptation around a frozen foundation model.
Takeaways & Limitations
Finite anchor sets cannot guarantee unchanged behavior on historical cases that are not represented among the anchors.
Abstract
from arXiv · showhide
Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. Because these contents jointly shape later execution, a harness update can disrupt previously reliable behavior even when the model is frozen. This raises a new question: how can an agent continually improve its state outside the model while retaining behavior acquired earlier? We formulate Harness Continual Learning (HCL), a new continual learning paradigm in which the harness evolves around a frozen foundation model, and define the resulting loss of earlier behavior as harness-level forgetting. We instantiate HCL with four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. We further introduce guarded harness evolution to separate update generation from state commitment. A Continual Optimizer proposes candidate harnesses from post-execution feedback, and a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity. Experiments on textual reasoning, multimodal perception, and open-world interaction demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings. Component ablations assess the contribution of each harness component, while controlled retention sweeps reveal measurable harness-level forgetting and show that the stability--plasticity trade-off can be explicitly adjusted.
1 INTRODUCTION
Harness Continual Learning shifts continual adaptation from model parameters to an evolving harness around a frozen foundation model. It formalizes harness-level forgetting and uses guarded evolution to balance capability accumulation, failure recovery, and retention.
- Harness-level forgetting: Harness-level forgetting occurs when coupled harness updates improve recent cases but disrupt earlier correct answers, valid tool calls, or successful action trajectories.Memory, skill, and routing changes can alter earlier execution behavior.
- Empirical findings: Experiments across textual reasoning, multimodal perception, and open-world interaction show capability accumulation and failure recovery alongside measurable harness-level forgetting.Historical-retention budgets shift the operating point between adaptation and retention.
- Empirical findings: Historical-retention budgets make the stability–plasticity trade-off controllable, while more permissive updates do not necessarily yield a stronger final harness.The results characterize retention as an explicit operating constraint on harness evolution.
- Harness Continual Learning: HCL shifts the learning object from model state to harness state around a frozen foundation model.The harness can evolve through prompts, memories, tool and skill specifications, and routing policies while the foundation model remains frozen.
- Guarded harness evolution: Guarded harness evolution separates candidate generation from commitment: a Continual Optimizer proposes candidate harnesses, while a Continual Evaluator checks current, historical, and validity criteria.The evaluator controls whether an update becomes part of the evolving harness state.
2 RELATED WORK
Related work establishes runtime harnesses as execution-oriented structures around foundation models, with interfaces, memories, and capabilities coordinated across agent systems. It also connects harness revision through execution feedback to continual learning’s retention problem, whose established formulation centers on model-level catastrophic forgetting.
- Agent Harnesses: Runtime harnesses place persistent execution contents around foundation models to convert inference into task-directed execution.The described contents include interfaces for inputs and observations, memory for interaction records and guidance, and capability registries.
- Agent Harnesses: ReAct, Toolformer, MRKL, HuggingGPT, MemGPT, Reflexion, and Voyager develop complementary interaction, capability-coordination, memory, feedback, or skill mechanisms.Together, these systems are characterized as forming a coupled execution pipeline.
- Harness Adaptation: Harness engineering revises prompts, programs, memories, tool-use policies, skills, and workflows using execution feedback, increasingly alongside configuration search and failure diagnosis.The cited work frames these revisions as part of sustained agent improvement.
- Continual Learning: Model-centric continual learning adapts models across non-stationary task or data streams while retaining earlier capabilities, with catastrophic forgetting arising when new knowledge disrupts prior model-encoded knowledge.The passage also notes representation-based approaches that learn features or prompts intended to remain useful across tasks.
3 HARNESS CONTINUAL LEARNING
Harness Continual Learning updates a jointly versioned harness around a frozen foundation model to acquire new behavior while retaining previously reliable responses, tool calls, and action trajectories. It combines four execution-facing components with guarded proposal–evaluation–commitment, making retention and validity conditions for deployment explicit.
- Definition: HCL sequentially updates a deployed harness around fixed model parameters while preserving previously reliable behavior.Reliable behavior includes correct responses, valid tool calls, and action trajectories satisfying an environment.
- Harness State: The harness comprises four jointly versioned components: Task Interface, Experience Memory, Capability Map, and Adaptive Router.Their prompts, processing rules, stored experience, reusable skills, and routing specifications jointly determine future task handling.
- Experience Memory: Experience Memory combines Raw Memory for concrete interactions with Abstract Memory for reusable knowledge generalized across tasks.Raw records preserve successful behavior and failures for replay and recovery, while summaries consolidate conventions, reasoning patterns, and errors to avoid.
- Capability Map: The Capability Map expands beyond external runtime operations by incorporating skills acquired through continual interaction.This connection between accumulated knowledge and inner capabilities enables the frozen-model agent to acquire, refine, and transfer skills across tasks.
- Guarded Harness Evolution: Guarded harness evolution isolates candidate generation from deployment and commits a candidate only after current-improvement, historical-retention, and validity checks.If the candidate fails, the existing harness remains deployed; the tolerance Bn adjusts the stability–plasticity balance.
- System-Level Formulation: HCL unifies model-centric continual-learning principles within a single evolving harness rather than treating replay, representation, architecture, optimization, and regularization as separate parameter-adaptation families.The formulation coordinates these functions at the system level while the harness evolves around the frozen model.
4 EXPERIMENTS
Experiments evaluate Harness Continual Learning across open-world interaction, textual reasoning, and multimodal perception with sequential task streams that measure capability accumulation, efficiency, retention, and forgetting. HCL improves continual performance with a frozen foundation model, while evaluator-controlled retention tolerances and component ablations expose the stability–plasticity trade-off and complementary harness contributions.
- Evaluation setup: HCL is evaluated on ALFWorld and Minecraft for capability accumulation, reuse, and failure recovery, plus controlled streams that measure harness-level forgetting and stability–plasticity.A single harness evolves sequentially around the same foundation model, with current and previously observed tasks evaluated after each stage.
- ALFWorld: 62.98% final average makes Plasticity-HCL the strongest ALFWorld profile, while Stability-HCL reaches 61.74% and substantially reduces average forgetting.Plasticity-HCL solves all Two-object episodes; Stability-HCL performs best on four of six categories.
- Minecraft: 83 environment actions let HCL complete all 50 Minecraft tasks versus 88 for MemRL and 91 for MemP, while the Static Harness plateaus after 15 tasks.HCL progresses from collection and crafting to persistent assets and coordinated multi-step execution, avoiding repeated diagnosis, crafting, and recovery actions.
- Multimodal perception: 68.92% final average and 0.22 forgetting make Stability-HCL the best multimodal profile, while Plasticity-HCL reaches 67.96%.Both profiles outperform Zero-shot and DGG overall, with the largest gains in detection and grounding; Zero-shot remains stronger only on VQAv2.
- Stability–plasticity control: 0.39 average forgetting at b = 0 rises to 3.45 at b = ∞, while final average peaks at 63.46% for b = 1 and reaches 60.13% when unrestricted.A separate historical test set explains why zero anchor loss does not guarantee zero forgetting.
- Component ablations: 63.41% final average for Full HCL shows complementary component contributions, while removing Memory updates increases forgetting to 0.83 and produces one of the largest performance decreases.Lower forgetting in some ablations reflects restricted adaptation rather than superior harness evolution.
5 CONCLUSION
The paper formulates Harness Continual Learning as a paradigm in which an agent harness, rather than model parameters, evolves through sequential experience. It unifies mutable harness components as evolving state and requires historical retention before deployment.
- Conclusion: Harness Continual Learning (HCL) makes the agent harness—not model parameters—the state that evolves through sequential experience.The harness is treated as a unified evolving state.
- Conclusion: The framework separates candidate generation from evaluation and commitment, making historical retention an explicit deployment condition.This separation governs whether a candidate harness is committed.
- Conclusion: Experiments show that harness evolution can accumulate capabilities and recover from failures.These findings support continual improvement through harness updates rather than model-parameter changes.
A IMPLEMENTATION AND EXPERIMENTAL SETTINGS · A.1 HARNESS AND EVALUATOR BOUNDARIES
The implementation defines explicit access and update boundaries for deployed harness components and the evaluation-only anchor set. Persistent execution-time contents are separated from transient and evaluation-only state, with only committed changes entering deployment.
- A.1 HARNESS AND EVALUATOR BOUNDARIES: Table 7 summarizes access and update boundaries for the deployed harness components and evaluation-only anchor set.The table is titled “Access and update boundaries of the deployed harness and anchor set.”
- A.1 HARNESS AND EVALUATOR BOUNDARIES: Hn contains only persistent execution-time contents, while in, zn, and yn are transient and An remains evaluation-only.This separation distinguishes deployed execution state from transient contents and evaluator-only state.
- A.1 HARNESS AND EVALUATOR BOUNDARIES: Component-level alternatives are evaluated sequentially, and only committed changes enter the deployed harness.Evaluation precedes commitment, preventing uncommitted alternatives from becoming deployed state.
A.2 EXPERIMENTAL SETTINGS
The experiments restrict validation cases and historical anchors to the Evaluator, reserving final test cases for reporting. Stability-HCL and Plasticity-HCL differ in retention budgets, while Minecraft and an independent textual sweep use specialized evaluation settings.
- Validation cases and historical anchors are restricted to the Evaluator, while final test cases are used only for reporting.
- Stability-HCL uses B_n = 0 and Plasticity-HCL uses B_n = ∞; candidates must improve validation performance without introducing an invalid outcome.Discrete metrics require improvement by at least one validation case, whereas continuous metrics require strict improvement.
- Minecraft evaluates skill-level retention with B_n = 0, while the textual sweep uses 40 proposal opportunities, 80 validation cases, 90% format compliance, and B_n ∈ {0, 1, 3, ∞}.The sweep requires two additional correct predictions and varies only B_n.
B COMPONENT ABLATION DETAILS
The ablation variants use a frozen Qwen3.5-4B under shared experimental conditions, while Table 9 defines which persistent harness updates each variant permits. Disabled components remain executable but keep initialized contents, whereas zero-shot removes the structured harness and sequential updates.
- Experimental setup: All ablation variants use frozen Qwen3.5-4B with shared task order, data allocation, evaluation criteria, and update schedule.These conditions are inherited from Section 4.5.
- Experimental setup: Table 9 specifies the permitted persistent updates for each ablation variant.The variants differ in which harness contents may persistently change.
- Ablation semantics: A disabled component remains available during execution but retains its initialized contents throughout the stream.Disabling therefore prevents persistent updating rather than removing execution-time availability.
- Ablation semantics: Zero-shot evaluates the frozen model without the structured HCL harness or sequential updates.It serves as the condition without the structured harness or stream updates.
B.1 ABLATION CONFIGURATIONS
The ablation configurations vary which harness components permit persistent updates, with fixed components marked separately. Disabling Memory updates also removes downstream skill creation from Abstract Memory.
- Update scope: Ablation variants differ by update scope: ✓ permits persistent component updates, whereas × keeps the component fixed.This configuration isolates the effects of allowing or preventing persistent updates.
- Memory updates: Disabling Memory updates removes a source of new skills distilled from Abstract Memory.The resulting variant captures both direct memory adaptation and its downstream effects.
- Memory updates: The Memory-disabled variant measures direct memory adaptation together with its downstream impact on skill formation.Its scope extends beyond memory changes because Abstract Memory can provide reusable skills.
B.2 FULL PER-TASK RESULTS … C.2 MULTIMODAL PERCEPTION
The paper reports full multimodal component ablations and clarifies that commitment counts are trajectory-specific rather than comparable acceptance rates. It also defines fixed anchor-success criteria for textual reasoning and multimodal perception.
- B.2 FULL PER-TASK RESULTS: Full component-ablation results are reported for the controlled multimodal stream, where “Committed” counts candidate updates entering the persistent harness.The results appear in Table 10.
- B.2 FULL PER-TASK RESULTS: Interface updates most visibly affect Caption and VQAv2, while disabling Memory updates primarily degrades Caption.The observed effects are task-dependent across the multimodal stream.
- B.2 FULL PER-TASK RESULTS: Fixing the Router causes its largest decline on VQAv2, whereas Capability updates have a smaller effect in this multimodal stream.The tasks rely less on long-horizon executable skills than the Minecraft curriculum.
- B.2 FULL PER-TASK RESULTS: Commit counts are trajectory-specific because each commitment changes the deployed harness and can affect subsequent feedback and proposals.Variants may not share a proposal sequence, so counts are not directly comparable acceptance rates or measures of update efficiency.
- C ANCHOR SUCCESS CRITERIA: Tables 11–13 define the fixed task-specific criterion q(H, a) in Eq. (10), applied to the same raw input under Hn and eHn+1.This criterion anchors success evaluation across task settings.
- C.1 TEXTUAL REASONING: Table 11 presents anchor success criteria for textual reasoning.It specifies the criteria for the textual-reasoning setting.
- C.2 MULTIMODAL PERCEPTION: Table 12 presents anchor success criteria for multimodal perception.It specifies the criteria for the multimodal-perception setting.
C.3 INTERACTIVE ENVIRONMENTS
Interactive-environment evaluation uses anchor success criteria and counts harness-level losses when previously successful anchors cross below their success thresholds. A RefCOCO IoU decline from 0.68 to 0.41 therefore counts as one loss, while improvement elsewhere does not offset it.
- Anchor success criteria define the evaluation basis for interactive environments.
- Historical loss counts an anchor only when it succeeds under Hn but fails under eHn+1.This criterion is specified by Eq. (11).
- 0.68 to 0.41 RefCOCO IoU contributes one loss after crossing the 0.5 threshold, and another anchor’s improvement does not offset it.