PaperlayerView exact revision on arXiv

Source-linked AI summary

Bayesian-Agent: Posterior-Guided Skill Evolution Across LLM Agent Harnesses

Xiaojun Wu, Cehao Yang, Honghao Liu, Xueyuan Lin, Wenjie Zhang, Zhichao Shi, Xuhui Jiang, Chengjin Xu, Jia Li, Jian Guo

arXiv:2606.08348v2cs.CL

TL;DR

Sparse, context-dependent trajectories make heuristic reflection and raw empirical rates brittle for maintaining reusable agent skills. Bayesian-Agent instead updates posterior belief states from verified trajectories and uses them to guide auditable actions and executable guardrails; results support incremental repair conditionally when harnesses expose verified, reusable failure evidence and a place for the skill layer to act.

  • Problem

    Maintaining reusable agent skills is difficult when evidence is sparse because individual failures may be accidental, context-specific, or reusable defects, making heuristic reflection and raw rates brittle.

  • Method

    Bayesian-Agent represents reusable skills as posterior belief states updated by verified trajectories, then renders uncertainty-aware actions, audit state, guardrails, and failure-mode patches across harnesses.

  • Results

    The results support a conditional operational conclusion: posterior-guided incremental repair is most useful when harnesses expose verified, reusable failure evidence and provide a place for the skill layer to act.

  • Takeaways & Limitations

    Prompt, context, and harness engineering can be treated as interventions on one conditional decision environment rather than unrelated heuristics.

  • Takeaways & Limitations

    Bayesian-Agent is best suited to repeated tasks with verifiers, recurring procedural failures, and controllable skill interfaces, while full online evolution can be order-sensitive under sparse evidence.

Abstract

from arXiv · show

LLM agents increasingly rely on prompts, tools, memory, SOPs, skills, and harness feedback, yet current self-evolution pipelines often update these assets through heuristic reflection or raw success counts. Such updates are brittle when trajectories are sparse, expensive, and context-dependent. We introduce Bayesian-Agent, a native and cross-harness framework that treats reusable agent skills as Bayesian evidence objects. Bayesian-Agent records verified trajectories, maintains posterior beliefs over skill reliability and failure modes, and turns those beliefs into auditable skill actions and model-facing guardrails. This posterior view provides a finite-sample alternative to raw empirical-rate skill updates and frames prompt, context, and harness engineering as inference over the external decision environment. On RealFin-Bench, Bayesian skill evolution matches or improves the raw empirical-rate control and yields large gains on native runs. In the incremental mode, incremental repair improves SOP-Bench from 80\% to 95\%, Lifelong AgentBench from 90\% to 100\%, and RealFin-Bench from 45\% to 65\%. Backend and model-scaling ablations further show that posterior-guided repair can operate across BA native, MiniSWEAgent, and Claude Code, provided the harness produces verifiable task artifacts. The source code is available at https://github.com/DataArcTech/Bayesian-Agent.

1 Introduction

Bayesian-Agent addresses sparse, context-dependent skill maintenance by treating reusable skills as posterior belief states updated from verified evidence. It provides native and cross-harness evolution with auditable actions and model-facing guardrails.

  • Motivation: Sparse failures can make reflection loops and raw empirical rates produce harmful or extreme skill updates.Bayesian-Agent instead retains evidence in a belief state with uncertainty-aware actions.
  • Approach: Bayesian-Agent treats reusable skills as Bayesian evidence objects with priors, posteriors, and finite-sample shrinkage analysis.The framework makes the bias–variance trade-off explicit under sparse verified outcomes.
  • Approach: The framework converts verified trajectories into posterior beliefs, separates audit state from model-facing instructions, and renders executable guardrails and failure-mode patches.This creates a persistent skill memory rather than restarting from a point estimate after each run.
  • Scope: Native and cross-harness evolution is supported through a native backend and adapters for GenericAgent, MiniSWEAgent, and Claude Code.The evidence boundary is shared across execution substrates.
  • Evaluation: The framework is evaluated on SOP-Bench, Lifelong AgentBench, and RealFin-Bench using controls, evolution modes, ablations, and case studies.The evaluation includes Bayesian-versus-raw-frequency comparisons and backend and model-scaling analyses.

2 Related Work

Prior agent research established tool-mediated reasoning, richer execution substrates, reusable skills, and Bayesian uncertainty modeling. Bayesian-Agent combines these directions by treating persistent skills and SOPs as evidence-bearing objects across tasks and harnesses.

  • LLM agents and harness engineering: ReAct and Toolformer established tool-mediated reasoning and action as a core agent pattern.Later systems expanded the execution substrate with memory, coordination, web interaction, and computer-use interfaces.
  • LLM agents and harness engineering: GenericAgent frames tools, memory, context, and self-evolution as engineering components of long-horizon execution.Bayesian-Agent shares this harness-centric perspective.
  • Bayesian and evidence-guided agent optimization: Bayesian-Agent extends these lines by modeling persistent skills and SOPs as evidence-bearing objects maintained across tasks and execution substrates.Its emphasis is on the skill layer rather than individual problem solutions.
  • Self-evolving agents and skills: Reflexion and ExpeL convert trajectories into verbal feedback or experiential knowledge, while Voyager builds reusable skill libraries.Recent work also studies procedural skills, SOPs, computer-use skills, and evolving memory.
  • Bayesian and evidence-guided agent optimization: Bayesian optimization allocates expensive evaluations by maintaining beliefs over uncertain outcomes.Probabilistic modeling and calibration provide complementary tools for representing uncertainty.

3 Method

The method places a Bayesian evidence layer between an execution harness and model-facing skill context. It updates posterior beliefs from verified trajectories, selects restricted auditable actions, and supports full or incremental evolution across backends.

  • Architecture: The harness supplies verified trajectories, while the Bayesian layer updates a skill registry, selects an action, and returns executable context.Audit state remains separate from model-facing text.
  • Problem formulation: The inference environment combines prompt and skill text, retrieved context, tool interface, and verifier feedback.These components are treated as one conditional decision environment around a fixed model.
  • Problem formulation: Bayesian-Agent estimates reusable-skill reliability from externally verified Bernoulli outcomes within task-and-harness feature cells.Optimization acts on external conditions through a restricted intervention set rather than changing model parameters.
  • Finite-sample shrinkage: The prior is a regularizer for sparse evidence, not a guarantee of better performance.The analysis assumes conditionally exchangeable verified Bernoulli outcomes, positive prior pseudo-counts, and squared loss.
  • Finite-sample shrinkage: Posterior shrinkage trades prior-directed bias for lower sampling variance, with lower fixed-parameter squared risk over an interior reliability interval.Empirical rates can remain preferable near boundaries when the prior is mismatched.
  • Evidence model: The system extracts task, cost, timing, verifier-failure, and metadata features, using post-execution evidence for audits and repairs.Only pre-execution features are queried before the current run.
  • Evolution modes: Full mode updates an initially empty registry online, whereas incremental mode initializes from an existing verified run and reruns only failed tasks.Incremental mode measures plug-in repair.
  • Posterior-guided skill actions: Posterior states drive explore, retire, patch, split, or compress actions, while model-facing context contains guardrails and repeated failure-mode patches.Posterior numbers remain in the audit state, and thresholds are fixed implementation defaults.

4 Experiments

The experiments compare Bayesian-Agent with raw empirical-rate evolution, test full and incremental repair across benchmarks and harnesses, and examine model scaling. Results favor posterior-guided incremental repair when failures are recoverable and harnesses produce verifiable artifacts, while full evolution and scaling remain conditional.

  • Bayesian versus raw frequency: 28/40 tasks were solved by Bayesian-Agent for both DeepSeek models, versus 15/40 and 18/40 for raw frequency in the native RealFin-Bench control.The comparison ran both methods from scratch on 40 RealFin tasks under otherwise fixed conditions.
  • Main benchmark results: 95%, 100%, and 65% were reached by BA-Inc on SOP-Bench, Lifelong AgentBench, and RealFin-Bench, respectively.With flash, BA-Inc improves the corresponding baseline results of 80%, 90%, and 45%.
  • Main benchmark results: BA-Full improved SOP-Bench from 80% to 95% and RealFin-Bench from 45% to 52% with flash, but Lifelong AgentBench fell from 90% to 85%.The results show that online evolution from scratch is not monotonic across benchmarks.
  • Backend ablation: Incremental repair improved RealFin-Bench across native, MiniSWEAgent, and Claude Code backends, with flash/pro results of 72.5%/77.5%, 70%/80%, and 87.5%/75%.The ablation reports a consistent incremental-repair advantage across the harness-complexity ladder, while full evolution is less stable in some settings.
  • Model scaling ablation: 70.0% RealFin-Bench accuracy was reached by DeepSeek V4 Pro, compared with 55.0% for DeepSeek V4 Flash and 45.0% for Qwen3.5-35B-A3B.The Qwen3.5-122B-A10B result was dominated by missing output files, indicating an adapter-contract failure rather than a clean reasoning-scale test.
  • Operational conclusion: Posterior-guided repair is most useful when harnesses expose verified, reusable failure evidence and provide a place for the skill layer to act.The paper concludes that stronger execution improves completion only when the harness contract turns behavior into valid artifacts.
  • Limitations: Bayesian-Agent is less suitable for one-off subjective tasks, highly nonstationary environments, or failures caused by unavailable tools or data.Full online evolution can also be order-sensitive under sparse evidence, while empirical rates may suffice with plentiful, stable observations.

5 Conclusion

Bayesian-Agent replaces stateless empirical patching with auditable posterior-guided skill updates under sparse verified evidence. The conclusion emphasizes stronger completion, incremental-repair gains, cross-harness evaluation, and a practical boundary around verifiable failures and controllable skill interfaces.

  • Conclusion: Bayesian-Agent treats reusable agent skills and SOPs as posterior belief states rather than stateless empirical patches.Verified evidence updates the belief state and supports inspectable actions.
  • Conclusion: Incremental repair is most effective when repeated procedural failures are verifiable and the harness exposes a controllable skill interface.The paper identifies this as a conditional practical lesson and a boundary for the reported gains.
  • Conclusion: BA-Inc gains were evaluated across native, MiniSWEAgent, and Claude Code backends on an increasing harness-complexity ladder.Table 3 reports baseline / BA-Full / BA-Inc accuracy, with detailed token accounting deferred to Appendix D.

Appendices

The appendices make the Bayesian model, experimental accounting, robustness checks, visual summaries, and task-level traces traceable. They also document fixed policy boundaries and identify limitations, ethics, and AI-use disclosure as dedicated material.

  • Method and policy: Appendix A details evidence features and policy boundaries, while Appendix B gives the finite-sample shrinkage and risk derivation.The deployed evidence model uses compact categorical features, and the policy emits explicit actions such as patch, split, compress, retire, or explore.
  • Experimental supplements: Appendices C–F provide complete token accounting, backend tables, visual benchmark summaries, and RealFin model-scaling analyses.These materials expand the accounting and visualizations underlying the main experimental tables.
  • Robustness and traces: Appendix G tests robustness to within-benchmark task order on the native backend, while Appendix H exposes task-level Bayesian-vs.-raw-frequency failure patterns.Appendix I adds before/after skill-evolution traces and full model-facing skill texts.
  • Limitations and disclosure: Appendix J contains limitations, ethics, and AI-use disclosure.The appendix organization explicitly separates these topics from the modeling and benchmark supplements.

B Finite-Sample Risk Derivation for Sparse Skill Cells

The derivation compares raw empirical success rates with Beta-shrunk posterior means for sparse skill cells, making their variance–bias trade-off explicit. It connects this finite-sample analysis to posterior-guided harness decisions and cost accounting.

  • Estimator setup: The analysis compares qF = S/n with the Beta-shrunk posterior mean qB = (α0 + S)/(α0 + β0 + n) for a fixed skill cell.The model assumes conditionally exchangeable outcomes after fixing the model, backend, skill version, and feature cell.
  • Risk comparison: qB has lower sampling variance but is biased toward the prior mean α0/A, whereas qF avoids prior bias but can be unstable with sparse evidence.This is the stated variance–bias trade-off between posterior shrinkage and empirical rates.
  • Finite-sample result: For n = 1, the shrinkage estimator’s lower-risk interval is approximately [0.092, 0.908], approaching [0.146, 0.854] as n grows.The empirical rate remains preferable when true reliability is extremely close to 0 or 1 and the prior is poorly matched.
  • Decision criterion: The posterior mean is the coherent reliability estimate under posterior squared loss, while fixed-θω sampling risk describes when shrinkage also lowers repeated-sample mean-squared error.The posterior-loss criterion and fixed-θω analysis support related but distinct claims.
  • Harness decisions: The deployed policy uses the posterior state with thresholds for patch, split, compress, and retire before durable skill edits.The direct control tests whether this posterior-versus-empirical trade-off improves task completion.
  • Cost accounting: Cumulative BA-Inc costs are 1.55M, 774k, and 6.26M tokens on SOP-Bench, Lifelong AgentBench, and RealFin-Bench with deepseek-v4-flash.Cumulative cost includes baseline tokens, unlike repair-only accounting.

D Full-Sample Backend Experiment Details

The backend experiment details separate full-sample execution costs from incremental repair costs across BA native, MiniSWEAgent, and Claude Code. Efficiency uses total tokens for full runs and repair-only tokens for incremental runs.

  • Backend comparison: The backend-complexity ladder comprises BA native, MiniSWEAgent, and Claude Code.Tables 6–8 provide the complete full-sample backend ablation.
  • Efficiency accounting: Full and baseline efficiency is solved tasks per million total tokens, while incremental efficiency is repaired successes per million repair-only tokens.A dash indicates that all baseline tasks were solved and no repair was required.
  • Visual summary: The visual DeepSeek-backbone summary compares accuracy for GA, BA-Full, and BA-Inc, then reports BA-Inc’s gain over GA for non-zero repair settings.The figure complements the aggregate values in Table 2.

F Model-Scaling RealFin Experiment Details

The model-scaling ablation holds task, backend, and optimization protocol fixed while varying model/provider configuration. Its results support a conditional scaling claim tied to verifiable artifact production.

  • Experimental design: The ablation fixes MiniSWEAgent, Bayesian full evolution, RealFin-Bench’s 40 tasks, verifier, and execution limits while varying model/provider configuration.This isolates model-and-adapter behavior under a common protocol.
  • Scaling result: RealFin accuracy increases from 45.0% to 55.0% and 70.0% along qwen3.5-35b-a3b, deepseek-v4-flash, and deepseek-v4-pro.The paper identifies this sequence as the cleanest positive scaling signal.
  • Failure interpretation: The qwen3.5-122b-a10b row has 37 failures with missing output files, identifying artifact production and verification alignment as the dominant improvement opportunity.The interpretation is explicitly framed as an adapter-contract bottleneck.
  • Scope boundary: The ablation supports a conditional claim that stronger models help Bayesian skill evolution when the harness turns behavior into verifiable task artifacts.Parameter count is only one factor in the interpretation.

G Robustness to Task Execution Order

Task-order reruns test whether Bayesian-versus-empirical comparisons depend on execution order, while accompanying traces show how evidence becomes inspectable skill edits. The results indicate stable benchmark-level performance with a small RealFin fluctuation.

  • Order-sensitivity design: Each benchmark reruns the same tasks in a different order with the native backend, deepseek-v4-flash, verifier, and full-evolution protocol fixed.Fresh skill state makes task order the only changed factor.
  • Robustness result: Bayesian-Agent remains at 100% on SOP-Bench and Lifelong AgentBench, while RealFin-Bench changes from 28/40 to 27/40.The frequentist control retains its original completion rate on all three benchmarks.
  • Aggregate comparison: Across 80 tasks, Bayesian accuracy changes from 85.0% to 83.8%, while its reordered advantage remains 17.5 percentage points.The reported fluctuation is small at the aggregate level.
  • Mechanism: Task order can alter the evidence-generating path because early skill actions change later contexts and outcomes, despite permutation-invariant update arithmetic.Both S/n and the Beta posterior are invariant to permutations of a fixed outcome multiset.
  • RealFin failure modes: RealFin traces show raw-frequency failures involving missing output files or blank OHLCV conversion crashes, whereas Bayesian runs retain guardrails for artifact and format validation.The Bayesian run is more expensive in this comparison.
  • Interpretability traces: Benchmark traces preserve links from verifier outcomes to evidence features, posterior audits, and model-facing edits such as patches, compressions, and retire/redesign actions.Figures 9–11 show representative before-and-after skill texts.

J.1 Limitations

Bayesian-Agent’s coverage and evaluation scope are limited, and deployment still requires oversight despite harness-side reliability mechanisms. The reported evidence also reflects backend-specific trade-offs and task-execution conditions.

  • Scope boundaries: Backend and model coverage remain limited, with cross-harness evaluation spanning the native backend, MiniSWEAgent, and Claude Code but not arbitrary harness/model pairs.The default backend uses a factorized categorical evidence model with Laplace smoothing rather than full Bayesian structure learning or model selection.
  • Evaluation trade-offs: Model scaling on RealFin-Bench is evaluated under MiniSWEAgent, with Qwen 122B-A10B treated as an adapter-contract anomaly rather than part of the main scaling path.Failed tasks are decomposed into missing requested output files versus created files whose verifier details fail.
  • Evaluation conditions: Task-order robustness is measured on the Bayesian-Agent native backend using deepseek-v4-flash, reporting solved tasks, accuracy changes, and advantage over the reordered frequentist control.The table defines |∆| as the absolute accuracy change from reordering and ∆BA−F as Bayesian-Agent’s reordered accuracy advantage.
  • Deployment considerations: Deployment still requires permission checks, logging, and human oversight because risks remain inherited from the base model, harness, tools, and benchmark data.Skill repair may reduce repeated operational failures but can also make an agent more persistent; the system preserves posterior audits and exposes failure-mode patches for inspection.
  • Evaluation trade-offs: Bayesian-Agent spends more inference budget than the cheaper raw-frequency controller while retaining posterior-weighted execution guardrails on sparse RealFin-Bench tasks.The comparison concerns native RealFin-Bench completion, where raw-frequency failures often involve missing output files or blank-field crashes.
Loading 2606.08348v2…