PaperlayerView exact revision on arXiv

Source-linked AI summary

Combee: Scaling Prompt Learning for Self-Improving Language Model Agents

Hanchen Li, Runyuan He, Qizheng Zhang, Changxiu Ji, Qiuyang Mang, Xiaokun Chen, Lakshya A Agrawal, Wei-Liang Liao, Eric Yang, Alvin Cheung, James Zou, Kunle Olukotun, Ion Stoica, Joseph E. Gonzalez

arXiv:2604.04247v1cs.AIcs.CLcs.LG

TL;DR

Prompt learning methods degrade when scaled naively to many parallel agent traces because aggregation overloads the context and loses fine-grained knowledge. Combee combines parallel-scan aggregation, augmented shuffling, and dynamic batch-size control, achieving substantial speedups while maintaining or improving quality across four benchmark settings at comparable cost.

  • Problem

    Naive parallel prompt learning suffers context overload, creating a trade-off in which larger batches reduce training time but degrade the learned context and quality.

  • Method

    Combee combines parallel-scan aggregation, augmented shuffling, and dynamic batch-size control to scale prompt learning across many agent trajectories.

  • Results

    Combee maintains or improves quality across AppWorld, Terminal-Bench 2.0, FiNER, and Formula while delivering substantial speedups at comparable cost, including over 17× faster training on Terminal-Bench 2.0.

  • Takeaways & Limitations

    Combee enables prompt learning to use high parallelism without the quality degradation observed in naive batching across the evaluated tasks and frameworks.

  • Takeaways & Limitations

    Experiments cover only ACE and GEPA, leaving integration with prompt-learning methods that maintain structurally different context artifacts for future validation.

Abstract

from arXiv · show

Recent advances in prompt learning allow large language model agents to acquire task-relevant knowledge from inference-time context without parameter changes. For example, existing methods (like ACE or GEPA) can learn system prompts to improve accuracy based on previous agent runs. However, these methods primarily focus on single-agent or low-parallelism settings. This fundamentally limits their ability to efficiently learn from a large set of collected agentic traces. It would be efficient and beneficial to run prompt learning in parallel to accommodate the growing trend of learning from many agentic traces or parallel agent executions. Yet without a principled strategy for scaling, current methods suffer from quality degradation with high parallelism. To improve both the efficiency and quality of prompt learning, we propose Combee, a novel framework to scale parallel prompt learning for self-improving agents. Combee speeds up learning and enables running many agents in parallel while learning from their aggregate traces without quality degradation. To achieve this, Combee leverages parallel scans and employs an augmented shuffle mechanism; Combee also introduces a dynamic batch size controller to balance quality and delay. Evaluations on AppWorld, Terminal-Bench, Formula, and FiNER demonstrate that Combee achieves up to 17x speedup over previous methods with comparable or better accuracy and equivalent cost.

2 Background and Motivation

Prompt learning updates reusable context artifacts from agent trajectories during deployment rather than fixing prompts in advance. Scaling its generate-reflect-update loop is challenging because naive aggregation trades the speed of parallelism for substantial loss of specific, high-value knowledge.

  • Prompt Learning: Prompt learning extracts task-relevant knowledge from trajectories, tool traces, or documents into reusable context artifacts without updating model weights.The studied generate-reflect-update loop executes tasks, reflects on trajectories, and updates a shared artifact for future iterations.
  • Relationship to Prompt Engineering: Unlike prompt engineering, prompt learning treats the prompt or context artifact as a living object that evolves during deployment through generate-reflect-update cycles.Prompt engineering instead crafts a fixed prompt manually or through offline search before inference-time deployment.
  • Failure of Naive Scaling: The degradation reflects lossy compression: aggregators retain broad patterns while discarding specific insights that disproportionately drive downstream accuracy.The effect persists even when all reflections fit within the model’s 128K context window, ruling out truncation as the simple explanation.
  • Quantitative Evidence: 264 to 21 Formula updates and 246 to 11 FiNER updates accompany accuracy drops from 87.0% to 72.5% and 76.0% to 70.6% as batch size rises to 100.Figure 2 shows monotonically decreasing context updates and accuracy with increasing batch size across both tasks.
  • Motivation: Naive scaling creates a speed–quality tension: large batches reduce wall-clock time but can collapse prompt-learning quality toward the no-context-learning baseline.Small batches offer only modest speedups, while the large batches needed for meaningful acceleration overload aggregation.

3 Design of Combee

Combee extends prompt learning to high-parallelism settings while maintaining quality by combining parallel scan aggregation, augmented shuffling, and dynamic batch-size control. Its Map-Shuffle-Reduce design aggregates reflections hierarchically, preserves dense information, and selects batch sizes to balance speed with learning quality.

  • Overview: Combee extends prompt learning from prior methods (Zhang et al., 2025a; Agrawal et al., 2025; Li et al., 2024) to high parallelism while maintaining quality.The framework is designed for scalable prompt learning through parallel generation and adaptation.
  • Parallel Scan Aggregation: Combee uses Map-Shuffle-Reduce: parallel agents generate reflections, augmented shuffling redistributes them, and hierarchical parallel scans combine them without context overload.Unlike naive scaling, which feeds all reflections into one prompt update, Combee first aggregates subgroup updates and then combines them into a single update.
  • Augmented Shuffling: Augmented shuffling duplicates each reflection p times, with default p = 2, then shuffles the expanded set so dense insights receive multiple learning opportunities at large batch sizes.This improves robustness of the parallel learning pipeline despite increased batch size, following the self-consistency principle (Wang et al., 2022).
  • Dynamic Batch Size Controller: Dynamic batch-size control profiles candidate delays, fits a power-law delay curve, and selects a batch size near diminishing marginal delay reductions.Parallel scans and augmented shuffling preserve quality across batch sizes, so selection primarily balances reduced per-epoch delay against diminishing returns and possible degradation from excessive sizes.

4 Results

Across AppWorld, Terminal-Bench 2.0, FiNER, and Formula, Combee scales ACE and GEPA prompt learning with comparable or better quality, substantially lower training delay, and comparable cost. Its parallel scan aggregation, augmented shuffling, and dynamic batch control prevent context-overload degradation and remain robust across tasks, settings, and model families.

  • Agentic benchmarks: 12× speedup at AppWorld batch 40 accompanies the highest average score and SGC across methods at comparable cost.Combee retains 6,887 playbook tokens versus 526 for naive batch 40, indicating that parallel scan aggregation preserves more reflection information.
  • Agentic benchmarks: Over 17× lower training time lets Combee at Terminal-Bench batch 30 recover most sequential quality, while naive larger batches degrade from context overload.Combee’s playbook contains 8,023 tokens, closer to the sequential baseline’s 9,067 than other batch settings.
  • Finance benchmarks: Combee consistently reaches the finance benchmarks’ Pareto frontier, matching or exceeding the best fixed-batch accuracy while training significantly faster than quality-matching setups.The quality–speed trade-off persists across GEPA and ACE, with large batches suffering context overload; on GEPA FiNER, batch 100 falls below the base LLM.
  • Ablation and robustness analysis: Ablations show that dynamic batch control avoids unnecessary delay, while augmented shuffling prevents quality fluctuations and significantly outperforms plain parallel scanning across subgroup sizes.Without dynamic control, fixed batches can require a small batch and increase delay with little quality change; without augmented shuffling, quality is significantly worse.
  • Ablation and robustness analysis: Combee transfers across model families, with GPT-OSS 120B showing superior quality over fixed-batch baselines at much reduced training time.The batch size controller and parallel scan aggregator transfer seamlessly across model families on Formula.

5 Related Work

Prior work studies inference-time external memory for improving LLM agents and parallel agent systems for concurrent task solving. Learning to Share and modern coding agents further explore information sharing, parallel decomposition, and concurrent execution to reduce redundancy and improve throughput.

  • Memory Mechanism for LLMs and Agents: Prompt learning uses external non-parametric memory to help language models and agents improve over time.Dynamic Cheatsheet demonstrates compact, evolving textual memory that accumulates reusable guidance from past experience, while ReasoningBank investigates a related direction.
  • Parallel Agents: Parallel-agent research coordinates multiple agents or workers to solve tasks concurrently.This line of work includes Hong et al. (2023) and Qian et al. (2024a).
  • Parallel Agents: Learning to Share (Fioresi et al., 2026) studies sharing useful intermediate information while avoiding redundant computation.Modern agentic coding systems, including Claude Code, OpenHands (Wang et al., 2024b), and SWE-agent (Yang et al., 2024), increasingly use parallel task decomposition and concurrent execution to improve throughput on complex workloads.

6 Conclusion

Combee is a scalable context-learning framework that enables parallel agents to efficiently acquire and consolidate knowledge. It combines parallel-scan aggregation, augmented shuffling, and dynamic batch-size control to address context overload while delivering substantial speedups across multiple benchmarks and domain-specific tasks.

  • Combee enables parallel agents to efficiently acquire and consolidate knowledge through scalable context learning.
  • Its parallel-scan aggregation, augmented shuffling, and dynamic batch-size control address context overload from naively scaling existing context-learning methods.
  • Combee delivers substantial speedups across AppWorld, Terminal-Bench 2.0, FiNER, and Formula.

Reproducibility Statement

The study documents its experimental setup and uses public benchmarks to support reproducibility, with source code planned for release upon publication.

  • The paper specifies its language models, datasets, and hyperparameters so readers with appropriate compute resources can reproduce the results.
  • All experiments use publicly available benchmarks.
  • The source code will be released upon publication.

A Use of Large Language Models (LLMs)

The work uses LLMs to evaluate proposed context-adaptation methods, while restricting paper-preparation use to writing polish and also using Claude Code and Cursor during experiment development.

  • LLMs are used for empirical evaluation of the proposed context-adaptation algorithms and system frameworks.
  • For paper preparation, LLMs are used only to polish writing, not to generate new text from scratch.
  • Claude Code and Cursor are used during development of the empirical experiments.

B Limitations and Future Work

Although Combee shows consistent improvements across its evaluation settings, its validation currently covers only ACE and GEPA, leaving integration with structurally different context artifacts for future work.

  • Limitations: Combee’s experiments focus on two base prompt learning frameworks, ACE and GEPA, despite the framework being designed as framework-agnostic.Both frameworks follow the generate-reflect-update paradigm.
  • Future Work: Future work will validate Combee with methods that maintain structurally different context artifacts, such as program libraries or retrieval-augmented skill stores.

C Extended Problem Formulation

The paper formalizes prompt learning as iterative execution, reflection, and context updating, then extends it to a Map–Reduce pipeline with parallel agents whose feedback updates a global context.

  • Single-threaded formulation: Single-threaded prompt learning executes an agent in environment E, reflects on trajectory τ_t to obtain feedback F_t, and updates context C_t to C_t+1.The process is defined as τ_t ∼ Exec(A, E | C_t), F_t = Reflect(τ_t, E | C_t), and C_t+1 = Update(C_t, F_t).
  • Prompt learning at scale: Prompt learning at scale adopts a Map–Reduce approach that launches parallel agents A_1, A_2, …, A_bs to interact with the environment and collect feedback.Each parallel agent produces a corresponding feedback signal for the iteration.
  • Prompt learning at scale: The collected feedback signals are aggregated through an aggregation function to update the global agent context.This aggregation-and-update step is the reduction stage of the scaled pipeline.

D Analogy to Distributed Training

The paper frames parallel prompt learning as analogous to distributed training: workers learn locally from separate data or tasks, then aggregate signals into globally useful adaptation. The analogy motivates treating contexts as scalable learning artifacts and importing distributed-systems principles without changing model parameters.

  • D Analogy to Distributed Training: Parallel prompt learning mirrors distributed training by having multiple workers independently process tasks and periodically aggregate locally generated learning signals.Distributed training aggregates local gradients, whereas prompt learning aggregates contextual adaptations rather than parameter updates.
  • D Analogy to Distributed Training: Agents acquire task-relevant rules, heuristics, plans, summaries, or structured artifacts during inference, which can be consolidated, accumulated, or shared across workers.These artifacts include prompts, memories, files, and playbooks, allowing downstream agents to benefit from others’ experience.
  • D Analogy to Distributed Training: Contexts function like gradients: accumulation resembles aggregation, while curation, compression, and distillation parallel averaging, compression, and delayed synchronization.This contextual learning process scales learning capacity without modifying model parameters.
  • D Analogy to Distributed Training: The analogy suggests designing large-scale prompt learning around parallelism, aggregation, communication efficiency, and consistency trade-offs.It positions context as a first-class medium for scalable learning.

E Qualitative Examples of Context Overload

The qualitative examples show that context composition matters: a 21-entry context containing finance calculations and error-handling guidance reaches 72.5% test accuracy, compared with 70.6% for an 11-entry context. The examples include both useful calculation formulas and potentially conflicting output-format, timing, compounding, and taxonomy guidance.

  • Context overload examples: The larger context includes formulas for purchasing-power parity, put-call parity, annuities, and interest-rate parity.These entries specify formulas and assumptions such as cash-flow timing and compounding conventions.
  • Context overload examples: The context also contains overlapping guidance on decimal-versus-percentage output, cash-flow timing, compounding assumptions, rounding, and ratio formatting.Several entries warn that the expected answer format and financial convention must be inferred from the question.
  • Context overload examples: The shorter context includes specialized taxonomy guidance for credit facilities, acquisition equity instruments, share-based compensation, debt components, and share issuance tags.These examples emphasize distinctions between related reporting concepts and tags.

CONTEXT CLUES & INDICATORS

The context contains 264 entries of tagging guidance, financial formulas, output conventions, and error-prevention rules, achieving 87.0% test accuracy. It emphasizes selecting context-specific tags and validating calculation assumptions, timing, units, and formatting.

  • Context clues map phrases such as “borrowings” and “available” to specific tags while favoring precise tags over generic alternatives.Examples distinguish total acquisition consideration from cash components and long-term debt fair value from generic debt fair value.
  • 87.0% test accuracy is achieved using 264 context entries.
  • Financial guidance specifies formulas, unit consistency, timing assumptions, rounding, decimal conversion, and plain-floating-point output across numerous calculations.Examples cover ratios, NPV variants, valuation models, option pricing, parity, and profitability measures.
  • Error-prevention rules require validating formula arrangements and payment timing rather than applying standard annuity, parity, or output conventions automatically.The guidance also warns against unnecessary trailing zeros and percentage formatting when decimal output is required.

COMMON MISTAKES TO AVOID

The section warns against mismatching US GAAP tags to the context, including errors involving specificity, taxonomy availability, transaction type, and financial-instrument meaning. It emphasizes distinguishing similar disclosures such as expenses versus cash payments, balances versus transactions, and facility capacity versus debt amounts.

  • Tag selection: Use exact, available taxonomy tags and select the most specific tag justified by the context, without abbreviating, generalizing, over-specifying, or choosing unavailable tags.This includes verifying tags against the provided taxonomy list and options.
  • Financial interpretation: Distinguish financial meaning and disclosure type: do not confuse fair-value hierarchy amounts with fair values, derivative instruments with independent instruments, or balance-sheet amounts with transactions.The warnings address underlying-instrument relationships, carrying values, and outstanding balances rather than transactional events.
  • Debt and credit facilities: Avoid using DebtInstrumentFaceAmount for revolving facilities, repayment transactions, or term-loan amounts embedded in credit-facility structures when capacity or repayment tags are appropriate.A revolving credit facility indicates a line-of-credit arrangement, while principal repayment describes an action rather than a measurement.
  • Expenses and fees: Separate expense recognition, cash payments, restructuring depreciation, and commitment-fee scope instead of applying tags based only on superficially similar wording.Examples include distinguishing paid rent from operating lease expense, restructuring-related depreciation from regular depreciation, and facility-wide fees from unused-capacity fees.
  • Ownership and equity disclosures: Match ownership, equity, and share-based compensation tags to the precise arrangement, distinguishing subsidiaries, joint ventures, collaborative structures, warrants, RSUs, ESPP shares, and stock issuances.The cautions cover voting-interest acquisitions, consolidated subsidiaries, equity-method ownership, treasury-stock sales, and new issuances that increase outstanding shares.

OTHERS

The section provides taxonomy guidance for selecting XBRL concepts in business acquisition and combination contexts.

  • Use BusinessAcquisitionEquityInterestsIssuedOrIssuableNumberOfSharesIssued when shares acquire ownership interests in another entity, such as through an exchange offer.
  • For cash payments in business acquisitions, use PaymentsToAcquireBusinessesGross when the context says “cash paid” or “purchase price” without mentioning netting against cash acquired.
  • The section also flags the need for care in business combination transactions.
Loading 2604.04247v1…