PaperlayerView exact revision on arXiv

Source-linked AI summary

Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents

Haoyu Zhao, Sitian Tu, Yuxuan Huang, Huichi Zhou, Zhangyue Yin, Jiahao Huang, Jingyuan Xu, Tianmiao Shi, Jin Shi, Nianxi Zhang, Yangmiao Zhang, Zhuolang Chan, Huiyan Hu, Chenchao Sheng, Yaqing Luo, Yue Chen, Huaxiu Yao, Meng Fang, Jun Wang

arXiv:2610.33956v1cs.LGcs.AIcs.CLcs.CV

TL;DR

Existing agent benchmarks often assess individual task performance rather than improvement from accumulated experience. CLE evaluates experience reuse across sequential professional workflows, and feedback plus revision improves most tested configurations while full ladder completion remains limited.

  • Problem

    Most existing agent benchmarks assess individual task performance or retrieval over histories rather than whether accumulated experience improves subsequent task execution.

    Source: §1 · p. 2

  • Method

    CLE evaluates agents on 120 five-level professional task ladders, where passing unlocks later assignments that build on earlier experience and submissions are checked against human-authored rubrics.

    Source: §1 · p. 2

  • Results

    16 of 17 configurations improve overall completion with feedback and revision, with a median increase of 5.33 percentage points; the best configuration completes 47.67% of levels in Trial 2.

    Source: §4.2 · p. 17

  • Takeaways & Limitations

    CLE provides a setting for developing and evaluating methods that acquire, retain, and reuse experience across realistic task ladders.

    Source: §6 · p. 20

    The Trial 1–Trial 2 comparison cannot isolate the contribution of knowledge retained across tasks; a matched comparison resetting harness state is needed.

    Source: §4.2 · p. 17

Abstract

from arXiv · show

Continual learning is the ability of LLM agents to improve over time by acquiring, refining, and applying knowledge from experience while retaining useful prior knowledge. It is essential for agents learning over long time horizons, yet current evaluations largely assess individual task performance rather than improvement through accumulated experience. We introduce Continual Learning Exam (CLE), a large-scale benchmark developed by 50+ domain experts to measure experience-driven improvement in realistic professional workflows. CLE comprises 120 human-authored tasks, each organised as a five-level ladder, yielding 600 levels spanning 89 professional roles across 12 occupational domains. Agents must pass each level to unlock the next, with later levels building on experience accumulated during earlier assignments. An evaluation agent assesses each submission against human-authored rubrics. We compare a single-submission setting with one that provides diagnostic feedback after failure and allows one revision. Across 17 harness-model configurations using frontier LLMs, feedback and revision yield a median completion gain of 5.3 percentage points. Yet the best-performing system clears only 47.7% of levels and completes all five levels in just 23.3% of tasks, highlighting the difficulty of sustained learning from experience for existing frontier LLMs. As such, the dataset can also be used to post-train LLMs to strengthen their continual learning capabilities.

1. Introduction

Existing agent benchmarks largely score isolated task performance rather than whether experience improves later work. CLE evaluates experience reuse across sequential professional assignments, and results show that feedback and revision help but sustained completion remains difficult.

1. Introduction

  • General capability benchmarks score independent problems, memory benchmarks test retrieval over histories, and persistent workflow state alone does not show improvement through experience.CLE is designed to evaluate cumulative improvement through sequential attempts with controlled feedback conditions.Source: §1 · p. 2
  • CLE comprises 120 five-level task ladders spanning 89 roles and twelve domains, with later assignments drawing on knowledge and experience from earlier work.The benchmark was created by 50+ industry experts and totals 600 levels.Source: §1 · pp. 2–3
  • CLE’s tasks connect assignments through five transferable knowledge forms: tools and skills, data dependencies, strategy, user context, and domain knowledge.These forms cover reusable procedures, prior information and artefacts, effective approaches, user preferences, and professional rules and conventions.Source: §1 · p. 2
  • The benchmark supplies 23,981 real-world input files and 19 stateful mock applications exposing 1,024 APIs for realistic workflow execution.Its evaluation agent checks submissions against 16,125 human-authored and human-verified rubric criteria.Source: §1 · pp. 2–3
  • In Trial 2 (K=2), across 17 configurations, Claude Code with Opus 5 completed only 47.67% of levels despite rubric-based diagnostic feedback and one chance to revise.Source: §1 · p. 3

2. Background and Preliminaries

The background frames continual learning as updating a learner from experience while retaining useful knowledge, then distinguishes levels of learning autonomy and related mechanisms. CLE’s task ladders instantiate sequential assignments where retained knowledge can be reused and refined.

2.1. Continual Learning

  • Continual learning updates a learner from accumulated experience to acquire knowledge while mitigating forgetting and supporting transfer to later tasks.In classical neural networks, updates adapt model parameters and may use retained state such as stored examples or parameter-importance estimates.Source: §2.1 · p. 4
  • For LLM agents, updates can change model parameters, persistent harness state, or both; harness state can retain memories, skills, and other knowledge for later tasks.Experience may include interaction records, tool results, supervision, rewards, or feedback.Source: §2.1 · pp. 4–5
  • CLE organizes each task as five sequential professional assignments, with passing a level unlocking the next and retained knowledge reused or refined across assignments.A level can contain multiple learning updates, which are distinct from the level index.Source: §2.1 · p. 5, Figure 2 · p. 5

2.2. Capability Hierarchy

  • The five cumulative capability tiers range from rule-governed learning through learning autonomy and cross-domain adaptation to research design and self-selected research problems.Each tier requires the capabilities of all preceding tiers.Source: §2.2 · p. 6, Figure 3 · p. 6

2.3. Related Concepts and Learning Mechanisms

  • Online learning names the sequential setting, while test-time training specifies optimization from test-time inputs; either can contribute to continual learning when useful adaptations persist.Online learning may update parameters, persistent state, or both.Source: §2.3 · p. 7
  • Case-based reasoning reuses retrieved precedents through adaptation, evaluation, and retention, whereas test-time training specifies an optimization step.An LLM agent can update its case store and selection policy while keeping its backbone fixed.Source: §2.3 · p. 7
  • Recursive self-improvement requires retained changes to how later improvements are generated, evaluated, or implemented; ordinary online learning, test-time training, or case reuse alone does not establish it.The distinction is between accumulating task knowledge and improving the process that acquires and refines that knowledge.Source: §2.3 · p. 7

3. Continual Learning Exam

CLE structures professional work as sequential, gated task ladders so agents must reuse knowledge across assignments. It combines rubric-based evaluation, persistent work environments, and expert-reviewed tasks to support evaluation of that experience-driven learning.

3.2. Agentic Evaluation

  • K=1 permits one submission without diagnostic feedback; K=2 adds rubric-category feedback after failure and one resubmission.A failed level blocks progression unless the agent passes within its submission budget.Source: Figure 4 · p. 8, §3.2 · p. 11

3.1. Benchmark Design

  • Each task is a five-level ladder of distinct assignments in one role and context, linked by transferable tools, data, strategies, user context, and domain knowledge.Later levels can draw on information, constraints, and procedures from earlier work; rubrics specify outcomes and constraints.Source: §3.1 · p. 8

3.3. Problem Formulation

  • The actor–environment interaction is modeled as a POMDP, with the harness using action, observation, and reward experience to update retained state for later actions.A level pass earns reward 1, and the run’s total reward equals the number of levels passed.Source: §3.3 · pp. 11, 13, 14

3.4. Data Construction

  • Domain experts draft tasks from their professional workflows; each draft undergoes an automated review and independent review by two randomly assigned human reviewers before acceptance.Rejections at either stage return the task for revision.Source: §3.4 · pp. 14–15

3.5. Data Overview

  • CLE contains 120 five-level tasks across 12 domains, with 600 levels and 508 English and 92 Chinese levels.The benchmark includes 23,981 real-world files spanning 77 extensions and 16,125 checklist items, averaging 26.9 per level; checklist counts increase across levels.Source: §3.5 · p. 15

4. Experiment

The evaluation compares 17 agent configurations under two submission budgets on CLE’s sequential task ladders. Feedback and revision usually improve completion, but results remain limited and vary across harnesses, later levels, and resource measures.

4.1. Evaluation Setup

  • The 17 configurations span four harnesses and eleven backbone models; each ran the fixed sequence of 120 ladders, with Trial 1 allowing one submission and Trial 2 allowing feedback and one revision.The evaluation agent achieved 96.86% rubric-item agreement with adjudicated human labels on a validation sample.Source: §4.1 · pp. 15–16
  • CLE reports cumulative level pass rates, overall completion across 600 levels, and total agent tokens, active runtime, and expenditure in USD.Levels never unlocked are scored as failed; active runtime excludes queueing delays, provider throttling, and cross-machine parallelism.Source: §4.1 · p. 16, Table 1 · p. 17

4.2. Results and Analysis

  • 47.67% of levels were completed by the best configuration, Claude Code with Opus 5, in Trial 2; it completed all five levels in 28 of 120 tasks.Across configurations, median cumulative pass rates fell from 62.50% at L1 to 8.33% at L5.Source: §4.2 · p. 17, Table 1 · p. 17, Figure 9 · p. 18
  • 16 of 17 configurations improved overall completion in Trial 2, with a median increase of 5.33 percentage points; Sonnet 5 gained 13.17 pp versus Opus 5’s 5.50 pp.The comparison combines diagnostic feedback with an additional revision opportunity.Source: §4.2 · pp. 17–18, Figure 9 · p. 18
  • Higher overall completion did not always extend to later levels: both Qwen3.8-Max configurations improved at L1–L3 but not L4–L5.Sonnet 5’s fully completed tasks rose from 8 in Trial 1 to 23 in Trial 2.Source: Table 1 · p. 17, §4.2 · p. 18
  • Hermes-Agent outperformed OpenClaw on all six shared backbones in both trials, with a mean advantage of 11.42 pp in Trial 2.For DeepSeek V4 Pro, Hermes-Agent gained 5.50 pp while OpenClaw declined 1.33 pp.Source: §4.2 · p. 18, Figure 9 · p. 18
  • At 33.17% completion, Codex with Terra cost $83.16 and took 20.20 hours, while Sol cost $210.83 and took 28.20 hours; Sol completed more full tasks, 21 versus 15.Equal overall completion can mask differences in full-task completion.Source: Table 1 · p. 17, §4.2 · p. 18
  • Hermes-Agent with Hy3 achieved 27.33% completion for $16.18 and 80.30 hours, placing it on the cost–completion frontier but not the runtime–completion frontier.Resource totals cover complete evaluation runs, whose workloads vary with levels reached and revisions performed.Source: §4.2 · p. 18, Figure 10 · p. 19

5. Related Work

Related benchmarks assess general task capability, memory and context learning, or experience reuse across task sequences. CLE positions itself as combining professional task execution, accumulated experience, an explicit learning protocol, and auditable grading.

5.1. General Agent Capability Benchmarks

  • General capability benchmarks score individual tasks, while CLE evaluates completion across connected professional assignments where experience can inform later work.Professional workflows add demands involving duration, environment continuity, and outcome verification.Source: §5.1 · p. 19

5.2. Memory and Context Learning Benchmarks

  • Memory and context-learning benchmarks test retention and application of prior information, whereas CLE evaluates knowledge reuse through professional task outcomes and leaves retention choices to each harness.Examples include conversational-history question answering, cross-session reasoning, and application of newly acquired knowledge.Source: §5.2 · pp. 19–20

5.3. Experience Accumulation and Reuse Benchmarks

  • Experience-accumulation benchmarks study transfer, memory use, skill evolution, or adaptation across task sequences; CLE links expert-authored professional assignments through tools, skills, data dependencies, strategies, user context, and domain knowledge.Its harness determines how experience is retained and reused, while a common protocol controls rubric-based feedback and revision.Source: §5.3 · p. 20
  • CLE’s positioning compares benchmarks across task design, learning protocol, and feedback and grading, combining professional task execution with experience reuse and auditable grading.The comparison uses nine criteria defined in the table notes.Source: §5.3 · p. 20

6. Conclusion

CLE evaluates continual learning in realistic human–AI cowork settings through five-level task ladders connecting successive assignments. Across 17 configurations, feedback and revision help most systems, but sustained completion remains difficult.

6. Conclusion

  • 47.67% of levels are completed by the best-performing system, despite diagnostic feedback and an opportunity to revise failed submissions.Feedback and revision improve most configurations, but the gains do not consistently extend to completing entire task ladders.Source: §6 · p. 20
  • CLE comprises 120 human-authored and human-verified tasks, each a five-level ladder connecting successive assignments through reusable information and artefacts.Stateful environments and human-defined rubrics support execution and verification throughout each ladder.Source: §6 · p. 20
  • CLE is presented as a setting for developing and evaluating methods that acquire, retain, and reuse experience across successive assignments.The conclusion identifies this as a direction for addressing the challenge CLE poses for current models and agent harnesses.Source: §6 · p. 20
  • The benchmark comparison distinguishes human-authored or verified tasks, stateful environments, multimodal inputs, tiered tasks, cross-level reuse, configurable attempts, milestone feedback, agentic judging, and audit trails.The comparison marks features as present, partial, or absent; its criteria describe benchmark characteristics rather than performance results.Source: Table 2 · p. 21, §6 · p. 21

B. Comparison of Related Learning Concepts

The appendix compares continual-learning perspectives by what they retain from experience and how they update model parameters or persistent state. It also describes an agent-assisted task-review process followed by independent human review.

B. Comparison of Related Learning Concepts

  • The comparison distinguishes learning settings by what they retain from experience and whether updates target model parameters θ_n or persistent state s_n.Table 3 uses arrows for update targets and equality for fixed components; its memory, reflection, and skill examples keep the backbone frozen.Source: Table 3 · p. 29, B. Comparison of Related Learning… · p. 29
  • Online learning uses sequential prediction, feedback, and updates, while test-time scaling allocates extra inference computation and can retain solutions or lessons for later updates.Curriculum learning sequences examples or tasks so later learning can build on earlier knowledge.Source: B. Comparison of Related Learning… · pp. 30–31
  • Parametric learning incorporates knowledge into model weights, in-context learning uses retained experience in context, and test-time training optimizes on test inputs with adaptations carried forward.The table represents these differences through parameter and state updates.Source: B. Comparison of Related Learning… · p. 30
  • Memory and retrieval-augmented generation make stored experience available later; case-based reasoning adapts prior solutions, while reflection uses feedback to revise knowledge.These approaches provide distinct ways to carry experience into subsequent work.Source: B. Comparison of Related Learning… · p. 31
  • Skill learning retains and adapts reusable procedures, while transfer learning and multi-task learning use knowledge across tasks or domains to support subsequent learning and retention.Meta-learning instead optimizes how a learner adapts using experience across tasks.Source: B. Comparison of Related Learning… · pp. 30–31
  • Experience-driven recursive self-improvement retains modifications that improve later improvement, while a Gödel Machine requires proof of greater expected utility before code changes.The appendix notes that experience can improve the learning mechanism itself.Source: B. Comparison of Related Learning… · p. 31

C. Agent Auto Review with Feedback

  • A reusable review skill derived from human expertise guides the task-review agent in inspecting complete ladders and returning evidence-grounded, actionable revision feedback.The illustrated interface returns a conference-style verdict, comments, and revision suggestions.Source: C. Agent Auto Review with Feedback · pp. 31–32, Figure 11 · p. 32

D. Human–Agent Grading Agreement

The grading-agreement analysis tests rubric-item judgments against human reference labels and checks repeatability across fresh evaluation sessions. The reported results show high agreement under the specified conditions.

D. Human–Agent Grading Agreement

  • 96.86% agreement with human reference labels is achieved by the evaluation agent with DeepSeek V4 Flash across 732 rubric items.The validation sample covers all five levels of 12 tasks and 60 submissions; Table 4 also reports pairwise agreement among human annotators and three agent judges.Source: D. Human–Agent Grading Agreement · p. 32, Table 4 · p. 33
  • Human annotators independently labeled each rubric item and resolved disagreements to establish reference labels for the evaluation-agent comparison.The same fixed submissions and environment evidence were provided to all judges.Source: D. Human–Agent Grading Agreement · p. 32, Table 4 · p. 33
  • 99.46% agreement is achieved across three fresh runs, with 553 of 556 rubric items receiving the same label in every run.The model, tools, rubrics, and submitted evidence were held fixed across runs.Source: D. Human–Agent Grading Agreement · p. 33, Table 5 · p. 33

E. Additional Benchmark Data Analysis

The additional analysis reports how task checklists and briefs change across levels and how often later-level specifications or rubrics refer to earlier-level files. Both analyses characterize increasing cross-level demands in CLE.

E. Additional Benchmark Data Analysis

  • Because a submission must satisfy every checklist item to pass, later levels require more conditions to be met simultaneously.The reported level-wise statistics show increasing checklist size and task-brief length.Source: E. Additional Benchmark Data Analysis · p. 33
  • Average checklist size rises from 18.74 items at L1 to 36.83 at L5, while average task-brief length grows from 246 to 466 words.The number of rubric sections also rises from 4.10 at L1 to 5.74 at L5; increases in checklist size and brief length are larger at upper levels.Source: E. Additional Benchmark Data Analysis · p. 33, Table 6 · p. 34
  • 88 of 120 task ladders contain an earlier-level file reference, and the number rises from 73 at L2 to 84 at L5.Figure 12 counts ladders whose task specification or evaluation rubric at each level refers to files from an earlier level.Source: E. Additional Benchmark Data Analysis · p. 33, Figure 12 · p. 34

F.1. Task Progression

Task progression remains difficult under both submission budgets, and strong first-level performance does not guarantee success at later levels. Conditional pass rates rise in aggregate, but reflect both selection among tasks that cleared earlier levels and experience carried forward.

F.1. Task Progression

  • In Trial 2, the median number of tasks ending after zero or one level falls only from 87 (72.50%) to 80 (66.67%), and only Claude Code with Opus 5 drops below half, at 52 tasks (43.33%).In Trial 1, every configuration has more than half its tasks ending after zero or one level.Source: §F.1 · p. 34
  • L1 performance correlates with overall completion in Trial 2 (ρ = 0.75), but less with L5 pass rate (ρ = 0.42); Qwen3.8-Max configurations exceed the L1 median yet fall below the L5 median.Their L1 pass rates are 74.17% with Hermes-Agent and 65.00% with OpenClaw, while L5 rates are 2.50% and 1.67%, respectively.Source: §F.1 · p. 34
  • Median conditional pass rates rise from L1→L2 to L4→L5 in both trials, but these rates use progressively selected task subsets and include experience from cleared levels, so they do not establish that later assignments are easier.Later bottlenecks remain: 8 of 17 Trial 2 configurations have a later transition below their L1→L2 rate, while the median denominator falls from 75 tasks to 14 at the final transition.Source: §F.1 · pp. 34–35
  • Figure 13 summarizes task counts by completed-level depth and Trial 2 cumulative pass rates across configurations, with the median and top overall-completion configuration highlighted.Each distribution row covers 120 tasks.Source: Figure 13 · p. 35

F.2. Feedback and Revision Gains

Feedback and revision improve pass rates, with gains varying by level and configuration. Hermes-Agent also outperforms OpenClaw on full-task completion across shared backbones, though resource use is higher and runs are not matched workloads.

F.2. Feedback and Revision Gains

  • 7.50 pp at L2 and 3.33 pp at L5 are the median Trial 2 pass-rate gains across configurations; the corresponding median relative increases are 22.50% and 75.00%.The L5 medians exclude OpenClaw with GLM 5.3 because its Trial 1 L5 pass rate is zero.Source: §F.2 · p. 35, Figure 15 · p. 37
  • Some configurations gain more at later levels than at L1: Claude Code with Sonnet 5 gains +16.67 pp at L4 and +12.50 pp at L5, versus +9.17 pp at L1.Hermes-Agent with DeepSeek V4 Pro loses 0.83 pp at L1 while gaining 9.17 pp at L5; differences compare complete gated runs and reflect both later-level access and success.Source: §F.2 · pp. 35–36
  • Figure 14 defines adjacent-level conditional pass rates, while Figure 15 displays Trial 2-minus-Trial 1 pass-rate changes in percentage points across tasks.Figure 15 colors increases blue, decreases red, and no change at the neutral midpoint.Source: Figure 14 · p. 36, Figure 15 · p. 37

F.3. Harness Comparison

  • Hermes-Agent has higher full-task completion than OpenClaw for all six shared backbones under both budgets, with a mean advantage rising from 2.36 pp in Trial 1 to 5.56 pp in Trial 2.In Trial 2, Hermes-Agent’s mean actor-token usage and cost are 1.90 and 1.71 times OpenClaw’s, respectively; totals reflect runs with different progression and revision workloads.Source: §F.3 · p. 36

F.4. Effect of Model Generation

Under a fixed Hermes-Agent setup, Gemini 3.7 Flash leads the three compared generations in both trials. The comparison does not show consistent performance improvement across model updates, even when feedback and revision are available.

F.4. Effect of Model Generation

  • Gemini 3.7 Flash achieves the highest overall and full-task completion in both trials; in Trial 1, overall completion is 25.33%, versus 21.00% for Gemini 3.6 and 21.17% for Gemini 3.8.The comparison holds the harness configuration, task set and order, tools, and state retention fixed.Source: §F.4 · p. 37, Table 7 · p. 38
  • Gemini 3.8 gains 9.50 percentage points in overall completion in Trial 2 but still trails Gemini 3.7 in overall completion (30.67% versus 33.67%) and fully completed tasks (14 versus 19).Across the three generations, performance does not improve consistently with model updates, even with diagnostic feedback and revision.Source: §F.4 · p. 37

Related papers

Loading 2610.33956v1…