PaperlayerView exact revision on arXiv

Source-linked AI summary

Aspire: Can Models Self-Evolve from Vague Goals?

Yuhao Wu, Jingyuan Zhang, Jiajun Shi, Yuxuan Zhang, Xinping Lei, Junting Zhou, Zexuan Wang, Yuchen Wu, Huan Zhou, Duo Wang, Yinzhu Piao, Yongchang Peng, Yunfeng Shi, Jin Chen, Zuo Wang, Jinkai Liu, Jiaheng Liu, Wenxuan Zhang, Shen Yan, Wenhao Huang, Ge Zhang

arXiv:2608.31111v1cs.CL

TL;DR

Existing LLM self-evolution usually optimizes human-specified tasks and metrics, leaving the harder problem of deciding what and how to learn under vague goals insufficiently studied. ASPIRE addresses this gap with hidden evaluation and a unified environment for model-weight and harness evolution. Vague goals shift effort toward interpretation, but retained weight gains are sparse and unstable, and the strongest evolved harness remains below the engineered Qwen-Agent reference.

  • Problem

    Existing LLM self-evolution starts from concrete tasks, evaluation scripts, and success metrics, leaving autonomous operationalization of broad capability goals underexplored.

  • Method

    ASPIRE gives agents only a natural-language capability goal, hides downstream evaluation, and supports data, training, validation, model-weight, and harness evolution in one interactive environment.

  • Results

    Vague goals redirect search toward goal interpretation; agents complete evolution loops more often than they retain improvements, while the strongest successor harness remains below a fixed reference.

  • Takeaways & Limitations

    Executing self-directed updates is not sufficient: progress must be judged against hidden, goal-aligned evaluation and the base model rather than narrow local signals alone.

  • Takeaways & Limitations

    Retained nonnegative improvement reflects safety rollback and selection, not evidence that every attempted update improves capability.

Abstract

from arXiv · show

Many important forms of human learning begin with a vague goal, such as "become a better physicist" or "improve at research." Learners must interpret the goal, identify capability gaps, decide how to learn, and determine whether they have actually improved. In contrast, existing work on LLM self-evolution typically begins with tasks and evaluation metrics specified by humans, reducing self-evolution to optimizing an explicit objective rather than deciding what and how to learn. We introduce ASPIRE, a benchmark for vague-goal-driven self-evolution. ASPIRE provides only a natural-language capability goal while downstream evaluation tasks remain hidden. The agent must operationalize the goal by choosing data and update methods, constructing training and validation signals, and deciding when to evaluate. ASPIRE supports both model-weight and agent-harness evolution in a unified interactive environment and evaluates the resulting systems on a hidden, expert-authored set of 520 items spanning six goals. Our experiments show that vague goals redirect search effort toward goal interpretation. Current agents routinely complete training and harness-editing loops, but weight-level gains remain sparse and unstable, and the strongest evolved harness remains below the engineered Qwen-Agent reference. Agents often train on mismatched data and trust narrow self-evaluations, so local gains fail to transfer to hidden evaluation and continued search and training can erase earlier improvements.

1 Introduction

ASPIRE studies self-evolution when agents receive broad capability goals instead of human-specified tasks and metrics. Agents must decide what to optimize, how to learn, and how to validate progress, yet gains often fail to transfer to hidden evaluation.

  • Motivation: Vague-goal learning couples decisions about what to improve, how to improve it, and how to verify improvement.This framing mirrors important forms of human learning that begin with broad capability directions rather than predefined benchmarks or rewards.
  • Motivation: Human-defined task specifications let agents optimize a fixed objective, whereas vague goals require agents to interpret goals and operationalize progress.Operationalization includes diagnosing gaps, decomposing sub-goals, and constructing learning and validation signals.
  • Benchmark: ASPIRE provides a natural-language capability goal while hiding downstream evaluation tasks, items, answers, and detailed feedback from the agent.The benchmark evaluates resulting systems on a hidden expert-authored set of 520 items spanning six goals.
  • Benchmark: ASPIRE supports both model-weight and agent-harness evolution through a unified interactive environment for data, training, self-evaluation, branching, and selection.The benchmark records decisions, actions, and self-evaluation trajectories across these evolution surfaces.
  • Findings: Vague goals redirect search toward goal interpretation and operationalization and produce lower aggregate outcomes than corresponding explicit-task references.The controlled comparison matches starting models, update tools, and compute budgets while changing goal specification.
  • Findings: Weight-level improvements above the base model remain rare and are not reliably retained through continued search.Agents routinely complete data selection, training, and checkpoint generation despite the sparse retained gains.
  • Findings: Harness evolution produces functional successors, but the strongest observed successor remains below the engineered Qwen-Agent reference.This comparison holds model weights fixed while evolving goal interpretation, tool use, and self-evaluation behavior.
  • Conclusion: The benchmark exposes a gap between executing updates and retaining target-aligned improvements.Its contribution combines target operationalization, sealed evaluation, a minimal interactive environment, and outcome-and-trajectory evidence.

2 Background

Most agent benchmarks assume that humans have already made a problem executable by specifying tasks, rewards, judges, and execution scaffolds. ASPIRE instead isolates the model’s role in translating broad deployment needs into objectives, feedback, and capability improvements.

  • Background: Most agent benchmarks begin with a specified task, reward or judge, and fixed execution scaffold.This setup supports controlled comparison but presupposes the work of making a problem executable.
  • Background: Forward-deployed engineers turn general-purpose models into working systems adapted to customer data formats, workflows, and operational constraints.Their success criterion concerns genuine use, continued operation, and improvement after failures rather than only demonstrations or benchmark scores.
  • Background: Deployment needs often require translating vague goals into concrete objectives, constructing reliable feedback, and creating usable execution systems.These layers are normally supplied by human engineering roles before agents are evaluated.
  • Background: ASPIRE focuses on whether models can determine what to learn, translate broad capability directions into data and training plans, and achieve growth through weight or harness updates.It isolates capability-improvement processes from related work on judging interaction experience or building and maintaining agent systems.

3 Aspire: Self-Evolution Benchmark and Interactive Environment

ASPIRE defines vague-goal self-evolution as a controlled search problem in which agents operationalize goals, evolve model weights or harnesses, and assess progress using hidden evaluation.

  • Hidden evaluation: ASPIRE keeps the evaluator controller-side while exposing only bounded aggregate outcomes, preventing the hidden benchmark from becoming an agent-visible task specification.The protocol withholds benchmark items, answers, rubrics, routing labels, and judge traces.
  • Task definition: A campaign fixes the evaluator, typed action contract, budget, and terminal selection rule while the agent translates the vague goal into data, update, and validation choices.The evaluator remains present throughout the campaign but is never part of the agent’s observation.
  • Interactive environment: Evolution proceeds through bounded rounds in which a fixed decision model can propose data operations, updates, validations, branches, and candidate states before a controller-mediated commit.A verified trained descendant may be explicitly promoted only between rounds; released observations contain public history, lifecycle status, legal requests, identifiers, and bounded budget information.
  • Evolution surfaces: ASPIRE separates evolution surfaces: weight candidates change model parameters, whereas harness candidates change the runtime instructions, tool policy, workflow, memory, and validation logic around fixed weights.The controller and evaluator remain fixed within the round while the component under study changes.
  • Hidden evaluation: The hidden evaluation set contains 520 expert-authored items across six goals, with review, difficulty screening, deduplication, overlap auditing, scorer binding, and immutable versioning.The six goals include scientific and academic reasoning, humanities and social-science knowledge, health and medical reasoning, mathematical reasoning, integrated reliability, and academic and scientific writing.
  • Safe retention: Safe retention keeps a selected weight checkpoint only when its raw score change is positive; otherwise the controller rolls back, while harness effects may remain negative.The nonnegative retained improvement is therefore a selection outcome rather than evidence that every attempted update improves capability.

4 Experiments and Analysis

Across RQ1–RQ3, vague-goal agents spend more effort operationalizing goals, often complete update loops, but rarely retain improvements that transfer to hidden evaluation. Narrow validation, mismatched data, and continued search can produce regression or proxy specialization, while evolved harnesses remain below the fixed reference.

  • RQ1: Vague-goal post-training: 27.07 versus 32.90 and 29.58 versus 36.23: vague-goal prompting scores below the official Claude Opus 4.8 Max and GPT-5.6 references overall, despite task-level reversals.Vague-goal Claude is higher on HumanEval and BFCL, while vague-goal GPT-5.6 is higher on GSM8K.
  • RQ1: Vague-goal post-training: 2,109 more decision-model thinking seconds and 0.61 more GPU-idle hours accompany 1.27 fewer active training and evaluation hours under vague goals.Agent-visible task-material and proxy-evaluation accesses become 2.98× and 2.39× as dense, while mean tool actions fall from 188.5 to 179.2.
  • RQ1: Vague-goal post-training: 0.57 versus 2.63 evaluations per training start: vague-goal Claude receives 4.6× less feedback than official GPT-5.6, while official Claude has nearly identical feedback density.The cross-system comparison is descriptive rather than a matched prompt effect; official Claude records much more reasoning volume than official GPT-5.6.
  • RQ2: Weight evolution: 1/12 final-only model–goal means exceeds its base score, with Qwen3.5-9B scientific and academic reasoning rising from 45.33 to 48.00.At the individual-run level, 3/24 final checkpoints exceed base scores; rollback retains the base model in the other 21 runs.
  • RQ2: Weight evolution: 28/30 adaptive-search cells produce evaluated checkpoints, but only one retained improvement survives rollback; Terra raises Qwen3.5-4B mathematics from 17.86 to 20.10.Only two cells have best evaluated checkpoints above base, and the retained Terra score was selected using repeated feedback on the same fixed evaluation slice.
  • RQ2: Weight evolution: 28/62 within-round transitions rise, yet monotonic trajectories can remain below base, and greater search volume does not ensure better outcomes.In five of six goals, the highest checkpoint among external decision models remains below base; blind continuation can also end in poor final checkpoints after 26.570–32.286 GPU-hours.
  • Failure modes: 30/32 dataset-import events use GSM8K or Hendrycks mathematics data, including runs targeting science, logic, and writing, while numeric-label SFT is associated with answer-format collapse.All 279 corresponding evaluation outputs are single-digit labels, and the five affected checkpoints score 0, 0, 0, 6.141, and 0; the comparison is not randomized causality.
  • RQ3: Harness evolution: 28.64 task macro and 27.65 example micro: the fixed Qwen-Agent reference exceeds all three valid evolved harnesses, with Sol closest at 27.22/25.97.Luna’s narrow eight-item validation improved from 7/8 to 8/8 while its hidden task-macro mean was 19.32, consistent with specialization to a narrow proxy.

5 Related Work

Prior self-evolution and agent benchmarks automate improvement toward human-specified objectives, while Aspire studies evolution when the task and evaluator are not operationalized in advance.

  • Self-training and self-evolution methods: Existing self-evolution methods construct learning signals through self-generated rationales, reward filtering, self-judgment, self-play, or problem generation.These methods reduce dependence on externally supplied data or feedback to varying degrees.
  • Automated post-training and ML-agent benchmarks: Post-training benchmarks give agents explicit models, evaluation scripts, budgets, and quantifiable objectives for data filtering, training, and search.Related systems also revise data strategies while measuring whether continued search retains earlier improvements.
  • From operationalized tasks to vague goals: Most automated engineering systems begin with an operationalized task, score, or evaluator rather than a broad research direction.Aspire instead withholds downstream tasks and measures whether agents turn broad directions into learning assessed on hidden evaluation items.
  • Memory, workflow, and skill evolution: Memory, workflow, and skill-evolution systems modify memories, finite-state workflows, harnesses, or verified trajectories to improve agent behavior.SkillLearnBench reports external feedback supporting continual gains, whereas self-feedback alone can cause recursive drift.

6 Conclusion

The paper argues that vague-goal self-evolution must be evaluated against hidden capability outcomes, not merely successful execution of training or editing procedures. Across its studies, agents often complete those procedures without reliably retaining improvements, while harness evolution remains below a fixed reference.

  • 6 Conclusion: Aspire’s studies associate vague-goal prompting with more goal-definition work and show that evaluated checkpoints occur far more often than retained improvements.The conclusion distinguishes closing the training loop from closing the capability loop.
  • 6 Conclusion: The highest successor-harness mean remains below a fixed reference under unchanged runtime weights.This comparison concerns harness evolution rather than weight updates.
  • 6 Conclusion: The conclusions are bounded by six goals, expert-authored evaluation coverage and scoring quality, one canonical adaptive-feedback run per cell, and controlled-access trace evidence.The paper identifies broader coverage, repeated runs, refreshed evaluation sets, and drift-controlled recursive evolution as future directions.

7 Contributions

The listed core and contributing authors are identified for the paper.

  • 7 Contributions: Yuhao Wu, Jingyuan Zhang, and Jiajun Shi are named as core contributors.
  • 7 Contributions: Yuxuan Zhang, Xinping Lei, Junting Zhou, Zexuan Wang, Yuchen Wu, Huan Zhou, Duo Wang, Yinzhu Piao, Yongchang Peng, Yunfeng Shi, Jin Chen, Zuo Wang, Jinkai Liu, and Jiaheng Liu are named as contributors.

A.1 Goal and Evaluation-Set Statistics

The evaluation set contains six vague goals and 520 expert-authored items, with contamination controls, blind screening, versioned prompts, and recovery checks supporting the hidden evaluation protocol.

  • Goal coverage: Six vague goals are each paired with one mutually exclusive group of items in the hidden evaluation set.Logic, reliability, and instruction following form one composite goal rather than three separately identified goals.
  • Evaluation-set size: 520 top-level evaluation items are newly authored by domain experts.The academic and scientific writing goal contains 20 top-level task bundles, while harness scoring can also report example-micro aggregation.
  • Contamination control: Contamination control removes exact and semantic duplicates and audits overlap against reference benchmarks before applying a second overlap gate during dataset registration.The hidden evaluation set is frozen under a manifest and SHA-256 after review, scorer binding, and local end-to-end validation.
  • Difficulty screening: Independent blind screening by Seed-2.0, GPT-5.2, and Gemini-3 records how many models fail each candidate, excluding retried timeouts and service errors from failures.L0–L3 denote the number of models that fail.
  • Prompt and evaluation visibility: A vague-goal prompt provides the capability goal, trained model, action contract, budgets, and aggregate-score rule without revealing evaluation content, answers, rubrics, or per-item feedback.Prompts are rendered from versioned setting profiles rather than written ad hoc for each run.
  • Run integrity: Creation and recovery checks prevent a vague-goal run from silently resuming under an explicit-task contract or a different prompt version.

B Complete RQ1 Score and Trajectory Accounting

Replacing explicit tasks with vague goals shifts agent effort toward interpretation and evaluation planning, while producing lower and less reliably transferable outcomes. The accounting shows more deliberation and idle time, but not necessarily more executed work.

  • Process accounting: Matched comparisons used the same decision model, base model, benchmark leaf, and trial index across 48 run pairs.Anti-cheat hits and runs returning only the fallback base were excluded.
  • Process accounting: 2,109 more seconds of decision-model thinking accompanied vague-goal runs, while active time fell by 1.27 hours per matched pair.Idle time increased by 0.61 hours, with absolute means moving from 0.6 to 1.2 idle hours and from 8.0 to 6.7 active hours.
  • Behavioral analysis: 2.98× higher reading-task-definition density and 2.39× higher evaluation-script inspection density indicate greater goal interpretation under vague prompting.Action mentions were normalized per 10,000 characters and interpreted only when hit rate and density moved consistently.
  • Behavioral analysis: 188.5 versus 179.2 mean actions per trajectory supports interpreting longer vague-goal traces as additional deliberation rather than more executed work.Training starts were counted only from concrete launch commands, not broad keyword matches.
  • Official-trajectory comparison: 3.54 evaluations and 6.17 training starts per vague-goal Claude trajectory exceeded official Claude references but remained below official GPT-5.6 frequencies.The official Claude averages were 2.89 evaluations and 5.14 training starts; GPT-5.6 averaged 39.21 and 14.91.
  • Outcome accounting: Only verified checkpoints enter the evaluated lineage, and retention uses the highest-scoring eligible checkpoint or rolls back to base.Evaluated, selected, and retained outcomes are reported separately because incomplete runs and rollback can otherwise obscure learning.

C.3 Final-Only Protocol: Vague-Goal Weight Evolution

The final-only protocol tests whether agents can turn vague goals into a single retained weight update without seeing hidden evaluation scores during development. Most final checkpoints fail to exceed base performance, and the one positive goal-level outcome is unstable at the item level.

  • Protocol: 24 runs crossed two model sizes and six goals, with each run allowed multiple jobs but only one final hidden-evaluation submission.The protocol used separate Run A and Run B executions for every model–goal pair.
  • Results: 48.00 was the only positive Mean, for scientific and academic reasoning on both runs, with only 26 correct items overlapping.Each run answered 36/75 items correctly, while 20/75 items flipped which run was correct.
  • Results: 35.735 to 18.609 and 39.010 to 25.640 were the raw final-checkpoint macro-average declines for 4B and 9B.These values are before rollback and summarize the final submitted checkpoints.
  • Failure modes: 5 submitted checkpoints used numeric-label MMLU SFT, with 21,000 single-digit training targets and final scores of 0, 0, 0, 6.141, and 0.The corresponding 279 evaluation outputs were also single digits.

C.4 Adaptive-Feedback Protocol Results

Adaptive feedback enabled frequent checkpoint evaluation, but improvements above base remained rare and retention remained stricter than local progress. Search profiles and proxy choices show substantial exploration alongside mismatched data and unstable trajectories.

  • Aggregate outcomes: 2/30 configuration–goal cells produced best evaluated checkpoints above base, and only 1/30 yielded a retained improvement.Qwen3.5-4B Self science rose from 44.00 to 45.33; Terra mathematics rose from 17.86 to 20.10 and was the sole retained improvement.
  • Aggregate outcomes: 28/30 configuration–goal cells produced at least one evaluated checkpoint, but among 62 transitions there were 28 increases, 13 ties, and 21 decreases.Later search beat the first evaluated checkpoint in 14/22 multi-checkpoint cells, while only Terra mathematics produced an eligible above-base checkpoint.
  • Search profiles: 8/22 multi-checkpoint trajectories were non-decreasing, while Luna, Terra, and Sol used 89.57, 91.73, and 76.56 settled GPU-hours respectively.Sol produced 33 evaluated checkpoints but none exceeded its corresponding base score.
  • Proxy and update choices: Qwen3.5-4B Self imported GSM8K or Hendrycks mathematics data in 30/32 events, including for science, logic, and writing.Qwen3.5-4B Self used GRPO in 4/12 plans, whereas Qwen3.5-9B Self used SFT in 9/11.
  • Evidence boundary: The artifact supports claims about exploration, checkpoint progression, selection, and rollback, but not item-level causal analysis.It excludes hidden evaluation items, raw model messages, command outputs, and per-item judge reasons.
  • Scope and accounting: Training GPU-hours exclude inference, judging, deployment, and idle controller time, and the adaptive-feedback design is not a multi-seed estimate.The protocol is a complete 5×6 design, but conclusions remain based on one run per configuration–goal cell.

D RQ3 Creator-Trajectory Case Analysis

The creator-trajectory analysis illustrates how harness evolution can improve local validation while failing or regressing on hidden tasks. The cases differ in failure mechanism: narrow stopping, missing output safeguards, and conservative selection.

  • Scope and evidence boundary: Three valid GPT-5.6 creator trajectories and one inspected hidden-set execution per frozen successor harness limit these examples to case evidence.Table 2 reports means over three executions, whereas the mechanism diagnoses come from one inspected execution.
  • Luna: Luna stopped after about 40 minutes when a narrow validation proxy improved from 7/8 to 8/8 on similar prompts.The edited harness added a fixed five-part template and improved structural completeness, but the hidden-set inspection identified reframing of direct questions.
  • Terra: Terra’s missing final-answer invariant allowed empty post-calculator responses to submit pre-tool fragments of 342 and 520 characters.These two inspected tasks scored lower after the harness change.
  • Sol: Sol tested four validation task types and rejected locally stronger variants for latency, repeated calls, fabricated numbers, or scorer misses.It selected the original Qwen-Agent policy with a completeness check for empty final responses.
Loading 2608.31111v1…