AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks

• AREX-2 scores 81.8 on MLE-bench Lite and 70.7 on Frontier-CS, plus 84.0 on BrowseComp, 52.6 on HLE, 92.2 on GAIA, and 93.8 on DeepSearchQA.The paper reports the highest score among compared systems on MLE-bench Lite and the highest among open-weight models on Frontier-CS; without new deep-research training data, it improves over previous AREX models. Figure 2 reports that AREX-2 (27B) matches or exceeds much larger models, and the agent keeps improving as its round budget grows.Source: Figure 2 · p. 2, §1 · p. 3
• Self-improvement means using more rounds on a task to produce a better solution, moving the iterative loop from the harness into the model.Existing systems often leave retry decisions and solution selection to the scaffold, while asking the model to produce one attempt at a time.Source: §1 · pp. 1–2
• Reflection improves the current solution, while long-horizon execution sustains iteration and preserves useful findings; both are needed for cumulative gains across rounds.Reflection determines the gain per productive round, while long-horizon execution determines how many rounds remain productive before stalling.Source: §1 · p. 2
• Conventional training data keeps successful outputs but discards intermediate attempts, feedback, and revisions, leaving long improvement trajectories largely absent.Because immediately passing solutions are easiest to collect, prolonged processes of measuring and revising are especially likely to be lost.Source: §1 · p. 3
• The authors hypothesize long-horizon reflection is domain-agnostic and choose machine-learning engineering and algorithmic programming for verifiable feedback, sustained improvement, and abundant source material.Validation scores or judge verdicts reveal whether revisions helped, and GitHub and online judges provide problems, code, and tests.Source: §1 · p. 3
• AREX-2 training data comes from executable tasks built from GitHub and online-judge material, multi-round agent revisions, and whole-trajectory selection by final score and task compliance.Selecting trajectories rather than successful individual rounds retains failures, regressions, and abandoned approaches so training can teach persistence after setbacks.Source: §1 · p. 3

2.1 Self-Improvement as Test-Time Scaling

• Productive improvement depends on reflection’s mean gain and long-horizon execution’s productive rounds; increasing the round budget helps only while rounds remain productive.A single submission may fall below the best score so far without reducing it, and gains tend to shrink as the score approaches its ceiling.Source: §2.1 · p. 4
• The method hypothesizes that using feedback, acquiring operational knowledge, and recovering from setbacks transfer across task families despite different tasks and scoring functions.Self-improvement is defined as more rounds leading to better solutions in expectation.Source: §2.1 · p. 4

2.2 Constructing Environments

• Teachers turn GitHub machine-learning repositories and online-judge programming problems into scored environments, then admit only executable tasks where a simple baseline trails a working reference.Repository tasks use a held-out metric; judge tasks use hidden tests, graded where possible to make each round informative.Source: §2.2 · pp. 4–5

2.3 Constructing Trajectories

• Trajectory agents receive hours and hundreds of tool calls, enabling them to establish baselines, measure performance, and revise solutions across rounds.The round budget is disclosed to the agent and matches the resource scaled at test time.Source: §2.3 · p. 5
• Agents acquire operational knowledge by inspecting code and documentation, searching for dependencies, and experimenting, with general and task-specific skills available during training and at test time.Task-specific skills exclude material about the task itself.Source: §2.3 · p. 5
• Agents receive scores and reports after each submission, and trajectories retain regressions so the model learns how to assess, revert, or respond to setbacks.The agent’s response to a lower-scoring round remains part of the training signal.Source: §2.3 · p. 5

2.4 Training

• Training keeps whole trajectories only when they achieve a reference-relative score threshold and satisfy task rules, then applies loss to progress-making decisions rather than failures or no-progress steps.Failures and regressions remain as context; the model is supervised on subsequent diagnosis, repair, strategy changes, experiments, and improved submissions.Source: §2.4 · p. 6
• AREX-2 fine-tunes Qwen3.8-27B on selected trajectories mixed with unchanged AREX deep-research data, making new trajectories the only change from the prior recipe.This design lets cross-domain deep-research gains over models using the previous recipe be read as transfer rather than as a result of new deep-research data.Source: §2.4 · p. 6

3.1 Experimental Setup

• The six benchmarks cover algorithmic programming, machine learning engineering, deep research, and general agentic reasoning, with Any Medal for MLE-Lite, F1 for DeepSearchQA, and accuracy elsewhere.Any Medal is averaged over three seeds; Frontier-CS uses its 188-task Agent Track, and MLE-Lite follows the OpenMLE protocol.Source: §3.1 · pp. 6–7

3.2 Overall Results

• 81.8 Any Medal on MLE-bench Lite leads Table 1, while 70.7 on Frontier-CS exceeds the strongest reported open-weight baseline by 16.0 points.At 27B parameters, AREX-2 is 8.1 points above Naive-N0.5-Flash on MLE-bench Lite and within 5.7 points of the strongest closed-weight system on Frontier-CS.Source: Table 1 · p. 7, §3.2 · p. 7
• 84.0 BrowseComp, 52.6 text-only HLE, 92.2 GAIA, and 93.8 DeepSearchQA; AREX-2 surpasses AREX (4B) and AREX (122B) on all four benchmarks.No new training trajectories are search tasks, and the deep-research training data is unchanged from the previous AREX recipe; AREX-2 leads reported models up to 40B parameters on BrowseComp, text-only HLE, and DeepSearchQA, but a gap remains to the strongest BrowseComp and HLE results.Source: §3.2 · pp. 7–8, Table 2 · p. 8

3.3.1 Round Scaling with Feedback

• 54.4 after one hour, 65.9 after two, and 70.7 after five: AREX-2 keeps improving on Frontier-CS, gaining 2.2 points in the final hour.DeepSeek-V4-Pro stops improving at 44.7 after two hours and DeepSeek-V4-Flash at 39.1 after three; AREX-2's gains shrink but remain positive through its five-hour budget.Source: Figure 4 · p. 9, §3.3.1 · p. 9

3.3.2 Round Scaling without Feedback

• 64.8 at 47 turns rising to 84.0 at 143 turns, AREX-2 improves on BrowseComp without correctness feedback and is more than 12 points ahead of AREX (122B) at about 140 turns.AREX-2 exceeds AREX's final accuracy with less than half the turns and has about three times its gain per turn, using 27B parameters versus AREX's larger model; its recursive process searches, reassesses, and refines without external correctness signals.Source: Figure 5 · p. 9, §3.3.2 · pp. 9–10

3.3.3 Stage-wise Ablation

• 28.8 rises to 41.2 with general skills and 68.2 with task-specific skills and more rounds; AREX-2 reaches 75.8 at the base round count and 81.8 with more rounds.With skills and rounds held constant, training adds 13.6 points; AREX-2 also beats the base model with fewer rounds by 7.6 points, then gains 6.0 more with additional rounds.Source: Figure 6 · p. 10, §3.3.3 · p. 10

4 Conclusion

• Training trajectories record an agent improving solutions over many rounds in environments compiled from machine-learning repositories and algorithmic problems.The paper defines self-improvement as turning more rounds on a task into a better solution: reflection raises gain per round, while long-horizon execution keeps rounds productive.Source: §4 · p. 10
• AREX-2 achieves leading results on MLE-bench Lite and Frontier-CS, and improves on deep-research tasks without adding new data.The conclusion reports continued gains through the fifth hour on Frontier-CS and on BrowseComp without a correctness signal.Source: §4 · p. 11
• Operational knowledge, training, and a larger budget each contribute to results, while performance improves with budget on Frontier-CS and BrowseComp.Frontier-CS gains continue through the fifth hour, and BrowseComp improves without any correctness signal.Source: §4 · p. 11
• The proposed next steps are to widen training domains, lengthen horizons, and use the model’s own trajectories as subsequent training data.Source: §4 · p. 11