Source-linked AI summary
NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness
NeoHorse Team, Guoliang Cao, Guohao Dai, Tianyu Guo, Kai Han, Hailin Hu, Zihan Jiang, Xiang Kuang, Boxun Li, Yulong Li, Zehua Pei, Yuchuan Tian, Jiamin Wang, Yu Wang, Yunhe Wang, Yihong Wu, Haiyang Xu, Shuo Zhang, Hang Zhou, Siyang Cheng, Jiayu Fan, Wei He, Qingrui Jiao, Hongguang Li, Zhiyuan Li, Runke Liu, Xi Liu, Xinchen Liu, Sinno Jialin Pan, Yi Ren, Liuyang Song, Chenyu Wang, Bei Yu, Quanlu Zhang, Xiangyu Zhang, Mengyu Zheng, Yingjie Zong
TL;DR
NeoHorse-1 addresses how an AI system can observe its capabilities and convert that evidence into subsequent learning. It uses routing-harness interaction data, structured quality evaluation, and capability-guided post-training to close an evaluation–selection–update loop. Across the reported benchmark suite, post-training improves macro-average scores at both 4B and 9B scales, while the authors present the result as an initial prototype requiring validation across further iterations and broader capabilities.
Problem
Recursive self-improvement requires a concrete mechanism for turning observations of system capabilities into the next round of learning.
Method
NeoHorse-1 combines heterogeneous routing harnesses, provenance-preserving trajectory data, routing-guided curricula, and capability-guided allocation to form an evaluation–selection–update loop.
Results
Across the evaluation suite, macro-average scores rise from 58.94 to 64.87 at 4B and from 65.60 to 69.04 at 9B, with broad gains at both scales.
Takeaways & Limitations
NeoHorse-1 provides an initial prototype showing that routing-harness operation can supply experience and feedback for harness-mediated recursive self-improvement.
Takeaways & Limitations
Validation covers mainly agentic, coding, tool-use, and instruction-following capabilities and reflects only a single evaluation–selection–update pass.
Abstract
from arXiv · showhide
Recursive self-improvement (RSI) requires a concrete mechanism through which an AI system observes its capabilities and converts that evidence into the next round of learning. We present NeoHorse-1, a family of agent-native models developed to explore this path through agentic post-training. Our system combines a heterogeneous model pool with intelligent routing, recording the predicted capability demand, selected service tier, and subsequent interaction for each user turn. These records are converted into training examples that preserve interleaved reasoning, tool calls, and harness context, and are admitted through structural validation, six-dimensional semantic evaluation, and subscene-level labeling. Routing signals organize supervised fine-tuning into a three-stage curriculum and extend to routing-guided on-policy distillation, where a teacher supervises student-generated responses under the same progression. Capability-guided allocation then converts evaluation feedback into the next training mixture, closing an evaluation-selection-update loop in which what the system learns to do shapes what it learns from next. Across eleven benchmarks covering harness-based agents, tool use, coding, and instruction following, post-training raises the macro-average from 58.94 to 64.87 at 4B and from 65.60 to 69.04 at 9B, substantially narrowing the aggregate gap between the post-trained 4B model and the 9B base model. NeoHorse-1 provides an initial prototype of this feedback-driven process and a path toward harness-mediated RSI across successive iterations.
1. Introduction
NeoHorse-1 explores harness-mediated recursive self-improvement by converting routed agent interactions into progressively organized training data and using capability feedback to shape later updates. Across evaluated tasks, post-training improves both model scales, with especially strong gains in harness-based and execution-intensive settings.
- System motivation: NeoHorse-1 combines heterogeneous model pools, routing harnesses, and recorded interactions to connect capability demand, model selection, and subsequent training.Routing records link estimated capability demand, the model used, and the resulting interaction.
- System motivation: Capability feedback allocates data for subsequent updates, while updated models return to the harness to generate new experience and expose changing strengths and limitations.The loop makes what the system learns influence what it learns from next.
- Post-training design: Recorded interactions become training examples that preserve historical and harness context, including interleaved reasoning, tool use, and visible responses.This keeps assistant behavior tied to the execution conditions in which it was produced.
- Results: 58.94 to 64.87 at 4B and 65.60 to 69.04 at 9B: agentic post-training raises macro-average scores across the evaluation suite.The largest gains appear on harness-based and execution-intensive evaluations, and the post-trained 4B model substantially narrows the aggregate gap to the 9B base model.
2. Related Work
Related work frames NeoHorse-1 at the intersection of trajectory-based post-training, on-policy distillation, harness design, routing, curriculum learning, and recursive improvement. The paper uses routing-derived capability demand to order training examples rather than treating the ultimately served model as a difficulty label.
- Agentic post-training: Trajectory-based SFT transfers planning and tool use from interaction records, while data composition, scale, synthetic tool-use data, rejection sampling, and preference optimization affect generalization.Prior systems include FireAct, AgentTuning, Agent-FLAN, AgentBank, and Llama 3.
- Agentic post-training: On-policy distillation lets students generate trajectories while teachers provide token-level logits on student-visited states, reducing the distribution gap of fixed-teacher imitation.This provides denser process supervision aligned with evolving student behavior.
- Harness-mediated learning: Harness and environment design affect agent performance and robustness by determining which tools, observations, feedback, and execution-grounded trajectories enter training.Related work also replaces fixed behavioral labels with rewards derived from tool execution and task outcomes.
- Routing and curriculum: Routing selects models or agents suited to requests, while curriculum learning orders examples by estimated difficulty; NeoHorse-1 uses request- and context-conditioned routing demand for this ordering.The paper avoids using the served model identity as the difficulty label because deployment policy and service availability can affect that choice.
- Recursive self-improvement: Recursive self-improvement links experience, evaluations, or generated artifacts to iterative improvement of models, scaffolds, or improvement procedures.Recent work studies task behavior, agent scaffolds, and training or research procedures as distinct improvement targets.
3. Data from Routing Harness
NeoHorse-1 constructs provenance-preserving agentic data from deployment-harness trajectories, organizes it across trajectory, user-turn, and subscene granularities, and filters it through structural and semantic quality controls. Routing and outcome records then support capability analysis, curriculum ordering, and data allocation.
- Data composition: Deployment-harness data preserves user requests, model responses, tool calls, environment observations, recovery attempts, and terminal outcomes across complete interaction trajectories.Public instruction, reasoning, tool-use, code, and preference data broaden capability coverage.
- Data organization: A user turn is the serialized unit between a user request and the next request or task termination, while a subscene groups adjacent turns sharing a local goal.These linked granularities preserve provenance between execution histories, learning examples, and semantic units.
- Data organization: Each user-turn record retains current-turn reasoning, tool calls, and observations while keeping earlier visible responses and tool interactions as context.Earlier-turn reasoning is omitted, and records remain linked to their parent trajectory and subscene.
- Quality control: Structural validation checks event order, payload readability, message structure, response presence, and closure of tool-call/result pairs before admission.Outputs are complete, partially recoverable, or quarantined; structural validity does not imply task success.
- Quality control: Six semantic dimensions—goal attainment, instruction adherence, tool use, evidence consistency, error recovery, and termination—are labeled PASS, WARN, FAIL, or NOT_EVALUATED with separate coverage tracking.Training admission, review, and quarantine policies operate over this structured representation rather than a single heuristic score.
- Semantic characterization: Scene, Goal, and Outcome characterize each subscene by linking user intent, expected achievement, and verifiable result for capability analysis and data allocation.Attributes are assigned at subscene level and retain derivation method and confidence.
- Routing signals: Routing records align predicted capability demand, selected service tier, and observed outcome, separating curriculum ordering signals from deficiency signals used for capability-guided allocation.The router estimates demand from the request, recent dialogue, prior routing decisions, and available execution state; service tiers range from C0 to C3.
4. Agentic Post-Training
NeoHorse-1 converts agent trajectories into context-preserving training examples and uses routing-estimated capability demand to schedule supervised fine-tuning and on-policy distillation. The resulting pipeline organizes both recorded responses and student-generated responses through staged curricula.
- 4.1. Agentic Supervision: Agent trajectories are organized into user-turn training units containing assistant responses, tool interactions, and historical context.A user turn extends from one user request through its following assistant and tool interactions; multiple assistant responses may be supervised in one sequence.
- 4.1. Agentic Supervision: Assistant reasoning, serialized tool calls, visible responses, and end-of-response tokens are supervised, while historical and non-assistant spans provide context without prediction loss.Causal attention lets each assistant response use earlier actions and tool results but not later ones.
- 4.2. Routing-Guided Curriculum Learning: Routing estimates capability demand and orders SFT examples into a curriculum instead of treating served model identity as the difficulty signal.The estimate uses the request and relevant interaction history available before the first supervised assistant response, because service selection can also reflect overrides, availability, and policy.
- 4.2. Routing-Guided Curriculum Learning: The three-stage SFT schedule progressively introduces higher-scored examples while reserving some lower-scored examples for later stages.Stages are roughly one third of the examples, and training keeps the same masked SFT objective without optimizer or learning-rate resets.
- 4.3. Routing-Guided On-Policy Distillation: On-policy distillation applies the same routing-based progression to recorded starting contexts, with a fixed teacher supervising responses generated by the student.The student generates responses containing reasoning, tool calls, or visible text from contexts sampled by stage-specific capability demand.
- 4.3. Routing-Guided On-Policy Distillation: Distillation minimizes response-normalized reverse KL on coarsened student and teacher distributions over shared top-K candidates plus a residual probability bin.Only the current student is updated; generated tokens, candidate IDs, and teacher scores remain fixed during optimization.
5. Results and Analysis
NeoHorse-1 improves performance across model scales and task types, with the strongest advantages on interactive execution, tool use, coding, and feedback-driven agent tasks. Larger models remain especially valuable when tasks require sustained state tracking, failure recovery, and strategy adaptation.
- Overall Results: Training provides broad improvements across agentic, coding, and instruction-following benchmarks at both the 4B and 9B scales.At 4B, NeoHorse-1-4B outperforms Qwen3.5-4B on every benchmark with available results; at 9B, NeoHorse-1-9B improves over Qwen3.5-9B on most available benchmarks.
- Overall Results: Improvement is strongest on harness-based agent tasks, tool interaction, execution-intensive evaluations, and selected coding benchmarks.Instruction-following performance remains comparatively stable at 9B, with one metric showing a minor decrease.
- Overall Results: NeoHorse-1-9B consistently outperforms NeoHorse-1-4B, with larger gains on multi-step interaction, function calling, and challenging coding than on instruction following.The scale gap is substantially smaller on tasks focused on compliance with explicit, relatively static instructions.
- Overall Results: NeoHorse-1-4B matches or exceeds Qwen3.5-9B on several benchmarks, narrowing part of the performance gap associated with model scale.The result indicates that post-training can compensate for part of the aggregate advantage normally associated with a larger model.
- Agent Trace Analysis: Larger-model advantages appear in execution behavior: iterative verification, recovery from failed actions, state maintenance, and strategy revision from environmental feedback.In a WorkBuddy trajectory, the 9B model completes an edit–test–inspect–repair cycle; in a PinchBench trajectory, it abandons an unavailable dependency and switches to standard Python libraries.
- Data Analysis: The routing-harness checkpoint improves the unweighted average across five comparable benchmarks by 6.26 points over public synthetic trajectories under the same routing-guided recipe.The largest gains are on HumanEval (+8.54) and τ2-Bench (+11.31), while function calling, instruction following, and coding also improve.
- Data Analysis: The development-suite average rises from 69.31 for the base model to 71.45 at the largest shown data scale, supporting capability-dependent allocation of agentic supervision.Data scale is measured in unique supervised tokens on a logarithmic axis.
6. Conclusion and Discussion
NeoHorse-1 operationalizes recursive self-improvement through harness-derived experience, routing feedback, and capability-guided training updates. Results show consistent post-training gains, but the evidence remains preliminary because evaluation covers limited capabilities and only one loop iteration.
- Contributions: NeoHorse-1 converts harness interactions into context-preserving training examples and uses routing scores for curriculum learning and on-policy distillation.The pipeline includes structural validation, six-dimensional semantic evaluation, and subscene-level labeling.
- Contributions: Capability-guided allocation converts evaluation feedback into the next training mixture, closing an evaluation–selection–update loop.Updated checkpoints return to the harness and generate new trajectories, routing records, and capability feedback for later training.
- Results: 58.94 to 64.87 at 4B and 65.60 to 69.04 at 9B: agentic post-training consistently raises macro-average scores across the evaluation suite.The post-trained 4B model also substantially narrows the aggregate gap to the 9B base model.
- Results: Scale remains beneficial after post-training, especially for iterative debugging, execution-failure recovery, and long-sequence state maintenance.The larger model retains a clear advantage on these tasks.
- Limitations: The study is an initial attempt rather than a definitive demonstration because it evaluates a limited capability range and only one evaluation–selection–update iteration.Whether gains accumulate across successive iterations remains untested.
A. Contributions
The contribution and affiliation lists identify the paper’s core contributors, additional contributors, and institutional affiliations. Names are organized by contribution category rather than summarized by research findings.
- Core Contributors: Core contributors are listed alphabetically by surname and include contributors from the numbered affiliation set.The list includes Guoliang Cao, Guohao Dai, Tianyu Guo, and additional named contributors.
- Contributors: Contributors are separately listed in a second category, including Siyang Cheng, Jiayu Fan, Wei He, and additional named contributors.The passage identifies this group as contributors rather than core contributors.
- Affiliations: The eight numbered affiliations span TokenRhythm Technologies, Infinigence AI, Tsinghua University, Peking University, The Chinese University of Hong Kong, Visionplus Capital, WX Capital, and Alibaba Group.The affiliation numbers map contributors to these organizations.
Case A: Daily Ticket Reporting under Temporal and Audit Constraints
Case A evaluates daily ticket reporting under exact desk/channel, temporal, and audit constraints. NeoHorse-1-9B matches the independently recomputed eligible records and preserves source references, while retaining a status-label issue.
- Task requirements: The task requires exact desk/channel matching, alias merging, latest eligible in-day updates, aggregate counts, and source-linked audit rows.Next-day updates must be excluded from the report.
- Outcome: NeoHorse-1-9B reports 50 eligible records and 19 tickets, matching independent recomputation while excluding the next-day update.It retains source-row references in the resulting artifact.
- Outcome: NeoHorse-1-9B’s status-summary table still contains two priority labels in place of status labels.This remains a localized artifact issue despite the correct totals and temporal filtering.
Case B: Implementing a Time-Leakage Auditor from Repository Requirements
Case B tests implementation of a time-leakage auditor against repository requirements. NeoHorse-1-9B follows the specified tolerance and output schema, unlike the comparison model.
- Task requirements: The auditor must allow a small temporal tolerance, record genuinely late features in rejection output, and continue exporting clean samples.The repository README specifies a five-minute tolerance.
- Outcome: NeoHorse-1-9B applies the README’s five-minute tolerance and preserves the required output schema.Its implementation satisfies the stated repository requirements.
- Comparison: Qwen3.5-9B uses a two-second tolerance, rejects valid sample s1, and changes the output structure.These deviations conflict with the repository’s specified tolerance and schema.
Case C: Supporting Sustained Two-Player Gomoku Interaction
The Gomoku case tests whether the generated HTML game sustains a 26-click two-player interaction. NeoHorse-1-9B preserves the board state and turn order, while Qwen3.5-9B accepts no moves because its click handler generates invalid board indices.
- The task is to use HTML to create a simple Gomoku game.
- After the same 26-click replay, NeoHorse-1-9B shows 13 black and 13 white stones in an ongoing position, whereas Qwen3.5-9B remains empty.
- Table 6 presents execution and artifact evidence for the Gomoku case.
- NeoHorse-1-9B preserves expected stone positions, colors, and turn order without captured runtime exceptions.
- Qwen3.5-9B accepts no moves because its click handler produces invalid board indices.