Source-linked AI summary
Prime Agent: A Self-Improving RLM Harness
Seth Karten, Alex L. Zhang, Kevin Thomas, Sebastian Müller, Elie Bakouch, Daniel Auras, Mika Senghaas, Fares Obeid, Konstantin Dunas, Johannes Hagemann, Sami Jaghouar
TL;DR
Long-horizon agency requires information and computation beyond a language model’s weights and active context. Prime Agent provides a persistent, expressive harness for programmatic processing, recursive coordination, and retained state, and reports stronger evaluation performance across several tasks. Its scope also includes author-identified concerns about externally reported results and unsafe refinement behavior.
Problem
Language models are bounded sequential processors, so long-horizon agency requires external information management and computation beyond model weights and active context.
Method
Prime Agent integrates persistent REPL computation, recursive subagents, direct communication, retained histories and memories, and standardized execution and accounting while leaving strategy construction to the model.
Results
Prime Agent improves ARC-AGI-3 performance from 30% to 95% and matches or exceeds native and popular harnesses across long-context coding, GPU-kernel generation, emulator construction, and autonomous nanoGPT speedruns.
Takeaways & Limitations
Persistent execution, recursive sessions, autonomous controls, recorded history, and Continual Harness support varied forms of test-time computation under standardized execution and accounting.
Takeaways & Limitations
The authors report that OpenAI self-reported ARC-AGI-3 performance and that native-harness reruns fell below published scores, so those values do not isolate a causal harness effect.
Abstract
from arXiv · showhide
Language models are sequential processors, but long-horizon agency requires external information and computation beyond model weights and active context. Prime Agent is an open-source harness for long-horizon evaluation and coding-agent workflows. A persistent IPython REPL follows the Recursive Language Model abstraction for programmatic context processing and test-time compute, while Continual Harness preserves histories, memories, skills, prompts, and subagent specifications across trajectories. Recursive subagents coordinate through direct agent-to-agent communication, and the Agents View lets humans inspect and manage daemon-backed sessions. Prime Agent standardizes execution, recovery, verification, and resource accounting while leaving strategy construction to the model. This low-friction, expressive membrane prevents harness failures from becoming model failures and pushes measurement toward the model's true maximal underlying capability. Prime Agent raises ARC-AGI-3 RHAE Best@1 from 30% to 95.5% and matches or exceeds native and popular harnesses across long-context coding, GPU-kernel generation, emulator construction, and autonomous nanoGPT speedruns. On Factorio, we find refinement allows for continuous technology progression and dedicated subagents enable parallelized work. Code is available at https://github.com/PrimeIntellect-ai/prime-agent.
1 Introduction
Prime Agent is an expressive, standardized harness that extends language models with persistent information management, computation, and recursive coordination for long-horizon evaluation. It improves measured performance while preserving model-directed strategy construction.
- Language models are bounded sequential processors whose next decisions depend on model weights and active context.
- Prime Agent combines programmatic context processing, revisable prompts and memories, reusable skills, and direct agent-to-agent communication.
- The harness standardizes reliable execution, recovery, verification, termination, and resource accounting while exposing a low-friction interface for model-constructed strategies.
- Prime Agent improves ARC-AGI-3 performance from 30% to 95%.
- The system matches or exceeds popular harnesses across long-context coding, GPU-kernel generation, and emulator construction, and sustains an 85.5-hour nanoGPT run with 19 validated records.
2 Prime Agent Architecture
Prime Agent organizes persistent state, programmatic computation, recursive sessions, human interaction, continual refinement, and long-horizon controls into one model-directed architecture. Its runtime preserves, coordinates, verifies, and accounts for work across continuations.
- Prime Agent separates information management from computation management, connecting retained state to code, tools, and recursive subagent sessions.
- The state hierarchy spans fixed model weights, active context, persistent REPL computation, and disk-backed retained state, with explicit operations moving information between levels.
- Programmatic computation with RLMs: Each session owns a persistent IPython REPL where Python execution, tool calls, and model inference provide test-time compute while intermediate values remain outside active context until selected.
- Programmatic computation with RLMs: The asynchronous rlm primitive creates subagent sessions with separate contexts and kernels, allowing parent computation to continue while results arrive through direct communication.
- Recursive orchestration and interaction: Daemon-backed sessions preserve recursive topology across detachment, restart, and recovery, while asynchronous queues support communication among parents, children, and siblings.
- Continual Harness: Continual Harness stores typed, revisable prompts, memories, skills, and subagent specifications, while refinement versions updates, preserves provenance, and enables rollback.
- Continual Harness: Self-improvement converts execution evidence into persistent harness state that changes later behavior while model weights remain fixed.
- Long-horizon control: Autonomous mode, persistent goals, and heartbeats provide budgeted continuation, agentic completion, and scheduled turns.
3 Evaluation
The evaluation tests whether Prime Agent’s standardized, expressive runtime supports test-time scaling, long-context information management, and persistent recursive execution across diverse long-horizon tasks. Results show competitive performance, sustained progress in persistent environments, and measurable behavioral differences from native harnesses, while some comparisons remain limited by external reference results and strict budgets.
- Evaluation scope: The evaluation spans interactive reasoning, long-context tasks, autonomous research, emulator construction, GPU-kernel generation, and persistent environments.The research questions cover test-time scaling, information management, and persistent recursive execution, with trajectory analyses examining allocation, retention, and recovery.
- Interactive reasoning at test-time scale: Additional output tokens and API cost produce sharply different progress rates across ARC-AGI-3 configurations, with stronger configurations improving across longer horizons while others plateau early.The curves are interpreted as evidence of model-dependent test-time scaling through an interface that does not impose one fixed workflow.
- Long-context information management: Prime Agent is competitive across long tasks and especially excels at long-running or long-context tasks, including autonomous operation.The long-context suite uses a persistent REPL to search, transform, summarize, and revisit information stored outside the active prompt.
- Multi-day autonomous research: DeepSeek V4 Pro created roughly six times more out-of-loop experiments per training run under Prime Agent than under Claude Code.Models also used persistent REPL experimentation and constructed programmatic benchmark interfaces, including Kimi K3’s probe function for roughly ninety screening experiments and all 19 validated records.
- Programmatic systems construction: Prime Agent and native harnesses remain close on PMPP-Hard, while Prime Agent achieves the same performance at substantially reduced cost and has an advantage token-for-token.The comparison reverses ordering between model groups, and the strict wall-clock budget does not expose the token-usage improvement.
- Persistent interaction and refinement: In a seven-day Sonnet 5 Factorio run, 23.4 million output tokens produced 24 completed technologies and 71% progress on advanced-circuit research without stalling.After a destructive reset, the session recovered and continued; the root created 633 depth-one subagents across 149 dispatch waves, with at most seven active concurrently.
4 Related Work
Prime Agent combines programmatic inference, persistent state, and direct communication to support extensible long-horizon agent workflows and human intervention.
- Prime Agent integrates persistent kernels, recursive sessions, recovery, and captured trajectories with revisable prompts, memories, skills, and subagent specifications.
- Direct agent-to-agent communication uses persistent family-scoped queues and exposes the session tree for human inspection and intervention.
5 Conclusion
Prime Agent unifies persistent execution, recursive sessions, autonomous controls, recorded history, and Continual Harness state for long-horizon work. Results span multiple task types, while models still underuse and struggle to allocate these capabilities effectively.
- Prime Agent unifies persistent execution, recursive sessions, autonomous controls, recorded history, and Continual Harness into one long-horizon substrate.
- Results across interactive reasoning, long-context tasks, autonomous research, systems construction, and persistent environments show support for varied test-time computation under standardized execution and accounting.
- Models still experience friction allocating subagents, managing retained information, and refining reusable state, leaving many harness capabilities underused.
- The paper expects model-harness co-learning to become a route to new long-horizon capabilities through training with Prime Agent or its components.
A Example Out-of-Loop Experiments in the nanoGPT Speedrun
The nanoGPT speedrun examples show agents using Prime Agent's persistent REPL to perform optimizer experiments outside the benchmark script, including coefficient search, numerical checks, and calibrated modeling.
- Each reproduced excerpt is an agent-created experiment run outside the benchmark’s training script during a nanoGPT speedrun.
- The examples instantiate model behavior that uses persistent REPL experimentation beyond the benchmark’s training script.
- Kimi K3 re-derived Newton–Schulz iteration coefficients with a global optimizer and checked bfloat16 rounding bit-exactly.
- DeepSeek V4 Pro built a calibrated toy training problem with Kronecker-Hessian minibatch noise and a natural-gradient oracle arm.
- GLM 5.3 debugged its SOAP implementation on CPU before any GPU screen by testing parameter updates for non-finite values.
B Example Programmatic Orchestration
Prime Agent's orchestration example launches persistent concurrent reviewer sessions, later recovers retained sessions, and communicates through explicit messages.
- The orchestration example admits independent subagents to audit an implementation and test its error-handling edge cases without waiting for immediate answers.
- Later orchestration recovers retained sessions and sends a follow-up message through the agent messaging system.
- A child is a persistent concurrent session rather than a stateless completion returned directly by rlm.
C LLM Usage Disclosure
Large language models assisted with manuscript preparation tasks, while the authors retained sole responsibility for the scientific and intellectual work.
- Language models assisted with code development, writing refinement, and formatting.
- The authors solely conducted the scientific claims, experimental design, and analysis.
- The authors solely provided the manuscript's intellectual contributions.