Source-linked AI summary
Composer 2 Technical Report
Cursor Research, :, Aaron Chan, Ahmed Shalaby, Alexander Wettig, Aman Sanger, Andrew Zhai, Anurag Ajay, Ashvin Nair, Charlie Snell, Chen Lu, Chen Shen, Emily Jia, Federico Cassano, Hanpeng Liu, Haoyu Chen, Henry Wildermuth, Jacob Jackson, Janet Li, Jediah Katz, Jiajun Yao, Joey Hejna, Josh Warner, Julius Vering, Kevin Frans, Lee Danilek, Less Wright, Lujing Cen, Luke Melas-Kyriazi, Michael Truell, Michiel de Jong, Naman Jain, Nate Schmidt, Nathan Wang, Niklas Muennighoff, Oleg Rybkin, Paul Loh, Phillip Kravtsov, Rishabh Yadav, Sahil Shah, Sam Kottler, Alexander M Rush, Shengtong Zhang, Shomil Jain, Sriram Sankar, Stefan Heule, Stuart H. Sul, Sualeh Asif, Victor Rong, Wanqi Zhu, William Lin, Yuchen Wu, Yuri Volkov, Yury Zemlyanskiy, Zack Holbrook, Zhiyuan Zhang
TL;DR
Agentic software-engineering models need to solve realistic, long-horizon coding tasks, but training and evaluation must closely match those workflows. Composer 2 combines coding-focused continued pretraining, reinforcement learning, matched infrastructure, and real-codebase benchmarks; it achieves frontier-level coding performance, while the authors note remaining room for improvement and benchmark-domain limitations.
Problem
The paper investigates how to reliably improve model performance on real-world coding through scalable training and benchmarks that better represent software-engineering work.
Method
Composer 2 combines coding-focused continued pretraining, large-scale reinforcement learning, matched Cursor-harness environments, and benchmarks derived from real software-engineering problems.
Results
61.3 on CursorBench: Composer 2 substantially improves over prior Composer models, while scoring 61.7 on Terminal-Bench and 73.7 on SWE-bench Multilingual, comparable to state-of-the-art systems.
Takeaways & Limitations
The work demonstrates that strong domain-specialized agentic coding models can be trained by scaling training while closely matching the target domain through benchmarks, infrastructure, and reward development.
Takeaways & Limitations
Public benchmarks can mismatch developer workflows and over-specify prompts, while Composer 2 still shows behaviors that could be improved and may be smaller than proprietary models of comparable ability.
Abstract
from arXiv · showhide
Composer 2 is a specialized model designed for agentic software engineering. The model demonstrates strong long-term planning and coding intelligence while maintaining the ability to efficiently solve problems for interactive use. The model is trained in two phases: first, continued pretraining to improve the model's knowledge and latent coding ability, followed by large-scale reinforcement learning to improve end-to-end coding performance through stronger reasoning, accurate multi-step execution, and coherence on long-horizon realistic coding problems. We develop infrastructure to support training in the same Cursor harness that is used by the deployed model, with equivalent tools and structure, and use environments that match real problems closely. To measure the ability of the model on increasingly difficult tasks, we introduce a benchmark derived from real software engineering problems in large codebases including our own. Composer 2 is a frontier-level coding model and demonstrates a process for training strong domain-specialized models. On our CursorBench evaluations the model achieves a major improvement in accuracy compared to previous Composer models (61.3). On public benchmarks the model scores 61.7 on Terminal-Bench and 73.7 on SWE-bench Multilingual in our harness, comparable to state-of-the-art systems.
1 Introduction
Composer 2 is a coding-specialized model for agentic software engineering, trained to improve long-horizon planning, coding intelligence, and interactive problem solving. It uses real-world-aligned training infrastructure and achieves strong results on CursorBench and public software-engineering benchmarks.
- Composer 2 targets agentic software engineering, emphasizing long-term planning, coding intelligence, and efficient interactive problem solving.
- The model combines continued pretraining with large-scale reinforcement learning to improve coding knowledge, reasoning, multi-step execution, and long-horizon coherence.
- Training uses the deployed Cursor harness, equivalent tools and structure, and environments designed to resemble real software-engineering problems.
- 61.3: Composer 2 achieves a major accuracy improvement over previous Composer models on CursorBench.
- 61.7 on Terminal-Bench and 73.7 on SWE-bench Multilingual: Composer 2 is comparable to state-of-the-art systems in the authors’ harness.
2 Background and Related Work
The paper frames software-engineering agents as systems that autonomously modify codebases through tool-mediated rollouts. Composer 2 investigates whether scaling training and matching training environments to real coding tasks can improve this capability.
- Software-engineering agents autonomously act on task prompts by producing tool-mediated rollouts that change an execution environment.
- Strong agents must explore non-trivially, write tests, and make minimal changes rather than solve only constrained programming problems.
- Composer 2 can read and edit files, run shell commands, search codebases, and search the web, with file changes as its most common outcome.
- The research program studies whether scaling training reliably improves real-world coding through continued pretraining, asynchronous reinforcement learning, and challenging benchmarks.
3 Continued Pretraining
Continued pretraining specializes a base model in coding knowledge and capabilities before agentic reinforcement learning. The authors evaluate how pretraining loss relates to downstream RL performance and use additional training components for production efficiency.
- Continued pretraining aims to improve coding knowledge and reliably enhance the final coding agent’s performance.
- The authors selected Kimi K2.5 as the base model using evaluations of codebase perplexity, coding knowledge, state tracking, and inference performance.
- The continued-pretraining recipe uses a code-dominated data mix across 32k-token training, 256k-token long-context extension, and targeted coding-task SFT phases.
- Figure 2 examines checkpoints trained at varying token budgets and shows a steady decrease in training perplexity.
- Cross-entropy loss after SFT predicts downstream RL performance across continued-pretraining compute levels followed by identical RL runs.
- Multi-Token Prediction layers are trained with self-distillation to support speculative decoding and faster production serving.
4 Reinforcement Learning
Composer 2’s reinforcement learning trains on realistic coding tasks with asynchronous policy-gradient infrastructure, KL regularization, and behavior-sensitive rewards. The training is designed to improve both coding performance and interactive efficiency across tasks of varying difficulty.
- RL training setup: RL training samples coding problems, generates multiple agent rollouts, and updates model weights according to solution quality.The environments emulate real Cursor sessions.
- RL training setup: The training distribution reflects common software-engineering use cases and includes aspects absent from popular AI coding benchmarks.Later training stages upsample harder data using rollout-turn and thinking-token heuristics.
- Asynchronous RL training: The asynchronous policy-gradient pipeline uses fixed-size rollout groups, single-epoch training, full-parameter updates, and separate rollout and training workers.Fast synchronization and in-flight updates reduce how off-policy long rollouts become.
- Asynchronous RL training: The KL regularizer uses k1 = −log r because the unbiased k3 estimator has sharply increasing variance when p and q diverge.The alternative k2 estimator avoids variance blow-up but is biased.
- Training outcomes: RL training improves both average and best-of-K performance rather than producing the expected trade-off between them.The curves are measured on a held-out evaluation set and CursorBench tasks.
- Agent behavior: A concave, increasing length penalty encourages quick solutions on easy tasks while allowing more effort on long-horizon tasks.The penalty combines thinking, tool-use, output, turn, and tool-call quantities, and supports efficient behaviors such as parallel tool calls.
5 Real-World Evaluation with CursorBench
CursorBench evaluates coding agents on realistic engineering tasks that are broader and less specified than public benchmarks. Its continuously refreshed design targets ambiguity, large-scale changes, and evolving developer workflows.
- Motivation: Public coding benchmarks can misalign with real-world utility because they omit developer workflows, over-specify prompts, and risk contamination or overfitting.The paper identifies domain mismatch, prompt over-specification, and data contamination and overfitting as primary factors.
- Benchmark design: CursorBench uses tasks drawn from actual coding sessions of the engineering team rather than curated public repositories.The suite also evaluates code quality, execution efficiency, and interactive agent behavior in realistic settings.
- Benchmark design: 181 median lines changed and 390 median prompt characters make CursorBench tasks broader and less specified than public benchmarks.SWE-bench variants require 7–10 median lines changed, while public benchmark descriptions range from 1,185–3,055 characters.
- Benchmark evolution: CursorBench-3 tasks involve changing more than twice as many files and lines of code on average as earlier iterations.Task types also increasingly include long-running command execution, experiment monitoring, and data analysis.
- Additional evaluations: The evaluation suite is complemented by targeted tests for ambiguous intent, instruction following, eager editing, and other coding-agent behaviors.These evaluations extend assessment beyond the primary CursorBench score.
6 Infrastructure
Composer 2’s infrastructure is designed for scalable, reliable training that closely matches deployment, while specialized parallelism, kernels, precision, and routing choices support efficient MoE training.
- Parallelism: Composer 2 decouples expert parallelism from tensor parallelism and uses context parallelism as the primary long-context scaling axis.The training setup uses EP=8, CP=2 during continued pretraining and EP=8, CP=8 during RL.
- Parallelism: Global sequence packing balances compute across data-parallel ranks during RL, where rollout lengths vary substantially across prompts.The algorithm accounts for the higher attention cost of longer sequences.
- Kernels: Training uses in-house CUDA, PTX, and ThunderKittens/ParallelKittens kernels with MXFP8 and NVFP4 precision on NVIDIA Blackwell GPUs.The kernels primarily optimize low-precision MoE training and blockscaled tensor-core matrix multiplication.
- Kernels: The MoE forward pass uses per-token NVFP4 scaling because per-tensor scaling made training batch-variant and caused RL divergence.The per-token scheme adds latency to quantization and the GEMM epilogue but proved more effective.
- Kernels: Forward and backward passes use different numerical formats: trainer NVFP4 supports inference matching, while higher-precision MXFP8 is used for backward computation.The backward pass runs only on the training cluster and is not a system-wide RL bottleneck.
- RL Infrastructure: Router replay aligns inference and trainer expert assignments so training log-probabilities match the distribution from which tokens were sampled.The router continues computing gating scores, and implausible replayed experts are filtered using a threshold derived from top-k selections.
7 Results
Composer 2 is evaluated in the production-like Cursor environment across internal and public software-engineering benchmarks. It achieves strong accuracy with comparatively favorable token and inference-cost efficiency, while evaluation protocols vary across third-party systems.
- Evaluation Setup: CursorBench evaluates agents directly in Anyrun using initialized codebases and prompts, matching the production execution setting.Accuracy is aggregated across multiple evaluation passes, alongside completion tokens, latency, and inference cost.
- CursorBench: 61.3% CursorBench-3 accuracy represents a 37% relative improvement over Composer 1.5 and a 61% improvement over Composer 1.Composer 2 is also described as competitive with the strongest frontier models while being significantly cheaper at inference.
- CursorBench: Composer 2 generates trajectories comparable in length to other models while maintaining frontier-level accuracy and token efficiency relative to similarly accurate frontier models.Token usage alone does not fully capture inference efficiency because active parameter counts differ.
- CursorBench: Composer 2 achieves a Pareto-optimal inference-cost trade-off, combining costs similar to smaller or low-effort models with accuracy competitive with much larger frontier models.The paper uses median inference cost per CursorBench task because FLOPs for API models were unavailable.
- Public Benchmarks: 61.7% on Terminal-Bench improves over Composer 1.5 by 13.8% and Composer 1 by 21.7%, with overall public-benchmark performance highly competitive with state-of-the-art models.Across both public benchmarks, successive Composer versions show consistent gains.
8 Conclusion
The paper concludes that continued pretraining and reinforcement learning can produce strong specialized coding models when training infrastructure and evaluation closely match the target domain. It also emphasizes substantial room for improvement as coding-agent tasks extend to longer horizons.
- Conclusion: Composer 2 demonstrates that continued pretraining and reinforcement learning can train strong specialized models for frontier-level agentic coding.The approach starts from a strong general-purpose model and combines domain benchmarking, harness and environment engineering, behavioral rewards, and infrastructure reliability.
- Limitations and Future Work: The 1.04T-parameter model with 32B active parameters still shows intelligence and coherence behaviors that can be clearly improved.The authors believe considerable architectural and algorithmic development remains possible.
- Limitations and Future Work: Future Composer iterations target coding tasks requiring hours of human time by improving long-term training signals and infrastructure for faithful long-horizon problems.The paper frames coding-agent horizons as expanding beyond interactive problems.
A Contributors
The Composer research team comprises the contributors listed in the paper.
- Contributors: The paper identifies the Composer research team and lists its contributors by name.The supplied contributor passage contains the team roster.
B Base Model Selection
The authors evaluated open-source base models using coding knowledge, state tracking, and codebase perplexity, while excluding agent benchmarks as poor predictors of final performance. Kimi K2.5 was selected based on strong overall results and infrastructure efficiency.
- Evaluation criteria: Three base models were evaluated on coding knowledge, state tracking, and codebase perplexity before selection.Coding knowledge used FreshBench, state tracking measured reconstruction after many diffs, and codebase perplexity used negative log-likelihood over an internal monorepo.
- Evaluation criteria: Agentic coding benchmarks were intentionally excluded because agentic and long-horizon capabilities can change substantially during reinforcement learning.
- Selection: Kimi K2.5 was selected after all three models performed well, based on strong general performance and additional efficiency considerations in the authors’ infrastructure.
- Selection: Table 2 reports the internal benchmark results, with negative log-likelihood measured over the internal codebase.
C.1 Streaming Prefix Detection
The reference detector identifies prefix-streaming failures by finding repeated, progressively extending copies of an initial seed within think blocks. It requires a minimum chain length and handles responses containing multiple think blocks.
- Task context: The task involves distinguishing malformed prefix streaming from normal incremental output and accounting for an interleave-stutter variant in chat responses.
- Prefix-chain detection: The prefix-chain algorithm requires at least ten characters of text and constrains the first line, used as the seed, to 2–50 characters.
- Prefix-chain detection: The algorithm locates successive occurrences of a newline-prefixed seed and records their starting positions until no further occurrence is found.
- Prefix-chain detection: Adjacent chunks form a valid extension sequence when each later chunk is longer than and starts with the preceding chunk.
- Detection pipeline: The detector searches each think block for a repeated prefix chain and flags the response when any block contains one.