PaperlayerView exact revision on arXiv

Source-linked AI summary

CWM: An Open-Weights LLM for Research on Code Generation with World Models

FAIR CodeGen team, Jade Copet, Quentin Carbonneaux, Gal Cohen, Jonas Gehring, Jacob Kahn, Jannik Kossen, Felix Kreuk, Emily McMilin, Michel Meyer, Yuxiang Wei, David Zhang, Kunhao Zheng, Jordi Armengol-Estapé, Pedram Bashiri, Maximilian Beck, Pierre Chambon, Abhishek Charnalia, Chris Cummins, Juliette Decugis, Zacharias V. Fisches, François Fleuret, Fabian Gloeckle, Alex Gu, Michael Hassid, Daniel Haziza, Badr Youbi Idrissi, Christian Keller, Rahul Kindi, Hugh Leather, Gallil Maimon, Aram Markosyan, Francisco Massa, Pierre-Emmanuel Mazaré, Vegard Mella, Naila Murray, Keyur Muzumdar, Peter O'Hearn, Matteo Pagliardini, Dmitrii Pedchenko, Tal Remez, Volker Seeker, Marco Selvi, Oren Sultan, Sida Wang, Luca Wehrstedt, Ori Yoran, Lingming Zhang, Taco Cohen, Yossi Adi, Gabriel Synnaeve

arXiv:2510.02387v1cs.SEcs.AIcs.LG

TL;DR

Reliable code generation requires models to understand executable behavior, not only static code patterns. CWM addresses this by mid-training an open-weights LLM on Python and Docker observation-action trajectories, then applying reasoning-focused post-training, and reports early benefits for code world modeling alongside strong coding and math performance. The release is intended as a research testbed rather than a production or general-chat system.

  • Problem

    Reliable high-quality code generation remains challenging, motivating models that capture what code does when executed rather than only its textual form.

  • Method

    CWM is mid-trained on Python execution traces and agentic Docker-environment trajectories, with post-training for reasoning across verifiable coding, math, and software-engineering tasks.

  • Results

    CWM’s world modeling data improves downstream performance in ablations, while the model reports strong results across coding, math, and reasoning tasks.

  • Takeaways & Limitations

    CWM provides open checkpoints and a testbed for studying world modeling, grounded reasoning, planning, and reinforcement learning for code generation.

  • Takeaways & Limitations

    CWM is not suitable for production or general-purpose chat use because its evaluations and chat optimization are limited.

Abstract

from arXiv · show

We release Code World Model (CWM), a 32-billion-parameter open-weights LLM, to advance research on code generation with world models. To improve code understanding beyond what can be learned from training on static code alone, we mid-train CWM on a large amount of observation-action trajectories from Python interpreter and agentic Docker environments, and perform extensive multi-task reasoning RL in verifiable coding, math, and multi-turn software engineering environments. With CWM, we provide a strong testbed for researchers to explore the opportunities world modeling affords for improving code generation with reasoning and planning in computational environments. We present first steps of how world models can benefit agentic coding, enable step-by-step simulation of Python code execution, and show early results of how reasoning can benefit from the latter. CWM is a dense, decoder-only LLM trained with a context size of up to 131k tokens. Independent of its world modeling capabilities, CWM offers strong performance on general coding and math tasks: it reaches pass@1 scores of 65.8% on SWE-bench Verified (with test-time scaling), 68.6% on LiveCodeBench, 96.6% on Math-500, and 76.0% on AIME 2024. To support further research on code world modeling, we release model checkpoints after mid-training, SFT, and RL.

1 Introduction

CWM addresses the challenge of reliably generating high-quality code by training a 32-billion-parameter open-weights model on execution-centered data. It is intended to support research on code generation with world modeling, including agentic coding and step-by-step Python execution simulation.

  • Reliable high-quality code generation remains challenging despite LLMs’ growing use in software development.
  • CWM targets code world modeling by teaching models how code execution changes program state, beyond learning code syntax alone.
  • CWM is mid-trained on Python execution traces and agentic Docker-environment trajectories containing observation-action interactions.
  • CWM uses a dense decoder-only architecture with 32 B parameters and supports contexts of up to 131 k tokens.
  • The release provides final and intermediate checkpoints to enable research on world modeling for reasoning, planning, and code generation.

2 Code world model datasets

CWM’s code world model datasets combine Python execution traces with large-scale agentic interactions in executable repository environments. The resulting data captures program states, executed actions, and software-engineering workflows at substantial scale.

  • CWM emphasizes code and code world modeling data across pre-, mid-, and post-training phases.
  • Executable repository images package configured environments for running repository code and tests, enabling isolated and repeatable data collection.
  • Function-level tracing produces over 120 M traced Python functions from intermediate interpreter events.
  • Python traces represent local variables and stack-frame metadata before execution, the executed Python line, and resulting variable states or events.
  • Natural-language traces add semantic descriptions and can compress trajectories by skipping less informative execution steps.
  • ForagerAgent data contains 3 M deduplicated trajectories from 10.2 k executable repository images, covering agentic coding interactions.

3 Examples of code world modeling

CWM examples illustrate agentic coding with environment feedback and early integration of execution-trace prediction into reasoning. They also demonstrate trace-based approaches to code generation and related future applications.

  • Agentic coding: CWM constructs and tests a competitive-programming solution by comparing predicted outputs with actual program execution.The example includes initial reasoning, input-output construction, and environment feedback.
  • Trace prediction: The paper presents trace prediction as a way to ground reasoning about code generation and execution without live execution environments.
  • Trace-based generation: CWM can predict traces for functions constrained by natural language or formal asserts, then construct function bodies from the generated actions.The paper relates this reasoning-via-tracing approach to logical reasoning about program execution.
  • Agentic coding: Figure 4 encodes tool calls in purple, environment feedback in orange, and reasoning in blue.
  • Trace prediction: With a trace example in the prompt, CWM uses execution-trace prediction to confirm the return value {1: 2, 2: 4} after natural-language reasoning.Special tokens such as <|frame_sep|> are encoded explicitly.
  • Additional examples: The examples also include self-correcting SWE behavior, comparisons of strict traces and natural language, trace-guided code generation, and termination prediction.

4 CWM: architecture, pre-training, and scaling laws

CWM combines a 32-billion-parameter dense Transformer with two-stage pre-training centered on code world modeling and long-context data, alongside compute-scaling experiments for hyper-parameter selection.

  • Architecture: CWM is a 32-billion-parameter dense decoder-only Transformer with alternating local and global attention and context windows up to 131,072 tokens.The architecture uses an interleaved 3:1 pattern of local and global attention blocks.
  • Architecture: CWM uses SwiGLU, RMSNorm, Rotary Positional Embedding, Grouped-Query-Attention, and full document-causal masking.
  • Two-stage pre-training: Pre-training begins with 8 T tokens of general data emphasizing coding, STEM, and knowledge, followed by 5 T tokens of code world model mid-training.The two stages differ in data mixture and maximum document length.
  • Two-stage pre-training: Mid-training introduces ForagerAgent and Python execution-tracing data to teach code world modeling capabilities.Additional code, reasoning, compiler, kernel, and formal-mathematics datasets are also included.
  • Training mixture: CWM-specific data makes up 30% of the mid-training mixture, while general code contributes 40% and rehearsal contributes 30%.The rehearsal fraction helps retain performance on standard evaluations.
  • Long-context training: About 30% of mid-training documents exceed 65 k tokens, motivating a maximum sequence length of 131 k tokens throughout mid-training.
  • Scaling laws: The compute budget is defined as C = M · D, with M representing non-embedding FLOP per token and D representing total training-token scale.The FLOP estimate accounts for both linear and self-attention terms.
  • Scaling laws: BS(C) = 30.17 · C^0.231, while the fitted learning-rate relation declines gradually with compute budget.The search evaluates batch size and learning rate across eight increasing compute scales.

5 Post-training: SFT, RL algorithms and environments

CWM’s post-training combines supervised fine-tuning with multi-task, multi-turn reinforcement learning across coding, mathematics, and software-engineering environments. The RL design adapts GRPO for asynchronous, long-horizon interaction while adding safeguards for exploration, trajectory quality, and bias control.

  • Post-training overview: SFT improves reasoning and instruction following before large-scale multi-task, multi-turn RL on coding contests, mathematics, and software-engineering environments.
  • RL algorithm: GRPO is adapted for multi-turn trajectories containing both model- and environment-generated tokens, with returns replacing single-turn rewards in advantage calculation.
  • RL algorithm: Asynchronous RL lets workers generate trajectories while trainers update the policy, increasing throughput relative to synchronous GRPO.
  • RL algorithm: The algorithm removes σ and trajectory-length normalization to avoid difficulty and length biases, while token-limit batching stabilizes optimization efficiency.
  • RL algorithm: Trajectory handling skips zero-advantage and stale samples, uses length-weighted returns, and rejects rare low-probability tokens to limit variance, off-policy data, and gibberish.
  • RL environments and outcomes: Agentic SWE RL substantially improves software reasoning and engineering capability, while reasoning-token design affects exploration and final performance during RL.

SWE RL

SWE RL trains agents to solve software issues through long-horizon tool interaction, while related coding, agentic-coding, and math environments provide verifiable multi-step tasks. Rewards and filtering emphasize executable correctness, format adherence, and useful training difficulty.

  • SWE RL: SWE RL uses iterative RL and rejection-sampled reasoning traces in SFT self-bootstrapping, improving data quality, format adherence, and SWE-bench Verified success rates.
  • Coding environments: Competitive programming RL supplies execution feedback on syntax errors, timeouts, and incorrect outputs across multiple programming languages and follow-up attempts.
  • Coding environments: Code trajectories receive reward 1 only when formatting requirements are met and the solution passes all unit tests within time and memory limits.
  • Coding environments: Agentic coding combines SWE-style reasoning and tool use with competitive-programming evaluation, explicitly prompting agents to write and run tests before producing solutions.
  • Math environment: Math trajectories receive reward 1 when formatting requirements are met and a verifier confirms equivalence between the predicted and ground-truth answers.

6 Code and infrastructure

CWM’s infrastructure combines distributed training and optimized inference with asynchronous rollout collection in isolated execution environments. Workers generate token-based agent-environment trajectories while trainers update and periodically redistribute the policy.

  • Training infrastructure: CWM training uses H100 GPUs with FSDP, tensor parallelism, FlashAttention-3, and additional optimizations for speed and memory efficiency.
  • Training infrastructure: Float8 linear layers achieve twice the nominal bfloat16 FLOPs on Hopper GPUs, while RL uses bfloat16 because float8 precision hurt performance.
  • Training infrastructure: Async tensor parallelism overlaps communication with computation, and activation checkpointing optimizes the memory-versus-recomputation tradeoff under a specified budget.
  • Rollouts and environments: Each rollout begins with an environment-provided state and observation, then advances through token-based actions and observations containing relevant rewards and environment feedback.
  • Asynchronous RL: Workers continuously produce trajectories and trainers update the policy asynchronously, with updated weights sent periodically while partially completed generations continue using old KV-caches.
  • Execution infrastructure: The execution service runs tens of thousands of code snippets per second in isolated containers and returns stdout, stderr, exit codes, and environment state as model feedback.

7 Experimental results

CWM performs strongly across agentic coding, execution-trace prediction, termination, complexity, and standard coding and math evaluations, while results also expose clear scope boundaries. Mid-training data combinations and larger inference budgets improve selected outcomes, but some benchmark gains do not directly transfer to real-world software.

  • 7.1 The impact of CWM data: Using all datasets together achieves the best performance across the ablation metrics, while ForagerAgent data alone improves agentic SWE-bench trajectory NLLs and adds 3.7% to SBV pass@1.Execution trace data significantly improves CruxEval input and output prediction but leaves SBV-related metrics unaffected.
  • 7.2 Agentic evaluation: 65.8% pass@1 resolve rate on SWE-bench Verified with test-time scaling outperforms similarly sized open-weight models and remains competitive with larger and proprietary models.Without test-time scaling, CWM achieves 53.9%.
  • 7.2 Agentic evaluation: 80.4% success at k = 40 is reached by pass@k, while best@k plateaus around k = 16 and majority voting plateaus at k = 24.At k = 16, best@k achieves a 65.8% resolve rate; majority voting reaches 58.4%.
  • 7.3 Execution trace prediction: 94.0% is CWM’s best execution-trace reasoning score, compared with 88% for full trace prediction and 87.3% for the SFT model using full trace prediction.Language reasoning traces average 1164 tokens, versus 497 tokens for full trace prediction; single-step prediction is not competitive.
  • 7.3 Execution trace prediction: CWM achieves more than 99% format matching, over 96% Observation/Action Exact Match, and over 97% Key (+Value) Match across trace and observation sources.These results indicate accurate prediction of execution traces, intermediate observations, and actions.
  • 7.5 Algorithmic complexity prediction: CWM’s reasoning performs strongly on complexity tasks, systematically outperforming other models in time-complexity prediction and generation, while ranking lower on some space-complexity metrics.CWM ranks second overall on official time-complexity prediction scores and first for space-complexity code-only pass@1.

8 Transparency, Risks & Limitations

CWM is released as a research model with automated risk assessments and explicit scope boundaries. The authors emphasize that its world-modeling capabilities remain an early research direction requiring further study.

  • Transparency: External LLMs support ForagerAgent interactions, trace-to-natural-language conversion, function tracing, and SFT.The listed base models include Llama3-70B-Instruct, Qwen3-235B-A22B, and Qwen3-32B-FP8.
  • Risks: CWM is assessed against capable open-source models for cybersecurity, chemical, and biological risk-related capabilities.The comparison set includes Qwen3-Coder-480B-A35B-Instruct, Llama 4 Maverick, and gpt-oss-120B.
  • Risks: The authors judge CWM unlikely to meaningfully increase cybersecurity or chemical-and-biological risks beyond the current open-source ecosystem baseline.Undesirable propensities are comparable to most open-source models, although gpt-oss-120B achieves substantially lower rates.
  • Risks: CWM falls within the “moderate” risk threshold defined for the assessed catastrophic domains.Assessment details are provided in the Code World Model Preparedness Report.
  • Release scope: CWM is released under a noncommercial research license, with limitations intended to reduce accidental misuse.The authors frame the release as a resource for exploring world modeling and reasoning in computational environments.
  • Limitations: The models are not suitable for production or general-purpose assistant use and have not received complete real-world, user-facing, or content-risk evaluations.CWM is also intended for English only, while extending world modeling beyond explicit Python execution remains future work.

9 Conclusion

The conclusion presents CWM as an open testbed for connecting language-level reasoning with executable semantics. It reports early benefits from world-modeling data while identifying broader planning and integration as open research directions.

  • Conclusion: CWM aims to bridge language-level reasoning and executable semantics through a learned transition function between states conditioned on actions.The vision targets coding and agentic uses of LLMs.
  • Conclusion: Ablations show that world-modeling data, Python execution traces, and executable Docker environments can directly benefit downstream task performance.The conclusion characterizes these findings as first steps rather than a completed solution.
  • Research directions: CWM provides a testbed for zero-shot planning, grounded chain-of-thought, and reinforcement learning with sparse, verifiable rewards.Python tracing also supports research on code generation, execution, correctness, and verification.
  • Research directions: More research is needed to consistently leverage world models in LLM pre-training and related reasoning and planning settings.The conclusion frames planning with code world models as central to the motivation for building CWM.

B CWM Examples

The examples illustrate CWM’s behavior in software-engineering agents and Python execution reasoning. They show recovery from editing errors, test-based verification, execution-trace advantages in one case, and debugging and code-generation applications.

  • Reasoning agent: In a bash-only SWE-bench setting, CWM recovers from incorrect edits by restoring the original file with git checkout and applying a correct sed edit.The example uses an environment without dedicated file-editing tools.
  • Reasoning agent: CWM runs tests to verify that a patch preserves existing functionality before submitting the patch and generating a summary.This behavior is illustrated in the final turns of SWE reinforcement learning.
  • Reasoning agent: With the edit tool, CWM performs extensive reasoning before modifying a file and receives surrounding-code feedback after the change.This is the default SWE RL setting described in the examples.
  • Python execution trace prediction: In one CruxEval example, execution-trace prediction tracks list modifications correctly while natural-language reasoning predicts the wrong return value.The contrast is presented as an inference-time comparison between the two reasoning modes.
  • Python execution trace prediction: CWM’s execution-trace capability is demonstrated as a neural Python debugger for steering predicted program states and return values.The authors suggest future tools such as skipping loops, jumping to lines, and predicting inputs for arbitrary states.
  • Python execution trace prediction: Execution traces can support code generation from assertions describing desired behavior, even when the target function definition is omitted.The authors consider joint tracing and code generation a possible result of training on traces without third-party source context.
  • Program Termination: CWM also illustrates termination reasoning by considering several concrete inputs before predicting termination on all inputs.The example is presented in the preliminary termination setting.

C RL algorithm

The RL procedure builds PPO training batches from multi-turn agent-environment trajectories while masking environment-generated tokens. Supporting infrastructure captures executable repository environments and includes illustrative SWE and execution-tracing behaviors.

  • RL algorithm: Each prompt produces G multi-turn trajectories containing agent actions and environment observations, with a mask distinguishing agent-generated tokens from environment tokens.The mask M_i,t equals 1 for agent-generated tokens and 0 for prompts and later observations.
  • RL algorithm: The advantage uses each trajectory’s total undiscounted return relative to a length-weighted mean return across the batch.The total number of agent-generated tokens provides the trajectory-length basis.
  • RL algorithm: Because workers continue rollouts during model updates, the behavior-policy distribution is difficult to characterize exactly when computing PPO importance ratios.The method nevertheless uses behavior-policy log probabilities for the ratio calculation.
  • RL algorithm: Trainer batches are formed by dequeuing trajectories until a fixed N-token limit, reducing batch-size variance and improving GPU utilization.Trajectories from one prompt may therefore span multiple gradient updates.
  • RL algorithm: The PPO objective masks environment tokens, clips the importance ratio, and sets N to 131 072, the model’s maximum context size.The ratio uses trainer and worker policy log probabilities.
  • Examples: The SWE examples show recovery from bad edits, test-based patch verification, and extended reasoning before file modification.These examples contextualize the multi-turn agent-environment trajectories used by the RL system.
  • Repository infrastructure: The Activ pipeline builds executable repository images by modifying GitHub Actions workflows, running them locally with act, and capturing built environments.The captured Docker image preserves repository dependencies for later execution.
  • Repository infrastructure: The pipeline reduces build complexity by selecting compatible versions, continuing past noncritical failures, and exiting on captured state, stalled progress, or timeout.It probes available test frameworks and injects pytest fixtures to capture container build state.

E.2 Quasi-random search for batch size and learning rate

The study estimates batch-size and learning-rate ranges across model scales using quasi-random Sobol sampling. Performance remains near-optimal within a broad range but degrades rapidly beyond it.

  • Sobol sequences generate batch-size and learning-rate candidates at each model scale, with scale-dependent sampling ranges.The batch-size range increases and the learning-rate range decreases as model scale grows.
  • Gray points show all evaluated batch-size/learning-rate candidates, while blue points mark candidates within 1% of the best validation loss.
  • The near-optimal batch-size and learning-rate range is broad across scales, but exceeding it causes rapidly degrading performance.

F RL data decontamination

The RL data pipeline decontaminates evaluation-sensitive data and uses verifiers for mathematical answers. CWM also defines structured formats for chat, tool use, and step-by-step execution-trace prediction.

  • RL data decontamination: RL prompts are decontaminated with MinHash LSH against the listed evaluation datasets spanning mathematics, code generation, scientific reasoning, commonsense, conversation, and general reasoning.The listed evaluations include AIME, LiveCodeBench, GPQA, ARC, ArenaHard, and MTBench.
  • RL data decontamination: Dockerized executable repositories are decontaminated against SWE-bench Verified by removing Docker images built from repositories in that benchmark.
  • RL data decontamination: Pairwise dataset-to-SWE-bench patch Jaccard similarities are verified to remain below 0.2, confirming no remaining unexpected instance-level contamination.
  • Verification: Mathematical RL answers are accepted when either a custom verifier or MathVerify finds the predicted and ground-truth expressions equivalent.The custom verifier expands equivalent-expression sets using normalization, numerical computation, and symbolic computation.
  • Interaction formats: Chat, reasoning, and tool interactions use explicit headers, think tags, tool wrappers, and end-of-turn markers to structure model inputs and outputs.Tool control returns to the model after an external tool produces its output.
  • Trace prediction: Trace prediction represents Python execution as observation-action frames, using dedicated delimiters for calls, lines, returns, exceptions, and arguments.Frames contain local variables and source-code actions, while JSON strings encode locals and return or exception values.

J RULER evaluation

RULER evaluates CWM, Qwen3-32B, and Gemma-3-27B at 32k and 128k sequence lengths. CWM outperforms Gemma3-27B at 128k but trails it at 32k, while both trail Qwen3-32B.

  • CWM outperforms Gemma3-27B at 128k sequence length but falls short at 32k.
  • Both CWM and Gemma3-27B perform worse than Qwen3-32B, whose full attention across all layers has higher computational costs for longer sequences.
  • The authors characterize CWM as a trade-off between efficiency and model performance in long-context evaluation.

K Agent capabilities learnt during RL training

During SWE reinforcement learning, the agent increasingly tests its code and improves localization of files relevant to resolving issues. Both behaviors improve substantially within 4,000 RL steps.

  • Testing behavior: RL training teaches the agent to test code more often over the course of rollouts.
  • Testing behavior: After 4,000 RL steps, trajectories containing at least one test increase from 57% to 74%.
  • File localization: The agent’s average file recall rises from 58% initially to over 66% after 4,000 RL steps.Recall is the percentage of files in the gold patch that the agent edited during a rollout.
  • File localization: Figure K.31b presents the learned improvement in localizing files relevant to solving the issue.
Loading 2510.02387v1…