CWM: An Open-Weights LLM for Research on Code Generation with World Models

• The authors argue that static, next-token code training is insufficient for mastering coding because models must understand execution effects on local state and codebase-level outputs.They identify code world modeling as typically absent before post-training.Source: §1 · p. 1
• CWM is mid-trained on Python execution traces and agentic Docker-environment trajectories to ground its predictions in software dynamics and provide a starting point for RL.Source: §1 · p. 1
• Python trajectories pair statements as actions with local-variable contents as observations, directly teaching how executing a line changes program state.The authors propose this semantics-focused training may help coding, verification, testing, and debugging.Source: §1 · pp. 1–2
• ForagerAgent generates large-scale Docker-environment trajectories for software-engineering scenarios, with shell-like commands or code edits as actions and environment responses as observations.The data is included during mid-training, rather than only at smaller scale during post-training as in some recent models.Source: §1 · p. 2
• CWM reaches pass@1 scores of 65.8% on SWE-bench Verified (with test-time scaling), 68.6% on LiveCodeBench-v5, 96.6% on Math-500, 76.0% on AIME 2024, and 94.3% on CruxEval Output.Source: §1 · p. 2
• CWM is a dense, decoder-only Transformer with 32 B parameters, sliding-window attention for up to 131 k tokens of context, and pre-, mid-, and post-training phases.With quantization, inference can run on a single 80 GB NVIDIA H100.Source: §1 · p. 2
• The release is intended to enable research on world models for agentic coding and Python-execution simulation, and includes final weights and intermediate checkpoints under a noncommercial research license.The report offers early supportive evidence, and its preparedness assessment concluded CWM is unlikely to increase catastrophic risks beyond those in the current model ecosystem.Source: §1 · p. 2

2.1 Executable repository images: building repositories at scale

• Running RepoAgent and Activ in parallel produced over 35 k executable repository images for repeatable code and test execution.RepoAgent sets up repositories and finds runnable tests; Activ repurposes GitHub Actions workflows by running them locally and capturing build state during test execution.Source: §2.1 · p. 3

2.2 Python tracing: neural code interpretation data

• Over 120 M Python functions were traced at intermediate interpreter events, including executed lines, returns, and exceptions.Input-output pairs are generated using fuzzing and prompting, and tracing captures Python interpreter stack frames.Source: §2.2 · p. 4
• Python traces encode each execution step as an observation of local variables and stack-frame metadata, the executed line as an action, and a subsequent observation of the resulting state.The traces are prefixed with source context, omit globals and external side effects, and use ellipses for unchanged variable values.Source: §2.2 · p. 4
• Additional traces include 70 k repository commits, 70 k CodeContests traces from 33 k snippets, and natural-language traces for 75 M functions and 110 k CodeContests examples.Repository tracing covers unit tests across more than 21 k traceable images; natural-language trajectories are retained only when their final predictions match the ground-truth trace.Source: §2.2 · pp. 4–5

2.3 ForagerAgent: agentic midtraining data generation

• 3 M ForagerAgent trajectories from 10.2 k images and 3.15 k repositories span issue-fix and mutate-fix tasks in a 55–45 split.Mutate-fix tasks introduce test-verified synthetic bugs, while issue-fix tasks use real GitHub issues and preceding commits. Trajectories are near-deduplicated, retained regardless of task success, and train prediction of both agent and environment turns, with loss stochastically masked for 50% of observations.Source: §2.3 · pp. 5–6, Table 1 · p. 6

Competitive programming

• CWM tests its competitive-programming solution by constructing input-output pairs and comparing its predictions with actual execution results.The example illustrates reasoning about environment dynamics, although CWM was not trained directly for this behavior.Source: §3 · p. 6, Figure 4 · p. 7

Trace prediction and reasoning

• With light prompting, CWM generalizes to combining natural-language reasoning with execution-trace prediction to confirm a code result.The authors envision trace prediction enabling grounded reasoning without live execution environments, and propose a future neural debugger that could predict inputs for arbitrary states or jump to future lines in constant time.Source: §3 · p. 6, Figure 5 · p. 8

Trace-guided program generation

• CWM can generate execution traces for functions constrained by natural language or formal asserts, then construct the function body from the executed statements.This reasoning-via-tracing approach recalls Hoare (1971) and Dijkstra (1976), who used logical reasoning about program execution to guide development.Source: §3 · p. 6

Additional examples

• Appendix B adds self-correcting SWE examples, compares strict trace prediction with natural language, and demonstrates trace-based generation and termination prediction.The authors argue that combining trace prediction and natural-language reasoning merits further exploration.Source: §3 · p. 9

4.1 Architecture and hyper-parameters

• CWM is a 32-billion-parameter dense decoder-only model with local and global attention blocks in a 3:1 pattern and windows of 8192 and 131072 tokens.It uses 48 query heads and 8 key-value heads, with scaled RoPE from mid-training onward to support long-context modeling.Source: §4.1 · p. 9

4.2 Two-stage pre-training

• Mid-training is the key stage for teaching code world modeling, adding 5 T tokens of CWM data after 8 T tokens of general pre-training.The mid-training mix includes ForagerAgent and Python execution traces, with 30% CWM-specific data, 40% general code, and 30% rehearsal; rehearsal helps retain standard-evaluation performance.Source: §4.2 · pp. 10–11
• About 30% of mid-training documents exceed 65k tokens, motivating a 131k-token maximum context and sequence-length bucketization across workers.Bucketization groups workers by document length to improve iteration speed while preserving dataset sampling probabilities.Source: §4.2 · p. 11

4.3 Scaling laws

• CWM scaling experiments fix the model-to-data ratio at D/M = 40, about eight times more data than Chinchilla compute-optimal scaling.The ratio matches the target 32B model pre-trained on 8 T tokens.Source: §4.3 · p. 11
• BS(C) = 30.17 · C^0.231, and the fitted learning-rate and batch-size equations diverge from prior work, likely because of different pre-training data.Across eight compute scales, near-optimal hyper-parameters span a broad range; batch size grows and learning rate declines as compute increases.Source: §4.3 · p. 12

5 Post-training: SFT, RL algorithms and environments

• Post-training first uses SFT to improve reasoning and instruction following, then applies large-scale multi-task, multi-turn RL to coding contests, math, and software engineering.The post-training builds on the code world model learned in earlier training stages.Source: §5 · p. 12

5.1 SFT

• SFT mixes internal and open-access instruction and reasoning data with mid-training rehearsal, agentic SWE trajectories, and external reasoning datasets.About 30% of the mix is mid-training rehearsal; the authors report that reasoning-trace benefits carry through to the final post-RL model.Source: §5.1 · p. 12
• SFT reasoning text is bracketed by special tokens whose start-token loss is masked, enabling default non-reasoning output and reasoning mode when the start token is injected.These reasoning tokens are discontinued during RL.Source: §5.1 · p. 12

5.2 RL algorithm

• The GRPO variant supports multi-turn returns and asynchronous training, while maximum-token batching and removing standard advantage and trajectory-length normalization target efficiency and bias.For multi-turn interactions, it masks model- and environment-generated tokens and uses summed returns in advantage calculation; batching by a token limit also reduces batch-size variance.Source: §5.2 · p. 13
• A rare-token, low-probability gibberish detector rejects suspect RL trajectories and stopped increasing gibberish generation.The detector was reported to perform better than detectors based on log probability and position alone.Source: §5.2 · p. 13

5.3 RL environments & data

• CWM’s RL covers agentic SWE, coding, agentic coding, and mathematics, with tasks defined by datasets and interactive environments.The environments are partially observable and use software-based outcome verification to produce a single terminal reward per rollout.Source: §5.3 · p. 14
• Replacing SFT reasoning tokens with clear-text <think> tags during RL produced shorter responses, higher starting entropy, and significantly improved final performance.The authors attribute long initial reasoning traces and slow early improvement to SFT reasoning data limiting RL exploration; the new tags let the model develop reasoning guided but not restricted by SFT.Source: §5.3 · p. 14

5.3.1 Agentic SWE

• Agentic SWE RL improves software reasoning and engineering through end-to-end tool use, with self-bootstrapped traces supporting stronger software-engineering capability before and after RL.The agent can interact for up to 128 turns across a 131k-token context. Its hybrid reward gives 1 when all hidden tests pass; otherwise, patch similarity above 0.5 yields 0 and lower similarity yields −1. Across three iterations, high-quality rejection-sampled traces are added to SFT.Source: §5.3.1 · pp. 14–15

SWE RL

• SWE-bench Verified pass@1 rose from 30% to 37% to 43% across SWE RL bootstrapping iterations.Each round restarts SFT from the midtraining checkpoint using new high-quality RL traces; the final traces join the joint SFT mix. Earlier offline pass rates serve as GRPO baselines in later rounds, enabling group size 1 and faster SWE RL training.Source: SWE RL · p. 16, Figure 9 · p. 16
• SWE RL uses 12.6 k unique training instances, divided into 6.9 k primary and 5.7 k secondary examples.Repository-issue pairs combine executable repositories with public issue and pull-request metadata and filtered SWE-Gym and R2E-Gym data. Hard zero-pass@1 instances receive hidden-test hints early, which raise pass@1 to approximately 30%, before hints are removed later.Source: SWE RL · p. 16

5.3.2 Coding

• Competitive-programming RL provides execution feedback on syntax errors, timeouts, and incorrect outputs, and ends when a solution is correct or attempts are exhausted.A correct trajectory must contain exactly one </think> tag and one markdown block, and pass all unit tests within time and memory limits; incorrect and correct trajectories receive rewards of −1 and 1, respectively.Source: §5.3.2 · pp. 16–17

5.3.3 Agentic coding

• Agentic coding combines SWE-style reasoning and tool use with competitive-programming tasks, asking agents to write and run tests to improve solutions.Unlike SWE RL, it has no submit tool: the final response is extracted for evaluation using Python or C++ execution environments.Source: §5.3.3 · pp. 17–18

5.3.4 Mathematics

• The mathematics RL prompt set contains 278 k public problem-answer pairs, after deduplication and filtering out problems solved in all 32 SFT non-reasoning attempts.A tool-enabled variant lets agents call the Python interpreter up to 4 times per episode, with a 10 s timeout per call; correctness uses a verifier that checks answer equivalence.Source: §5.3.4 · p. 18

5.4 Joint RL

• Joint RL samples 40% of tasks from software engineering, 40% from competitive programming, and 20% from mathematics.Rehearsal batches make up 1/3 of training data and use negative log likelihood loss scaled by 0.1. Training has three stages with changing task distributions; code and math use a soft length limit that grows from 8 k to 64 k over 10 k steps.Source: §5.4 · pp. 19–20

6.1 Techniques for efficient training

• FP8 linear layers achieved twice the nominal FLOPs of bfloat16 on Hopper GPUs, while RL used bfloat16 because FP8 hurt performance.Training used H100s with FSDP and TP, plus FlashAttention-3; the FP8 recipe used e4m3 and dynamic row-wise scaling, with specialized backward-pass setups.Source: §6.1 · p. 20
• Async-TP overlapped tensor-parallel communication with computation, while AutoAC8 and vocab- and loss-parallel helpers reduced memory use.For TP, the FP8 recipe used sub-row-wise scaling and FP8 all-gathers to align quantization with shards and improve throughput; asynchronous TP made duplicate backward all-gathers add zero overhead.Source: §6.1 · p. 21

6.2 RL systems

• RL rollouts comprise token-based agent-environment interactions: environments start episodes and step through actions, observations, rewards, and optional context switches.A start method creates an initial state and prompt from a dataset sample; a step transitions state and returns information visible to the agent.Source: §6.2 · p. 21
• Rollout workers send completed trajectory batches to trainers and receive periodic weight updates, continuing partial trajectories with old KV-caches to sustain GPU utilization.Workers can update weights between trajectories, between steps, or during generation, avoiding inference blocking but allowing trajectories to use mixed weights.Source: §6.2 · pp. 21–22, Figure 15 · p. 23
• FastGen supports batched inference, CUDA graphs, paged attention, chunked prefills, host-side KV-cache, and tensor parallelism; moodist transfers model shards from trainers to workers.Moodist uses InfiniBand between GPU and CPU memory; worker-side concatenation and weight application can overlap generation, reducing trainer involvement in model transfer.Source: §6.2 · p. 22
• An internal execution service runs code in parallel isolated environments and returns execution feedback, while custom tool-based containerized environments support agentic RL tasks.The service handles tens of thousands of snippets per second; the agentic environment supports persistent shell sessions, plugins, and reward or benchmarking infrastructure.Source: §6.2 · pp. 22–23

7.1 The impact of CWM data

• Using all mid-training datasets together gives the best ablation results; ForagerAgent improves agentic SBV NLLs and adds 3.7% to SBV pass@1.PR data helps oracle SBV NLLs and SBV pass@1 but not CruxEval or agentic SBV trajectory NLLs; execution-trace data improves CruxEval input/output prediction but leaves SBV metrics unaffected.Source: §7.1 · pp. 23–24, Table 4 · p. 24

7.2 Agentic evaluation

• On SWE-bench Verified, CWM achieves pass@1 resolve rates of 65.8% with test-time scaling and 53.9% without it, averaged over 4 runs; alternative harnesses are lower but remain reasonable.Source: §7.2 · p. 24, Table 5 · p. 26
• 35.1%: CWM’s Aider Polyglot accuracy is comparable to Qwen3-32B’s 40.0% and Gemini 2.0 Pro’s 35.6% in the whole-file edit format; Terminal-Bench accuracy is 26.25%.On Terminal-Bench, CWM ranks below o4-mini but above Gemini 2.5 Pro. CWM was not optimized for the diff edit format and is not competitive with top-performing models using it.Source: Table 6 · p. 26, §7.2 · p. 26, Table 7 · p. 27

7.3 Execution trace prediction

• 94.0%: CWM’s best CruxEval-output score comes from natural-language reasoning, compared with 88% for full trace prediction and 87.3% for SFT with full traces.Reasoning traces average 1,164 tokens versus 497 for full trace prediction; single-step prediction is not competitive with classic few-shot prompting. Trace-format matching exceeds 99%, observation/action exact match exceeds 96%, and key (+value) match exceeds 97%.Source: Table 8 · p. 27, §7.3 · p. 27, Table 9 · p. 28

7.4 Program termination prediction

• ∼0.94 pass@1: with reasoning, CWM and Qwen3-32B reach comparable HaltEval-prelim performance, although Qwen3 performs better with direct prediction and prompted chain-of-thought.HaltEval-prelim contains 115 terminating and 115 non-terminating translated Python programs. Its small, self-contained programs and balanced classes limit how directly the results generalize to real-world software.Source: Table 10 · p. 28, §7.4 · pp. 28–29

7.5 Algorithmic complexity prediction

• CWM systematically outperforms the compared models across time-complexity prediction and generation metrics; on space-complexity generation, it leads on code-only pass@1 and ranks second on the remaining metrics.On time-complexity prediction, CWM also ranks second overall on the official leaderboard; for time-complexity generation, it achieves the best pass@1, best@1, and all@1 among the compared models.Source: Table 11 · p. 29, §7.5 · p. 29

7.6 Code and mathematical reasoning

• CWM is competitive with similarly sized baselines on LiveCodeBench, performs slightly worse across the reported math benchmarks, and gains up to 11% on AIME24 with majority voting at k = 10.Short-3@k achieves performance comparable to majority voting while significantly reducing computational cost for a given k.Source: Table 12 · p. 30, §7.6 · p. 30, Figure 18 · p. 31

7.7 Non-reasoning evaluations

• Without reasoning, CWM typically outperforms Gemma-3-27B, is similar to Qwen2.5-32B, and trails Qwen3-32B; on LoCoDiff, it beats DeepSeek-R1 0528 and gpt-oss-120B.On LoCoDiff, CWM is competitive with GPT-5 and Gemini 2.5 Pro across short and long sequences, but trails Claude Sonnet 4 by a significant gap; all models degrade as sequence length increases. The section defers RULER results to §J.Source: §7.7 · p. 30, Table 13 · p. 31, Figure 18 · p. 31

8.1 Transparency on external models and data

• External LLMs were used only for environment interaction, trace conversion, function tracing, and SFT, with mitigations applied to training data.Llama3-70B-Instruct and Qwen3-235B-A22B supported ForagerAgent; Qwen3-32B-FP8 converted Python traces; Llama3-70B-Instruct supported function tracing; and DeepSeek-R1 trajectories entered SFT through OpenMathReasoning and OpenCodeReasoning. Mitigations included algorithmic-bias filtering and cybersecurity protections, and no external LLM tokens were used beyond these contexts.Source: §8.1 · p. 31

8.2 Code World Model Preparedness Report

• The preliminary assessment compared CWM with Qwen3-Coder-480B-A35B-Instruct, Llama 4 Maverick, and gpt-oss-120B on catastrophic Cyber and Chemical & Biological risks.The authors also included a preliminary propensity evaluation and plan to expand it in future assessments.Source: §8.2 · p. 32
• CWM is judged unlikely to meaningfully raise cybersecurity or chemical/biological risks above the open ecosystem baseline and falls within Meta’s “moderate” catastrophic-risk threshold.The assessment also found undesirable propensities at rates comparable to most open-source models, though gpt-oss-120B had substantially lower rates.Source: §8.2 · p. 32

8.3 Limitations & future research

• CWM is released under a noncommercial research license, is not suitable for production or user-facing use, and has not been fully evaluated for real-world performance.The authors recommend caution when deploying or using the research models.Source: §8.3 · p. 32
• CWM is not a general-purpose assistant: it lacks thorough chatbot optimization and content-risk evaluation, may produce inappropriate or inaccurate responses, and may be weak outside code and reasoning.System-level protections including Llama Guard, Prompt Guard, and Code Shield are available, but alone are unlikely to suffice for production; CWM is intended for English only, with other-language performance unevaluated and unoptimized.Source: §8.3 · p. 32
• Current code-world-model data collection focuses on explicit Python execution; other languages and symbolic execution remain future work.The authors identify prompting or fine-tuning to leverage world-model knowledge across tasks and planning with formal or informal inference as research directions.Source: §8.3 · p. 32

9 Conclusion

• CWM’s ablations show that world-modeling data, Python execution traces, and executable Docker environments can directly benefit downstream task performance.The authors say more research is needed to consistently leverage world models during pre-training across tasks.Source: §9 · p. 33
• The authors propose that world models can support zero-shot planning, grounded chain-of-thought, and reinforcement learning with sparse, verifiable rewards.They describe Python tracing as enabling research on reasoning about code generation, execution, correctness, and verification.Source: §9 · p. 33
• Agents familiar with environment dynamics may focus reinforcement learning on actions that lead to rewards, while reasoning about action consequences may enable more efficient interactions and more complex tasks.The authors present these as potential benefits of world models, not established outcomes.Source: §9 · p. 33
• The authors envision Code World Models bridging language-level reasoning and executable semantics, enabling models to reason about the consequences of their actions.They argue that this could allow models to scale to more complex tasks.Source: §9 · p. 33

Core contributors

• Core contributors include Jade Copet, Quentin Carbonneaux, Gal Cohen, Jonas Gehring, Jacob Kahn, Jannik Kossen, Felix Kreuk, and Emily McMilin.The passage lists these eight names under Core contributors.Source: Core contributors · p. 34

Senior core contributors

• Senior core contributors are Taco Cohen, Yossi Adi, and Gabriel Synnaeve.The passage lists these three names under Senior core contributors.Source: Senior core contributors · p. 34

Reasoning agent

• In a bash-only SWE-bench Verified example, CWM recovers from incorrect edits by restoring the original file with git checkout, then makes a correct edit with sed.The bash-only environment is more challenging than one with dedicated tools such as file editing.Source: B CWM Examples · p. 42
• CWM runs tests to verify a patch preserves existing functionality before submitting it and generating a summary.Source: B CWM Examples · p. 42
• In the default SWE RL setting, CWM reasons extensively before editing, and the edit tool returns surrounding code as feedback.Source: B CWM Examples · p. 42

Python execution trace prediction

• Trace prediction correctly tracks list mutations that natural-language reasoning misses, while natural-language reasoning solves a complex statement that trace prediction gets wrong in one step.The authors suggest combining trace prediction’s groundedness with natural-language reasoning’s flexibility as a direction for future research.Source: B CWM Examples · p. 42
• CWM’s execution trace prediction supports neural debugging; proposed future capabilities include skipping loops in constant time, jumping to arbitrary lines, and predicting inputs to reach arbitrary states.The passage presents these capabilities as interesting future work.Source: B CWM Examples · p. 42
• For CruxEval output prediction experiments, the paper illustrates execution trace prediction.Source: B CWM Examples · p. 43
• Given behavior-consistent asserts and a pretend import without a function definition, CWM generates actions consistent with the desired function.The authors suggest that jointly tracing and generating code may relate to tracing data that omits third-party library source context; they identify improving code generation with trace prediction as future work.Source: B CWM Examples · p. 43

Program Termination

• CWM considers concrete inputs before generalizing that a program terminates on all inputs.Source: B CWM Examples · p. 43

C RL algorithm

• PPO advantages subtract a length-weighted batch mean return from each trajectory’s undiscounted total return.The batch comprises G rollouts, and the length measure is the total number of agent-generated tokens.Source: C RL algorithm · p. 43
• The behavior-policy log probabilities come from the inference backend when each token is sampled, despite asynchronous rollouts making the true behavior distribution difficult to describe mathematically.These worker-side log probabilities are importance-weighted against trainer-computed policy probabilities in the clipped PPO loss.Source: C RL algorithm · pp. 43, 53
• Training batches are queued and packed up to N = 131,072 tokens, masking environment-generated tokens and computing policy log probabilities on trainer nodes.The fixed token limit reduces batch-size variance and supports GPU utilization without over-allocating memory; trajectories from one prompt may span gradient updates.Source: C RL algorithm · p. 43

D Activ image-building pipeline

• Activ captures a successful GitHub Actions CI environment as a standalone Docker image by modifying workflows, running them locally with act, and identifying the container holding repository dependencies.For Python repositories, injected pytest fixtures detect the unit-test container; other repositories use workflow checks for framework executability before capture.Source: Figure D.29 · p. 53
• Activ reduces cross-platform build matrices to one compatible configuration and uses continue-on-error workflows and early exits to capture environments or stop stalled or timed-out runs.The pipeline executes approximately 500 repositories in parallel in secure, isolated virtual environments.Source: D Activ image-building pipeline · p. 53

E.1 Derivation of per-token compute formula

• A linear layer with N weights costs 6N FLOP across forward and backward passes, with backward computation approximately twice as expensive as forward.Forward computation costs 2N FLOP, and backward computation adds approximately 4N FLOP.Source: §E.1 · p. 54
• Causal self-attention costs 6SdL FLOP per token across L layers, after halving the attention computation to account for causality.The attention multiplications cost 12BS²d FLOP per layer for forward and backward passes before the causal reduction and division by BS tokens.Source: §E.1 · p. 54

E.2 Quasi-random search for batch size and learning rate

• At each scale, Sobol sequences generated batch-size and learning-rate candidates, with the sampled batch-size range increasing and learning-rate range decreasing as model scale grew.The two-dimensional candidates were rescaled to ranges that varied with model scale.Source: §E.2 · p. 54
• Figure E.30 shows a broad optimal batch-size/learning-rate range at each scale, with performance degrading rapidly beyond it.Gray points mark all evaluated candidates; blue points mark those within 1% of the best validation loss at that scale.Source: Figure E.30 · p. 54, §E.2 · p. 54

F RL data decontamination

• MinHash LSH decontaminates RL prompts against math, code-generation, scientific, commonsense/general-reasoning, and conversation evaluation benchmarks.The listed suites include AIME, LiveCodeBench, GPQA, HellaSwag, SimpleQA, ArenaHard, and MTBench.Source: F RL data decontamination · pp. 54–55
• Dockerized-repository RL data excludes SWE-bench Verified repositories (Jimenez et al., 2024) and checks residual patch overlap using instance-pair Jaccard similarity below 0.2.The procedure removes Docker images built from SWE-bench Verified repositories before checking for unexpected instance-level contamination.Source: F RL data decontamination · p. 55

G Mathematical expression comparison for RL

• Math RL answers are accepted when either the custom verifier or MathVerify deems the prediction equivalent to ground truth.The custom verifier grows equivalent-expression sets using string and LaTeX normalization, numerical and symbolic computation, and SymPy simplification, including recursively for structured objects.Source: G Mathematical expression comparison… · p. 55

H Prompting guide

• Chat messages use a start-header/role/end-header structure, with an assistant header followed by two newlines as the inference prompt prefix.The first conversation token is expected to be <|begin_of_text|>, and <|eot_id|> marks the end of the assistant reply.Source: H Prompting guide · p. 56
• Reasoning mode is enabled through the system prompt and uses <think>…</think> to delimit internal reasoning from the external answer.The prompt should end with a leading <think> followed by a newline.Source: H Prompting guide · p. 56
• Tool calls use a <tool: $TOOL> $CONTENT </tool> format, with available tools announced in the system prompt and control returning to the model after tool output.User code detects calls and responds using the tool’s role.Source: H Prompting guide · p. 56
• Trace prediction requires source-code context and a frame sentinel, then represents execution as frames pairing local-variable observations with source-line actions.Frame types cover function calls, line steps, returns, and exceptions; locals and return or exception values are JSON-encoded, and end-of-text marks execution end.Source: H Prompting guide · p. 57

I Formal mathematics datamix

• Formal-mathematics training uses Lean 4 datasets for proof-state transitions, proof formalization, and theorem-statement formalization.Sources include LeanUniverse (Aram H. Markosyan, 2024), Goedel-Pset (Lin et al., 2025), Compfiles (CompFiles authors, 2025), Lean Workbook (Ying et al., 2025), miniF2F (Zheng et al., 2022), and ProofNet (Azerbayev et al., 2023).Source: I Formal mathematics datamix · p. 57

J RULER evaluation

• CWM outperforms Gemma3-27B at 128 k sequence length but falls short at 32 k.The comparison is against post-trained models.Source: J RULER evaluation · p. 58
• Both CWM and Gemma3-27B achieve worse performance than Qwen3-32B.The passage notes that Qwen3-32B uses full attention across all layers.Source: J RULER evaluation · p. 58
• Qwen3-32B’s full attention incurs significantly higher computational costs, particularly for longer sequences, making CWM a trade-off between efficiency and performance.Source: J RULER evaluation · p. 58

K Agent capabilities learnt during RL training

• File recall is the percentage of gold-patch files edited by the agent during a rollout, formalized as the intersection of edited-file sets divided by the gold-patch file count.The numerator counts files edited by both the agent and the gold patch; the denominator counts files edited in the gold patch.Source: K Agent capabilities learnt during RL… · p. 58
• Testing rose from 57% of trajectories at RL training start to 74% after 4,000 steps, with tests run on at least one rollout turn.Figure K.31a shows the share of trajectories in which the agent runs tests on at least one turn.Source: Figure K.31 · p. 58, K Agent capabilities learnt during RL… · p. 58
• Average file recall increased from 58% at RL training start to over 66% after 4,000 steps.Recall is the percentage of files in the gold patch that the agent edited during a rollout.Source: Figure K.31 · p. 58, K Agent capabilities learnt during RL… · p. 58