MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement

1 Introduction

• A hybrid sparse MoE backbone, multimodal encoders, and agent-centric mid-training provide a multimodal foundation and expand exploration space for later agentic tasks.The backbone interleaves Local Sliding Window Attention with Global Attention.Source: §1 · p. 3
• The framework scales training throughput, environment and harness diversity, and groupwise agentic grading to produce finer reward signals and guide more accurate, efficient solutions.The grading compares solutions within groups rather than assigning identical rewards to all solutions that pass binary tests.Source: §1 · p. 3
• Unified trajectories, high-concurrency rollout, decoupled control and data planes, and training-inference alignment support scalable mixed-task agentic RL.Sample mixing, dynamic sampling, and partial rollout stabilize per-task sample composition in training batches.Source: §1 · p. 3
• Increasing RL steps improves Pro and Flash across coding, general workflows, visual tasks, and cybersecurity, with gains associated with stronger complex problem solving and daily co-work performance.Reported domains include DeepSWE, AutomationBench, MiMo Visual Coding, and MiMo Cyber Bench.Source: §1 · pp. 3–4
• The release includes a distilled model, verified multi-domain environments, an end-to-end RL framework, and a composable harness, with experiments showing consistent gains across tasks and harnesses.The authors intend these resources to lower barriers to reproducing and extending large-scale RL experiments.Source: §1 · p. 4

2.1 Overall Architecture

• The text backbone repeats blocks of N SWA layers followed by a GA layer, with sparse MoE feed-forward networks and a dense first block.The sliding window size is 128, and the backbone also includes an SWA-and-dense-FFN multi-token prediction module.Source: §2.1 · p. 4

2.2 MiMo-ViT

• MiMo-ViT alternates sink-augmented SWA with periodic GA and alternates row-major and column-major serialization to propagate visual information across spatial axes.The design reduces high-resolution visual-processing cost while achieving performance comparable to GA ViTs.Source: §2.2 · p. 4

2.3 Audio Encoder

• Audio tokenization uses a causal hybrid-attention Transformer and RVQ, then groups frames into patches processed with within-patch bidirectional attention before projection into backbone embeddings.The tokenizer represents each frame with 20 discrete audio tokens; the patch encoder reduces the sequence rate from 25 Hz to 6.25 Hz.Source: §2.3 · p. 5

2.4 Speculative Decoder

• The speculative decoder uses five dense-FFN Transformer layers to draft seven tokens from backbone features and an anchor token for parallel backbone verification.Draft layers use SWA; tokens attend within their block and to at most 1,024 preceding backbone context positions.Source: §2.4 · pp. 5–6

3.1 Pre-Training Setup

• The pretraining corpus combines text sources with vision data including OCR, GUI, video, and visual coding, plus speech-text, ASR, and audio-captioning data.The text sources include public web content, books, academic papers, code, and STEM materials.Source: §3.1 · pp. 6–7
• Pretraining first trains the language backbone on text, then jointly trains the full model on omni-modal data, extending context from 32K to 256K tokens.Flash uses 48T total pretraining tokens and Pro 30T, with different allocations between the text and omni-modal stages.Source: §3.1 · p. 7

3.2 Mid-Training Setup

• Mid-training bridges broad multimodal pretraining and RL with agent-centric data, long-context support, and an optimization setup adapted for large batches.Its mixture includes agent trajectories across coding, general, visual, and research tasks, alongside text, code, and image, video, and audio data.Source: §3.2 · p. 7
• For mid-training, Muown replaces AdamW on hidden weight matrices, while embeddings, the LM head, and MoE router remain on AdamW; no loss spike was observed.MXFP4 quantization-aware training is also used to adapt to low-precision computation while preserving model quality.Source: §3.2 · p. 7

4 Scaling Reinforcement Learning

• After a short SFT stage, RL scales training computation, environments and agent harnesses, and grader computation.The report associates this three-dimensional scaling with a fundamental leap in model capabilities.Source: §4 · p. 7
• Figure 3 tracks DeepSWE v1.1 average@3 score against cumulative RL cost.The left panel presents this score-cost relationship.Source: Figure 3 · p. 8
• Figure 3 breaks down training, rollout, and grader cost shares for MiMo-V2.6-Pro and MiMo-V2.6-Flash.These cost categories are shown in the right panel.Source: Figure 3 · p. 8

• DeepSWE average@3 rose from 58.4 to 72.6 for MiMo-V2.6-Pro and from 48.7 to 65.7 for MiMo-V2.6-Flash over RL.Source: §4.1 · p. 8
• Each step rolls out 25K sequences from 1,568 prompts at group size G=16, totaling 2.7B–3.7B training tokens.The rollout batch is approximately 110K–150K tokens per sequence.Source: §4.1 · p. 8
• For MiMo-V2.6-Pro, rollout uses 43.8% of compute, training 43.5%, and grading 12.7%.Source: §4.1 · p. 8
• Partial rollout leaves in-flight sequences to resume in a later phase, while dynamic sampling filters groups that are all-pass or all-fail.Resuming requires rebuilding the KV cache after policy updates; a Sample Mixer manages batches across tasks with different rollout durations and pass rates.Source: §4.1 · p. 9

4.2.1 Code Agent Tasks

• Coding tasks use specification–test alignment, eight-run stability checks, and rollout audits to target accurate, stable rewards.Rollout audits flag disagreements between observed rewards and independent assessments of solution correctness.Source: §4.2.1 · pp. 10–11

4.2.2 General Agent Tasks

• General-agent environments use resettable local workspaces and software mocks, with agent-built tasks assessed by atomic rubric items and reviewed rollouts.Rubrics combine code-based checks for deterministic properties with LLM-based checks for open-ended content.Source: §4.2.2 · pp. 11–12

4.2.3 Visual Agent Tasks

• Visual tasks cover open-ended design and high-fidelity replication, using groupwise comparisons for aesthetic quality and similarity metrics plus LLM judging for fidelity.Source: §4.2.3 · pp. 12–13

4.2.4 Cybersecurity Agent Tasks

• Cyber tasks ask agents to trigger the target vulnerability; sanitizer type-and-location matching provides deterministic verification, with source and compiled harness supplied.Source: §4.2.4 · p. 13

4.2.5 Multi-Harness Training

• Decoupled mini-harnesses provide controllable combinations of interaction mechanisms, supporting analysis and transferable task-solving strategies.Production harnesses are tightly coupled and include prompts or requirements outside the task-completion reward signal.Source: §4.2.5 · p. 14

4.2.6 Reward Hacking Mitigation

• Reward-hacking defenses address solution leakage through mid-training alignment examples, environment cleanup, adversarial screening, and iterative repairs.Hack-agent checks uncover exploit paths and are rerun after cleanup until no successful exploit is found in the environments.Source: §4.2.6 · pp. 14–15
• Detected reward-hacking trajectories stayed below 2% throughout training for both MiMo-V2.6-Flash and MiMo-V2.6-Pro.The process combines environment preparation and hack-agent screening with offline trajectory audits during training.Source: §4.2.6 · pp. 15–16, Figure 6 · p. 16

4.3.2 Groupwise Advantage Redistribution (GAR, Online Grading)

• GAR compares mixed-outcome trajectories, resets confirmed-hack rewards to zero, and redistributes positive advantage toward higher-quality passing solutions.The grader examines repository context and candidate patches across five dimensions before ranking trajectories.Source: §4.3 · p. 16, §4.3.2 · p. 18

4.3.1 Groupwise Reward Synthesis (GRS, Offline Rubrics)

• GRS reuses task-grounded solution and behavior rubrics with binary tests, distinguishing passing rollouts by implementation quality and problem-solving behavior.Rubrics are built by analyzing offline rollouts, but requirements are grounded in the task rather than limited to behaviors observed in those rollouts.Source: §4.3.1 · pp. 16–18

4.3.2 Groupwise Advantage Redistribution (GAR, Online Grading)

• Online groupwise grading sustained pass-rate gains through step 52 while turns stayed roughly stable and token length grew gradually.The comparison used code-only RL with MiMo-V2.6-Flash, batch size 128, and token-mean loss aggregation.Source: §4.3.2 · p. 18, Figure 8 · p. 19

4.3.3 Behavioral Regularization

• Behavioral regularization combines a pass-rate-gated, prompt-relative length penalty with segment-level penalties for format and tool-call errors.Batch-level advantage rebalancing adjusts flagged and unflagged token advantages while limiting excess negative optimization pressure.Source: §4.3.3 · pp. 19–20

5.1 Training Setup

• Training used 1,568 prompts with 16 rollouts each, GRPO with asynchronous partial rollouts, a frozen router, prompt-mean loss aggregation, and per-token importance sampling.Positive and negative importance-sampling bounds were tuned independently at runtime based on policy entropy.Source: §5.1 · pp. 20–21

5.2 Evaluation Settings

• Evaluation covered code, cybersecurity, general-agent, and visual-agent benchmarks, combining public suites with internal MiMo benchmarks.Configurable-reasoning baselines were evaluated at their highest supported setting.Source: §5.2 · pp. 21–22

5.3 RL Performance

• Held-out harness mean Pass@1 rose from approximately 50% to 66%, while training-harness performance also improved during multi-harness RL.The held-out harnesses were codex, claude code, and mini-swe-agent; the gap between mean training and held-out performance narrowed.Source: §5.3 · p. 22, Figure 10 · p. 23

5.4 Router Freezing for Stable RL

• With a trainable router, expert-load CV rose from 0.78 to 2.0 and peak load from 6× to 16×; freezing kept load metrics flat as benchmark performance grew.The cold-expert fraction rose from 0.5% to 22% with a trainable router; restoring initial router parameters recovered balance without changing benchmark performance.Source: Figure 11 · p. 23, §5.4 · p. 23

5.5 RL Failure Analysis

• Training runs were interrupted by infrastructure, rollout, MoE-imbalance OOM, and host-memory failures.One expert-parallel rank received over 30× the mean token load despite a relatively balanced full batch; late Flash packing also caused CPU OOM.Source: Figure 12 · p. 24, §5.5 · p. 24

5.6 Broadening Capabilities via MOPD2

• MOPD2 combines autonomous student rollouts with prefix-conditioned single-turn distillation, using teacher-rollout or SFT-data prefixes and student-generated continuations.SFT-Prefix OPD uses demonstration context for open-domain tasks with difficult-to-design RL rewards; the student generates its own continuation rather than imitating a fixed response.Source: §5.6 · pp. 24–25, Figure 13 · p. 25
• The MiMo-V2.6 series substantially improves over MiMo-V2.5 and achieves performance comparable to frontier models across domains.MOPD2 extends the RL-trained model to tasks where training-time verification is challenging, including long-horizon game development, scientific research, and embodied intelligence.Source: §5.6 · p. 25

6.1 Agentic RL with Fine-grained Learning Signals

• The infrastructure defines a Sample → Sequence → Context → Segment trajectory hierarchy and applies configurable masking, advantage shaping, monitoring, and early stopping to refine credit assignment.Only model-generated turns contribute to loss; penalties can drop contexts, zero sequence advantages, or reject samples.Source: §6.1 · pp. 26–27

6.2 Harness Pool and Payload Porter: Large-Batch RL with Multiple Harnesses

• Persistent multi-tenant actor pools host concurrent Agent Loops and harnesses, while the Payload Porter stores heavy trajectory payloads separately from lightweight scheduling metadata.Packing fetches only the context-parallel windows needed by each rank, avoiding full-batch aggregation on the driver and dense padded intermediates.Source: §6.2 · pp. 28–29

6.3 Sample Mixer: Stable Asynchronous Mixed-task RL

• The Sample Mixer combines adaptive concurrency, source scheduling, predictive dispatch, and replay to maintain the target training distribution despite large workload differences.Across 25 sources, mean generated tokens and active rollout duration vary by 90× and 66×, respectively.Source: §6.3 · pp. 29–31

6.4 Training/Inference Consistency and Optimization

• Training–inference consistency replays rollout expert choices and top-p candidate sets during training, while context caching retains dialogue KV state and transfers only newly introduced images.These mechanisms align discrete execution choices and avoid resending historical visual inputs within a policy version.Source: §6.4 · pp. 31–32
• 31.3% higher average accepted length with block-6 DFlash than MTP, and approximately 6% higher global average throughput with block-6 than block-8, are reported for rollout acceleration.RL-adapted FP8 DFlash achieves approximately 10.3% higher per-node throughput than the baseline on the long-context workload.Source: §6.4 · p. 32

7 Open Foundations for Agentic RL

• The report releases the distilled model, curated environments with verifiers, an end-to-end RL framework, and a composable mini-harness to support reproducible agentic RL research.It also establishes domain-specific GRPO baselines and reports gains across multiple domains.Source: §7 · p. 33

7.1 Distillation from MiMo-V2.6

• The released MiMo-V2.6-Distill-Qwen-9B is produced by SFT of Qwen3.5-9B on 77.4 billion tokens of MiMo-generated data, including 27.2 billion loss tokens.The distill checkpoint improves over Qwen3.5-9B across reported evaluations, including SWE-bench Pro from 32.0 to 44.6.Source: §7.1 · p. 33

7.2 RL with Open Environments

• The released environments span coding, cybersecurity, general-domain, and visual tasks, with approximately 7k tasks across four training sets and approximately 1k music-generation tasks.The general-domain set focuses on knowledge work.Source: §7.2 · p. 34, Table 5 · p. 34
• RL improves on the SFT checkpoint in all 11 reported evaluations spanning four domains, including SWE-bench Verified from 61.1 to 66.2 and MiMo Cyber Bench (mini) from 31.3 to 47.0.Internal music-benchmark scores also rise from 45.7 after SFT to 52.5 after RL.Source: §7.2 · p. 34, Table 6 · p. 35
• Multi-harness RL further improves every one of 21 dataset–harness pairs, with additional MiMo Code Bench (mini) gains of 1.8–9.3 percentage points across seven harnesses.Evaluation includes four jointly optimized mini-harnesses and three additional held-out harnesses.Source: §7.2 · p. 34, Table 7 · p. 35

Qwen3.5-9B SFT RL

• 61.7 → 64.0 → 72.4 is the reported Table 6 trajectory across Qwen3.5-9B, SFT, and RL, alongside qualitative improvements in visual design and page completeness.RL examples include refined typography and a more complete HR dashboard; the figure presents all three models on the same prompts.Source: Qwen3.5-9B SFT RL · p. 36, Figure 17 · p. 36

8 Conclusion

• MiMo-V2.6 scales RL batch size and throughput, environment and harness diversity, and groupwise grading, supported by asynchronous training, mixed-task rollout infrastructure, and training–inference consistency.The report states that these advances enable sustained long-horizon optimization while improving solution quality and token efficiency.Source: §8 · pp. 36–37
• Evaluations across public and internal benchmarks show competitive performance against frontier models across a broad range of agentic tasks.The report describes consistent RL gains across domains and harnesses as evidence for the value of the released resources as a shared foundation.Source: §8 · p. 37
• The release combines MiMo-V2.6-Distill-Qwen-9B with curated environments, verifiers, an end-to-end RL framework, and composable mini-harnesses to support reproducible research.The report frames this as a practical step toward model self-improvement.Source: §8 · p. 37

• The authors thank the Xiaomi Data Platform, CloudML, NGK, MiChat, Mify, MiKS, LLM-Plus teams, and others not explicitly listed.Source: A Contributions and Acknowledgments · p. 43
• Authors within each role are listed in reverse alphabetical order by first name.Source: A Contributions and Acknowledgments · p. 43
• The section lists core contributors separately from contributors.Source: A Contributions and Acknowledgments · p. 43