PaperlayerView exact revision on arXiv

Source-linked AI summary

QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management

Weizhou Shen, Ziyi Yang, Chenliang Li, Zhiyuan Lu, Miao Peng, Huashan Sun, Yingcheng Shi, Shengyi Liao, Shaopeng Lai, Bo Zhang, Dayiheng Liu, Fei Huang, Jingren Zhou, Ming Yan

arXiv:2512.12967v1cs.CL

TL;DR

QwenLong-L1.5 addresses the lack of mature post-training systems for long-context reasoning and memory management. It combines synthesized multi-hop data, stabilized RL, and memory-augmented processing, achieving performance comparable to flagship models and substantial gains over its baseline.

  • Problem

    Long-context research lacks a mature post-training recipe combining challenging reasoning data, RL methods tailored to long-context training, and architectures for inputs beyond the context window.

  • Method

    The paper unifies scalable data synthesis, tailored reinforcement learning, and a memory-augmented agent architecture in a Qwen3-30B-A3B-Thinking-based model.

  • Results

    QwenLong-L1.5 achieves performance comparable to GPT-5 and Gemini-2.5-Pro, surpassing its baseline by an average of 9.90 points.

  • Takeaways & Limitations

    The acquired long-context reasoning ability extends beyond long-context benchmarks to mathematics, natural science, tool-using agents, and other general domains.

  • Takeaways & Limitations

    The current reward function is effective for clear-cut correctness tasks but less effective for open-ended and subjective tasks where correctness is multifaceted.

Abstract

from arXiv · show

We introduce QwenLong-L1.5, a model that achieves superior long-context reasoning capabilities through systematic post-training innovations. The key technical breakthroughs of QwenLong-L1.5 are as follows: (1) Long-Context Data Synthesis Pipeline: We develop a systematic synthesis framework that generates challenging reasoning tasks requiring multi-hop grounding over globally distributed evidence. By deconstructing documents into atomic facts and their underlying relationships, and then programmatically composing verifiable reasoning questions, our approach creates high-quality training data at scale, moving substantially beyond simple retrieval tasks to enable genuine long-range reasoning capabilities. (2) Stabilized Reinforcement Learning for Long-Context Training: To overcome the critical instability in long-context RL, we introduce task-balanced sampling with task-specific advantage estimation to mitigate reward bias, and propose Adaptive Entropy-Controlled Policy Optimization (AEPO) that dynamically regulates exploration-exploitation trade-offs. (3) Memory-Augmented Architecture for Ultra-Long Contexts: Recognizing that even extended context windows cannot accommodate arbitrarily long sequences, we develop a memory management framework with multi-stage fusion RL training that seamlessly integrates single-pass reasoning with iterative memory-based processing for tasks exceeding 4M tokens. Based on Qwen3-30B-A3B-Thinking, QwenLong-L1.5 achieves performance comparable to GPT-5 and Gemini-2.5-Pro on long-context reasoning benchmarks, surpassing its baseline by 9.90 points on average. On ultra-long tasks (1M~4M tokens), QwenLong-L1.5's memory-agent framework yields a 9.48-point gain over the agent baseline. Additionally, the acquired long-context reasoning ability translates to enhanced performance in general domains like scientific reasoning, memory tool using, and extended dialogue.

1 Introduction

QwenLong-L1.5 presents a post-training recipe for long-context reasoning that combines scalable data synthesis, specialized reinforcement learning, and memory mechanisms. It reports substantial benchmark gains, including performance comparable to GPT-5 and Gemini-2.5-Pro.

  • The paper identifies a post-training gap: existing work lacks an end-to-end recipe combining long-context data synthesis, tailored RL, and architectures beyond context capacity.
  • QwenLong-L1.5 builds on Qwen3-30B-A3B-Thinking and unifies data synthesis, training methods, and memory-augmented agent architectures.
  • Its synthesis pipeline constructs verifiable multi-hop questions from atomic facts and relationships, moving beyond simple retrieval or single-hop RAG tasks.
  • QwenLong-L1.5 surpasses its Qwen3-30B-A3B-Thinking-2507 baseline by an average of 9.9 points and reaches performance comparable to GPT-5 and Gemini-2.5-Pro.
  • On 1M∼4M-token tasks, its memory-agent framework outperforms the baseline’s single-pass configuration by 15.26 points and its memory-agent configuration by 9.48 points.

2 Preliminary

The method formulates long-context reasoning as RL and combines GRPO-based optimization with a memory agent for contexts that exceed full-attention capacity. The memory workflow processes chunks sequentially, updates compact state, and generates the final answer from accumulated memory.

  • 2.2 Group Relative Policy Optimization (GRPO): GRPO samples response groups and estimates advantages through group-wise reward normalization, avoiding a separate value network.
  • 2.2 Group Relative Policy Optimization (GRPO): For long-context inputs, GRPO is used because PPO with value-network-based generalized advantage estimation is computationally prohibitive under quadratic attention complexity.
  • 2.2 Group Relative Policy Optimization (GRPO): DAPO-style token-level normalization gives each token in a group equal contribution, preserving signals from valuable long responses and penalizing undesirable lengthy outputs.
  • 2.3 Memory Agent: The memory-agent paradigm reframes ultra-long reading comprehension as sequential decision-making rather than single-pass inference.
  • 2.3 Memory Agent: The memory agent separates core-question reasoning from formatting instructions, partitions documents into chunks, and updates memory and navigation plans at each step.
  • 2.3 Memory Agent: The recurrent process folds global context into a compact representation, after which the final answer integrates accumulated memory with the original formatting instructions.

3 Long-Context Data Construction

The data-construction system scales long-context RL training with filtered corpora, synthesized questions, and verification procedures designed to require dispersed evidence and multi-hop reasoning. It covers graph-based, numerical, and general reasoning tasks through specialized synthesis methods.

  • The pipeline retains 14.1k high-quality training samples from 42.7k synthesized examples after difficulty filtering, deduplication, and test-set decontamination.
  • 3.1 Corpus Collection and Preprocessing: The end-to-end process collects long documents, synthesizes difficult questions with dispersed evidence, and verifies grounding and contextual robustness.
  • 3.1 Corpus Collection and Preprocessing: The corpus spans heterogeneous sources including code, academic literature, professional documents, and general knowledge and literature.
  • 3.2 Question-Answer Synthesis: Knowledge-graph synthesis mines cross-document relationships to construct challenging multi-hop reasoning questions.
  • 3.2 Question-Answer Synthesis: The synthesis framework also generates corpus-level numerical questions and uses multi-agent self-evolution for viewpoint analysis and long in-context learning.

4 Long-Context Post-Training

The post-training recipe combines balanced long-context data sampling, task-specific advantage estimation, entropy-controlled optimization, and multi-stage training to stabilize long-context RL. These strategies improve benchmark performance while addressing reward imbalance, training instability, and the distinct similarities between correct and incorrect long-context reasoning paths.

  • Long-Context Post-Training: Long-context RL uses multi-stage length extension because abruptly switching from short-input reasoning to multihop grounding can destabilize training.Reasoning output length generally increases with input context length, motivating adaptation to varying output requirements.
  • Memory Management: QwenLong-L1.5 trains specialized full-context and memory-management experts separately because mixing their data harms RL efficiency and training stability.The resulting approach uses specialized-expert training followed by model merging.
  • Multi-Task Reinforcement Learning: Task-balanced sampling stabilizes entropy dynamics, but its combination with GRPO still permits substantial response-length growth and only moderate overall gains.The sampler addresses distributional imbalance across heterogeneous long-context task clusters.
  • Multi-Task Reinforcement Learning: 2.55 points: task-specific advantage estimation improves average performance over GRPO, particularly on dense-reward MRCR tasks.Separate task-batch reward variance isolates dense- and sparse-reward tasks for more accurate advantage estimation.
  • Negative Gradient Clipping: Long-context tasks require locating relevant information before reasoning, making correct and incorrect responses more similar than in short-context settings.This similarity motivates clipping high-entropy negative responses or tokens and later using sequence-level negative-gradient clipping.
  • Adaptive Entropy-Controlled Policy Optimization (AEPO): 3.29 points: AEPO improves average performance over GRPO on Qwen3-4B-Thinking-2507 while balancing exploration and exploitation on the larger model.AEPO masks negative-advantage samples when entropy is high and reintroduces them when entropy falls below the target range.

5 Experiments

QwenLong-L1.5 improves long-context reasoning across standard, generalization, and ultra-long memory-agent evaluations, with gains concentrated on complex information-integration tasks and longer contexts.

  • Overall Performance Evaluation: 71.82 average score demonstrates QwenLong-L1.5-30B-A3B’s broad advantage, including competitiveness with Gemini-2.5-Pro at 72.40 and state-of-the-art MRCR performance at 82.99.It surpasses DeepSeek-R1-0528 at 68.67, Gemini-2.5-Flash-Thinking at 68.73, and its baseline at 61.92.
  • Analysis of Performance on Specific Task Categories: Performance gains are most pronounced on multi-hop reasoning and information-aggregation tasks that require connecting discontinuous or distributed evidence.The model is comparable to flagship models on Longbench-V2, Frames, and LongBench-V1-QA, and scores 81.25 on CorpusQA versus GPT-5’s 81.56.
  • Quantifying the Impact on Long-Context Performance: +9.90 points over Qwen3-30B-A3B-Thinking-2507, with the largest gains on MRCR (+31.72), CorpusQA (+9.69), and LongBench-V2 (+6.16).These benchmarks have average context lengths of 36.5K, 92.8K, and 85.5K tokens, respectively.
  • Generalization Benefits from Long-Context Enhancement: The model avoids degradation on out-of-distribution general knowledge and reasoning tasks, while improving AIME25 by +3.65 and GPQA-Diamond by +0.90.The evaluation describes this stability alongside training data occupying a distinct semantic space from the evaluation sets.
  • Generalization Benefits from Long-Context Enhancement: +15.60 points on LongMemEval shows a substantial gain in dialogue memory, while Memory-KV improves by +5.80 on BFCL-V4.The reported gains extend to agentic memory and extended-conversation state maintenance.
  • Ultra-Long Context Evaluation: The memory-agent framework reaches 14.29 on CorpusQA at 4M tokens and remains more than 18 points above the baseline across reported MRCR ranges.On MRCR, it scores 34.87 at 128K–512K and 22.53 at 512K–1M.
  • Performance Dynamics of Progressive Long-context Post-training: Multi-stage training yields a unified model with both full-context and memory-agent capabilities, restoring full-context performance to 71.18 while raising memory-agent performance to 22.53.Specialized memory-RL initially lowers full-context performance to 68.53, while merging and later full-context RL recover and improve both capabilities.

6 Conclusion

QwenLong-L1.5 combines scalable data synthesis, tailored reinforcement learning, and memory-agent architecture to improve long-context reasoning and extend it to ultra-long contexts.

  • 6 Conclusion: QwenLong-L1.5 unifies a scalable data synthesis pipeline, tailored RL methodologies, and a memory-augmented agent architecture in a comprehensive post-training recipe.The system improves multi-hop reasoning and information aggregation, generalizes to mathematics, tool-use agents, and dialogue, and extends operation to the 1M–4M-token regime.

7 Limitations and Future Works

The paper identifies limitations in data coverage, synthesis scalability, credit assignment, and reward modeling, and outlines corresponding future directions.

  • Data Coverage and Synthesis: The current synthesis pipeline is not optimized for long-input, long-output applications and remains exclusively text-based.Future work targets broader task taxonomies and multimodal data.
  • Data Scaling: Automated data synthesis remains constrained by proprietary-model API quotas and the computational cost of serving large open-source models.A closed-loop data flywheel is proposed to reduce reliance on these external resources.
  • Reinforcement Learning: AEPO stabilizes training but does not fundamentally solve credit assignment because the current method applies one advantage signal to an entire reasoning step.The proposed direction is token-level credit assignment within thinking trajectories.
  • Reward Model: The reward function is effective for clear-cut question-answering tasks but less effective for open-ended tasks with multifaceted correctness.The authors propose rubric-based reward models to provide richer signals for complex human preferences.

A Detailed Results on LongBench-V2 and LongBench-V1 QA Subsets

Detailed LongBench analyses show broad gains over the baseline, with the strongest improvements concentrated in selected context-length and multi-hop reasoning subsets.

  • Overall Pattern: The reported gains are concentrated in specific areas aligned with the post-training strategy.The fine-grained analysis covers LongBench-V2 and LongBench-V1 QA subsets.
  • LongBench-V2: +10.23 points on the Medium-length LongBench-V2 subset mark the largest reported gain across context-length subsets.The Medium subset covers 32K∼128K tokens, a focus area of the synthetic data generation.
  • LongBench-V2: QwenLong-L1.5 improves over its baseline across all LongBench-V2 difficulty and context-length sub-dimensions.The detailed tables report gains relative to the base model.
  • LongBench-V1 QA: +7.00 points on Musique and +9.00 points on NarrativeQA demonstrate gains in multi-hop reasoning and extended-document information integration.QwenLong-L1.5 is also reported as comparable to Gemini-2.5-Pro on several key reasoning tasks, while Qasper decreases by 3.00 points.

B QwenLong-L1.5 Synthetic Data Cases

The synthetic-data cases span diverse reasoning formats, including numerical calculation, in-context learning, viewpoint analysis, financial extraction, and multi-fact reasoning.

  • Case Coverage: The constructed cases include numerical calculation, long in-context learning, viewpoint analysis, multi-fact reasoning, hypothetical scenarios, temporal reasoning, and causal analysis.These categories are presented as examples of the long-context RL data synthesis pipeline.
  • Financial Reasoning: Other cases require extracting and aggregating financial information from annual-report documents.One question asks for differences between segment operating expenses, while another concerns segment-performance definitions.
  • Long In-context Learning: Long in-context learning cases can require exact tensor-shape derivation from model-conversion and quantization implementation details.The example specifies dimensions, quantization settings, and target layers such as wqkv and w2.
  • Multi-fact Reasoning: The examples pair questions with source documents and, in some cases, explicit reasoning chains linking distributed facts across documents.The multi-fact example connects multiple Assassin’s Creed and Watch Dogs facts to identify an unlockable character outfit.

C.1 General Performance Gain

The case analysis contrasts QwenLong-L1.5 with its baseline on difficult reasoning tasks, highlighting hypothesis refinement and strategic adaptation alongside failure modes in negative rollouts.

  • General Performance Gain: The analysis characterizes QwenLong-L1.5’s improvement as hypothesis refinement and strategic adaptation compared with Qwen3-30B-A3B-Thinking.The associated conclusion emphasizes self-correction and high-level algebraic modeling.
  • Transition from Heuristic Guessing to Algebraic Synthesis: QwenLong-L1.5 refines the hypothesis from local minima to a global-minimum interpretation and derives candidates {8, 32, 200, 512}.The baseline instead selects {18, 72, 98} and obtains the incorrect sum 188.
  • Negative Rollouts: High-entropy negative rollouts can invalidate an initially correct path through erroneous self-correction or failure to rectify a detected error.These behaviors are illustrated in separate rollout examples.
  • Negative Rollouts: Frequent blank-line occurrences after reaching a correct answer can degrade reasoning quality and produce an incorrect final answer.This is presented as another high-entropy negative rollout pattern.

D Prompt Template

The memory-agent prompt directs sequential reading of article sections, updating self-contained memory and planning while prioritizing information relevant to the problem.

  • The agent reads article sections in their original order and updates memory with information useful for answering the problem.It receives the problem, previous memory, and the current section as inputs.
  • The prompt requires updated memory to remain self-contained and sufficient to answer the problem without additional context.
  • The agent must critically revise prior memory and plans when current-section information reveals inaccuracies or suboptimal content.
  • Because reading is strictly one-way, relevant information must be fully integrated into memory rather than merely referenced for later retrieval.
  • The prompt instructs the agent to prioritize factual relevance and avoid content that does not directly contribute to solving the problem.
Loading 2512.12967v1…