PaperlayerView exact revision on arXiv

Source-linked AI summary

Free(): Learning to Forget in Malloc-Only Reasoning Models

Yilun Zheng, Dongyang Ma, Tian Liang, Jiahao Xu, Xinting Huang, Lihui Chen, Haitao Mi, Yan Wang

arXiv:2602.08030v2cs.AIcs.CL

TL;DR

Longer test-time reasoning may fail to improve accuracy and can degenerate; Free()LM adds a LoRA-based forgetting mechanism to prune redundant context. Across six benchmarks and model scales, it reports a 3.3% average boost over top-tier reasoning baselines.

  • Problem

    Reasoning models can benefit from test-time compute, but longer thoughts do not reliably improve accuracy and may cause degeneration.

    Source: §1 · p. 1

  • Method

    Free()LM uses a plug-and-play LoRA adapter that switches between reasoning and cleaning modes to identify and prune redundant context.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    3.3% average boost over top-tier models across six benchmarks, with evaluations spanning model scales from 8B to 685B.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings support active context pruning as a way to sustain reasoning on long-horizon tasks where standard models collapse.

    Source: §1 · p. 2, §5 · p. 10

    The authors caution against claiming a definitive intelligence boost, while describing reasoning sustainability as supported.

    Source: §3.4 · p. 8

Abstract

from arXiv · show

Reasoning models enhance problem-solving by scaling test-time compute, yet they face a critical paradox: excessive thinking tokens often degrade performance rather than improve it. We attribute this to a fundamental architectural flaw: standard LLMs operate as "malloc-only" engines, continuously accumulating valid and redundant steps alike without a mechanism to prune obsolete information. To break this cycle, we propose Free()LM, a model that introduces an intrinsic self-forgetting capability via the Free-Module, a plug-and-play LoRA adapter. By iteratively switching between reasoning and cleaning modes, Free()LM dynamically identifies and prunes useless context chunks, maintaining a compact and noise-free state. Extensive experiments show that Free()LM provides consistent improvements across all model scales (8B to 685B). It achieves a 3.3% average improvement over top-tier reasoning baselines, even establishing a new SOTA on IMOanswerBench using DeepSeek V3.2-Speciale. Most notably, in long-horizon tasks where the standard Qwen3-235B-A22B model suffers a total collapse (0% accuracy), Free()LM restores performance to 50%. Our findings suggest that sustainable intelligence requires the freedom to forget as much as the power to think.

1 Introduction

Excessive test-time reasoning can cause accuracy drops and degeneration rather than continued gains. Free()LM adds a mechanism to prune obsolete context while reasoning continues.

Introduction

  • Among 31 Qwen3-8B instances reaching the context limit, 26 (84%) were trapped in repetitive loops, and degeneration reached 100% by 48k tokens.Degeneration rises sharply once thinking tokens exceed 16k; excessive reasoning typically stops helping when tokens occupy about 70–90% of the context window.Source: §1 · pp. 1–2
  • Standard reasoning models passively accumulate useful and obsolete steps without a mechanism to discard them, producing noise and eventual degeneration as thinking grows.The paper describes this as a “malloc-only” architecture and illustrates the reasoning process eventually crashing as context accumulates.Source: §1 · pp. 1–2, Figure 1 · p. 1
  • Free()LM uses a plug-and-play LoRA Free-Module that switches between unmerged reasoning mode and merged cleaning mode to identify and prune useless context segments.In cleaning mode, it outputs prefix and suffix commands to remove large redundant chunks; reasoning mode remains equivalent to the original backbone.Source: §1 · p. 2
  • Free()LM gains 3.3% on average across six reasoning benchmarks; DeepSeek V3.2-Speciale gains 2.3% accuracy on IMOanswerBench, and Free()LM restores Qwen3-235B from 0% to ∼50% on tasks requiring >80k thinking tokens.The reported model scale range is 8B to 685B; the standard Qwen3-235B model has 0% accuracy on these long-horizon tasks.Source: §1 · p. 2

2 Method

Free()LM adds a plug-and-play LoRA Free-Module that alternates between reasoning and cleaning to prune redundant context. Its training pipeline synthesizes candidate pruning operations and filters them by whether cleaned contexts preserve or improve solution accuracy.

2.1 Architecture & Inference

  • Free()LM alternates between unmerged reasoning and merged cleaning modes, using the Free-Module to identify and prune redundant context before reasoning resumes.In reasoning mode, the backbone generates tokens; in cleaning mode, it scans context and issues pruning commands. The cycle is illustrated in Figure 2.Source: §2 · p. 2, §2.1 · p. 2, Figure 2 · p. 3
  • Cleaning commands use prefix-and-suffix string anchors to identify spans for pruning, allowing long redundant chunks to be removed with few command tokens.The module emits structured JSON commands, which an external executor applies; the main experiments use re-prefilling, while KV-cache pruning is an alternative not natively supported by vLLM [8].Source: §2.1 · p. 3

2.2 Training: Learning to Forget

  • ∼1% gains on Qwen3-8B from self-correction or external-LLM ICL, even with prompt optimization and Gemini-2.5-Pro, motivated explicit training for context pruning.The authors describe identifying necessary history versus obsolete noise as a complex capability that must be learned.Source: §2.2 · pp. 3–4
  • 1,000 DeepMath-103k trajectories were split into sequential 1k-token chunks for Gemini-2.5-Pro synthesis, conditioning each pruning step on the already-cleaned history.This produced ∼8,000 candidate instances; rejection sampling with 8 independent rollouts retained 6,648 instances only when pruning preserved or improved accuracy.Source: §2.2 · p. 4, Figure 3 · p. 4

3 Experiments

Experiments test whether active forgetting improves long-horizon reasoning while preserving performance on shorter tasks. They assess benchmark results, cross-model transfer, reasoning-length effects, serving costs, and deletion behavior.

3.1 Settings

  • Experiments use Qwen3-8B, Qwen3-30B-A3B, and Qwen3-235B-A22B backbones, comparing vanilla, heuristic-compression, and prompted ICL baselines.H2O and ThinkCleary are evaluated only on Qwen3-8B because they are incompatible with vLLM.Source: §3.1 · p. 5
  • The benchmark suite tests long-horizon reasoning on AIME 24&25, BrUMO25, HMMT, BeyondAIME, HLE, and IMOAnswerBench, plus short-chain general reasoning on BBH, MMLU-Pro, MMLU-STEM, and GPQA-Diamond.The latter benchmarks serve as controls for whether pruning preserves standard-model performance on general-purpose queries.Source: §3.1 · p. 5

3.2 Main Results

  • 48.14% average Pass@1 on Qwen3-8B beats vanilla’s 44.24% (+3.9%), with Free()LM improving benchmark accuracy using a more compact context.On Qwen3-235B-A22B, Free()LM yields an 11.7% relative improvement on HLE while reducing context length by 27.5%.Source: Table 1 · p. 6, §3.2 · p. 6
  • Free()LM’s average response length is 13.8k versus Gemini-2.5-Pro’s 15.8k, while H2O and TC neither improve accuracy nor reduce response length.The paper attributes heuristic failures to disrupted reasoning chains and repetitive loops.Source: §3.2 · p. 6

3.3 Cross-Model Generalization

  • The Qwen3-8B Free-Module improves IMOAnswer performance on Qwen3-235B-A22B by +1.5% and on DeepSeek-V3.2-Speciale by 2.3%, reducing response tokens by 45.99% there.On IMOAnswerBench, the 235B Free-Module gain is +1.75%; the results motivate a plug-and-play context-pruning service without model-specific adaptation.Source: Table 3 · p. 7, §3.3 · p. 7

3.4 Analysis on Reasoning Length

  • Beyond 80k-token trajectories, vanilla Qwen3-235B-A22B accuracy falls to 0%, while Free()LM recovers to approximately 50% by compressing context 40%–50%.The paper cautions that the limited tail sample does not establish a definitive intelligence boost; compression brings 100k+ trajectories into a 40k–70k range.Source: §3.4 · pp. 7–8, Figure 4 · p. 7

3.5 System Performance

  • Free()LM increases per-sample latency by 56% but reduces KV cache usage by 45%, from 6.14 GB to 3.34 GB.The authors estimate that KV Cache Pruning could cut latency overhead to approximately 20%, but have not implemented it in vLLM.Source: §3.5 · p. 8, Table 4 · p. 8

3.6 Case Study

  • Free()LM removes redundant self-correction without regeneration, whereas Gemini deletes essential coordinate setup and triggers a prune-then-regenerate cycle.The case study uses an AIME2425 example to illustrate the difference.Source: §3.6 · p. 8

4 Related Work

Free()LM addresses context accumulation, where unbounded intermediate thoughts crowd out information needed for later reasoning. The paper situates it at the intersection of KV-cache compression, context-window expansion, and CoT overthinking.

4.1 KV Cache Compression

  • Free()LM uses a trainable Free-Module to prune semantic redundancy in reasoning traces, unlike throughput-focused KV-cache compression methods.Prior methods bound cache memory through token eviction [22], saliency clustering and layer-wise budgets [23] [24], reconstruction [25], or low-bit quantization [26]; lossy compression can distract attention and hurt precise reasoning.Source: §4.1 · p. 9

4.2 Long Context Window

  • Free()LM complements context-window expansion by freeing obsolete information rather than extending the window.Prior approaches extend context through positional-encoding changes [28] [29], distributed attention [31], or training [32], while performance can still degrade as context grows [33].Source: §4.2 · p. 9

4.3 Overthinking

  • Free()LM cleans reasoning context during inference, unlike methods that mainly limit output length.CoT [35] can produce redundant steps [36], including dead ends, repeated computations, and over-verification; o1-style models may use 10k–100k+ tokens per problem, and redundancy can occur even in trivial arithmetic. Prior approaches use budgeted generation [38], token-importance pruning [13], entropy-based stopping, or conciseness training such as O1-Pruner and DeepCompress.Source: §4.3 · p. 9

5 Conclusion

Free()LM adds a free() operation to malloc-only reasoning, using active context pruning to maintain a noise-free state and improve reasoning across model scales and long-horizon tasks.

5 Conclusion

  • Standard LLMs’ passive, append-only context accumulation lets redundant tokens overwhelm the model and eventually collapse long-horizon reasoning.Free()LM completes the memory-management cycle with a free() operation.Source: §5 · p. 9
  • 3.3% average gain over leading baselines and a new SOTA on IMOanswerBench demonstrate Free()LM’s overall results.The paper reports that its plug-and-play Free-Module maintains a noise-free state.Source: §5 · p. 10
  • 28% accuracy for Free()LM, versus 0% for standard Qwen3-235B-A22B, shows restored performance on complex long-reasoning tasks.The paper attributes the restoration to precisely pruning redundancy.Source: §5 · p. 10
  • Active context pruning enhances reasoning performance across all model scales, from 8B to 685B.The paper characterizes this as the first systematic evidence of that effect.Source: §5 · p. 10
  • Free()LM points toward test-time scaling through mastering forgetting, not merely expanding the context window.The paper frames the shift as moving from “malloc-only” to “malloc + free.”Source: §5 · p. 10

A Experimental Details · A.1 Prompts

Free()LM uses a consistent prompt for training-corpus preparation and alignment evaluation, with structured JSON output guiding the Free-Module’s responses.

A.1 Prompts

  • The study uses the same prompt for training-corpus preparation and alignment evaluation.The prompt’s system component identifies redundant reasoning steps, while its user component supplies the chain-of-thought context for processing.Source: §A.1 · p. 13, Figure 6 · p. 13
  • A JSON schema guides the Free-Module’s output, and both vLLM and Gemini natively support structured output.The prompt design is illustrated in Figure 7.Source: §A.1 · p. 13

A.2 Training Details

Free()LM is trained to delete from visible backbone-generated reasoning, matching inference-time inputs, with a frozen backbone and a trainable LoRA Free-Module. Training uses scale-specific infrastructure and shared optimization settings.

A.2 Training Details

  • Free()LM disables thinking-mode prompts so the Free-Module learns deletion from visible backbone-generated reasoning, matching inference-time conditions.The backbone does not expose explicit reasoning traces as supervision.Source: §A.2 · p. 13
  • All models train for 5 epochs at a 1 × 10−5 learning rate and global batch size 16, using AdamW, FlashAttention-2, and BF16 precision.Qwen3-8B uses TRL and DeepSpeed ZeRO Stage 3; Qwen3-30B-A3B and Qwen3-235B-A22B use Megatron-LM and ZeRO Stage 2.Source: §A.2 · p. 13
  • LoRA is applied to all linear layers with rank r = 128, scaling factor α = 256, and dropout 0.1; backbone parameters stay frozen while Free-Module parameters update.Source: §A.2 · pp. 13–14

A.3 Evaluation Details

Free()LM resumes generation from the refined reasoning trace by prefilling the prompt with the model’s prior response. It reuses cached key–value states for the unchanged prefix, prefilling only the altered suffix after a free() operation.

A.3 Evaluation Details

  • Prefilling appends the model’s prior response to the chat-templated prompt, then reruns it so generation continues conditionally on the refined reasoning trace.Inference resumes from the updated context rather than restarting from scratch.Source: §A.3 · p. 14
  • After a free() operation, vLLM reuses cached key–value states for the unchanged prefix and prefills only the altered suffix, reducing recomputation overhead.This avoids recomputing attention for the entire context.Source: §A.3 · p. 14

Related papers

Loading 2602.08030v2…