Source-linked AI summary
Think Anywhere in Code Generation
Xue Jiang, Tianyu Zhang, Ge Li, Mengyang Liu, Taozhi Chen, Zhenhua Xu, Binhua Li, Wenpin Jiao, Zhi Jin, Yongbin Li, Yihong Dong
TL;DR
Upfront reasoning can be insufficient during code implementation and cannot adapt computation to positions with different difficulty. Think-Anywhere enables on-demand reasoning at arbitrary token positions through cold-start training followed by RLVR. Across four code-generation benchmarks, it achieves state-of-the-art performance, generalizes across LLMs, and invokes thinking at high-entropy positions; its special-token variant remains constrained by limited post-training data.
Problem
Upfront thinking may miss complexity revealed during implementation and cannot adaptively allocate reasoning effort across code positions with different difficulty.
Method
Think-Anywhere combines cold-start training that teaches reasoning invocation patterns with RLVR that reinforces autonomous discovery of when and where to invoke thinking.
Results
Think-Anywhere achieves state-of-the-art performance across LeetCode, LiveCodeBench, HumanEval, and MBPP, with consistent generalization across LLM families and sizes.
Takeaways & Limitations
Models learn to invoke reasoning at high-entropy positions, supporting adaptive computation and greater transparency into code-generation decisions.
Takeaways & Limitations
Limited post-training data constrains the special-token variant from fully learning the semantics of its new tokens.
Abstract
from arXiv · showhide
Recent advances in reasoning Large Language Models (LLMs) have primarily relied on upfront thinking, where reasoning occurs before final answer. However, this approach suffers from critical limitations in code generation, where upfront thinking is often insufficient as problems' full complexity only reveals itself during code implementation. Moreover, it cannot adaptively allocate reasoning effort throughout the code generation process where difficulty varies significantly. In this paper, we propose Think-Anywhere, a novel reasoning mechanism that enables LLMs to invoke thinking on-demand at any token position during code generation. We achieve Think-Anywhere by first teaching LLMs to imitate the reasoning patterns through cold-start training, then leveraging outcome-based RL rewards to drive the model's autonomous exploration of when and where to invoke reasoning. Extensive experiments on four mainstream code generation benchmarks (i.e., LeetCode, LiveCodeBench, HumanEval, and MBPP) show that Think-Anywhere achieves state-of-the-art performance over both existing reasoning methods and recent post-training approaches, while demonstrating consistent generalization across diverse LLMs. Our analysis further reveals that Think-Anywhere enables the model to adaptively invoke reasoning at high-entropy positions, providing enhanced interpretability.
1 Introduction
Think-Anywhere addresses the limits of upfront reasoning by allowing models to invoke thinking during code generation, where complexity emerges and varies across positions. It is trained through cold-start demonstrations and RLVR, and achieves strong benchmark performance with adaptive, interpretable reasoning behavior.
- Upfront thinking can miss implementation-stage complexity, causing bugs when new problems emerge during coding.
- Code positions differ in difficulty, so fixed upfront reasoning cannot precisely allocate computation to simple boilerplate versus algorithmic decisions and edge cases.
- Think-Anywhere lets models invoke reasoning at any token position based on immediate context and local complexity.
- Cold-start training teaches reasoning invocation patterns, while RLVR enables autonomous exploration of where to trigger reasoning.
- Think-Anywhere reaches state-of-the-art performance across LeetCode, LiveCodeBench, HumanEval, and MBPP, generalizing across LLM families and sizes.Ablations find cold-start plus RLVR optimal and token-level thinking superior to line-level thinking; models tend to think at higher-entropy positions.
2 Related Work
Prior work improves LLM reasoning through prompting, planning, interleaved thinking, and code-generation post-training. However, these approaches predominantly use upfront reasoning or require fixed interleaving, motivating more flexible invocation during implementation.
- Chain-of-Thought and related prompting or search methods elicit intermediate reasoning steps before an answer.
- Self-Planning applies problem decomposition and planning before code generation.
- Interleaved Thinking places reasoning during implementation but requires thinking at every sub-step, creating unnecessary overhead and limited flexibility.
- Code-generation post-training includes distillation from stronger reasoning models and reinforcement learning using task-specific data or verifiable execution signals.
- Existing post-training methods predominantly retain upfront thinking, preserving the limitations associated with separating reasoning from implementation.
3 Methodology
THINK-ANYWHERE replaces strictly upfront reasoning with a generation process that can insert thinking blocks at dynamically selected positions during code generation. It is trained through cold-start imitation followed by RLVR exploration, with dedicated trigger tokens and structured rewards supporting the mechanism.
- Defining THINK-ANYWHERE: Unlike upfront thinking, THINK-ANYWHERE permits reasoning at positions where local code-generation complexity requires deliberation.Upfront thinking strictly separates a complete reasoning trace from subsequent code generation, making additional reasoning during implementation difficult.
- Defining THINK-ANYWHERE: THINK-ANYWHERE decomposes generation into code segments and interleaved thinking blocks whose number and positions are dynamically determined during generation.The mixed sequence includes an initial thinking block, code segments, and optional thinking blocks between those segments.
- Cold Start for THINK-ANYWHERE: Cold-start training uses constructed samples to teach models the structural pattern of invoking thinking blocks within code.Strong reasoning LLMs generate demonstrations, malformed formats are filtered, and approximately 5,000 samples are used for LoRA supervised fine-tuning.
- Cold Start for THINK-ANYWHERE: Dedicated trigger tokens combine semantic content for “think anywhere” with delimiter structure to make on-demand reasoning more reliable.The method addresses ambiguity from multi-token ordinary delimiters and initializes special-token embeddings from semantic and structural sources.
- RLVR for THINK-ANYWHERE: RLVR reinforces autonomous discovery of thinking positions using GRPO and a hierarchical reward combining reasoning-structure and code-correctness signals.GRPO uses group-level statistics instead of a separate value model, while the structure reward checks for initial thinking and at least one embedded THINK-ANYWHERE block.
4 Experiments
Experiments evaluate THINK-ANYWHERE across code benchmarks, model families, mathematical reasoning, and controlled ablations. The results support strong performance, generalization, and benefits from token-level, two-stage reasoning.
- Main Results: THINK-ANYWHERE achieves the best average score across four code-generation benchmarks, reaching 70.3% and improving 9.3% over the base model.The comparison covers LeetCode, LiveCodeBench, HumanEval, and MBPP.
- Cross-Domain Generalization: A code-trained THINK-ANYWHERE model improves mathematical pass@1 on AIME 2024 from 5.3% for the base model and 6.0% for GRPO to 17.3%.The passage also reports improvements on AIME 2025 and HMMT 2025.
- Generalization Across Models: THINK-ANYWHERE consistently outperforms the base model and GRPO across LLaMA and Qwen models spanning 1.5B to 8B parameters.The reported improvement reaches up to +13.9% over the base model.
- Ablation Study: The complete cold-start-plus-RLVR pipeline outperforms training with either stage alone, while line-level thinking underperforms token-level thinking.Removing upfront thinking causes only a moderate 2.8% drop, whereas padding the reasoning content also degrades performance.
- Thinking Position Analysis: THINK-ANYWHERE is invoked where disabling thinking produces predominantly higher entropy, indicating placement at uncertain code-generation positions.The analysis also examines the syntactic categories where thinking is most frequently invoked.
- Further Analysis: THINK-ANYWHERE outperforms GRPO across all reported sampling values of k in pass@k analysis.This analysis evaluates whether the method expands the model’s capability boundary.
5 Conclusion
THINK-ANYWHERE lets LLMs invoke reasoning at any token position during code generation, unlike strictly upfront approaches. Experiments report SOTA performance, broad generalization, and adaptive reasoning at high-entropy positions.
- THINK-ANYWHERE enables LLMs to invoke thinking at any token position during code generation.
- Unlike conventional upfront thinking, THINK-ANYWHERE allows models to deliberate where complexity arises during implementation.
- Extensive experiments across mainstream benchmarks show SOTA performance and strong generalization across different LLMs.
- Models learn to invoke thinking at high-entropy positions, supporting adaptive computation based on local complexity.
A Token Cost Breakdown
THINK-ANYWHERE reduces total reasoning-token usage by replacing lengthy upfront thinking with shorter upfront reasoning plus modest on-demand blocks.
- THINK-ANYWHERE’s upfront thinking phase is substantially shorter than GRPO’s and CoT’s across benchmarks.
- The additional <thinkanywhere> tokens remain modest, producing a net reduction in total reasoning token usage.
- Table 6 reports THINK-ANYWHERE cost as upfront thinking length plus <thinkanywhere> block length.
B Thinking Block Statistics Across Training Stages
Cold-start SFT establishes THINK-ANYWHERE block use, while RL refines it into more concise, targeted reasoning and improves pass@1.
- Table 7 tracks average frequency and length of <thinkanywhere> blocks across training stages.
- The base model never invokes thinking blocks, while prompting alone produces very few blocks with abnormally long lengths.
- Cold-start SFT teaches a normal frequency and length of <thinkanywhere> blocks, establishing a foundation for RL training.
- After RL training, block frequency and length decrease slightly relative to SFT while pass@1 improves substantially.
- RL refines on-demand reasoning into more concise and targeted deliberation rather than simply increasing thinking-token counts.