HeavySkill: Heavy Thinking as the Inner Skill in Agentic Harness

• The paper abstracts agentic harnesses into parallel thinking followed by summarization, a two-stage pattern also used by Kimi K2, PaCoRe, and LongCat-Flash-Thinking-2601.The introduction contrasts this decomposition with methods that elicit multiple trajectories using inline thinking tags before summarizing them (Hsu et al., 2025; Zheng et al., 2025b; Wen et al., 2025).Source: §1 · pp. 1–2
• Across STEM, coding, and general tasks, heavy thinking substantially outperforms single reasoning; on verifiable STEM tasks, Heavy-Pass@k ≥ Heavy-Mean@K ≥ Vote@K ≥ Mean@k.The study also reports effectiveness and limitations across diverse model scales and task domains.Source: §1 · p. 2
• Stronger models can approach Pass@k under heavy thinking, while deliberation compares trajectory differences and functions as an implicit verifier.The paper attributes effective identification and synthesis of correct reasoning paths to sequential deliberation.Source: §1 · p. 2
• HEAVYSKILL offers a training-free framework that separates parallel reasoning from sequential deliberation and caches trajectories for iterative refinement.The model revisits and synthesizes prior attempts during deliberation.Source: §1 · p. 2
• Ablations identify trajectory quality and diversity as keys; deliberation relies on the general capability of its model, and RLVR improves Heavy-Mean@k and Pass@k by optimizing breadth and depth.The authors suggest separately optimizing the thinking and deliberation models may yield additional gains.Source: §1 · p. 2

2.1. Workflow of Heavy Thinking

• The workflow generates multiple reasoning trajectories in parallel, then uses a second LLM for sequential deliberation that aggregates them.The summary model receives a serialized memory cache derived from the parallel trajectories.Source: §2.1 · p. 2

2.2. Serialized Memory Cache

• The serialized cache stores candidate trajectories, pruning them to fit model length limits and shuffling them to reduce positional bias.The resulting serialized context is supplied to sequential deliberation.Source: §2.2 · p. 2

2.3. Iterative Deliberation

• From iteration 2 through N, deliberation modifies the cache by concatenating content from the previous sequential-deliberation output.The number of summary contents generated in the previous iteration is denoted K(t−1).Source: §2.3 · p. 2

2.4. Readable Skill for Agentic Harness

• The HEAVYSKILL document specifies conditional activation, independent diverse agents, critical synthesis with re-derivation if needed, and domain-consistent answer-only output.It activates for complex reasoning rather than simple factual queries or casual conversation, and asks the deliberation model to avoid superficial concatenation.Source: §2.4 · p. 3
• The plain-text skill works across Claude Code and custom harnesses without modification, decoupling heavy thinking from specific orchestration infrastructure.The orchestrator reads the skill, spawns parallel agents, collects their outputs as serialized context, and performs deliberation in a subsequent generation step.Source: §2.4 · pp. 3–4

3.1. Setups

• The experiments use parallel reasoning followed by summary generation, with both phases defaulting to the same model unless otherwise specified.Main settings use N=1, K=8 or 16 parallel trajectories, and K(1)=4 summaries; evaluation reports M@K, P@K, V@K, HM@K, and Heavy-Pass@K across closed- and open-weight models.Source: §3.1 · p. 4

3.2. Evaluations on STEM Tasks

• HM@4 consistently surpasses M@K across models and STEM benchmarks, with frontier models often approaching near-perfect scores.Table 1 compares HM@4 with M@K, P@K, and V@K.Source: §3.2 · p. 4, Table 1 · p. 5
• Increasing parallel width and sequential deliberation continues to improve performance rather than causing a plateau, supporting test-time compute scaling on complex reasoning tasks.The passage characterizes reasoning width and deliberation depth as multipliers of the base model’s capability, without retraining.Source: §3.2 · p. 4
• Heavy thinking frequently outperforms V@K, especially on BeyondAIME, HMMT, and GPQA-Diamond, though ceiling effects limit gains on AIME25 for some strong models.The passage attributes occasional parity or slight underperformance on AIME25 to models already scoring above 90.Source: §3.2 · p. 4
• With a sufficiently capable LLM in deliberation, HP@4 exceeds raw-thinking potential P@K in nearly half of experimental trials; HM@4 frequently approaches P@K for frontier models such as DeepSeek V3.2 and GPT-5 Thinking.The authors say deliberation can synthesize cross-trajectory insights to generate correct solutions absent from any single raw reasoning path.Source: §3.2 · p. 5

3.3. Evaluations on General Reasoning Tasks

• GPT-OSS-20B improves from M@K 69.7% to HM@4 85.5% on LiveCodeBench, while R1-Distill-Qwen-32B rises from 35.7% to 69.3% on IFEval.Gains are reported on tasks with clear logical or programmatic constraints.Source: §3.3 · pp. 5–6
• On Arena-Hard, HM@4 gains are marginal or sometimes negative, suggesting less benefit for preference-oriented tasks than for correctness-oriented tasks.The passage notes that averaging multiple responses may not match stylistic preferences favored by a reward model or judge.Source: §3.3 · p. 6
• On IMO Answer Bench, GLM 4.6 reaches HP@4 86.0% versus P@K 75.1%, illustrating that summary deliberation can recover answers missed in raw sampling.The passage says HP@4 is the highest metric across nearly all general-reasoning benchmarks.Source: §3.3 · p. 6

4.1. Can Sequential Deliberation Revises Parallel Thinking?

• For queries below 0.5 parallel pass rate, deliberation corrected over 500 of approximately 1,400 unresolved cases; above 0.5, success exceeded 98%, with about 30 degradations.The analysis used K = 16 parallel samples and sequential deliberation without iteration.Source: §4.1 · pp. 6–7

4.2. Model Selection for the Role of Summary

• Across AIME25 and HMMT25-Feb, HM@K outperforms M@K regardless of which tested model performs sequential deliberation.The sequential-stage models are R1-Distill-Qwen-7B, R1-Distill-Qwen3-8B, and Qwen2.5-32B-Instruct, paired with R1-Distill-Qwen-7B for parallel reasoning.Source: §4.2 · p. 7, Figure 3 · p. 7
• Qwen2.5-32B-Instruct improves deliberation despite not independently excelling at these problems (Yang et al., 2025a) 2, suggesting analysis, synthesis, and summarization matter more than peak intrinsic reasoning.The authors suggest larger instruction-following models can scale the deliberation stage without specialized reasoning capability being the main driver.Source: §4.2 · p. 7

4.3. Effectiveness of Iteration Deliberation

• As iterations increase, HM@K rises while HP@K degrades, exposing a trade-off between collective reasoning gains and interference from earlier deliberation.The iterative setup uses K = K(1) = · · · = K(N) = 8, N = 4, and tests three model scales.Source: §4.3 · pp. 7–8, Figure 4 · p. 8

4.4. Adaptation to Agentic Tool Use

• On AIME25, GPT-OSS-20B achieves 90.0% accuracy with HM@4 versus 83.3% with V@4, and HM@4 surpasses V@4 across all tested models and datasets.The tested models are Qwen3-8B, Qwen3-32B, and GPT-OSS-20B; parallel reasoning uses Python-interpreter feedback.Source: §4.4 · p. 8

5.1. Parallel Reasoning in LLM

• Parallel-reasoning methods spawn independent trajectories for aggregation or synchronize at fixed intervals, making their branching and merging schedules insensitive to evolving reasoning progress (Brown et al., 2024; Zheng et al., 2025b; Wen et al., 2025; Hsu et al., 2025).The paper frames these static approaches as lacking adaptivity.Source: §5.1 · p. 8
• MCTS and Tree of Thoughts enable more granular parallelization but rely on handcrafted heuristics and external verifiers, while heavy thinking offers flexible test-time scaling (Zhang et al., 2024a; Yao et al., 2023; DeepMind, 2025; Bai et al., 2025; StepFun-AI, 2025).This paper examines heavy thinking’s implementation and effectiveness across tasks and domains.Source: §5.1 · p. 9

5.2. Test-time Scaling in LLM Post-training

• Post-training test-time scaling extends reasoning chains for self-correction, reflection, critique, and verification, while RLVR uses automatically checkable outcome rewards or step-wise signals; this paper applies RLVR to heavy thinking.The cited approaches include OpenAI’s o1, DeepSeek R1, and Gemini (Jaech et al., 2024; Guo et al., 2025; DeepMind, 2025).Source: §5.2 · p. 9

6. Conclusion

• The paper investigates heavy thinking as a test-time-scaling strategy and distills its workflow into a readable skill for agentic harnesses.The framework centers on parallel reasoning and sequential deliberation to explain how test-time computation translates into task performance.Source: §6 · p. 9
• Heavy thinking consistently outperforms traditional Best-of-N strategies and can approach Pass@K in models with high intrinsic reasoning potential.The authors also report detailed analysis of heavy thinking’s effects.Source: §6 · p. 9
• RLVR can substantially improve models’ reasoning capabilities.The authors identify more granular analysis of heavy-thinking trajectories within RL frameworks as future work.Source: §6 · p. 9

A. Impact of Parallel Trajectories

• The study used R1-Distill-Qwen3-8B on AIME25 and HMMT25-Feb, generating 256 parallel trajectories per problem and comparing four selection strategies.The strategies were Random, Max-Diversity, Max-Length, and Max-Answer-Num.Source: A. Impact of Parallel Trajectories · p. 13
• Max-Answer-Num significantly outperformed Random, Max-Diversity, and Max-Length, using majority-vote consensus to increase correct reasoning paths in the candidate set.The resulting candidate set provides a more robust foundation for synthesis, summarization, and final logical deduction.Source: A. Impact of Parallel Trajectories · p. 13
• Accuracy rose as the number of parallel trajectories K increased for all four strategies; Max-Diversity matched Random, while Max-Length performed worst.The authors interpret these results as indicating that explicit diversity optimization adds little marginal gain and that longer outputs do not necessarily improve reasoning quality.Source: A. Impact of Parallel Trajectories · p. 13

B. Advancing Heavy Thinking via RLVR

• Because HP@K can significantly outperform HM@K in some heavy-thinking scenarios, the section asks whether RLVR applied to heavy-thinking trajectories can raise reasoning capability.Source: B. Advancing Heavy Thinking via RLVR · p. 13
• Experiments reuse parallel reasoning trajectories, selecting queries with pass rates in [0, 0.625] and sampling K ∈ {8, 16} trajectories as serialized memory caches.The setup uses R1-Distill-Qwen-7B, VeRL (Sheng et al., 2025), and GSPO (Zheng et al., 2025a).Source: B. Advancing Heavy Thinking via RLVR · p. 13
• HM@4 improves by approximately 10% during the first 100 training steps, with growth on both training and test sets.Figure 6 presents these results.Source: B. Advancing Heavy Thinking via RLVR · p. 13
• K = 8 remains more stable, while K = 16 undergoes significant entropy collapse after 100 steps, possibly due to sequence-length limits truncating training signals.The authors say RLVR heavy-thinking dynamics require further exploration.Source: B. Advancing Heavy Thinking via RLVR · p. 13

• The agentic-harness skill file for heavy thinking is presented across Figures 8–10.Source: C. Prompt and SKill for Heavy Thinking · p. 13, Figure 8 · p. 16, Figure 9 · p. 17
• The heavy-thinking prompt is provided as a serialized memory-cache prompt (Figure 7).Source: C. Prompt and SKill for Heavy Thinking · p. 13, Figure 7 · p. 15