PaperlayerView exact revision on arXiv

Source-linked AI summary

ThinkFuse: Trajectory-Aware Test-Time Fusion for Small Reasoning Models

Myunghoon Kang, Jungseob Lee, Jaehyung Seo, Heuiseok Lim

arXiv:2610.07803v1cs.AIcs.CL

TL;DR

Small reasoning models can struggle to recover from erroneous reasoning paths, while existing fusion triggers may rely on local uncertainty signals that miss trajectory trends. ThinkFuse selectively fuses auxiliary reasoning using trajectory-aware uncertainty calibration, and it reports consistent gains over standalone models and test-time fusion baselines across five benchmarks.

  • Problem

    Existing fusion methods rely on local uncertainty signals that may mistake transient uncertainty for instability, while small reasoning models struggle to generalize on mathematical and coding tasks.

    Source: §1 · p. 1

  • Method

    ThinkFuse uses EWCA-based segment uncertainty and an adaptive threshold calibrated against the primary model’s uncertainty history to selectively fuse auxiliary reasoning paths.

    Source: §1 · pp. 1–2

  • Results

    Across five benchmarks, ThinkFuse consistently outperforms standalone models and existing test-time fusion baselines, with gains across model-family combinations and robustness with a smaller primary model.

    Source: §1 · p. 2, §9 · p. 9

  • Takeaways & Limitations

    The results suggest that effective collaborative reasoning depends on deciding when to intervene and what to retain.

    Source: §9 · p. 9

    The main experiments use a fixed segment length and PPL scoring rule, with representative alternatives examined in ablations.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Small reasoning models (SRMs) have shown strong performance on complex reasoning tasks by generating extended chain-of-thought trajectories, but they often fail to recover once their reasoning enters an erroneous path. Existing test-time fusion methods rely on local fusion signals to determine when to trigger fusion, which can be misled by transient uncertainty fluctuations and may reinforce unstable reasoning trajectories. We propose ThinkFuse, a training-free test-time fusion framework that selectively intervenes in unreliable reasoning segments. ThinkFuse compares segment-level uncertainty shifts with trajectory-level uncertainty trends to identify unstable reasoning points and fuse auxiliary reasoning paths into the primary model's trajectory. Extensive experiments demonstrate that ThinkFuse outperforms baselines on mathematical and knowledge-intensive reasoning benchmarks, with consistent gains across model-family combinations, and remains robust with a smaller primary model. Our analysis shows that ThinkFuse requires fewer fusion triggers and generates fewer tokens, highlighting the efficiency of selective triggering. Our code is available at https://github.com/js-lee-AI/ThinkFuse.

1 Introduction

ThinkFuse addresses the underexplored question of when test-time fusion should intervene by calibrating segment uncertainty against the reasoning trajectory’s history. It reports stronger benchmark performance and more efficient selective triggering, while distinguishing its approach from prior fusion, collaboration, and uncertainty-guided methods.

1 Introduction

  • Small reasoning models struggle to generalize on mathematical and coding tasks, while existing fusion policies may mistake transient uncertainty for instability because they do not track trajectory-level uncertainty.Prior fusion work has explored granularity, but when to intervene and what evidence should justify fusion remain underexplored.Source: §1 · p. 1
  • ThinkFuse uses EWCA-based segment uncertainty and an adaptive threshold informed by historical uncertainty and fusion frequency to selectively invoke auxiliary reasoning.It compares segment-level uncertainty shifts with trajectory-level trends to identify unstable points.Source: §1 · pp. 1–2
  • ThinkFuse outperforms test-time scaling and fusion baselines, gains consistently across model-family combinations, and remains robust with a smaller primary model.Its analysis also reports fewer fusion actions and fewer generated tokens.Source: §1 · p. 2
  • Its analysis reports fewer fusion actions and fewer generated tokens, supporting the efficiency of selective fusion triggering.The source describes this as an analysis of ThinkFuse’s behavior.Source: §1 · p. 2

2.1 Test-Time Ensembling and Fusion for Large Language Models

  • Prior test-time fusion work varies the intervention granularity, from answer- or response-level aggregation to token, span, segment, and adaptive-unit fusion.ThinkFuse frames temporal dynamics—when to invoke auxiliary computation—as a gap in granularity-focused approaches.Source: §2.1 · p. 2

2.2 Large Model Guided Reasoning Correction and Collaboration

  • Prior collaboration approaches include offline chain-of-thought supervision, external verification, cost-aware routing, and trainable token-level routing; ThinkFuse instead fuses reasoning paths on the fly.Its auxiliary model is invoked when segment-level uncertainty indicates deviation in the primary model’s ongoing trajectory.Source: §2.2 · p. 3

2.3 Uncertainty-Guided Test-Time Scaling

  • Unlike uncertainty-guided methods that schedule computation within one model or verify every draft, ThinkFuse delegates unreliable segments to an auxiliary model and selects continuations by trajectory compatibility.The cited comparisons include certainty-based budgeting, confidence-filtered parallel traces, momentum-smoothed uncertainty, and draft verification.Source: §2.3 · p. 3

3 Methodology

ThinkFuse selectively fuses reasoning segments when segment-level uncertainty is high relative to an adaptive trajectory-aware threshold, then uses the primary model to choose a compatible continuation.

3.1 Overview

  • ThinkFuse combines selective triggering, trajectory-aware uncertainty calibration, and multi-model fusion to intervene at unstable reasoning points.It compares primary and auxiliary segments using primary-model perplexity and continues with the more compatible segment.Source: §3.1 · p. 3

3.2 Segment-level Fusion Monitoring

  • Fixed-length segments are monitored with EWCA uncertainty, which aggregates token-level uncertainty using position and accumulated-uncertainty weights.The method avoids relying directly on isolated token signals that can be sensitive to local lexical or discourse fluctuations.Source: §3.2 · pp. 3–4
  • An adaptive threshold uses historical trajectory uncertainty and cumulative fusion frequency to target localized uncertainty spikes while penalizing excessive fusion.The threshold is based on an updated mean and deviation scale, with a quadratic modulation for accumulated fusion.Source: §3.2 · p. 4

3.3 Trajectory compatibility scoring

  • When triggered, ThinkFuse generates an auxiliary segment and selects between it and the primary segment using primary-model trajectory compatibility.The selected segment updates the context, from which primary-model generation continues until the trajectory reaches a terminal state.Source: §3.3 · p. 4

4 Experimental Setup

Experiments use six open-weight reasoning models, compare ThinkFuse with standalone, fusion, and test-time-compute baselines, and evaluate mathematical and knowledge-intensive benchmarks under a shared protocol.

4.1 Models

  • Six open-weight models spanning 1.2B–4B parameters cover multiple model families and both reasoning-specialized and hybrid or instruction-following models.The model pool is intended to test complementary reasoning trajectories without relying on larger proprietary models.Source: §4.1 · p. 5

4.2 Baselines

  • Baselines include standalone models, Cool-Fusion and AdaFuse, and Qwen3-4B self-consistency with K ∈{3, 5} sampled paths.Standalone evaluates gains beyond each model’s original capability; self-consistency uses majority voting over normalized answers, falling back to the first sample if no majority emerges.Source: §4.2 · p. 5

4.3 Datasets

  • Evaluation covers MATH-500, GSM8K, and AIME 2024 for mathematics, plus GPQA-Diamond and NQ-Open for knowledge-intensive reasoning.These benchmarks span competition mathematics, grade-school arithmetic, expert scientific reasoning, and open-domain factual question answering.Source: §4.3 · p. 5

4.4 Implementation Details

  • Methods share a 16,384-token generation budget and decoding configuration, and are evaluated on consistent fixed random subsamples of each benchmark.Unparseable answers within the budget count as incorrect; auxiliary candidate generation runs in parallel with primary generation whenever applicable.Source: §4.4 · p. 5

5 Main Results

THINKFUSE improves Qwen3-4B’s standalone accuracy across all five benchmarks, with the Qwen3-4B × Ministral pair achieving the best result on four. The gains also extend to a smaller primary model.

5 Main Results

  • THINKFUSE with Qwen3-4B improves over its standalone performance on every benchmark and achieves the best result on four of five with Ministral.The largest reported gains are +20.0 on AIME24, +10.6 on GPQA, and +10.0 on NQ-Open.Source: §5 · pp. 5–6, Table 2 · p. 6
  • Cool-Fusion and AdaFuse often underperform the standalone primary, while self-consistency and redundant same-model fusion do not improve consistently.With Qwen3-4B × Ministral, Cool-Fusion drops from 89.0 to 31.0 on MATH-500 and from 54.0 to 11.6 on GPQA.Source: §5 · p. 6
  • With Qwen3-1.7B primary paired with EXAONE-4.0, THINKFUSE raises GPQA from 39.4 to 53.5, gains +2.0 on MATH-500 and +0.5 on GSM8K, ties NQ-Open, and differs by one problem on AIME24.Standalone Qwen3-1.7B uses recommended top-k 20, while THINKFUSE retains the shared decoding setting, favoring the standalone baseline.Source: §5 · p. 6
  • The results support selectively fusing complementary auxiliary trajectories at uncertain points rather than relying on frequent or redundant fusion.The authors connect this design to more stable and effective test-time fusion than existing baselines.Source: §5 · pp. 6–7

6 Analysis

The analysis examines fusion dynamics, compute efficiency, and auxiliary compatibility, finding that selective fusion and reasoning-specialized auxiliaries best align with the framework’s trajectory-fusion design.

6.1 Fusion Dynamics and Downstream Performance

  • THINKFUSE uses auxiliary tokens more sparsely than Cool-Fusion while retaining a more substantial thinking process, alongside consistent accuracy gains.Cool-Fusion fuses the most auxiliary tokens yet has the shortest thinking-stage outputs; AdaFuse’s lower auxiliary usage also shortens reasoning-intensive outputs.Source: §6.1 · p. 7

6.2 Compute Efficiency

  • THINKFUSE achieves the highest accuracy per TFLOP, with TFLOPs comparable to AdaFuse and below Cool-Fusion, but has longer wall-clock time.The authors attribute the time gap to CPU-GPU data transfer during PPL scoring rather than additional computation.Source: §6.2 · p. 7, Table 3 · p. 7

6.3 Auxiliary Compatibility Beyond Reasoning Models

  • Instruction-tuned and non-thinking auxiliaries produce limited, task-dependent gains, while all non-reasoning auxiliaries degrade GPQA performance.The authors conclude that reasoning-specialized auxiliaries remain essential to fully leverage trajectory-aware fusion.Source: §6.3 · pp. 7–8, Table 4 · p. 8

7 Ablation Study

The ablations show that fusion budgeting, segment length, compatibility scoring, and uncertainty choice each affect THINKFUSE’s results.

Fusion budget penalty ablation

  • The soft fusion-budget penalty outperforms the no-budget variant, which falls below the standalone baseline on MATH-500 and GSM8K.The no-budget variant reaches 88.5% on MATH-500 and 90.5% on GSM8K; the authors say the soft budget discourages excessive interventions.Source: §7 · p. 8, Table 5 · p. 8

Effects of segment length

  • The default segment length T = 8 performs best, while T = 4 falls below the standalone MATH-500 baseline and longer segments diminish gains.The authors suggest longer segments delay necessary interventions, allowing incorrect reasoning paths to propagate.Source: §7 · p. 8, Table 5 · p. 8

Effects of compatibility selection rule

  • Primary PPL scoring outperforms Average PPL, which reaches 80.5% on MATH-500 and only the 92.0% standalone baseline on GSM8K.The authors say blending the auxiliary distribution into the primary score misaligns compatibility selection with the primary reasoning path.Source: §7 · pp. 8–9, Table 5 · p. 8

Token-level uncertainty estimation

  • Entropy preserves gains on both evaluated datasets, whereas the margin-based confidence gap reaches 98.6% on GSM8K but falls below baseline on MATH-500 at 87.5%.The authors therefore use entropy as their default token-level uncertainty metric.Source: §7 · p. 9

8 Case Study

An AIME 2022 example contrasts a baseline trapped by an incorrect coordinate choice with THINKFUSE selecting an auxiliary segment that preserves symbolic variables and both constraints. The conclusion reports consistent gains across five benchmarks, while the limitations bound the evaluation and deployment settings.

8 Case Study

  • In the AIME 2022 example, THINKFUSE selects a symbolic auxiliary segment that enforces both distance-to-line constraints, unlike the baseline’s dead-end quadratic.The baseline prematurely grounds D; the selected segment keeps D and C symbolic before solving.Source: §8 · p. 9

9 Conclusion

  • Across five benchmarks, THINKFUSE outperforms standalone models and test-time fusion baselines, with gains attributed to complementary auxiliary trajectories rather than additional computation alone.The conclusion characterizes the framework as a practical, training-free approach to scaling small reasoning models at test time.Source: §9 · p. 9

Limitations

  • Serving an auxiliary model alongside the primary model can increase memory use and latency, although selected-segment intervention and KV-cache reuse avoid an extra full-context pass.Quantization, parallel serving, and cache-aware deployment may further reduce practical overhead.Source: Limitations · pp. 9–10
  • The evaluation covers open-weight reasoning models from 1.2B–4B and mathematical, scientific, and knowledge-based benchmarks, leaving larger, closed, code, multilingual, and long-context settings untested.The paper says additional evaluation is needed to determine whether the behavior holds in those settings.Source: Limitations · p. 10
  • THINKFUSE assumes the primary model produces coherent segments; highly unstable or repetitive reasoning may leave uncertainty-based triggering unable to find useful intervention points.The paper suggests alternative decoding policies or stronger primary-model selection criteria for such cases.Source: Limitations · p. 10
  • Fusion depends on auxiliary segments complementing the primary model; shared failure modes or task-mismatched strengths may prevent performance gains.Task-specific auxiliary selection or model-pool routing is proposed as future work to reduce this dependence.Source: Limitations · p. 10
  • Segment-level intervention requires identifying explicit reasoning spans and aligning text from different tokenizers, with broader chat templates and tokenizer combinations potentially needing extra engineering.The implementation uses <think> tag normalization and shared-prefix text alignment.Source: Limitations · p. 10
  • The main experiments use a fixed segment length and PPL scoring rule, while broader searches over threshold and budget settings may yield additional gains.The stated focus is testing uncertainty-guided fusion without heavy task-specific tuning.Source: Limitations · p. 10

Ethical Statements

The paper describes its benchmark data, model and software licenses, intended research use, and use of AI assistants for language polishing. It reports that the curated datasets pose minimal risk of personally identifiable information or offensive content.

Privacy and Languages

  • The experiments use publicly available English academic datasets, which the authors describe as posing minimal risk of personally identifiable information or offensive content.The datasets include mathematical, scientific reasoning, and open-domain factual question-answering benchmarks.Source: Ethical Statements · p. 10

Licenses and terms

  • The evaluated reasoning models and primary serving and evaluation tools are available under licenses permitting research use or permissive open-source use.The listed model licenses include Apache 2.0 and MIT; the named tools are also described as permissively licensed.Source: Ethical Statements · p. 10

Intended use

  • THINKFUSE is intended for research on training-free test-time fusion of small reasoning models, and the authors plan to release code and evaluation scripts for reproducibility.The stated research goal is improving reasoning-trajectory reliability and studying when and what to fuse.Source: Ethical Statements · pp. 10–11

Use of AI assistants

  • AI assistants were used for grammar correction and minor language polishing, and the authors reviewed and verified those edits and take responsibility for the manuscript.Source: Ethical Statements · p. 11

A Detailed Experimental Setup

The appendix records implementation settings, uncertainty and budget-penalty variants, cross-tokenizer alignment, and the benchmark sampling protocol. Experiments use the same fixed examples across methods and model configurations for each benchmark.

A.1 Hyperparameter settings

  • Table 7 summarizes settings shared across methods and additional settings specific to each baseline or comparison method.Source: §A.1 · p. 14

A.2 Uncertainty Signals and Budget-Penalty Details

  • The confidence-gap variant assigns greater uncertainty when the top two candidate-token probabilities are similar; the no-budget variant removes only the soft-budget penalty while retaining the EMA trigger.With λ = 0, its threshold is θ_k = µ_(k−1) + τσ_(k−1), and fusion still occurs only when U_p^k ≥ θ_k.Source: §A.2 · p. 14

A.3 Cross-Tokenizer Alignment

  • For cross-tokenizer comparisons, shared-prefix text alignment finds a common decoded prefix, while reasoning tags are normalized to <think> for fusion and restored before PPL computation.Alignment reduces the effective segment length by 1–2 tokens on average.Source: §A.3 · p. 14

A.4 Benchmark sampling protocol

  • Each method is evaluated on the same fixed random subsample of 200 examples per benchmark, with AIME 2024 and GPQA-Diamond evaluated in full.The subsample is drawn once per benchmark with seed 42; MATH-500, GSM8K, and NQ-Open use strict subsamples of their full splits.Source: §A.4 · p. 14

B Bootstrap Confidence Intervals

Appendix analyses report bootstrap uncertainty, additional cross-family pairs, a repeated-sampling diagnostic, fusion-position patterns, and a comparison with post-hoc critique. These passages characterize robustness and intervention behavior without supplying detailed cell values for the tables.

B Bootstrap Confidence Intervals

C Additional Model-Pair Results of THINKFUSE

  • Both additional cross-family auxiliary pairs improve over standalone Qwen3-4B on every benchmark, matching the pattern of the main-text pairs.The configurations use EXAONE-Deep and DeepSeek-R1 as auxiliaries under the same protocol as the main experiments.Source: C Additional Model-Pair Results of… · p. 14, Table 9 · p. 16

D Additional Sampling Diagnostic

  • The Qwen3-4B pass@5 oracle diagnostic counts an instance as correct if any of five independent samples is correct and is not treated as a main baseline.It measures a repeated-sampling upper bound rather than self-consistency.Source: D Additional Sampling Diagnostic · p. 15, Table 10 · p. 16

E Fusion position distribution by correctness

  • In MATH-500, incorrect answers have more dispersed fusion positions across early and middle thinking stages, while GPQA shows a smaller correctness-related difference.Figure 3 displays normalized trigger positions for Qwen3-4B with Ministral-3B-Reasoning as auxiliary.Source: E Fusion position distribution by… · pp. 15–16, Figure 3 · p. 16

F Comparison with Post-Hoc Critique

  • THINKFUSE intervenes at segment boundaries during thinking, whereas post-hoc critique waits for completed trajectories and revises the answer after disagreement.THINKFUSE can replace an uncertain intermediate segment while retaining reusable reasoning context.Source: F Comparison with Post-Hoc Critique · p. 16

G Supplementary Comparisons with Uncertainty-Guided Test-Time Scaling Methods

This section compares ThinkFuse with uncertainty-guided test-time scaling methods that pursue different objectives. ThinkFuse improves over MUR variants and remains ahead of SpecReason in the reported comparisons.

G Supplementary Comparisons with Uncertainty-Guided Test-Time Scaling Methods

  • MUR and SpecReason both use uncertainty to guide test-time scaling, but MUR scales computation within one model while SpecReason uses a larger model to score drafted steps.ThinkFuse instead calibrates intervention against trajectory history and targets accuracy beyond the stronger model.Source: G Supplementary Comparisons with… · pp. 16–17, §G.2 · p. 17

G.1 Comparison with MUR

  • ThinkFuse improves over both MUR variants on both benchmarks, while SpecReason improves over its generator but remains below standalone Qwen3-4B and ThinkFuse.The comparisons use a 16K-token budget; SpecReason accepts 15.6% of proposed steps on GPQA, and most AIME24 failures end in repetitive reasoning rather than budget truncation.Source: §G.1 · p. 17, Table 12 · p. 17, §G.2 · p. 17
  • MUR allocates computation to uncertain steps within one model, whereas ThinkFuse uses fixed-length segments without altering the natural reasoning flow.The authors attribute part of the performance gap to MUR's prompted step segmentation, which inserts explicit markers into the trace.Source: §G.1 · p. 17

G.2 Comparison with SpecReason

  • SpecReason has a different goal from ThinkFuse: it verifies each small-model draft step to approximate its base model's quality at lower cost.The evaluation uses DeepSeek-R1-Distill-Qwen-1.5B as generator and Qwen3-4B as verifier, with a threshold of 7 and greedy verifier scoring.Source: §G.2 · p. 17

Related papers

Loading 2610.07803v1…