PaperlayerView exact revision on arXiv

Source-linked AI summary

HyperThink: Text-to-Parameter Hypernetworks for Efficient Reasoning

Donggyun Kim, Jack Lu, Chanwoo Kim, Mengye Ren, Seunghoon Hong

arXiv:2610.03039v1cs.CLcs.LG

TL;DR

Long thinking traces can support multi-step reasoning but add substantial inference-time cost, while direct non-thinking may sacrifice reliable intermediate computation. HYPERTHINK replaces the long trace with a query-conditioned parameter update, and improves the low-latency region of the accuracy–latency trade-off, most strongly near non-thinking operation.

  • Problem

    Long thinking traces can improve multi-step reasoning but add substantial inference-time overhead, while direct non-thinking can sacrifice reliable intermediate computation on multi-step problems.

    Source: §1 · p. 1

  • Method

    HYPERTHINK uses a hypernetwork to predict query-conditioned updates to a small subset of base-model parameters, with a vector-quantized bottleneck restricting updates to reusable patterns.

    Source: §1 · p. 2

  • Results

    HYPERTHINK improves the low-latency region of the accuracy–latency trade-off on mathematical and general reasoning tasks, with strongest gains near non-thinking operation.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    A single query-conditioned parameter update can recover useful instance-specific reasoning behavior without requiring long explicit thinking traces.

    Source: §1 · p. 2

    Efficiency improvements to LLM reasoning could lower the barrier to deploying reasoning models for harmful purposes.

    Source: Ethics Statement · p. 11

Abstract

from arXiv · show

Long-form thinking traces can substantially improve the multi-step reasoning performance of large language models (LLMs), but they introduce high inference-time overhead, with latency dominated by sequential decoding. We propose HyperThink, a text-to-parameter approach that amortizes this reasoning computation into a single query-conditioned parameter update: a lightweight hypernetwork reads the question and predicts updates to a small subset of the base LLM's parameters, while a vector-quantized decoder constrains them to a finite set of reusable patterns to improve robustness and transfer. Trained end-to-end on outputs from the base model itself, HyperThink eliminates long thinking traces at test time: after one hypernetwork forward pass, the adapted model generates a concise step-by-step solution and final answer without an intermediate trace, using far fewer tokens while retaining strong reasoning performance. Empirically, HyperThink improves the low-latency region of the accuracy-latency trade-off on mathematical and general reasoning tasks, with its strongest gains in the near-non-thinking regime.

1 Introduction

Long thinking traces support multi-step reasoning but impose substantial inference-time cost, while direct non-thinking can sacrifice reliable intermediate computation. HYPERTHINK instead predicts query-conditioned parameter updates so an adapted model can answer without a long trace, with its clearest gains in low-latency settings.

1 Introduction

  • Long thinking traces can improve multi-step reasoning but often dominate the token budget, making inference costly for latency-sensitive or cost-constrained settings.The traces are generated autoregressively before the response.Source: §1 · p. 1
  • Direct non-thinking inference may produce plausible answers through shortcuts rather than reliably performing the intermediate computation needed for correctness.This motivates preserving much of the benefit of explicit traces at or near the cost of decoding short step-by-step responses.Source: §1 · p. 1
  • HYPERTHINK predicts query-conditioned parameter shifts that adapt a frozen base model to produce a concise step-by-step solution and final answer without a long thinking trace.Figure 1 contrasts direct response generation, trace-first generation, and the query-conditioned update approach.Source: §1 · p. 2, Figure 1 · p. 2
  • A vector-quantized bottleneck restricts updates to finite reusable patterns, and training uses the base model’s thinking-mode responses as supervision.Inference needs one lightweight hypernetwork pass followed by standard decoding; the base LLM remains frozen with only a small number of extra parameters.Source: §1 · p. 2
  • Across reasoning benchmarks, HYPERTHINK’s clearest gains are in the low-latency region, particularly on mathematical and question-answering tasks.Its single non-autoregressive, query-conditioned update targets the near-non-thinking regime, where native non-thinking is fast but can sacrifice accuracy and full thinking is too slow for strict latency budgets.Source: §1 · p. 2

2 Preliminaries

The preliminaries frame problem solving as autoregressive response generation evaluated by extracted final answers. They distinguish direct non-thinking generation from trace-first thinking, whose additional autoregressive tokens increase latency.

2.1 Large Language Models for Problem Solving

  • For problem solving, an LLM autoregressively generates a response to a query, is trained to assign high probability to correct responses, and is evaluated by extracting and matching final answers.The evaluation applies a rule-based answer extractor to the response and compares the extracted answer with the dataset ground truth.Source: §2.1 · p. 3

2.2 Inference Modes

  • Thinking mode generates a long intermediate trace before its step-by-step response, whereas non-thinking mode conditions the response directly on the query and can still include solution steps.The hidden trace is usually much longer than the response and must be generated autoregressively, adding significant latency; non-thinking sacrifices that scratchpad and can therefore be much faster.Source: §2.2 · p. 3

3 HYPERTHINK

HYPERTHINK reframes thinking as query-conditioned parameter modulation rather than explicit token generation, using a hypernetwork to steer the base model toward direct responses.

3.1 Thinking Process as Parameter Updates

  • A query-conditioned shift is intended to approximate how thinking tokens steer decoding, without generating or conditioning on those tokens.The shift is intended to capture the resulting change in activations and KV-cache.Source: §3.1 · pp. 3–4
  • A trained hypernetwork amortizes query-specific parameter shifts, applying them to the frozen base model before direct response generation.This replaces per-query optimization that would still require generating thinking tokens at inference.Source: §3.1 · p. 4

3.2 Bias-Only Adaptation

  • Because predicting full LLM weight updates is impractical, HYPERTHINK predicts shifts only to selected layer biases; one 0.6B-model example uses 90K parameters (0.015%).Biases are a lightweight adaptation subspace; LoRA and Prompt Tuning are also possible in principle.Source: §3.2 · p. 4

3.3 Hypernetwork Architecture

  • A frozen text encoder and bias encoder contextualize learnable layer-specific parameter tokens, then a VQ decoder maps them to bias updates.The decoder quantizes representations using layer-wise codebooks, constraining updates to reusable learned prototypes.Source: §3.3 · pp. 4–5, Figure 2 · p. 5

3.4 Training Objective

  • Training freezes the base model and updates the hypernetwork and VQ codebook using teacher-response likelihood, VQ stabilization, and a code-usage regularizer.The usage regularizer encourages batch-wise code assignments toward uniformity, increasing code-use entropy and mitigating collapse.Source: §3.4 · pp. 5–6

4 Experiments

Experiments find that HYPERTHINK improves reasoning performance in low-latency settings, with results spanning math and general reasoning and ablations identifying the value of its VQ bottleneck and bias adaptation.

4.1 Experimental Setup

  • The setup evaluates math and general reasoning with multiple backbones, benchmark-specific training and evaluation, five baselines, and five-sample accuracy, Pass@5, and FLOPs measurements.The reported decoding uses temperature 0.6 and nucleus sampling with p = 0.95.Source: §4.1 · p. 7

4.2 Main Results

  • HYPERTHINK generally improves over non-thinking and distillation baselines on math benchmarks, and beats budget-controlled thinking at similar compute; the reported tables cover Qwen3-0.6B and SmolLM3-3B.The authors also report stronger out-of-distribution MATH-500 transfer than System 2 Distillation and TokenSkip.Source: §4.2 · p. 7, Table 1 · p. 8, Table 2 · p. 8
  • On Qwen3-0.6B math reasoning tasks, HYPERTHINK achieves a significantly better accuracy–latency trade-off than Budget-Controlled Thinking Mode and System 2 Distillation, while improving performance over Native Non-Thinking with a slight latency increase.Figure 4 plots average Pass@5 against latency; HYPERTHINK directly generates response tokens, so its latency is comparable to Native Non-Thinking.Source: Figure 4 · p. 8, §4.2 · p. 8
  • On CommonsenseQA and BIG-Bench Hard, HYPERTHINK generally improves over native non-thinking at low latency; on Olmo-3-7B-Think, it matches evaluated thinking-mode points at about 7–9% of their latency.High-budget thinking remains substantially stronger on search-intensive Olympiad problems, while native non-thinking can remain competitive or stronger on executable code generation.Source: §4.2 · p. 8, Figure 5 · p. 9, Figure 6 · p. 9

4.3 Ablation Studies

  • Component ablations show that query-specific adaptation without the VQ decoder generalizes less well to MATH-500, while the discrete bottleneck acts as a regularizer against overfitting.The comparison includes full fine-tuning, global bias-only tuning, an encoder without VQ, and full HYPERTHINK.Source: §4.3 · p. 9, Table 3 · p. 9
  • LoRA and Prompt Tuning variants underperform bias adaptation, supporting the authors’ choice of bias-only tuning.The comparison uses rank 8 for LoRA and 32 prompt tokens for Prompt Tuning.Source: §4.3 · p. 9, Table 4 · p. 9
  • The learned VQ codes align with math subjects and subject-related keywords, suggesting that individual codes capture semantic structure in the problem space.The examples include Algebra-related problems for one code, Probability for another, and Precalculus for a third.Source: §4.3 · p. 10

5 Related Work

Prior work reduces reasoning cost through shorter or parallel traces, distillation, and hypernetworks, while HYPERTHINK uses query-conditional updates rather than static student weights or task-level adapters.

5 Related Work

  • CoT improves reasoning but adds inference latency, motivating methods that parallelize steps, prune tokens, or constrain trace length.Distillation can compile reasoning into efficient behavior, but the cited approaches produce a single static set of student weights.Source: §5 · p. 10
  • HYPERTHINK differs from static student distillation by applying dynamic, query-conditional parameter updates to a frozen base model.It shares the goal of distilling System 2 into System 1 reasoning (Yu et al., 2024).Source: §5 · p. 10
  • Prior hypernetwork work generates task-specific adapters or instance-conditioned parameters, while HYPERTHINK targets LLM reasoning rather than few-shot visual classification.Task-specific adapters may be reused across problems within a task (Mahabadi et al., 2021; Charakorn et al., 2025).Source: §5 · p. 10

6 Conclusion

HYPERTHINK amortizes long thinking traces into query-conditioned updates to a frozen base model. It improves the low-latency accuracy–latency trade-off, especially near non-thinking inference.

6 Conclusion

  • HYPERTHINK amortizes long thinking traces into a small, structured parameter shift predicted for each query.The update lets the base model operate in non-thinking mode while emulating the reasoning ability of thinking mode.Source: §6 · p. 10
  • A bias-predicting hypernetwork and vector-quantized bottleneck constrain per-query adaptations to finite reusable reasoning prototypes.Source: §6 · p. 10
  • HYPERTHINK improves the low-latency accuracy–latency trade-off on mathematical and general reasoning tasks, with strongest gains near the non-thinking regime.Source: §6 · p. 10

Ethics Statement

The ethics statement notes that making LLM reasoning more efficient could lower deployment barriers for harmful uses, a concern it says applies broadly to inference cost-performance improvements.

Ethics Statement

  • The paper addresses efficiency improvements to LLM reasoning on mathematical reasoning tasks.Source: Ethics Statement · p. 11
  • The authors say lower reasoning costs could lower the barrier to deploying models for harmful purposes.Source: Ethics Statement · p. 11
  • The stated deployment concern is not unique to HYPERTHINK but applies broadly to research improving inference cost-performance.Source: Ethics Statement · p. 11

Reproducibility Statement

The statement identifies where experimental and architectural details are documented. The appendices distinguish HYPERTHINK from related adaptation and reasoning methods and specify the baselines used for comparison.

Reproducibility Statement

  • Base LLMs, training datasets, and evaluation benchmarks are described in Section 4; architecture and hyperparameters are detailed in Section 3 and Appendix C.The base models are publicly available, and code release is planned upon publication.Source: Reproducibility Statement · p. 11

A Related Work

  • Unlike gradient-based test-time adaptation or recurrent latent reasoning, HYPERTHINK predicts a single query-conditioned update in one hypernetwork pass.The base LLM remains frozen, and the adapted model decodes a short response without a recurrent latent trajectory.Source: A Related Work · p. 15
  • HYPERTHINK distills the base model’s query-specific thinking trace into a temporary update, unlike distillation from persistent documents or task descriptions.Its update is per-query and VQ-regularized rather than a static student or reusable document-level adapter.Source: A Related Work · p. 15

B Further Details on Baselines

  • The baselines span unconstrained and budget-controlled thinking, native non-thinking, System 2 distillation, and compressed-trace generation with TokenSkip.TokenSkip uses query-conditioned compressed CoT generation with rank-8 LoRA adapters while keeping the base LLM frozen.Source: B Further Details on Baselines · p. 16

C Further Details on Implementation

This section specifies HyperThink’s encoder, parameter targets, and shared training and evaluation setup.

C Further Details on Implementation

  • HyperThink uses a frozen pretrained LLM text encoder and a three-block MM-DiT bias encoder, with a 256-entry VQ codebook of latent dimension 512.The text-encoder backbone varies by base model: Qwen3-Embedding-0.6B for Qwen3-0.6B, and the SmolLM3-3B or Olmo-3-7B-Think model itself for the corresponding experiments.Source: C Further Details on Implementation · p. 16
  • The method adapts biases in the last eight Qwen3-0.6B blocks or the last half of SmolLM3-3B and Olmo-3-7B-Think blocks.These choices yield 90,122, 516,096, and 614,400 adapted parameters, respectively.Source: C Further Details on Implementation · p. 16
  • Training uses AdamW with a 1 × 10^-4 learning rate and batch size 16; training runs use two NVIDIA H200 GPUs and evaluation uses one.For HyperThink, λVQ = 100 and λUse = 0.1; other hypernetwork and training settings are shared across backbones unless noted.Source: C Further Details on Implementation · p. 16

D Complete Results on General Reasoning Experiments

This section presents complete quantitative general-reasoning results for SmolLM3-3B and Olmo-3-7B-Think across four benchmarks, reporting accuracy, Pass@5, and inference FLOPs.

D Complete Results on General Reasoning Experiments

E Additional Ablation Studies

The additional ablations examine which bias layers to adapt and whether the VQ bottleneck improves held-out transfer rather than training-set fit; the section also provides inference-mode examples.

E Additional Ablation Studies

  • The selected target biases exclude k proj and adapt later blocks: the last eight for Qwen3-0.6B and the last half for SmolLM3-3B and Olmo-3-7B-Think.The last-eight setting performs best on Qwen3-0.6B MATH-500, while the last-half setting performs best on SmolLM3-3B GSM8K-Test and MATH-500.Source: E Additional Ablation Studies · p. 18
  • The VQ and continuous variants perform nearly identically on GSM8K training data, while VQ improves held-out GSM8K-Test and MATH-500 accuracy, especially under the GSM8K-to-MATH-500 shift.The authors interpret this pattern as regularization of the query-to-parameter mapping through reuse of finite update prototypes, supporting transfer rather than increased training-set fit.Source: E Additional Ablation Studies · p. 18

F Example Generation for Each Inference Mode

  • The section provides generation examples for Thinking Mode, Native Non-Thinking Mode, System 2 Distillation, and HYPERTHINK.The supplied passage identifies the compared modes but gives no further example outcomes.Source: F Example Generation for Each… · p. 18

F.1 GSM8K Examples

The GSM8K example computes Jill’s annual salary by combining her weekly teaching and coaching earnings, then multiplying the total by 50 weeks.

F.1 GSM8K Examples

  • Teaching pays 700 dollars per week and coaching pays 450 dollars per week, for combined weekly earnings of 1,150 dollars.The example computes these rates as 20 × 35 and 30 × 15, respectively.Source: §F.1 · p. 19
  • The worked response states Jill’s annual salary as $57,500.The example’s final answer is also shown as 57500.Source: §F.1 · p. 19
  • Multiplying 1,150 dollars per week by 50 weeks gives annual earnings of 57,500 dollars.The example presents the multiplication as the final annual-earnings calculation.Source: §F.1 · pp. 19–20

Native Non-Thinking Mode

The section presents examples of Native Non-Thinking Mode and System 2 Distillation outputs, including salary and MATH-500 answers and derivations with differing final values.

Native Non-Thinking Mode

  • In the Jill example, the response adds her $20 and $30 hourly earnings to get $50 per hour and calculates $125,000 from 2,500 total hours per year.The 2,500 hours come from 50 weeks at 35 teacher hours plus 15 coach hours per week; multiplying 2,500 hours by $50 per hour gives $125,000.Source: Native Non-Thinking Mode · pp. 20–21

System 2 Distillation

  • The System 2 Distillation example reports Jill’s annual salary as $57,500 after calculating weekly earnings from both jobs.It gives the weekly earnings as $700 for teaching and $450 for coaching, then multiplies their $1,150 total by 50 weeks.Source: System 2 Distillation · p. 21

F.2 MATH-500 Examples

  • The MATH-500 example ends with a = -256, matching the product of roots 32 derived from the logarithmic condition and Vieta’s formula.The solution identifies the product as -a/8 and equates it to 32.Source: §F.2 · p. 25

Native Non-Thinking Mode

  • The native non-thinking cubic response derives a root product of 32 from the logarithmic condition, then states that Vieta’s product is a/8.The supplied response does not include a final value for a.Source: Native Non-Thinking Mode · pp. 26–27
  • The Native Non-Thinking Mode output concludes with a boxed answer of 256.Source: Native Non-Thinking Mode · p. 27

System 2 Distillation

  • In System 2 Distillation, the logarithm sum of 5 implies r_1r_2r_3=32, which is then combined with Vieta’s product relation.The later derivation states r_1r_2r_3=-a/8.Source: System 2 Distillation · pp. 28–29
  • System 2 Distillation ends with a=-256, but an earlier derivation in the same subsection gives 256 using the opposite sign for the Vieta root product.The later derivation uses r_1r_2r_3=-a/8 and sets the product to 32; the earlier derivation uses a/8 and reaches 256.Source: System 2 Distillation · pp. 27–29

Related papers

Loading 2610.03039v1…