Source-linked AI summary
Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models
Lok-Lam Ieong, Chia-Chien Chen, Chih-Kai Yang, Yu-Han Huang, An-Yu Cheng, Hung-yi Lee
TL;DR
Reasoning remains a limitation for large audio-language models, and enhancing Chain-of-Thought without additional training is an open question. This paper introduces inference-time hidden-state steering and finds generally improved CoT performance across four models and four benchmarks, with gains up to 4.4% absolute accuracy.
Problem
It remains unclear whether Chain-of-Thought reasoning in large audio-language models can be enhanced at inference time without extra training.
Method
The paper derives reasoning-oriented steering directions from CoT versus non-CoT hidden states and injects them during decoding using three variants.
Results
Across four LALMs and four speech-based benchmarks, steering generally improves CoT performance, with up to 4.4% absolute accuracy gains over CoT.
Takeaways & Limitations
Text-derived steering transfers to speech-based reasoning, offering stable improvements with few samples and supporting training-free, data-efficient model steering.
Takeaways & Limitations
Steering effectiveness depends on scaling strength and layer position, with instance-specific steering more sensitive to hyperparameter choices.
Abstract
from arXiv · showhide
Chain-of-thought (CoT) prompting has been extended to large audio-language models (LALMs) to elicit reasoning, yet enhancing its effectiveness without training remains challenging. We study inference-time model steering as a training-free approach to improve LALM reasoning. We introduce three strategies using diverse information sources and evaluate them across four LALMs and four benchmarks. Results show general accuracy gains up to 4.4% over CoT prompting. Notably, we identify a cross-modal transfer where steering vectors derived from few text samples effectively guide speech-based reasoning, demonstrating high data efficiency. We also examine hyperparameter sensitivity to understand the robustness of these approaches. Our findings position model steering as a practical direction for strengthening LALM reasoning.
1. Introduction
This work investigates training-free hidden-state steering to enhance chain-of-thought reasoning in large audio-language models. Across four models and four speech-based benchmarks, steering generally improves CoT performance while enabling generalized and cross-modal interventions.
- Motivation: Reasoning remains a fundamental limitation of large audio-language models despite their strong auditory perceptual capabilities.This limitation prevents LALMs from fully realizing their potential as foundations for universal and interactive auditory intelligence.
- Motivation: Existing LALM CoT improvements rely on supervised reasoning data or reinforcement learning, requiring additional supervision and substantial training cost.The introduction frames training-free enhancement as an alternative to these costly methods.
- Method: The proposed framework derives reasoning-oriented steering directions from differences between CoT and non-CoT hidden states and injects them during decoding.It treats steering as a representation-level intervention for improving CoT reasoning in LALMs.
- Method: Three variants are introduced: Vanilla Steering uses instance-specific vectors, while SGS and TGS derive shared directions from spoken and text-only auxiliary data, respectively.TGS transfers text-derived steering directions to speech-based reasoning; a qualitative example shows improved intermediate reasoning and corrected final prediction.
- Results: 4.4% absolute accuracy gains over CoT are achieved across four advanced LALMs and four speech-based benchmarks.Vanilla steering also surpasses self-consistency under a comparable computational budget while requiring fewer decoding operations.
- Contributions: The study demonstrates generalized and cross-modal steering directions while providing empirical insights into steering stability and data efficiency.These findings position model steering as a training-free approach for strengthening LALM reasoning.
2. Methodology
The methodology extracts steering vectors from hidden-state differences and injects them during decoding as a training-free intervention. It introduces Vanilla Steering, Speech-derived Generalized Steering, and Text-derived Generalized Steering, including transfer from text-only data to spoken reasoning tasks.
- Overview: Model steering has extraction and injection phases, with three proposed strategies for deriving steering vectors.Extraction derives vectors, while injection applies them during generation.
- Extraction strategies: Vanilla Steering dynamically constructs a sample-specific vector from hidden states under different prompts for the same input without supervision.This training-free inference-time intervention requires additional forward passes for every test sample.
- Extraction strategies: SGS computes one shared steering direction from an external auxiliary spoken dataset using Difference-in-Means and reuses it across test sets.The shared vector is applied uniformly to test samples, reducing the sample-specific overhead of Vanilla Steering.
- Extraction strategies: TGS derives a shared direction entirely from text-only inputs and transfers it at inference time to spoken reasoning tasks.This setup assesses whether text-derived steering can improve CoT performance when spoken extraction data is less accessible.
- Injection phase: During decoding, the extracted vector is scaled by α, injected into selected layers at every token position, and applied with norm-preserving rescaling.The intervention modifies hidden states throughout decoding before standard forward computation and decoding continue.
- Motivation: Steering reinforces CoT-related hidden-state activations to encourage more structured reasoning when multimodal models do not reliably follow CoT instructions.The steering vector is derived from CoT-induced state differences.
3. Experimental Setups
The experiments cover four advanced LALMs, four spoken reasoning benchmarks, and comparisons against default, chain-of-thought, and self-consistency baselines. Steering uses disjoint external or development data, with hyperparameters selected on spoken GSM8K.
- Models and decoding: The study evaluates Voxtral, Phi4-mm, Qwen2.5, and AF3, using greedy decoding unless otherwise specified.These are Voxtral-mini-3B, Phi4-Multimodal-Instruct, Qwen2.5-Omni-7B, and Audio Flamingo 3.
- Baselines: Baselines include Normal default performance, direct CoT prompting, and self-consistency with three temperature-0.5 outputs aggregated by majority vote.Self-consistency approximates Vanilla Steering’s three-forward-pass cost but remains more expensive because every pass performs full generation.
- Steering data: BeyondAIME provides 100 strictly disjoint external samples for SGS and TGS; SGS verbalizes math questions and synthesizes speech with IndexTTS2.The synthesized dataset Ds_ext receives manual quality checks, while TGS uses the original BeyondAIME forms Dt_ext.
- Hyperparameter tuning: Hyperparameters are tuned on disjoint spoken GSM8K development data by searching α ∈[0.025, 0.2] and k ∈1, . . . , 5, then selecting the best development configuration.The tuned parameters are the number of steered last layers k and scaling factor α.
- Evaluation benchmarks: Evaluation uses College, High School, and Elementary Mathematics from VoxEval plus ReveAL-CoT from SpeechR, following each benchmark’s official protocol.The VoxEval benchmarks span varying math difficulty, while ReveAL-CoT targets spoken scientific reasoning.
4. Results
Steering improves CoT accuracy across most evaluated settings, with gains varying by model and method. TGS provides the largest average improvement and remains effective with few textual samples, while Vanilla Steering is more sensitive to hyperparameters.
- Overall results: 11 of 12 model–method combinations show positive average-accuracy gains over CoT, with AF3 and Voxtral improving by +4.4% and +4.3%, respectively.Other models also benefit from multiple steering variants, though effects vary across models.
- Overall results: Vanilla Steering achieves higher overall accuracy than self-consistency on three of four models under the same three-forward-pass budget.Self-consistency performs three full generation processes, whereas Vanilla Steering requires one generation pass after vector extraction.
- Overall results: SGS and TGS improve average accuracy on all models, sometimes matching or outperforming Vanilla Steering, including TGS on Phi-4-mm and AF3.Both variants extract general steering vectors without instance-specific information.
- Overall results: 2.5% is TGS’s average gain over CoT, exceeding Vanilla Steering’s 1.9% and SGS’s 1.4%.TGS derives steering vectors purely from textual data, indicating that reasoning-relevant directions can transfer to spoken tasks.
- Hyperparameter sensitivity: Vanilla Steering peaks at small α values and degrades rapidly as α increases, whereas SGS and TGS remain stable across a wider α range.Instance-specific vectors can be over-amplified, while aggregated directions produce smoother representation shifts.
- Data efficiency: SGS accuracy rises with more spoken samples before saturating around 40, while TGS reaches near-peak performance with few textual samples such as 10.TGS is more data-efficient when spoken data is limited; performance also varies nonmonotonically with the number of steered last layers.
5. Conclusion
The paper presents inference-time model steering as a training-free method for improving Chain-of-Thought reasoning in large audio-language models, with consistent gains across four models and four spoken reasoning benchmarks. It also contrasts strong but less stable instance-specific steering with generalized steering directions.
- Conclusion: Inference-time model steering is evaluated as a training-free approach to enhance Chain-of-Thought reasoning in large audio-language models.The intervention operates at the representation level.
- Conclusion: Across four LALMs and four spoken reasoning benchmarks, representation-level intervention consistently improves CoT performance.The conclusion reports consistent improvement across both model and benchmark sets.
- Conclusion: Instance-specific steering achieves strong gains but is sensitive to hyperparameters and less stable than generalized steering directions.The passage explicitly contrasts the strength and stability of these steering approaches.