Source-linked AI summary
Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Models
Zhifei Xie, Ziyang Ma, Zihang Liu, Kaiyu Pang, Hongyu Li, Jialin Zhang, Yue Liao, Deheng Ye, Chunyan Miao, Shuicheng Yan
TL;DR
Speech interfaces struggle with the latency and listener burden of completing or verbalizing lengthy reasoning before an answer. Mini-Omni-Reasoner interleaves silent reasoning and spoken response tokens using aligned training data and a hierarchical Thinker-Talker architecture. On Spoken-MQA, it reports higher arithmetic and reasoning accuracy with substantially shorter responses than Qwen2.5-Omni-3B.
Problem
Speech consumes reasoning sequentially, so completing or verbalizing a full reasoning trace can delay the answer or burden listeners with verbose content.
Source: §1 · p. 2Method
Mini-Omni-Reasoner interleaves internal reasoning and spoken tokens, supported by causally aligned Spoken-Math-Problems-3M data and a hierarchical Thinker-Talker architecture.
Source: §1 · pp. 2–3, §3.1 · p. 6Results
77.25% arithmetic accuracy and 68.1% reasoning accuracy were achieved versus 64.9% and 64.0% for Qwen2.5-Omni-3B, while responses shortened to 42.9 from 116.1 words.
Source: §1 · p. 3Takeaways & Limitations
The framework produces concise spoken responses while retaining structured internal reasoning for real-time interaction.
Source: §1 · pp. 2–3The conventional thinking-before-speaking approach can incur first-token delay or cognitive overload from exposing reasoning before the answer.
Source: §2.2 · p. 4
Abstract
from arXiv · showhide
Reasoning is essential for effective communication and decision-making. While recent advances in LLMs and MLLMs have shown that incorporating explicit reasoning significantly improves understanding and generalization, reasoning in LSMs remains in a nascent stage. Early efforts attempt to transfer the "Thinking-before-Speaking" paradigm from textual models to speech. However, this sequential formulation introduces notable latency, as spoken responses are delayed until reasoning is fully completed, impairing real-time interaction and communication efficiency. To address this, we propose Mini-Omni-Reasoner, a framework that enables reasoning within speech via a novel "Thinking-in-Speaking" formulation. Rather than completing reasoning before producing any verbal output, Mini-Omni-Reasoner interleaves silent reasoning tokens with spoken response tokens at the token level. This design allows continuous speech generation while embedding structured internal reasoning, leveraging the model's high-frequency token processing capability. Although interleaved, local semantic alignment is enforced to ensure that each response token is informed by its preceding reasoning. To support this framework, we introduce Spoken-Math-Problems-3M, a large-scale dataset tailored for interleaved reasoning and response. The dataset ensures that verbal tokens consistently follow relevant reasoning content, enabling accurate and efficient learning of speech-coupled reasoning. Built on a hierarchical Thinker-Talker architecture, Mini-Omni-Reasoner delivers fluent yet logically grounded spoken responses, maintaining both naturalness and precision. On the Spoken-MQA benchmark, it achieves a +19.1% gain in arithmetic reasoning and +6.4% in contextual understanding, with shorter outputs and zero decoding latency.
1 INTRODUCTION
Mini-Omni-Reasoner addresses the latency and verbosity of speech reasoning by interleaving internal reasoning with continuous spoken responses. It combines this formulation with aligned training data and reports stronger, shorter Spoken-MQA responses than the base model.
Motivation
- The “thinking-before-speaking” paradigm delays spoken answers or exposes verbose reasoning because speech is consumed sequentially over time.Unlike text, speech does not allow listeners to scan or skip extended reasoning, making delayed or verbose responses problematic for real-time interaction.Source: §1 · pp. 1–2
Thinking-in-Speaking
- Mini-Omni-Reasoner interleaves high-frequency internal reasoning with semantically informative spoken tokens to reduce latency while preserving reasoning depth.The framework decouples internal inference timing from speech emission timing, enabling concurrent latent reasoning and verbal response generation.Source: §1 · p. 2, Figure 1 · p. 2
Interleaved Generation
- The method exploits the throughput mismatch between model inference and audio playback by allocating generation bandwidth between speech and reasoning tokens.The system emits spoken tokens at 20 per second and uses a 2:8 speech-to-reasoning token ratio within the Thinker-Talker architecture.Source: §1 · pp. 2–3
Training Data
- Spoken-Math-Problems-3M provides 3 million audio-based mathematical reasoning samples with temporally aligned verbal and symbolic reasoning streams.The pipeline defers substantive verbal content, frontloads internal reasoning, and verifies that no verbal content precedes its logical derivation.Source: §1 · p. 3
- The aligned dataset addresses overshooting, where speech advances ahead of reasoning and can produce premature or hallucinated answers.Its tokenized and interleaved streams are checked for causal consistency between reasoning and spoken output.Source: §1 · p. 3
Results
- The model improves Spoken-MQA arithmetic accuracy from 64.9% to 77.25% and reasoning accuracy from 64.0% to 68.1%, while reducing response length from 116.1 to 42.9 words.These results are reported against Qwen2.5-Omni-3B and are presented as evidence for concise, real-time interaction with preserved correctness.Source: §1 · p. 3
2 INVOLVING REASONING IN SPOKEN DIALOGUE MODELS
The section first describes the modular Thinker-Talker speech pipeline, then contrasts delayed thinking-before-speaking with token-level thinking-in-speaking for lower-latency spoken reasoning.
2.1 THINKER-TALKER PIPELINE
- The Thinker-Talker pipeline converts audio into linguistic tokens, generates responses with the Thinker, maps them to audio with the Talker, and decodes streamed segments for real-time interaction.Its modular separation supports integration of reasoning within the Thinker while preserving streaming speech synthesis.Source: §2.1 · pp. 3–4
2.2 THINKING-BEFORE-SPEAKING
- Thinking-before-speaking generates latent reasoning before the verbal response, either keeping reasoning silent or verbalizing it fully.Silent reasoning improves clarity and efficiency but causes first-token delay, while full verbalization makes listeners hear the reasoning before the answer and can create cognitive overload.Source: §2.2 · p. 4
2.3 THINKING-IN-SPEAKING
- Thinking-in-speaking alternates p response tokens with q reasoning tokens, allowing speech to stream while reasoning continues.The Talker converts only response segments into audio and remains silent for reasoning tokens.Source: §2.3 · pp. 4–5
- The interleaved design exploits faster token generation than audio rendering to provide continuous, low-latency interaction, with the (p, q) ratio tunable to throughput and deployment constraints.Figure 2 shows audio tokens initializing the Thinker, silent reasoning tokens guiding generation, and answer tokens streaming through the Talker into speech.Source: §2.3 · p. 5, Figure 2 · p. 5
3 MINI-OMNI-REASONER
Mini-Omni-Reasoner enables real-time spoken reasoning by interleaving response and reasoning tokens within a hierarchical Thinker–Talker system. It supports this formulation with semantically aligned spoken-math data and staged training that transfers reasoning from text to speech.
3 MINI-OMNI-REASONER
- Mini-Omni-Reasoner combines interleaved reasoning and response generation with a hierarchical Thinker–Talker architecture for real-time spoken dialogue.The Thinker performs reasoning and produces response tokens, while the Talker generates speech conditioned on those outputs.Source: §3 · p. 5, §3.1 · p. 6
3.1 THE PIPELINE OF MINI-OMNI-REASONER
- The Thinker alternates two response tokens with eight reasoning tokens, using control markers and padding to maintain continuous speech and stable alignment.The 2 vs. 8 ratio balances response latency with deeper internal reasoning, while masked markers avoid unstable placement and padding supports downstream Talker alignment.Source: §3.1 · p. 6
3.2 SPOKEN-MATH-PROBLEM DATASET
- The dataset addresses anticipation drift by delaying reasoning-grounded response content and enforcing semantic precedence between reasoning and speech tokens.An asynchronous alignment scheme allows contextual openings, while substantive response content follows prior reasoning rather than appearing prematurely.Source: §3.2 · p. 7
- GPT-based sequence verification filters interleaved examples for premature reasoning content and semantic or temporal inconsistency before training.Only sequences that pass screening are retained, preserving the intended reasoning-before-speaking logic under token-level interleaving.Source: §3.2 · pp. 7–8
- Spoken-Math-Problems-3M contains 3M math instances converted into speech-oriented prompts, paired with reasoning traces and concise responses for interleaved training.Questions receive original and conversational rewrites, while answers preserve reasoning-before-response structure with reasoning roughly twice as long as the response.Source: §3.2 · p. 8
3.3 TRAINING METHODOLOGY
- Training progresses from audio-adapter alignment and mathematical pretraining to textual interleaving, acoustic interleaving, and Talker training.The Thinker is progressively adapted across modalities, then frozen while the Talker learns to synthesize fluent speech from interleaved outputs.Source: §3.3 · pp. 8–9, Figure 4 · p. 9
4 EXPERIMENTS
The experiments evaluate Mini-Omni-Reasoner across spoken reasoning benchmarks, latency, training stability, and qualitative paradigms. It achieves strong arithmetic and contextual reasoning while producing shorter spoken responses and maintaining stable training behavior.
4.1 TRAINING SETUP
- The model is trained from Qwen2.5-Omni-3B components with an audio encoder, linear adapter, Qwen2.5-3B language model, 3 million samples, six epochs, and 32 H100 GPUs.Training uses data parallelism, batch size 64, and cosine learning-rate decay with a 2e-4 maximum learning rate.Source: §4.1 · pp. 9–10
4.2 BENCHMARK
- The benchmark separates arithmetic computation from contextual reasoning, including single-step AddSub and SingleOp tasks and multi-step GSM8K and SVAMP tasks.The evaluation compares cascade, speech-model, and Qwen2.5-Omni-3B foundation-model baselines.Source: §4.2 · p. 10, §4.3 · p. 10
4.3 BASELINES
- The experiments compare against cascade systems, mainstream speech models, and Qwen2.5-Omni-3B under standard and step-by-step prompting.This setup evaluates performance across pipeline-based and end-to-end reasoning-capable alternatives.Source: §4.3 · p. 10
4.4 PERFORMANCE ON SPOKEN-MQA
- 77.25% arithmetic and 68.1% reasoning averages surpass the strongest baselines by +4.0% and +4.1%, respectively, while matching or exceeding several larger or cascade models.The model reaches 92.9% short-form arithmetic, 66.1% long-form arithmetic, 85.9% single-step reasoning, and 60.5% multi-step reasoning.Source: §4.4 · pp. 10–11
4.5 RESPONSE LATENCY COMPARISON
- 42.9 spoken words are less than half of Qwen2.5-Omni-7B’s 116.13 words, although Mini-Omni-Reasoner trails that model on complex multi-task reasoning.Interleaving reduces user-audible content to 25% of total generation length, supporting faster information delivery.Source: §4.5 · pp. 11–12, Figure 5 · p. 12
4.6 TRAINING ANALYSIS
- Training and validation losses decrease coherently from roughly 0.5–0.6 toward 0.1 over more than 200,000 steps without divergence or erratic behavior.The curves support the viability of alternating speaking and thinking during training.Source: §4.6 · pp. 12–13, Figure 6 · p. 12
4.7 CASE STUDIES
- The case study contrasts speaking without reasoning, full reasoning before speech, and interleaved thinking-in-speaking, with the proposed model combining correctness and shorter spoken outputs.Concise phrases such as “after calculating” summarize computation and halve overall response time relative to the full-spoken-reasoning approach.Source: Figure 7 · p. 13, §4.7 · p. 14
5 RELATED WORK
Related work spans speech language models, long-chain reasoning, and reasoning-efficiency methods. Mini-Omni-Reasoner addresses speech’s time-linear output constraint by internalizing reasoning without synthesizing the full reasoning chain.
Speech LLMs
- Speech language models reduce reliance on traditional ASR–text generation–TTS pipelines, but many still verbalize long reasoning chains that increase latency for complex queries.Mini-Omni-Reasoner instead internalizes reasoning as inner thinking while retaining speech reasoning capabilities.Source: §5 · p. 14
Inference Scaling and Long-Chain Reasoning
- Chain-of-thought, test-time scaling, reinforcement learning, and multimodal reasoning methods extend structured reasoning, but they primarily originate in text or broader multimodal settings.The cited methods include CoT, o1, DeepSeek-R1, DAPO, GSPO, SELF-EVOLVING, and LATENT REASONING.Source: §5 · p. 14
Reasoning Efficiency
- Text-oriented efficiency methods shorten or adapt reasoning because users can browse outputs, whereas speech generates time-linear outputs whose long chains directly increase latency.This modality difference motivates token-level thinking-in-speaking rather than unnecessary speech synthesis.Source: §5 · p. 14
6 CONLUSION
Mini-Omni-Reasoner introduces token-level thinking-in-speaking, interleaving silent reasoning with spoken responses through a dedicated dataset and architecture. On Spoken-MQA, it improves arithmetic and contextual reasoning while reducing output length and eliminating decoding latency.
6 CONLUSION
- Mini-Omni-Reasoner interleaves unspoken reasoning tokens with spoken response tokens to produce fluent, timely speech while preserving logical and semantic coherence.The framework is supported by the SPOKEN-MATH-PROBLEMS-3M dataset and alignment strategies for interleaved reasoning–speech generation.Source: §6 · p. 15