Source-linked AI summary
ISCSLP 2026 CoT-TTS Challenge: Chain-of-Thought Reasoning for Context-Aware Text-to-Speech
Wei Xue, Junlan Feng, Shilei Zhang, Yue Wang, Ruosong Yang, Bei Liu, Liumeng Xue, Sitong Cheng, Jiahao Pan, Weizhen Bian, Boyi Kang, Bin Long
TL;DR
CoT-TTS addresses the difficulty of choosing delivery from context when controllable TTS commonly relies on explicit prompts and contextual style reasoning is often implicit. The challenge provides contextual tracks, data, and an evaluation protocol; its lightweight baseline produces meaningful reasoning and some context-aware controllability.
Problem
Controllable TTS often depends on explicit style prompts, while contextual synthesis commonly leaves the link between context understanding and speaking-style choice implicit.
Source: §2 · p. 2Method
The challenge evaluates explicit reasoning and scene-consistent speech from textual or acoustic context, supported by a media-derived dataset and a unified evaluation protocol.
Source: §2 · p. 3, §7.1 · p. 4Results
The 0.6B Qwen3-based baseline produces meaningful reasoning analyses and demonstrates a certain degree of controllability in context-aware CoT-TTS.
Source: §7.2 · p. 5Takeaways & Limitations
The challenge provides a benchmark for studying context understanding, speaker intention, emotional change, and translating reasoning into acoustic realization.
Source: §2 · p. 3
Abstract
from arXiv · showhide
Recent advances in text-to-speech (TTS) have greatly improved speech naturalness, speaker similarity, and controllability. However, most existing controllable TTS systems still rely on explicit user-provided style prompts, making it difficult to automatically determine how a sentence should be spoken in long and complex conversational scenarios. This proposal introduces the ISCSLP 2026 CoT-TTS Challenge, which aims to evaluate whether a system can infer the intended speaking manner from contextual information and generate speech consistent with both the reasoning output and the surrounding scene. The challenge contains two tracks: text-context-aware CoT-TTS and audio-context-aware CoT-TTS. We construct a large-scale bilingual training set from speech-rich media and provide carefully filtered evaluation data for leaderboard comparison. Each system is required to output both a chain-of-thought reasoning analysis and the generated speech waveform. The official evaluation combines objective metrics, multimodal LLM-based evaluation, and human subjective assessment. To facilitate reproducibility, we provide inference code together with a fine-tuning recipe for a 0.6B Qwen3-based model trained via a three-stage strategy. This challenge is expected to support research on context understanding, chain-of-thought reasoning, and expressive speech generation for applications such as film dubbing, audiobook production, virtual characters, and spoken dialogue agents. Further information about the associated challenge is available at:https://iscslp2026-cot-tts.github.io/challenge-website/
1. Session/Challenge Organizers
The section identifies proposed session organizers and provides brief biographies describing their roles and research interests.
1. Session/Challenge Organizers
- Junlan Feng is an IEEE Fellow and Chief Scientist at China Mobile, with interests in AI, big data, and communications-industry AI technologies.Her research particularly focuses on developing the Jiutian AI Platform.Source: §1 · p. 1
- Shilei Zhang leads speech technology at China Mobile Jiutian Research and works on speech recognition, voiceprint recognition, and speech large language models.His interests particularly focus on speech technology development for the Jiutian AI Platform.Source: §1 · p. 1
- Ruosong Yang is an AI Technical Research Senior Executive at China Mobile (Hong Kong) Innovation Research Institute, with interests in large language models and multi-agent systems.His interests particularly focus on LLM-based multi-agent applications.Source: §1 · p. 1
2. Background, Motivation, and Potential Impact
The section motivates CoT-TTS as a way to infer speaking manner from context and make the reasoning behind speech style explicit. It proposes a challenge and dataset to evaluate this capability.
2. Background, Motivation, and Potential Impact
- Most controllable TTS still relies on user-provided style prompts, which are time-consuming to set sentence by sentence in conversations, dubbing, and virtual characters.CoT-TTS uses preceding context to reason about delivery and produce speech matching the scene.Source: §2 · p. 2
- Prior contextual speech synthesis often leaves the link between context understanding and style control implicit, while cascaded systems can add latency and information loss [8] [9].Related systems also differ in role-playing focus or caption-style controls, and often do not explicitly evaluate reasoning [14] [17].Source: §2 · p. 2
- DailyDialog lacks speech recordings, while DailyTalk and its annotation extensions retain relatively simple scenarios and limited scale.The section also describes media- and situated-dialogue datasets, including NCSSD, as efforts toward richer settings.Source: §2 · p. 2
- EmoSpeech has about 16 hours of audio, DNASpeech about 18 hours, and ActorMind [14] a limited number of roles and scenes.These scope constraints motivate a larger dataset with chain-of-thought annotations for CoT-TTS.Source: §2 · p. 3
- The challenge evaluates whether systems infer speaking manner from textual or acoustic context, produce explicit reasoning, and generate scene-consistent speech.It provides two tracks, a bilingual training set, an evaluation protocol, and a reproducible baseline.Source: §2 · p. 3
3. Tentative List of Authors / Teams Who Could Contribute Papers
The section presents a tentative list of potential contributing teams, framed as outreach targets rather than confirmed participants.
3. Tentative List of Authors / Teams Who Could Contribute Papers
- Potential outreach communities include speech synthesis, speech foundation models, spoken dialogue systems, and multimodal speech processing.The section describes these as relevant communities, not confirmed submissions.Source: §3 · p. 3
- The proposed team list reflects preliminary interest and research relevance rather than confirmed participation..Source: §3 · p. 3
- Table 1 presents a tentative list of potential participating teams.The list is based on preliminary interest and the relevance of existing research directions.Source: Table 1 · p. 3, §3 · p. 3
4. Tentative Challenge Timeline
The tentative timeline is aligned with the official ISCSLP 2026 schedule. Its milestones are organized around proposal acceptance and the paper submission and notification dates.
4. Tentative Challenge Timeline
- The model submission deadline will align with the paper submission deadline, and evaluation will finish before paper acceptance notification.The final rankings are planned to be announced alongside accepted challenge papers.Source: §4 · p. 3, Table 2 · p. 3
- The challenge launch is planned after proposal acceptance.The timeline is designed to follow the official ISCSLP 2026 schedule.Source: §4 · p. 3
- Table 2 shows the tentative challenge timeline.Source: Table 2 · p. 3, §4 · p. 3
5. Planned Format of the Challenge Session
The planned two-hour session moves from a challenge overview to winner presentations, then a panel discussion and awards.
5. Planned Format of the Challenge Session
- The two-hour session opens with an overview of the task setup, dataset construction, evaluation methodology, and overall results.Source: §5 · p. 3
- Representatives of winning teams in each track and model category are invited to present their approaches and findings.Source: §5 · p. 3
- A panel discussion on future directions and an award ceremony conclude the session.Source: §5 · p. 3
6. Recommended Expert Reviewers
The section recommends expert reviewers with relevant expertise in speech and spoken-language research.
6. Recommended Expert Reviewers
- The recommended reviewers are described as having expertise in speech synthesis, speech generation, speech understanding, and spoken language processing.Source: §6 · p. 4
7. Additional or Non-Standard Resources Needed
The challenge provides a large bilingual training dataset, a three-stage Qwen3-based baseline, and a website for challenge information and resources.
7.1. Training Dataset
- Training samples are built from speech-rich media by structuring utterances and scenes, annotating emotion and reasoning, and filtering for context and audio quality.The pipeline uses speaker-embedding similarity to reduce diarization-related errors.Source: §7.1 · p. 4, Figure 1 · p. 4
- The released training set contains around 3 million segments, roughly 16K hours of speech, and an English-to-Chinese ratio of around 54:46.Source: §7.1 · p. 4, Table 5 · p. 4
- Released audio preserves realistic acoustic variation, while metadata and annotations are generated from normalized and denoised versions.Source: §7.1 · p. 5
7.2. Baseline
- The baseline fine-tunes a 0.6B Qwen3-based model using parameter-efficient adaptation on RTX 4090 GPUs.The authors report meaningful reasoning analyses and some context-aware style controllability under this lightweight setup.Source: §7.2 · p. 5
- Its three-stage recipe aligns modalities, trains the context-aware CoT-TTS task with auxiliary tasks, then fine-tunes on a higher-quality subset.The final stage is intended to improve reasoning-generation stability and speech-style control.Source: §7.2 · p. 5
7.3. Challenge Website
- The website provides challenge descriptions, track definitions, dataset access, baseline code, submission instructions, announcements, and evaluation details.The submission portal and public leaderboard are to be added before the official submission period.Source: §7.3 · p. 5
8. Challenge Description
The challenge tests context-aware CoT-TTS through text- and audio-context tracks, requiring reasoning and speech outputs that fit the context and reference speaker. It pairs filtered bilingual evaluation data with objective, LLM-based, and human assessment.
8. Challenge Description
- Systems infer speaking manner from context, produce reasoning, and generate speech consistent with the reasoning and surrounding scene; cascaded systems are disallowed.Leaderboards include unrestricted and parameter-constrained categories, the latter for systems below 1B parameters.Source: §8 · p. 5, §8.7 · p. 7
8.1. Track 1: Text-Context-Aware CoT-TTS
- Track 1 uses speaker-labeled dialogue transcripts, target text, and reference speech to infer style, emotion, tone, and rhythm.The contexts come from dramatic and narrative sources where prior dialogue informs the speaker’s state and intent.Source: §8.1 · p. 5
8.2. Track 2: Audio-Context-Aware CoT-TTS
- Track 2 replaces textual history with continuous multi-speaker audio context for style inference and natural continuation of the dialogue.This setting targets cases where prior speech is available but reliable transcripts or structured scene descriptions may not be.Source: §8.2 · pp. 5–6
8.3. Input and Output Specifications
- Each sample supplies track-specific context, target text, and reference speech for speaker timbre; systems return both reasoning analysis and a speech waveform.Generated speech should preserve the reference timbre and match the inferred style.Source: §8.3 · p. 6
8.4. Evaluation Data
- The evaluation pool starts at about 3 million non-overlapping bilingual samples and is filtered for context, audio quality, deduplication, text quality, and audio–CoT consistency.Human review assesses transcription accuracy, scenario richness, reasoning accuracy, and audio–reasoning consistency; the final set contains 600 Chinese and 600 English samples.Source: §8.4 · p. 6, Figure 2 · p. 6
8.5. Evaluation Method
- The final score weights objective, LLM-based, and human evaluation at 30%, 20%, and 50%, respectively.Objective measures include naturalness, quality, intelligibility, speaker similarity, prosody, expression, and efficiency; LLM scoring includes context understanding, reasoning coherence, speech–reasoning consistency, and informativeness.Source: §8.5 · pp. 6–7
8.6. Submission Requirements
- Submissions include executable inference code, the runtime environment, model files, and a system description to support validity checks and reproducibility.Inference must produce both reasoning and speech outputs.Source: §8.6 · p. 7
8.7. Other Rules
- Rules prohibit hidden-test-label inspection and require offline evaluation; systems exceeding RTF 3.0 will not advance to official evaluation or ranking.The parameter-constrained category counts all inference-time models and modules toward its fewer-than-1B-parameter limit.Source: §8.7 · p. 7
B. Data Copyright Statement
The challenge dataset uses publicly accessible media while leaving ownership and copyright with the original materials’ respective owners. Only processed speech segments and metadata or annotations will be released, subject to non-commercial research-use terms.
B. Data Copyright Statement
- Copyright in the original videos, audio programs, podcasts, films, dramas, and other source materials remains with their respective owners.The organizers do not claim ownership of those materials.Source: B. Data Copyright Statement · p. 11
- The release excludes original videos, complete audio programs, and full-length source materials, providing processed speech segments and corresponding metadata or annotations instead.Preparation includes segmentation, clipping, normalization, resampling, quality filtering, transcription, and annotation.Source: B. Data Copyright Statement · p. 11
- Participants may use the dataset only for non-commercial research, development, and evaluation related to the challenge.Commercial use, redistribution, and attempts to reconstruct, identify, or redistribute original source materials are not permitted.Source: B. Data Copyright Statement · p. 11
- The dataset will use a non-commercial research-use license following the spirit of Creative Commons Attribution-NonCommercial 4.0, and organizers may remove or modify items over copyright, privacy, or licensing concerns.Participants must follow the organizers’ data usage terms.Source: B. Data Copyright Statement · p. 11
C. LLM Prompts for Data Construction
The data-construction prompts refine scene boundaries using temporal continuity and topic consistency, and generate context-aware Chinese emotion tags from dialogue and audio-event information. They specify constrained inputs and outputs for both tasks.
C.1. Scene Boundary Refinement
- Scene-boundary refinement asks an editor to identify scene splits using temporal continuity and topic consistency from utterances with relative timestamps.A split index marks the utterance that starts a new scene; index 0 and the total utterance count are excluded.Source: §C.1 · p. 11
- The scene-refinement prompt restricts split decisions to temporal continuity and topic consistency.It excludes index 0 and the total number of utterances from the returned split indices.Source: §C.1 · p. 11
- The scene-refinement prompt requests an ascending JSON array of split indices, with a time-gap threshold, block start time, and utterance list as inputs.The example output is.Source: §C.1 · p. 11
C.2. Emotion Annotation
- Emotion annotation uses the dialogue window, target utterance, original emotion tag, and audio-event information to generate a context-aware Chinese emotion tag.The input template also includes the target speaker and a history note.Source: §C.2 · p. 11
- The emotion tag should be a short Chinese phrase describing emotional intensity, attitude, or speaking manner rather than a bare emotion word.The output field is “emo_tag.”Source: §C.2 · p. 11
- Emotion judgments use only the current dialogue window, with conservative choices when the evidence is insufficient.Source: §C.2 · p. 11
C.3. Reasoning Analysis
The reasoning-analysis prompt takes the target utterance and contextual information and requests an analysis using the current dialogue window. The processed-data format records segment content, annotations, audio features, and timing information.
C.3. Reasoning Analysis
- The reasoning-analysis prompt takes the target speaker and utterance, refined emotion tag, history note, and dialogue window, and instructs analysis using only the current dialogue window.Source: §C.3 · p. 11
- Its requested analysis is organized into language act, scene semantics, cognition/persona/motivation, expected outcome, and emotion trajectory dimensions.Each dimension is shown with a need_cot flag and text field.Source: §C.3 · p. 11
D. Processed Data Format
- The processed-data format includes segment identifiers, text, original and normalized speaker labels, emotion tags, duration and active duration, loudness, expressive intensity, timing, and best-reference similarity.It also includes a cot_text field described as chain-of-thought style analysis.Source: D. Processed Data Format · p. 11
- Dialog-segment records include segment identifiers, text, speaker labels, emotion tags, audio features, and start and end times.Source: D. Processed Data Format · p. 11