PaperlayerView exact revision on arXiv

Source-linked AI summary

Qwen-Audio-3.1-ASR Technical Report

Chuanmeng Bian, Daren Chen, Peixin Chen, Zhigao Chen, Chong Deng, Zhiyun Fan, Zhifu Gao, Bo Gong, Qing Gu, Jiajun He, Yawei Hu, Yunjie Ji, Wei Ju, Jingbei Li, Xiangang Li, Xu Li, Zengxi Li, Zheng Li, Chengdong Liang, Baiji Liu, Yaming Liu, Ying Liu, Bin Ma, Yiping Peng, Yuezhang Peng, Zhendong Peng, Yu Pu, Yang Shi, Xin Shu, Jian Tang, Biao Tian, Peiyao Wang, Tianzi Wang, Wen Wang, Wupeng Wang, Cheng Wen, Yuzhong Wu, Zijian Xia, Yunchong Xiao, Nan Yang, Jianwei Yu, Jixing Yu, Binbin Zhang, Lei Zhang, Sitong Zhao, Guangdong Zhou, Yuan Zhou, Jianheng Zhuo

arXiv:2610.28711v1cs.SDcs.CL

TL;DR

A transcript alone omits speaker-and-timing structure and audio information beyond words. Qwen-Audio-3.1-ASR extends an existing audio-language model with speaker-diarized transcription and general audio understanding; evaluations report competitive recognition and strong results across the broader task families.

  • Problem

    A transcript alone captures only part of a recording, omitting who spoke and when as well as acoustic information not recoverable from words.

    Source: §1 · pp. 2–3

  • Method

    Qwen-Audio-3.1-ASR builds on Qwen-Audio-3.0-ASR and adds speaker-diarized transcription and, in its Next variant, general audio understanding.

    Source: §1 · p. 3

  • Results

    Evaluations show competitive recognition across Chinese, English, multilingual, and dialectal benchmarks, with strong results on speaker-diarized transcription and audio understanding tasks.

    Source: §6 · pp. 28–29

  • Takeaways & Limitations

    The family combines multilingual and dialectal recognition controls with structured speaker-aware transcription and broader audio analysis in selected variants.

    Source: §6 · pp. 28–29

    Speaker-diarized transcription and general audio understanding are unavailable in the Message ASR profile and streaming variant.

    Source: §4.3 · p. 16, §4.2 · p. 16

Abstract

from arXiv · show

We present Qwen-Audio-3.1-ASR, a family of audio-language models that builds upon its predecesor Qwen-Audio-3.0-ASR with improved speech recognition and expanded capabilities in speaker-diarized transcription and general audio understanding. The family retains support for 30 languages and 16 Chinese dialects, together with instruction-controlled capabilities including historical-context conditioning, hierarchical hotword customization, and native transcription polishing. Speaker-diarized ASR produces structured outputs containing speaker labels, timestamps, and recognized text, maintaining consistent speaker labels across turns and the representation of overlapping speech. The general audio understanding capabilities cover audio-, speech-, and music-captioning, temporal grounding and sound event detection, as well as audio question answering and multiple-choice reasoning. These tasks broaden the application scope from recognizing spoken words to interpreting speakers, acoustic events, and their temporal relations. To accommodate different deployment requirements, the family includes variants for asynchronous file transcription, low-latency streaming, interactive voice input, and non-streaming recognition, with speaker-diarized transcription and general audio understanding available in certain variants. Evaluations span conventional ASR, multilingual and dialectal speech, speaker-diarized transcription, and general audio understanding, demonstrating the capabilities of the Qwen-Audio-3.1-ASR family across transcription and broader audio analysis tasks.

1 Introduction

The introduction frames ASR as insufficient for recovering speaker and timing structure or acoustic information beyond words. Qwen-Audio-3.1-ASR extends recognition with speaker-diarized transcription and general audio understanding, while targeting recognition quality across deployment settings.

1 Introduction

  • Audio systems must address what was said, who spoke and when, and what the recording conveys beyond its transcript.The introduction motivates these needs for multi-party conversations and interpretation of speech, environmental sounds, and music.Source: §1 · pp. 2–3
  • Prior audio-language systems span multilingual recognition, multimodal audio perception, captioning, timestamped transcription, and speech generation.The introduction cites Qwen3-ASR [6], Qwen3-Omni, MOSS-Audio [8], and FireRedAudio [9] as examples.Source: §1 · p. 2
  • Production-oriented systems must balance recognition quality, structured outputs, audio understanding, and deployment configuration as distinct concerns.The introduction identifies multilingual and dialectal speech, terminology, conversational context, speaker consistency, timing, and latency as relevant demands.Source: §1 · p. 3
  • The family adds speaker-diarized transcription and general audio understanding to improved speech recognition, retaining support for 30 languages and 16 Chinese dialects.Speaker-diarized outputs include recognized text, speaker labels, and timestamps; general audio understanding is available in the Next variant.Source: §1 · p. 3
  • The report evaluates speech recognition, speaker-diarized transcription, and audio understanding using task-appropriate protocols.It describes architecture, training, deployment variants, and experiments in subsequent sections.Source: §1 · p. 3

2 Model Architecture

The architecture extends an audio-language framework to transcription, speaker-diarized outputs, and general audio understanding. Instructions and optional textual context shape decoder outputs from acoustic representations, while supported capabilities vary by model variant and deployment profile.

2 Model Architecture

  • The core framework combines an audio encoder, audio adapter, and LLM-based decoder, with an auxiliary CTC branch supplying preliminary recognition hypotheses.The adapter bridges audio representations to the decoder, and task instructions and optional context specify desired output behavior.Source: §2 · pp. 3–4, §2.1 · p. 4

2.1 Audio Encoding and Instruction-Conditioned Decoding

  • The decoder generates task-dependent output from audio embeddings, instructions, and optional textual conditions such as language, history, hotwords, or polishing instructions.For audio understanding, instructions can instead specify a description, target event, or question about the recording.Source: §2.1 · p. 4

2.2 Speaker-Diarized Transcription

  • Speaker-diarized transcription associates each segment's recognized text with a recording-local speaker label and start and end times.Speaker labels remain consistent across turns, and temporal intervals may overlap for different speakers.Source: §2.2 · p. 5

2.3 General Audio Understanding

  • General audio understanding includes captioning, temporal grounding, and audio question answering across speech, environmental sounds, and music.Grounding localizes a single target interval or retrieves all matching intervals; question answering includes open-ended and multiple-choice formats.Source: §2.3 · p. 5

2.4 Deployment Variants

  • The report summarizes five deployment variants and their supported capabilities, including speaker-diarized ASR and general audio understanding.The variants table uses checkmarks for supported capabilities and dashes for unsupported ones.Source: §2.4 · p. 5

3 Training

Training is described as a six-stage progression from encoder pretraining through multi-teacher distillation. The report also points to a deployment-variant capability summary and an audio-encoder pretraining pipeline.

3 Training

  • Training comprises six stages: audio encoder pretraining, audio-language model pretraining, LLM cooldown adaptation, supervised fine-tuning, reinforcement learning, and multi-teacher on-policy distillation.The stages align acoustic and linguistic representations, develop instruction following, optimize task performance, and consolidate multiple teachers' capabilities.Source: §3 · p. 5
  • Table 1 summarizes five deployment variants and their supported capabilities, using checkmarks for support and dashes for unsupported capabilities.Its caption defines SD as speaker-diarized ASR and AU as general audio understanding.Source: Table 1 · p. 6
  • Figure 4 presents the audio encoder pretraining pipeline.The supplied figure caption does not specify the pipeline stages.Source: Figure 4 · p. 6

3.1 Pretraining of Audio Encoder

The audio encoder is trained first with self-supervision on unlabeled speech and then with supervised audio-text data. The resulting encoder initializes the audio encoder used in subsequent Audio LLM pretraining.

3.1 Pretraining of Audio Encoder

  • The encoder pretraining uses two stages across a corpus of tens of millions of audio hours: BEST-RQ self-supervision followed by supervised AED training.The first stage uses large-scale unlabeled speech; the second uses labeled audio-text data.Source: §3.1 · p. 6
  • BEST-RQ masks acoustic frames and trains the encoder to predict discrete targets generated from unmasked features using a fixed random projection and frozen codebook.Compatible Transformer layers are initialized from a pretrained Qwen3 LLM, while the non-streaming encoder uses bidirectional attention for speech encoding.Source: §3.1 · p. 6
  • Supervised AED training optimizes an autoregressive decoder to predict reference transcriptions token by token from encoder representations.After this stage, the AED decoder is discarded and the trained encoder initializes the Qwen-Audio-3.1-ASR audio encoder.Source: §3.1 · pp. 6–7

3.2 Pretraining of Audio LLM

Audio LLM pretraining aligns acoustic representations with the LLM’s text space, combining Qwen’s linguistic knowledge with learned audio–text alignment.

3.2 Pretraining of Audio LLM

  • Approximately 20 million hours of multilingual ASR and interleaved text–audio data train the audio adapter and LLM decoder while the pretrained encoder remains frozen.Cross-entropy over target text tokens establishes audio–text alignment for subsequent ASR adaptation.Source: §3.2 · p. 7
  • The resulting Audio LLM provides multilingual knowledge, world knowledge for ambiguous entities and homophones, and instruction-following capability for flexible textual conditions.The decoder is initialized from Qwen’s language-model component, with vision-specific components unused.Source: §3.2 · p. 7

3.3 LLM Cooldown Adaptation

A cooldown adaptation stage uses higher-quality audio–text data to refine audio–language alignment and broaden language, dialect, and task coverage before supervised fine-tuning.

3.3 LLM Cooldown Adaptation

  • Approximately one million hours of higher-quality audio–text data support cooldown adaptation, which emphasizes balanced language and dialect coverage, transcription quality, and broader tasks.Its mixture includes conventional ASR, audio understanding, speaker diarization, speech question-answering, and dialogue data.Source: §3.3 · p. 7
  • Multilingual and dialectal ASR data are rebalanced, and samples whose retranscribed hypotheses have word error rate greater than 0.3 are removed.VibeVoice retranscriptions are compared with existing references as a quality filter; they do not replace those references.Source: §3.3 · p. 7
  • Audio understanding, speaker diarization, and speech question-answering and dialogue broaden the objectives beyond plain transcription and prepare for task-specific fine-tuning.These tasks are combined with language and dialect rebalancing.Source: §3.3 · p. 7
  • In the preceding Audio LLM pretraining stage, the pretrained audio encoder is frozen while the audio adapter and LLM decoder are optimized with cross-entropy.Source: §3.3 · p. 7

3.4 Supervised Fine-tuning (SFT)

Supervised fine-tuning adapts the audio-language model to instruction-controlled recognition, speaker-diarized transcription, and, in the Next variant, general audio understanding. Its training examples extend beyond plain ASR while retaining multilingual, dialectal, and application-oriented recognition coverage.

3.4 Supervised Fine-tuning (SFT)

  • SFT jointly updates the audio encoder and adapter, keeps base LLM parameters frozen, and adapts the decoder through LoRA using conversational instruction–audio–response examples.The autoregressive cross-entropy objective covers multilingual, dialectal, code-switching, noisy, far-field, and application-oriented speech.Source: §3.4 · pp. 7–8

3.4.1 Recognition-oriented SFT

  • Specified-language instructions train monolingual transcripts in the requested language and preserve each supported language’s written form in multilingual speech.The mixture includes target-language lists and diverse dialectal and code-switching examples.Source: §3.4.1 · p. 8
  • Recognition controls include domain-entity examples, native transcript polishing, historical-context conditioning, and hierarchical hotwords that favor acoustically supported P0 candidates over broader P1 candidates.Polishing removes fillers and repetitions while preserving the speaker’s final intent; historical context may be incomplete or incorrect, so audio remains primary evidence.Source: §3.4.1 · pp. 8–9

3.4.2 Speaker-diarized ASR

  • Speaker-diarized targets serialize newline-separated [SPK|start-end]text segments, with recording-local speaker labels, audio-relative timestamps, and overlapping intervals.Film and television annotations are cross-validated with specialized models, and simulated meetings broaden conversational and acoustic coverage.Source: §3.4.2 · p. 9
  • Speaker-ID tokens receive a cross-entropy weight of 10 while other supervised tokens retain weight 1, substantially improving DER and cpWER without changing targets or token counts.Speaker-ID tokens are 2.90% of supervised tokens but contribute 1.97% of unweighted loss; Figure 5 illustrates category shares and reweighting.Source: §3.4.2 · pp. 9–10, Figure 5 · p. 10
  • Quantizing timestamps from 1 ms precision to an 80 ms grid matching the audio representation yields faster loss convergence and better DER and cpWER under the same data mixture.Figure 6 compares training loss with original and quantized timestamps.Source: §3.4.2 · p. 10, Figure 6 · p. 11

3.4.3 General Audio Understanding

  • The Next variant adds audio-understanding examples spanning speech, environmental sounds, and music, including captioning, temporal grounding, and audio question answering.Questions use open-ended or multiple-choice reference answers.Source: §3.4.3 · p. 10
  • Across audio-understanding tasks, the model autoregressively predicts reference responses conditioned on audio and task instructions, covering semantic descriptions, temporal annotations, and audio-grounded answers.Source: §3.4.3 · p. 11

3.5 Reinforcement Learning (RL)

Reinforcement learning complements token-level training by optimizing sequence-level recognition and diarization measures. The paper introduces asynchronous audio-aware training infrastructure and task-specific rewards for ASR and speaker-diarized ASR.

3.5 Reinforcement Learning (RL)

  • RL optimizes transcription accuracy and speaker-diarized ASR after SFT, using sequence-level recognition, diarization, and temporal-consistency measures.The compact dataset includes hard cases, long utterances, hallucination-related examples, hotword and keyword cases, and regular ASR data.Source: §3.5 · p. 11
  • FunVerl extends verl with audio preprocessing, multimodal rollouts, and task-specific rewards, asynchronously decoupling response generation from policy optimization.A trajectory queue separates rollout generation from trainer consumption, while a policy-version threshold limits stale trajectories; Figure 7 depicts the framework.Source: §3.5 · pp. 11–12, Figure 7 · p. 12
  • GRPO [19] generates response groups for each audio–instruction input, normalizes rewards within each group into a shared token advantage, and optimizes a clipped objective with KL regularization.The objective uses a fixed reference policy and requires no separately trained value network.Source: §3.5 · p. 12

3.5.1 ASR Reward

  • ASR rewards combine WER-based accuracy with sample-dependent keyword, hotword, context, and hallucination criteria, routed according to each sample’s source.Keyword recall and accuracy balance recognition against insertions; hallucinated spans are penalized by length.Source: §3.5.1 · pp. 12–13

3.5.2 Speaker-diarized ASR Reward

  • Speaker-diarized ASR rewards use DER for speech activity and speaker consistency, cpWER for transcript and speaker assignment, and tcpWER for temporal alignment.Together these components score words, speaker labels, and timestamps in the structured response.Source: §3.5.2 · p. 13

3.6 On-policy Distillation

On-policy distillation consolidates capabilities from task-specialized teachers into one audio-language student using token-level feedback on student-generated responses. Task routing and a mixed training set preserve the distinct output requirements of audio and text tasks.

3.6 On-policy Distillation

  • On-policy distillation transfers transcription, speaker-diarization, and audio-understanding capabilities from specialized teachers to one student using token-level feedback on student-generated responses.Unlike reinforcement learning’s sequence-level task-metric feedback, the teacher evaluates responses generated by the student.Source: §3.6 · p. 14

3.6.1 Learning from Student-generated Responses

  • Student-generated rollouts let the teacher guide prefixes the student actually visits, including prefixes containing its own mistakes, addressing distribution mismatch with fixed reference responses.The k3 objective averages token-level penalties over student responses; trajectories are refreshed as the student evolves, with policy-version control limiting lag in asynchronous execution.Source: §3.6.1 · p. 14

3.6.2 Teacher Models and Task Routing

  • Task-based routing assigns examples to specialized ASR, diarization, audio-understanding, or Qwen3.5 text-and-dialogue teachers, which supervise the same student.Their supervision targets respectively include speech-to-text, speaker labels and timing, audio-grounded responses, and language modeling and conversational capabilities.Source: §3.6.2 · p. 14

3.6.3 Training Data Mixture

  • The training mixture combines ASR, speaker-diarization, and audio-understanding data with text and dialogue data, preserving each audio task’s distinct output requirements.Text and dialogue examples are distilled from Qwen3.5; dialogue inputs retain preceding conversational context to help preserve general language and conversational capabilities during audio specialization.Source: §3.6.3 · p. 15

4 Deployment

Qwen-Audio-3.1-ASR provides deployment profiles for multilingual recognition, streaming, context-aware voice input, speaker-diarized transcription, and broader audio understanding. Capabilities vary by profile, with some specialized tasks limited to non-streaming variants.

4 Deployment

  • Deployment profiles cover file transcription, low-latency streaming, context-aware voice input, speaker-diarized transcription, and general audio understanding, with some capabilities variant-dependent.The family also adds explicit selection between dialect transcription and dialect-to-Mandarin translation.Source: §4 · p. 15

4.1 Multilingual and Dialectal ASR

  • The recognition interface supports 30 languages and 16 Chinese dialectal varieties across eight major dialect regions, with inference-time language selection and original writing systems for multilingual transcripts.For Chinese dialectal speech, users can choose dialect-preserving transcription or translation into written Mandarin.Source: §4.1 · p. 15

4.2 Low-Latency Streaming Recognition

  • Streaming recognition provides provisional transcripts from incremental audio, then refreshes them using the complete utterance after endpoint detection.Chunk size and right-context length configure the latency–accuracy trade-off; shorter contexts prioritize responsiveness, while longer contexts provide more acoustic evidence before updates.Source: §4.2 · p. 15, Figure 9 · p. 16

4.3 Message ASR for Context-Aware Transcription

  • Message ASR combines historical topic and entity cues, hierarchical hotwords, and optional native polishing to produce readable transcripts while preserving intended meaning.Polishing can remove non-semantic fillers and redundant repetitions, resolve explicit self-corrections, and normalize punctuation and formatting; these controls do not add speaker-diarized or general audio understanding outputs to this profile.Source: §4.3 · p. 16

4.4 Speaker-Diarized Transcription

  • The non-streaming Flash and Next variants return recognized text with recording-local speaker labels and temporal boundaries for multi-speaker recordings.Speaker labels persist within a recording, and speaker intervals may overlap; labels do not identify real-world people.Source: §4.4 · pp. 16–17

4.5 General Audio Understanding

  • The Next variant supports instruction-conditioned audio captioning, temporal grounding, and open-ended or multiple-choice questions about speech, environmental sounds, music, and event relations.These tasks use broader acoustic evidence than transcription alone, including localization of one or multiple matching intervals.Source: §4.5 · p. 17

5 Evaluations

Evaluations span conventional and multilingual ASR, dialect ASR and translation, and industry-specific entity recognition. Recognition results are strong across several benchmarks, while relative performance varies by language, task, and domain.

5 Evaluations

  • The evaluation combines public benchmarks with internal tests for dialects, long-tail terminology, hotwords, transcription polishing, and streaming recognition.Public coverage includes conventional and multilingual ASR, speaker-diarized transcription, general audio understanding, and temporal grounding.Source: §5 · p. 17

5.1.1 Chinese and English ASR Benchmarks

  • Qwen-Audio-3.1-ASR has the lowest error rate on four of seven unified-pipeline Chinese and English test sets, reducing error versus each strongest baseline by 0.13–0.32 percentage points.Fun-ASR-Flash has the lowest error rate on the other three test sets: AISHELL-1, AISHELL-2, and WeNetSpeech-net.Source: §5.1.1 · p. 18, Table 2 · p. 19

5.1.2 Multilingual ASR Benchmarks

  • On GigaSpeechBench’s seven East and Southeast Asian languages, Qwen-Audio-3.1-ASR has the lowest error rate on every language and a 15.09% macro average versus 22.50% for Qwen-Audio-3.0-ASR.Improvements appear across all seven languages, with particularly large absolute reductions on Japanese, Malay, and Thai.Source: §5.1.2 · p. 18, Table 3 · p. 20
  • On Common Voice 15, Qwen-Audio-3.1-ASR has a 3.74% macro average and the lowest error on five of eight languages; on FLEURS, its 3.85% compares with 4.94% for Qwen-Audio-3.0-ASR.On FLEURS, Azure MAI-Transcribe-2 has the lowest macro average at 3.25%; Qwen-Audio-3.1-ASR has lower error rates on all seven Asian languages.Source: §5.1.2 · pp. 18–19, Table 3 · p. 20

5.2 Chinese Dialect ASR and AST

  • On the selected public dialect benchmarks, Qwen-Audio-3.1-ASR’s macro-average CER is 4.55%, versus 7.15% for Doubao-ASR and 4.61% for Tencent Hy-ASR-3.0-preview.It has the lowest CER on six subsets, with a small macro-average advantage over Tencent and a substantial improvement over Doubao.Source: §5.2 · pp. 19–20, Figure 10 · p. 21
  • On internal dialect ASR, Qwen-Audio-3.1-ASR has the lowest CER on 11 of 16 subsets and a 10.38% macro average, versus 20.23% for Doubao-ASR and 17.13% for Tencent Hy-ASR-3.0-preview.Its largest improvement is on Wenzhou: 19.21% CER versus 68.22% and 64.74% for the two baselines.Source: §5.2 · p. 20, Figure 11 · p. 21
  • On internal dialect-to-Mandarin AST subsets matched to each baseline’s documented dialect coverage, Qwen-Audio-3.1-ASR achieves macro-average semantic sentence accuracy of 88.03% versus 78.69% for Doubao-ASR and 89.03% versus 77.53% for Tencent Hy-ASR-3.0-preview.Source: §5.2 · p. 22, Figure 12 · p. 22

5.3 Industry-Domain Entity Recognition

  • Across 15 industry domains, Qwen-Audio-3.1-ASR has the highest entity recall in nine and ties in two, with recall ranging from 94.9% to 99.5%.Tencent retains higher recall in four domains; the comparison does not isolate the contribution of entity mining and data synthesis from other model changes.Source: §5.3 · p. 22

5.4 Hotword Customization

Across six evaluation subsections, the report examines hotword customization, long-audio context, native polishing, two-stage message ASR, speaker-diarized transcription, and general audio understanding. Results cover recognition quality, output readability, latency, diarization, and audio benchmarks.

5.4 Hotword Customization

  • Conditioned Qwen-Audio-3.1-ASR has the highest recall among evaluated systems in every P0 category, ranging from 97.95% to 99.46%.P1 recall improves for subject terms and trending buzzwords, but the model does not lead those categories; P0 candidates receive stronger conditioning than P1 candidates.Source: §5.4 · pp. 22–23, Table 4 · p. 24

5.5 Long-Audio Context Modeling

  • Conditioning each segment on recently recognized session text improves recognition consistency for previously mentioned entities and technical terms in representative examples.This history supplies context from preceding segments without requiring a manually maintained hotword list.Source: §5.5 · pp. 23–24, Table 5 · p. 24

5.6 Native Transcription Polishing

  • Native polishing raises readability from 2.53 to 3.44, versus 3.51 for the cascade, while its faithfulness rating is 3.44 versus 3.47 for the cascade.Polishing removes fillers and repetitions, incorporates final corrections, and standardizes formatting in a single decoding pass without a separate refinement call.Source: §5.6 · pp. 24–25, Figure 14 · p. 25

5.7 Streaming Display and Non-streaming Refresh in Message ASR

  • In streaming outputs, Qwen Message has 63.93% mean industry-term recall versus Doubao’s 46.28%; after refresh, recall is 92.82% versus 78.27%.The report also finds lower Qwen Message latency on all three reported metrics under the tested conditions.Source: Table 6 · p. 25, §5.7 · p. 26, Figure 16 · p. 27

5.8 Speaker-diarized ASR

  • Across 12 dataset–metric combinations, the two Qwen variants collectively achieve the lowest error rates; both beat the Fun-ASR cascade on all four test sets for DER and cpWER.Their relative strengths vary by dataset and metric; the reported comparison uses approximately four-minute audio chunks.Source: §5.8 · pp. 26–27, Table 7 · p. 27

5.9 General Audio Understanding

  • Qwen-Audio-3.1-ASR-Flash-Next achieves the best performance on seven of nine audio-understanding benchmarks, including all five specialized tasks.It scores 81.60% accuracy on MMAU, 0.70 percentage points above Gemini-3.1-Pro, and 82.20% mIoU on Clotho-Moment, 10.41 points above the next-best result.Source: §5.9 · p. 28, Table 8 · p. 28

6 Conclusion

Qwen-Audio-3.1-ASR combines speech recognition with speaker-diarized transcription and general audio understanding. The family supports multilingual and dialectal recognition, configurable deployment workflows, and evaluation results across transcription and broader audio tasks.

6 Conclusion

  • Qwen-Audio-3.1-ASR combines improved speech recognition with speaker-diarized transcription and general audio understanding.The family builds on a Qwen Mixture-of-Experts backbone and large-scale speech pretraining.Source: §6 · p. 28
  • The family supports 30 languages and 16 Chinese dialectal varieties, alongside industry-domain entity recognition, historical-context conditioning, hierarchical hotwords, and single-pass polishing.Users can select dialect transcription or translation into Mandarin.Source: §6 · pp. 28–29
  • Non-streaming Flash and Next variants provide speaker labels, timestamps, and recognized text, including overlapping speech; Next also supports captioning, temporal grounding, sound event detection, and audio question answering.Deployment variants serve asynchronous file transcription, low-latency streaming, and context-aware voice input; Message ASR combines streaming display with an utterance-end refresh.Source: §6 · p. 29

Related papers

Loading 2610.28711v1…