FunASR: A Fundamental End-to-End Speech Recognition Toolkit

• FunASR is an open-source speech recognition toolkit designed to bridge academic research and industrial applications.Source: §1 · p. 1
• Paraformer is a non-autoregressive end-to-end model trained on 60,000 hours of manually annotated Mandarin speech.Source: §1 · p. 1
• The toolkit provides training and fine-tuning workflows for Transformer, Conformer, and Paraformer models across several speech datasets.Source: §1 · p. 1
• FunASR adds timestamp prediction and hotword customization to Paraformer while releasing FSMN-VAD and CT-Transformer for speech-recognition services.Source: §1 · p. 1
• The paper reports Mandarin experiments, while FunASR also supports English, French, German, Spanish, Russian, Japanese, and Korean.Source: §1 · p. 1

• ModelScope manages FunASR models, including Paraformer, FSMN-VAD, and CT-Transformer.Source: §2 · p. 2

Academic pipeline

• The academic run.sh pipeline covers data preparation, feature extraction, dictionary generation, training, inference, and scoring.Source: §2 · p. 2

Industrial pipeline

• The industrial pipeline separates inference through infer.sh from fine-tuning through finetune.sh.Source: §2 · p. 2

Deployment runtime

• FunASR supports deployment on CPU, GPU, Android, and iOS through Libtorch, ONNX, and TensorRT backends.Source: §2 · p. 2
• AMP quantization accelerates the inference runtime for application deployment.Source: §2 · p. 2

Paraformer

• Paraformer is a single-step non-autoregressive model whose predictor generates acoustic embeddings and whose training-time sampler incorporates target embeddings to capture token interdependence.The sampler is inactive during inference, when acoustic embeddings are used directly by the decoder.Source: §3.1 · p. 2

Timestamp Predictor

• Timestamp prediction redesigns the Paraformer predictor with transposed convolution and LSTM upsampling, then derives token durations and silence from CIF weights.FunASR also releases TP-Aligner, a smaller encoder-based force-alignment-like model that takes speech and transcription as input.Source: §3.2 · p. 2
• Paraformer-TP outperforms the Kaldi force-alignment system on AISHELL and remains within 10ms of the hybrid system on industrial data.The one-pass approach is reported to reduce computation and time overhead for commercial usage.Source: §3.2 · p. 2

Hotword Customization

• Contextual Paraformer adds a hotword embedder and decoder multi-head attention to customize recognition using named entities.The hotword embedding is combined with the decoder representation through attention and dimensionality reduction before subsequent processing.Source: §3.3 · p. 3
• Approximately 58% F1 improvement on the AISHELL-1 named entity subtest and an average 10% improvement on industrial customization tasks are reported for hotwords.The evaluation uses CER and F1-score, with test sets spanning AISHELL entity utterances and industrial AI and common domains.Source: §3.3 · p. 3

VAD and Text Postprocessing

• FSMN-VAD filters invalid voice to improve CER, while CT-Transformer performs real-time punctuation and disfluency removal with competitive F1 and faster inference.CT-Transformer uses controllable time delay, fast decoding, and history pruning; VAD evaluation covers meeting and video data.Source: §3.4 · p. 3, §3.5 · pp. 3–4

ASR evaluation

• Paraformer slightly outperforms Conformer without language-model decoding in recognition accuracy.The comparison removes Conformer’s language model and joint-CTC decoding for fairness.Source: §4.1 · p. 4

Runtime performance

• 12x faster inference gives Paraformer a substantial speed advantage over autoregressive models.The speedup is measured by GPU RTF with batch size 1.Source: §4.1 · p. 4

ASR evaluation

• 1.95%, 2.85%, and 6.97% CERs are reported on AISHELL, AISHELL-2, and WenetSpeech, respectively.Paraformer-large was trained on a 60,000-hour Mandarin industrial corpus.Source: §4.1 · p. 4

Fine-tuning

• 7.4% and 8.7% relative improvements are achieved on the AISHELL development and test tasks after fine-tuning.Fine-tuning on AISHELL and 200 hours of industrial-domain data improves both general and domain-specific recognition.Source: §4.1 · p. 4
• Domain-keyword recall increases from 76.7% to 96.8%, while CER decreases from 11.25% to 10.10% after logistics-domain fine-tuning.The results cover both short and long logistics test sets.Source: §4.1 · p. 4

Runtime performance

• 40% faster inference is achieved with AMP quantization without significant recognition degradation.The runtime evaluation uses Paraformer-large on an Intel Xeon CPU with one thread.Source: §4.2 · p. 4

Conclusions

• FunASR openly provides industrially trained speech recognition models and supports deployment in real-world applications.The toolkit includes Paraformer-large, FSMN-VAD, and CT-Transformer.Source: §5 · p. 4