Source-linked AI summary
agentic-ger: terminology recovery in long-form speech using global context
Yanqiao Zhu, Wupeng Wang, Zhifu Gao, Xiangang Li, Xie Chen
TL;DR
Long-form speech recognition still has difficulty transcribing domain-specific terminology accurately and consistently. Agentic-GER uses full-transcript context to find suspicious terms and check candidate edits against selectively re-transcribed speech, improving terminology recognition across the evaluated configurations, with a reported 36.8% relative B-CER reduction over Whisper on Chinese speech.
Problem
Domain-specific terms in long-form audio remain difficult to transcribe accurately and consistently because rare terms are susceptible to acoustic confusion and contextual dilution across segments.
Source: §1 · p. 1Method
Agentic-GER uses the full transcript to locate suspicious spans, selectively re-transcribes their audio, evaluates candidate edits using the resulting hypotheses and global context, and carries accepted edits into subsequent rounds.
Source: §1 · p. 2, §2.2 · p. 2, §2.3 · p. 2Results
Takeaways & Limitations
Substitution reductions contribute most to the terminology gains, while deletion improvements are smaller.
Source: §5 · p. 4The authors leave recovery of omitted terminology for future work, and note that omissions may lack textual cues for transcript-based discovery.
Source: §4.2 · p. 3, §5 · p. 4
Abstract
from arXiv · showhide
Recent advances in speech language models have improved automatic speech recognition (ASR) for long-form audio. However, accurately and consistently transcribing domain-specific terminology remains challenging. Motivated by the world knowledge and contextual capability of large language models (LLMs), we propose Agentic-GER, an LLM-based agent for terminology correction in long-form speech. The agent uses global context from the full transcript to identify suspicious terms and resolve ambiguous hypotheses. It selectively re-transcribes the source speech to check candidate corrections, and uses accepted edits to guide subsequent decisions. Experiments with four LLMs and two ASR systems on GigaSpeechBench show consistent terminology improvements in both Chinese and English, with and without thinking. On Chinese speech, Agentic-GER achieves up to a 36.8% relative reduction in biased character error rate (B-CER) over the Whisper baseline.
1. INTRODUCTION
Domain-specific terms remain difficult to transcribe consistently in long-form audio, and correction must connect error discovery with evidence from selected speech segments. Agentic-GER uses the full transcript to guide those checks and improves terminology recognition across the reported configurations.
1. INTRODUCTION
- Long-form ASR struggles to transcribe domain-specific terminology accurately and consistently because rare terms face acoustic confusion and contextual dilution across segments.These factors can produce inconsistent hypotheses for the same audio.Source: §1 · p. 1
- Earlier generative error-correction approaches use textual evidence, source speech, retrieved entities, or contextual memory to revise ASR hypotheses without modifying the ASR model.The paper frames long-form correction as requiring a link between error discovery and correction.Source: §1 · p. 1
- The agent scans the full transcript for suspicious spans, re-transcribes selected audio, evaluates candidate edits using the hypotheses and global context, and carries accepted edits into later rounds.Recovered terminology can inform subsequent decisions.Source: §1 · p. 2
- Agentic-GER lowers terminology error rates in all 32 configurations, with a 36.8% relative B-CER reduction over Whisper on Chinese speech.The evaluation spans four LLMs, two languages, two initial ASR systems, and settings with and without thinking.Source: §1 · p. 2
2. METHOD
Agentic-GER uses transcript-wide context to find suspicious terms, checks selected audio through re-transcription, and iteratively applies supported edits while recording decisions.
2. METHOD
- Agentic-GER takes audio and a timestamped transcript, finds suspicious terms using the full transcript, and checks them by re-transcribing fixed-boundary segments before editing.The topic, domain, and observed term forms extracted from the initial transcript provide fixed context for later decisions.Source: §2 · p. 2
2.1. Error Discovery via Global Context
- The agent proposes suspicious spans using the topic summary, current transcript, and prior decisions, guided by cues such as inconsistent entity spellings or unlikely technical contexts.It proposes up to k spans per round, each with a segment, substring, and inspection reason.Source: §2.1 · p. 2
2.2. Re-transcription and Evaluation
- For each proposed segment, the agent obtains a second hypothesis and makes a minimal keep-or-edit decision using pronunciation compatibility, re-transcription evidence, and global context.In the illustrated case, an earlier “stethoscope” mention helps resolve a re-transcription of “deathscapes” to “stethoscopes.”Source: §2.2 · p. 2, Figure 2 · p. 2
2.3. Iterative Correction
- After evaluating candidates against the same transcript, the agent applies accepted changes, records edits and keep decisions in working memory, and repeats until a stopping limit is reached.The updated transcript and recorded decisions inform subsequent rounds.Source: §2.3 · p. 2
3. EXPERIMENTAL SETUP
The experiments use GigaSpeechBench audio in Chinese and English, compare two initial ASR systems and four LLM correctors, and score terminology errors alongside overall transcription accuracy.
3. EXPERIMENTAL SETUP
- The evaluation uses 524 Chinese and 387 English GigaSpeechBench audio files, totaling approximately 120 hours per language across 12 domains.Initial transcripts come from FunASR-Realtime and Whisper-Large-v3.Source: §3 · p. 3
- Four LLMs are evaluated with and without thinking, yielding 32 configurations, and all use Qwen3-ASR-1.7B for segment re-transcription without task-specific fine-tuning.The correctors include two dense models and two mixture-of-experts models.Source: §3 · p. 3
- Terminology accuracy is measured with B-CER for Chinese and B-WER for English, while overall CER and WER assess effects on full-transcript accuracy.Terminology scores count substitutions, deletions, and insertions associated with annotated terms and are macroaveraged over 12 domains.Source: §3 · p. 3
4. RESULTS AND ANALYSIS
Agentic-GER improves terminology recognition across evaluated settings, with larger gains on Chinese; global-context iteration reduces terminology and overall error where local GER raises overall CER. Thinking effects vary by model and increase correction cost.
4.1. Recovering Domain Terminology
- For Whisper on Chinese speech, overall CER falls 8.7–18.3% relative to baseline; across other configurations, overall CER/WER changes range from a 3.4% decrease to a 0.5% increase.Source: §4.1 · p. 3
- 36.8% relative B-CER reduction: Max lowers Whisper’s Chinese B-CER from 35.07% to 22.16%, with terminology gains across all evaluated LLMs and both ASR systems.Max performs best in all four ASR/language combinations without thinking.Source: Table 1 · p. 3, §4.1 · p. 3
4.2. Error-Type Analysis
- Substitution errors fall more than deletions in both languages; for Whisper, reductions are 21.3% versus 3.0% on Chinese and 6.4% versus 0.4% on English.Substitution reductions contribute most to gains across all 32 configurations.Source: Table 2 · p. 3, §4.2 · p. 3
- Omitted terms can leave no textual cue for inspection; deletions account for 43.1% of Whisper’s English terminology errors versus 14.7% in Chinese, a pattern hypothesized to contribute to smaller English gains.Source: §4.2 · pp. 3–4
4.3. The Value of Global Context
- With thinking, local GER reduces B-CER by 2.57% but increases overall CER, whereas Agentic-GER reduces B-CER by 17.27% and lowers both metrics.Local GER uses each segment’s original text and ASR hypothesis without global context or iterative updates.Source: Table 3 · p. 4, §4.3 · p. 4
4.4. Thinking for Terminology Recovery
- Thinking improves Qwen3.8-27B and Gemma4-31B across all four ASR/language combinations, but Max regresses in three; on Whisper-Chinese, it changes relative B-CER reduction by +8.5 points for Qwen3.8-27B and −9.7 for Max.Source: Table 4 · p. 4, §4.4 · p. 4
- For Gemma4-31B on Whisper-Chinese, thinking raises correction RTF from 0.42 to 5.71 and output-token use from 2.6 million to 38.3 million.Input-token use rises from 161 million to 198 million.Source: §4.4 · p. 4
5. CONCLUSION
Agentic-GER uses global context and accepted edits to guide terminology correction in long-form speech. Experiments show improvements across 32 configurations, while future work targets omitted terminology and expanded tools.
5. CONCLUSION
- Across all 32 configurations, terminology improves; substitutions contribute most, deletions improve less, and thinking benefits 27B/31B models but yields limited or negative gains for Flash and Max.Experiments were conducted on GigaSpeechBench.Source: §5 · p. 4
- The full transcript helps locate suspicious spans and assess candidate replacements alongside ASR re-transcriptions; accepted edits inform subsequent decisions.Source: §5 · p. 4
- Future work will expand the tool set, investigate omitted-terminology recovery, and evaluate the framework on additional settings.Proposed tools include web search for unfamiliar terms and end-to-end multimodal models for assessing candidate corrections.Source: §5 · p. 4