Training-Free Contextual ASR via SpeechLLM-Based Error-Aware Selective Retrieval

1. INTRODUCTION

• A large terminology dictionary can introduce irrelevant candidates, increasing computational cost and potentially causing over-biasing.Prior selection approaches include named-entity, phonetic, learned, ranking, relevance-estimation, and localized-alignment strategies, some requiring separately trained components.Source: §1 · p. 1
• Contextual biasing can improve recognition of domain-specific terms, but directly supplying many candidates risks burdening recognition with irrelevant biasing terms.SpeechLLM approaches have also incorporated pronunciation-aware context and bias-word information.Source: §1 · p. 1
• Prior ASR error detection has guided correction or re-decoding, but has not primarily been used to decide whether an external terminology dictionary should be queried.The paper identifies a need for training-free selection that targets regions likely to contain both domain terms and recognition errors.Source: §1 · p. 1
• A pretrained SpeechLLM jointly generates an ASR hypothesis and localizes likely domain-term error spans, which gate phonologically similar terminology retrieval before the same model re-recognizes the audio.The approach requires no task-specific model training.Source: §1 · pp. 1–2
• Across medical, air traffic control, and financial speech, the method substantially reduced dictionary queries while improving relevant-candidate recall and ranking and second-pass ASR performance.The framework's cross-domain evaluation supports the reported improvements across all three domains.Source: §1 · p. 2

2. PROPOSED METHOD

• The framework uses localized error spans to gate terminology retrieval, then re-recognizes the audio using the first-pass hypothesis and retrieved terms with the same SpeechLLM.No task-specific parameter updates are made.Source: §2 · p. 2

2.1. Joint ASR and Domain-Term Error Localization

• The SpeechLLM jointly generates a first-pass hypothesis and domain-term error spans, using audio-text in-context demonstrations to guide localization.Spans are normalized, limited to at most two words, and numeric-only spans are discarded; demonstrations cover error-free and three error-containing cases.Source: §2.1 · p. 2

2.2. Terminology Retrieval from Localized Spans

• Each localized span independently queries the external dictionary for phonologically similar terms using Acoustic Neighbor Embeddings.Terms are ranked across localized spans, and retrieval is skipped when no span is localized.Source: §2.2 · p. 2

2.3. Second-Pass Contextual Re-recognition

• For second-pass re-recognition, the same SpeechLLM receives the audio, first-pass hypothesis, and retrieved terms as contextual biasing information.The hypothesis supplies textual context while the retrieved terms provide contextual biasing.Source: §2.3 · p. 2

3.1. Experimental Setup

• Evaluation covered MedSyn, ATCOSIM, and Earnings; MedSyn used a 7,906-utterance sample, and dictionaries retained rare target-domain unigrams while excluding number words, short words, and non-alphabetic tokens.Dictionaries were fixed across evaluation utterances, with no utterance-level reference information used during retrieval; the SpeechLLM and ANE embedder were used without parameter updates or additional training.Source: §3.1 · p. 3

3.2. Domain-Term Error Localization

• Localization baselines included all ASR 1-grams, random spans, named-entity extraction, and confidence spans; SpeechLLM inputs varied across ASR-only, audio-only, combined, and joint generation configurations.Localization was assessed with span count, Error Precision/Recall, General Recall, and Domain Recall.Source: §3.2 · p. 3
• Joint ICL selected 0.99 spans per utterance with 51.9% Error Precision, 76.1% Error Recall, and 86.0% Domain Recall, versus 13.37 spans, 8.1%, and 99.1% for All ASR 1-grams.Its Domain Recall exceeded its General Recall of 47.6%.Source: §3.2 · p. 3

3.3. Terminology Retrieval and Contextual ASR

• On MedSyn, Joint ICL cut queries by 92.6% versus All ASR 1-grams while raising R@10 from 53.2% to 61.8% and MRR@10 from 31.9% to 47.9%.Random spans used the same query count but achieved 6.1% R@10.Source: Table 5 · p. 4, §3.3 · p. 4
• With top-10 retrieved candidates on MedSyn, Joint ICL reduced WER from 7.30% to 5.67% and B-WER from 41.43% to 28.05%, while U-WER fell from 2.80% to 2.72%.Top-10 outperformed top-50; the authors suggest lower-ranked candidates can introduce less relevant biasing terms.Source: Table 6 · p. 4, §3.3 · p. 4

3.2. Domain-Term Error Localization

• Domain Recall was 22.8% with Audio only, 66.4% with ASR only, and 84.4% with Audio + ASR; audio-text ICL raised Error Precision from 46.7% to 51.9% with similar recall.Joint ICL and Baseline ASR first-pass WERs were 7.30% and 7.30% on MedSyn.Source: §3.2 · p. 4

3.4. Cross-Domain Validation

• Across ATCOSIM and Earnings, Joint ICL reduced queries and led R@10/MRR@10; second-pass WER and B-WER were lowest, with B-WER falling from 46.40% to 30.65% and 50.93% to 45.66% versus Baseline ASR.On Earnings, queries fell from 32.71 to 1.38 per utterance while R@10 rose from 31.8% to 35.8%.Source: Table 7 · p. 4, §3.4 · p. 4
• Joint ICL first-pass WERs were 14.91% on ATCOSIM and 17.97% on Earnings, compared with Baseline ASR at 14.81% and 17.97%, respectively.Together with MedSyn results, the authors report that joint localization does not substantially degrade first-pass transcription accuracy across domains.Source: §3.4 · p. 4

• Across domains, the framework substantially reduces dictionary queries while improving relevant terminology-candidate recall and ranking and second-pass ASR performance.Source: §4 · p. 4
• SpeechLLM-based domain-term error localization selectively triggers terminology retrieval in the training-free framework.Source: §4 · p. 4
• The same SpeechLLM performs second-pass re-recognition using the first-pass ASR hypothesis and retrieved terms.Source: §4 · p. 4