Source-linked AI summary
Automatic estimation of verbal fluency index in people with Motor Neuron Disease using ASR alignment and pause modelling
Bahman Mirheidari, Leslie Ing, Daniel Blackburn, Sharon Abrahams, Christopher McDermott, Heidi Christensen
TL;DR
Cognitive impairment screening in MND is complicated by co-occurring speech difficulties, and the paper targets the gap in automatic ECAS VFI estimation for this population. It develops an ASR- and VAD-based pipeline for VFI estimation, with results supporting comparable estimation from P-words and S-words using the generation phase alone.
Problem
Method
Results
The study reports comparable VFI estimation performance for P-words and S-words, achievable using only the generation phase.
Source: §6 · p. 5Takeaways & Limitations
Automated VFI scoring could reduce clinician supervision and support scalable remote assessment for people with MND.
Source: §1 · p. 1Dysarthric populations exhibit elevated WERs, and the authors identify supplementary tasks as a future direction for these populations.
Source: §6 · p. 5
Abstract
from arXiv · showhide
Monitoring cognitive impairment (CI) in motor neuron disease (MND) is essential for timely treatment and care, yet challenging due to co-occurring speech difficulties. The Edinburgh Cognitive and Behavioural ALS Screen (ECAS) provides a robust metric for CI assessment, with the Verbal Fluency Index (VFI) a central element. Building on recent advances in automated speech analysis, this study proposes a system for estimating VFI. It leverages a unique MND dataset and combines ASR (WhisperX) and VAD (Silero) with refined timestamping to predict the VFI and extract several clinically interpretable measures. Our approach outperformed systems based on traditional acoustic features and self-supervised embeddings, evaluated using multiple regression algorithms. Clinically inspired features consistently outperformed the other sets, with the best models achieving strong results (P-words: R2 0.9, NRMSE 0.05; S-words: R2 0.8, NRMSE 0.08), demonstrating the feasibility of automated VFI estimation.
1. Introduction
Cognitive impairment assessment in MND is complicated by speech and motor deficits, and VFI is designed to reduce the effect of slowed speech on fluency scoring. This study proposes an automated VFI pipeline using speech recordings and clinician-scored data.
1. Introduction
- The ECAS uses an articulation-speed-adjusted VFI to distinguish cognitive retrieval speed from raw word counts, which can be confounded by slowed speech.Up to 50% of ALS patients develop CI, and 15% develop FTD.Source: §1 · p. 1
- Manual VFI calculation requires a one-minute word-generation test followed by clinician verification and timing of a controlled reading phase.The process is time-consuming, while routine cognitive assessment for people with MND is not widely available in the UK.Source: §1 · p. 1
- The study develops an ASR- and VAD-based pipeline to estimate VFI and produce timing, pause, and word-level information, using clinician-calculated ECAS scores as ground truth.The authors investigate whether generation-phase data alone may suffice for reliable VFI scores.Source: §1 · p. 1
2. Related work
Prior work spans acoustic analysis, automated fluency scoring, and advanced AI methods for dysarthria and disease progression. The paper identifies automatic ECAS VFI estimation in people with MND as an unaddressed target.
2. Related work
- Traditional acoustic features capture speech characteristics associated with ALS dysarthria and CI, while multimodal approaches combine acoustic, facial, and linguistic features.Prior work reports that speaking rate and pause frequency can be sensitive to disease progression, and combined modalities achieved optimal performance.Source: §2 · p. 2
- Earlier ASR-based fluency scoring achieved near-perfect agreement with manual scoring, while later work addressed limitations and improved temporal accuracy.Other cited work applies multi-task learning to dysarthria detection, transformers to disease-progression estimation [16], and multimodal biomarkers to bulbar-decline detection.Source: §2 · p. 2
- No prior study specifically targeted automatic ECAS VFI estimation in people with MND, despite VFI’s design for separating motor and cognitive impairment.The proposed pipeline prioritizes precise timing and word-level assessment and uses only the fluency-generation phase, avoiding controlled reading.Source: §2 · p. 2
3. Automatic VFI scoring pipeline
The pipeline combines ASR timestamps with VAD outputs to refine word and pause boundaries, then extracts clinical features and estimates VFI from the generation phase. This adjustment addresses ASR’s difficulty with isolated word production and pause segmentation.
3. Automatic VFI scoring pipeline
- ASR timestamps alone are insufficient because isolated fluency words and pauses between or after words are often misidentified or omitted.VAD outputs are combined with ASR timestamps to produce refined word and pause boundaries.Source: §3 · p. 2, Figure 1 · p. 2
- The four-stage pipeline transcribes words, adjusts timestamps, extracts clinical features from correct words and their gaps, and applies regression to estimate VFI.It operates on the verbal-fluency generation phase without the controlled reading phase.Source: Figure 1 · p. 2
- The alignment procedure matches ASR and VAD segments, but dysarthric speech can produce recognition errors and multiple overlapping regions that require a segment-selection strategy.The figure description gives “pod” for “part” and “parapem” for “pardon person” as examples.Source: §3 · p. 2
4. Experimental setup
The experimental setup uses an MND cohort and evaluates speech alignment, feature extraction, and regression approaches. It compares alternative recognition and VAD systems and includes acoustic, embedding, and clinically interpretable features.
4.1. Dataset
- The CognoMND dataset includes 42 enrolled participants with MND, of whom 20 scored below 4 on ALSFRS-R, indicating dysarthric speech.Figure 2 compares manual segmentation with ASR, VAD, and adjustment strategies for typical and dysarthric speech.Source: §4.1 · p. 2, Figure 2 · p. 3
4.2. Automatic Speech Recognition
- WhisperX produced more accurate word-level timestamps than Wav2Vec2, HuBERT, NeMo, and Whisper, largely through phoneme-level forced alignment.The study reports WhisperX results and measures ASR performance using WER.Source: §4.2 · p. 3
4.3. Voice Activity Detection
- The VAD evaluation tested Silero, FSMN, and CNN-BiLSTM and summed four types of speech/pause segmentation errors, normalized by audio duration.The error types are speech false alarms, speech misses, pause false alarms, and pause misses.Source: §4.3 · p. 3
4.4. Clinical feature extractor
- Feature extraction covered traditional acoustics, Wav2Vec2 and HuBERT embeddings, and clinical measures such as correct-word count, word duration, pause ratio, and disfluencies.SpaCy, GeoText, and NLTK processing supported word identification, repetition detection, and filler flagging.Source: §4.4 · p. 3, Table 2 · p. 3
4.5. Regression models
- The regression comparison included classical models and transformer- or CNN-based methods, with classical models consistently outperforming deep-learning counterparts.The authors attribute this to the small dataset; models used 5-fold cross-validation and were evaluated with NRMSE and R².Source: §4.5 · p. 3
5. Results
The ASR and segmentation evaluations identify task-specific recognition behavior and Silero as the best tested VAD system. The final clinical-feature models estimate VFI for both word sets, with a reported performance gap for dysarthric speakers.
5.1. ASR and adjustment performance
- Fine-tuning reduced P-word WER, whereas prompting worked better for S-words, likely reflecting training-data prevalence and acoustic differences.The fine-tuning used the CognoSpeak corpus (n > 400).Source: §5.1 · pp. 3–4
- WER was 21.5% for P-words and 23% for S-words; dysarthric speakers had higher errors, especially for S-words (typical: 15%; dysarthric: 24%).Removing fillers improved performance by 14% and 18.7% for P-words and S-words, respectively.Source: §5.1 · p. 4
- Silero had the lowest adjusted segmentation error rates among tested VAD systems: 6.9% for P-words and 11.8% for S-words.Table 4 reports weighted speech/pause segmentation errors.Source: Table 4 · p. 4, §5.1 · p. 4
5.2. Regression models evaluation
- NRMSE was 0.060 for typical versus 0.117 for dysarthric speakers, suggesting ASR word errors and segmentation inaccuracies disproportionately reduced S-word estimation accuracy for dysarthric speech.The scatter plots aggregate estimates across all five test folds.Source: Figure 3 · p. 4, §5.2 · p. 4
- P-words reached R² 0.856, NRMSE 0.053, while S-words reached R² 0.832, NRMSE 0.083; Clin-Man and Clin-Auto LASSO models did not differ significantly.Table 5 reports VFI estimation results for the top models; the comparison used 10× 5-fold cross-validation with different random seeds.Source: Table 5 · p. 4, §5.2 · p. 4
6. Conclusions
The study concludes that automated VFI estimation using ASR, VAD, and interpretable clinical features is feasible. P-words perform comparably to S-words, and estimation is achievable from the generation phase alone.
6. Conclusions
- An ASR- and VAD-based pipeline with interpretable clinical features produced accurate regression models for VFI estimation.The conclusion characterizes this as establishing feasibility.Source: §6 · p. 5
- The authors propose supplementary tasks, including paragraph reading and picture descriptions, particularly for dysarthric populations with elevated WERs.Source: §6 · p. 5
7. Generative AI Use Disclosure
The authors report that generative AI tools were used only for language editing, clarity refinement, and paragraph summarisation.
7. Generative AI Use Disclosure
- Generative AI use was limited to language editing, clarity refinement, and paragraph summarisation, without influence on the study's experiments or interpretation.The authors state that study design, data collection, experiments, analysis, and conclusions were conducted by the authors.Source: §7 · p. 6