PaperlayerView exact revision on arXiv

Source-linked AI summary

Choosing a PEFT Variant for Per-Patient Dysarthric ASR: A Single-Speaker Case Study on Two ASR Bases

Bernard Muller, László Tóth, LaVonne Roberts

arXiv:2609.02735v1cs.CLcs.SD

TL;DR

Per-patient dysarthric ASR lacks systematic evidence on which PEFT variant to use in the speaker-dependent setting. This study compares seven LoRA-family methods on two production ASR bases for one Hungarian speaker and finds that simpler LoRA matches DoRA while real 4-bit QLoRA performs worse. The results support LoRA as the production lead within the tested single-speaker regime.

  • Problem

    PEFT variants have not been systematically compared for speaker-dependent, per-patient dysarthric ASR, despite per-patient adapters being a production architecture.

  • Method

    The study compares seven LoRA-family variants on Whisper-large-v3 and Qwen3-ASR-1.7B using one post-stroke Hungarian speaker under a fixed per-patient recipe.

  • Results

    Across three seeds, LoRA and DoRA show no significant difference, while real 4-bit NF4 QLoRA is worse on every seed and saves no peak VRAM at this scale.

  • Takeaways & Limitations

    LoRA is adopted as the simpler, cheaper per-patient production lead, with DoRA retained as an alternative with no detected accuracy difference.

  • Takeaways & Limitations

    The conclusions are limited by evaluation on one speaker and one language, and by untested deeper adapter targets or higher ranks on some backbones.

Abstract

from arXiv · show

Per-patient adapters are the preferred production architecture for dysarthric automatic speech recognition (ASR), yet parameter-efficient fine-tuning (PEFT) variants have not been compared in the speaker-dependent, per-patient regime. We present a single-speaker case study comparing seven LoRA-family methods (LoRA, QLoRA, AdaLoRA, DoRA, LoHA, VeRA, VB-LoRA) on two production bases (Whisper-large-v3 with Hungarian fine-tuning, and a multilingual Qwen3-ASR-1.7B checkpoint) for one post-stroke Hungarian male speaker (S1, 409 utterances; severe dysarthria on auditory-perceptual clinical assessment). Attention-projection adapters substantially improve CER on both bases. Across three seeds, a paired bootstrap detects no significant LoRA-DoRA difference (p>0.5; 13.86/13.90 % CER on Whisper, 28.10/28.33 % on Qwen3-ASR), so we adopt the simpler, cheaper LoRA. Real 4-bit (NF4) QLoRA is worse on every seed and both bases (14.56/30.09 % CER) with no memory saving at this scale, and LoHA, VeRA, VB-LoRA and AdaLoRA do not reach the LoRA family, though LoHA still gives an 18.6 % relative CER reduction on Whisper. On the same base, full fine-tuning is more accurate (11.43 % CER), but a 115 MB LoRA that also adapts the feed-forward blocks reaches within 0.66 pp of it at approximately 3.7 % of the per-patient storage. A 6-point enrollment grid shows about 5 min of patient audio captures 45.6 % of the zero-shot-to-30-min CER reduction, with further gains at 10 and 30 min (caveat: one speaker, one language, severe post-stroke dysarthria). Training scripts and recipes will be released, source-available under a research-use licence, on publication.

1. Introduction

This paper addresses the unstudied choice of PEFT variant for speaker-dependent, per-patient dysarthric ASR. It compares seven LoRA-family methods across two production ASR architectures under a fixed per-patient setting.

  • Motivation: Per-patient adapters keep the shared ASR base untouched while storing and shipping a small patient-specific model.Storage and training costs are incurred separately for each patient.
  • Research gap: The study targets a gap: PEFT variants have not been systematically compared for speaker-dependent dysarthric speech adaptation.Prior work examined different adaptation questions, including speaker-disjoint evaluation, adapter fusion, and children’s ASR.
  • Study scope: The paper compares LoRA, QLoRA, AdaLoRA, DoRA, LoHA, VeRA, and VB-LoRA on Whisper-large-v3 and Qwen3-ASR-1.7B using the same speaker, data, and recipe.This is presented as the first speaker-dependent, per-patient comparison of these seven variants on dysarthric speech.
  • Study scope: Attention adapters are evaluated alongside feed-forward adaptation, with the latter reaching within 0.66 pp CER of full fine-tuning at approximately 3.7 % of per-patient storage.The comparison identifies encoder attention as the dominant LoRA lever.
  • Related alternatives: The paper also examines alternative adaptation mechanisms, including FiLM, but defers a proper multi-patient comparison of storage, CER, and multi-task retention.The FiLM comparison is not treated as another configuration in the controlled variant study.

3. Method

The study uses one post-stroke Hungarian speaker and evaluates adapter variants on two ASR bases with fixed-budget, reproducible training and speaker-internal data splits. The method fixes the training recipe while varying the PEFT configuration.

  • Data: The corpus contains 409 utterances from one post-stroke Hungarian male speaker, with severe dysarthria described by auditory-perceptual clinical assessment.The audio cannot be redistributed under the data-sharing agreement.
  • Data: The speaker-internal split contains 262 training utterances, 40 validation utterances, and 107 evaluation utterances.The training pool includes 195 read-sentence and 67 narrative utterances.
  • ASR bases: The bases are a Hungarian-fine-tuned Whisper-large-v3 checkpoint and an internal multilingual Qwen3-ASR-1.7B production checkpoint.Zero-shot CER on the S1 test split is 29.46 % for Whisper and 49.46 % for Qwen3-ASR.
  • Training protocol: All configurations use fixed-budget AdamW training for five epochs, with the final-epoch model evaluated and no early stopping or checkpoint selection.The principal varied factor is the PEFT variant.
  • Data: The evaluation references are text-disjoint from training and validation, and the speaker’s data were excluded from the Hungarian warm-base pool and the Qwen3-ASR checkpoint.The same evaluation set was used across the seven-variant screening and subsequent comparisons.
  • Adapter setup: Adapters target attention projections across the models, while other weights remain frozen according to the specified configuration.The method uses matched module names across the models and fixed decoding and normalization procedures.
  • Reproducibility: Training scripts, per-variant configurations, and evaluation runners are released source-available, but the corpus, trained adapters, and per-utterance outputs remain restricted.Results on the internal Qwen3-ASR checkpoint are reproducible in recipe but not from publicly available files alone.

4. Results

Across two production ASR bases, the LoRA family was the only group that adapted usefully, with LoRA and DoRA tied after reseeding while real 4-bit QLoRA consistently lagged. Additional analyses show Qwen3 Hungarian fine-tuning can regress dysarthric recognition, whereas expanding LoRA targets approaches full fine-tuning accuracy with much less storage.

  • Cross-base variant ranking: LoRA, DoRA, and QLoRA were the only variants that adapted usefully on either production base.The remaining variants, including AdaLoRA, VeRA, and VB-LoRA, under-performed on both bases.
  • Multi-seed head-to-head: No significant LoRA–DoRA difference was detected across three seeds on either base, so LoRA was adopted for simplicity and cost.LoRA used half the Whisper training time of DoRA and had no magnitude-vector overhead.
  • Multi-seed head-to-head: Real 4-bit NF4 QLoRA lost to LoRA on every seed and both bases, with no peak-VRAM saving at this model scale.The reported penalties were +0.69 pp CER on Whisper and +1.99 pp on Qwen3.
  • NeMo-backbone negative result: Both tested NeMo backbones regressed rather than improved under simple LoRA recipes.The authors restrict this negative result to the tested recipes and do not claim the failure is structural.
  • Picking the warm base: Qwen3 Hungarian fine-tuning preserved clean-Hungarian performance while worsening dysarthric CER, reaching a +28.75 pp increase at the 75 % checkpoint.The degradation was attributed to a narrow 47-speaker command-style Parkinson’s corpus that did not transfer to S1’s post-stroke read sentences.
  • Same-base comparison: 11.43 % CER was achieved by full fine-tuning, versus 13.86 % for LoRA, while full fine-tuning required a 3.1 GB per-patient checkpoint instead of a 63 MB adapter.Encoder-frozen decoder fine-tuning performed substantially worse at 22.96 % CER.
  • Adaptation-surface ladder: 12.09 % CER was reached by adding feed-forward projections to LoRA, within 0.66 pp of full fine-tuning at 115 MB.This configuration used 28.8 M parameters, roughly 3.7 % of full-model storage.

5. Discussion

The discussion favors LoRA for speaker-dependent per-patient dysarthric ASR, while showing that variant performance, warm-base choice, and enrollment requirements remain regime- and scope-sensitive.

  • 18.6 % relative CER reduction makes LoHA the closest alternative on Whisper, but AdaLoRA, VeRA and VB-LoRA remain near zero-shot performance.The discussion attributes the broader gap to the roughly 80-step per-patient data budget and method-specific convergence demands.
  • LoRA and DoRA are statistically tied, while real 4-bit QLoRA is worse on every seed and saves no peak VRAM at this model scale.The production template therefore defaults to LoRA, pending multi-speaker validation.
  • +23.55 pp regression on Qwen3 contrasts with a 29.46 % Hungarian-fine-tuned Whisper zero-shot CER versus ∼32.24 % for raw Whisper v3.The authors therefore state that warm-base selection must be made per base.
  • 45.6 % of the zero-shot-to-30-minute CER reduction is captured by a 3–5 min enrollment session, with further gains at 10 and 30 min.The reported CERs are ∼22–23 % at 3–5 min, 18.87 % at 10 min, and 14.17 % at 30 min.
  • The study’s conclusions are constrained to one speaker, one language, severe dysarthria, a fixed five-epoch budget, and greedy decoding.Multi-patient generalisation and the FiLM-versus-LoRA fleet-Pareto question are deferred.

6. Practical recipe and release

The release provides variant-specific training recipes and per-patient adapter deployment tools, while restricting patient-derived data and artifacts under a research-use framework.

  • The repository bundles separate training recipes for each PEFT variant, with a drop-in plain-LoRA replacement and an enrollment-grid runner.It also includes the locked PEFT-variant Pareto chart and summary table.
  • The patient-side runtime loads a shared base plus a small per-patient adapter, while an adapter manager switches among many patient adapters at inference time.The base weights are cached once and shared across patients; only the low-rank adapter is stored per patient.
  • The scripts, recipes, configurations, plotting tools, and enrollment runner will be source-available on publication under a research-use licence.The licence restricts field of use, so the release is described as source-available rather than open-source.
  • The S1 corpus, trained adapter weights, and per-utterance result JSON files are not openly redistributed because the data-sharing agreement requires the participant’s written consent.Aggregate summary tables, Pareto coordinates, and the paper’s main results are public.
  • LoRA r=16 is the default production template on both Whisper-large-v3 and the Qwen3-ASR production checkpoint.The choice reflects LoRA and DoRA being statistically tied, with LoRA selected as simpler and cheaper; 4-bit QLoRA is not recommended.

7. Conclusion and future work

The study compares seven LoRA-family variants across two production ASR bases in a single-speaker, per-patient dysarthric-ASR case study. It favors simpler LoRA after finding no significant LoRA–DoRA difference, poorer QLoRA results, and substantial scope limits requiring multi-patient validation.

  • Conclusion: 13.86 % / 13.90 % CER on Whisper and 28.10 % / 28.33 % on Qwen3-ASR showed no significant LoRA–DoRA difference across three seeds.The paired bootstrap gave p > 0.5 on both bases.
  • Conclusion: 14.56 % / 30.09 % CER made true 4-bit NF4 QLoRA worse on every seed, with no memory saving at this scale.The authors therefore prefer simpler, cheaper LoRA for production, while retaining DoRA as an alternative.
  • Conclusion: LoHA, AdaLoRA, VeRA, and VB-LoRA did not reach the LoRA family within the approximately 80-optimizer-step budget, although LoHA reduced Whisper CER by 18.6 % relatively.The comparison used seven LoRA-family variants across two production ASR bases.
  • Future work: Approximately 5 minutes of patient audio captured 45.6 % of the zero-shot-to-30-minute CER reduction, with substantial further gains through 30 minutes.The enrollment-time grid was evaluated on a single severe post-stroke speaker.
  • Future work: Fleet-level recommendations remain limited by the study’s single-speaker scope and await multi-patient validation across aetiologies.Future work also proposes deeper LoRA targets, higher-rank LoHA or DoRA, and broader storage-quality-retention evaluation.

Ethics and data statement

The study analyzes one participant’s self-recorded voice data, contributed under written informed consent and a research-use-only data-sharing agreement. The participant is pseudonymous rather than anonymous because the paper discloses reuse in related work.

  • Consent and data use: S1 recorded and contributed his own voice under written informed consent and a signed Data Sharing Agreement for research use only.The agreement prohibits redistribution of the audio and its derivatives.
  • Ethics: Because the study involved one participant’s self-recorded speech without clinical intervention, no ethics-committee approval was required or sought.The statement specifies that there were no other human participants.
  • De-identification: The participant is pseudonymous rather than anonymous because disclosure of identity overlap with related work could permit a determined reader to infer identity.Accordingly, the authors do not describe the data as anonymised.

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

Generative AI was used as an accessibility tool throughout the research process, while the authors retained responsibility for scientific decisions, interpretations, and conclusions.

  • AI-assisted research: Claude Opus 5 supported experimental design, code development, data analysis, statistical computation, figure generation, and manuscript drafting.The first author communicates exclusively through an AI-assisted eye-gaze interface due to motor neurone disease.
  • Author responsibility: The authors made all scientific decisions, interpretations, and conclusions despite using generative AI throughout the research process.The statement distinguishes assistive use from scientific responsibility.

CRediT authorship contribution statement

The contribution statement assigns conceptualization, methodology, software, analysis, investigation, visualization, drafting, resources, data curation, validation, supervision, and project administration across three authors.

  • Contributions: Bernard Muller contributed conceptualization, methodology, software, formal analysis, investigation, visualization, and the original draft.
  • Contributions: László Tóth contributed resources, data curation, validation, and review and editing.
  • Contributions: LaVonne Roberts contributed project administration, supervision, and review and editing.

Declaration of competing interest

The authors declare no known competing financial interests or personal relationships that could have influenced the work. The Scott-Morgan Foundation develops the described system and retains commercial rights to released code, while authors receive no personal income from it.

  • The authors declare no known competing financial interests or personal relationships that could have influenced the work.
  • The Scott-Morgan Foundation develops the per-patient dysarthric-ASR system and reserves commercial rights on released code under a research-use licence.No author receives personal income from the code.

Data availability

The S1 corpus, trained adapters, and per-utterance outputs are restricted and are not openly redistributed. Bona fide researchers may request access subject to consent and a research-use agreement, while training materials will be released under a research-use licence.

  • The S1 corpus, trained adapters, and per-utterance outputs are not openly redistributed because they are restricted under a signed Data Sharing Agreement.
  • Bona fide researchers can request access subject to the data subject’s written consent and a research-use agreement mirroring the original.
  • Training scripts, per-variant configurations, and Pareto and enrollment runners will be released source-available under a research-use licence on publication.

Funding

The work was conducted as part of The Scott-Morgan Foundation’s research programme and received no specific grant from public, commercial, or not-for-profit funding agencies.

  • The work received no specific grant from public, commercial, or not-for-profit funding agencies.
Loading 2609.02735v1…