Source-linked AI summary
InterCorrect: Intersection-Aware Correction of Demographic Model Merging for Fair ASR
Ashley E. Bravo-Bravo, Yuchen Zhang, Haralambos Mouratidis, Ravi Shekhar, Monorama Swain
TL;DR
ASR disparities can be especially difficult to address when speakers belong to multiple demographic groups, and parameter-level merging has been underexplored for fairness-oriented ASR. This paper merges subgroup-adapted connectors and transfers corrections for selected demographic intersections to a global model. On Fair-Speech, demographic merging improves performance, and WER-guided TIES achieves the best overall WER, though lower average WER does not always mean lower subgroup disparity.
Problem
ASR disparities can limit reliability for underrepresented speakers, and subgroup improvements may not transfer to people at demographic intersections; fairness-oriented ASR research had largely overlooked parameter-level merging.
Source: §1 · p. 1Method
The approach merges subgroup-adapted connectors into a global model, then applies correction vectors for cross-axis demographic pairs selected using subgroup WER or task-vector conflict.
Source: §1 · p. 1, §2.2 · p. 2Results
Takeaways & Limitations
Most settings reduce subgroup disparity across demographic axes, but TIES with Method 1 can increase the WER gap for age and socioeconomic background despite improving average performance.
Source: §3.4 · p. 4Intersection correction does not uniformly reduce within-axis disparity: TIES with Method 1 increases the WER gap for age and socioeconomic background.
Source: §3.4 · p. 4
Abstract
from arXiv · showhide
Automatic Speech Recognition (ASR) systems often show uneven performance across demographic groups, and errors can be especially difficult to address for speakers belonging to multiple demographic groups. This work studies demographic-aware model merging for fair Speech-LLM-based ASR. Starting from a SLAM-ASR-based model, we fine-tune only the connector on demographic-specific subsets and merge the resulting subgroup-adapted connectors into a global model. We then identify critical cross-axis demographic pairs using subgroup WER and task-vector conflict, and apply intersection-specific correction vectors to the global merged model. Experiments on Fair-Speech show that global demographic merging improves overall WER over the base model, while intersection correction provides additional gains for several merging strategies. In particular, TIES with WER-based correction achieves the best overall WER, reducing it from 7.38\% to 5.13\%. Subgroup and disparity analyses further show that the proposed approach improves performance across demographic axes, while highlighting that lower average WER does not always imply reduced subgroup disparity.
1. INTRODUCTION
ASR disparities can limit reliability for underrepresented speakers, and improvements for individual demographic groups may not transfer to speakers at demographic intersections. This study asks whether parameter-level model merging can mitigate demographic and intersectional disparities in Speech-LLM-based ASR.
1. INTRODUCTION
- Because demographic attributes can overlap, gains for individual groups may not transfer to speakers belonging to multiple groups.The paper tests whether intersection-specific correction can address cases subgroup-level adaptation alone may miss.Source: §1 · p. 1
- Prior ASR merging work focused on robustness and task adaptation, while fairness-oriented ASR research had largely overlooked parameter-level merging for demographic bias mitigation.Existing demographic disparity strategies include balancing, augmentation, fine-tuning, contrastive learning, and adapter fusion, but may require additional data, specialized losses, explicit balancing, or adaptation modules.Source: §1 · p. 1
- 7.38% to 5.13%: WER-guided TIES intersection correction reduces overall WER by 30.5% relative to the base model while improving targeted intersections.The study combines demographic-specific connector fine-tuning and merging with correction vectors for selected cross-axis demographic pairs.Source: §1 · p. 1
2. METHODOLOGY
The method trains a base Speech-LLM-based ASR model, merges connectors adapted to demographic subgroups, then transfers corrections estimated for selected cross-axis intersections to the global model.
2.1. Base ASR Model Training
- The base SLAM-ASR model connects a speech encoder to an LLM through a linear connector, while keeping the encoder and LLM frozen and training only the connector.The encoder is Whisper-largev3 and the LLM is TinyLlama 1.1B.Source: §2.1 · p. 2
2.2. Global Demographic Model Merging
- The global demographic model merges connector adaptations fine-tuned on demographic subgroup subsets using linear, Task Arithmetic, TIES, Breadcrumbs, or DARE-TIES strategies.Linear merging averages subgroup-adapted connector parameters; the other strategies combine updates relative to the base model, with TIES resolving update interference through trimming, sign election, and sign-consistent merging.Source: §2.2 · p. 2
2.3. Intersection-Corrected Model
- Intersection correction selects cross-axis pairs by poor subgroup WER or conflicting updates, fine-tunes pairwise merged models on intersection data, and transfers their resulting correction vectors to the global model.The three variants use WER-selected pairs, conflict-selected pairs, or both.Source: §2.3 · p. 2
3. EXPERIMENTS AND DISCUSSION
Across five demographic axes, global merging improves overall and subgroup WER, while intersection correction adds gains for several strategies. Disparity and intersection analyses show that these gains vary across groups and correction variants.
3.1. Experimental Setting
- The evaluation uses Fair-Speech, with 18 demographic subgroups across five axes and stratified 70/15/15 training, validation, and test splits.All adaptations reuse training subsets, and pairs and hyperparameters are selected on validation data.Source: §3.1 · p. 3
3.2. Overall Results
- 5.13% is the best overall WER, achieved by TIES with Method 1; global merging reduces WER from 7.38% to 6.38%–6.77% across strategies.Intersection correction adds gains over global merging, but the magnitude varies; stronger subgroup-update conflicts are associated with greater correction effectiveness.Source: Table 1 · p. 3, §3.2 · p. 3
3.3. Subgroup Performance Analysis
- Global merging improves average subgroup WER across all five demographic axes; TIES+M1 and Breadcrumbs+M3 also lower WER on several individual axes.TIES+M1 reduces age WER from 6.27% to 4.71%, gender from 6.62% to 5.26%, and ethnicity from 6.08% to 4.77%; Breadcrumbs+M3 lowers age and SES WER.Source: §3.3 · p. 3, Table 2 · p. 3
3.4. Demographic Disparity Analysis
- Most settings reduce max–min WER disparity, but TIES with Method 1 increases the age and socioeconomic-background gaps.Positive relative gap reduction means a smaller subgroup gap than the base model; negative values mean the gap increases.Source: §3.4 · pp. 3–4, Figure 1 · p. 4
3.5. Intersection Analysis
- Corrections yield larger gains on selected than non-selected intersections, and most variants reduce WER on selected groups.On non-selected intersections, TIES with Method 1 and Method 3 reach 87.91% and 88.67%, respectively; Model Breadcrumbs with Method 3 reaches 84.96%.Source: §3.5 · p. 4, Figure 2 · p. 4
4. CONCLUSION
Demographic model merging improves overall and demographic-level ASR performance, while intersection correction provides further gains for several strategies.
4. CONCLUSION
- TIES with WER-based correction achieves the best overall WER.Intersection-aware corrections use subgroup WER and task-vector conflict; the analyses show targeted intersectional improvements while largely preserving non-targeted performance.Source: §4 · p. 4
- Demographic model merging improves overall and demographic-level ASR performance.Source: §4 · p. 4
- Intersection correction provides further gains for several merging strategies using the same training data.Source: §4 · p. 4
6. COMPLIANCE WITH ETHICAL STANDARDS
The research used human subject data from the open-access Fair-Speech Dataset released by Meta AI.