InterBias-SV: Compound Conditions in Speaker Verification

• A joint-condition error can be compared with an additive prediction only when its marginal and common-reference terms use comparable sampling and scoring.Source: §1 · p. 1
• The audit identifies three families with same-corpus terms, but code and metadata do not establish all matching and provenance requirements.Source: §1 · p. 1
• The benchmark organizes compound-condition evaluation around 17 experiments, 4,068 scored records, and scripts that reconstruct tabulated summaries.Source: §1 · p. 2
• Small positive mean contrasts coexist with large setting-level deviations, while near-chance performance and small reference gaps constrain interpretation.Source: §1 · p. 2
• InterBias-SV contributes an evaluation framework and empirical analysis, not a new encoder or newly collected speech dataset.Its usefulness depends on making measurements and their limits reproducible.Source: §1 · p. 2

• SVeritas includes compound stressors, while InterBias-SV focuses on comparing joint error with an additive prediction from its reference terms (Baali et al., 2025).Source: §2 · p. 2
• SUPERB evaluates representations across downstream tasks (Yang et al., 2021), and frozen mean-pooled representations differ from systems with trained speaker heads.Source: §2 · p. 2
• The demographic analysis tracks how reported gaps vary across conditions, without treating subgroup EERs as a fairness assessment at a shared deployment threshold.Source: §2 · p. 2
• Table 1 compares reported analyses of related resources, not an exhaustive account of what their data could support.Source: Table 1 · p. 2

Task and scoring

• The intended task is text-independent speaker verification using frozen L2-normalised embeddings, cosine-scored trials, and reported EER, minDCF, and AUC.EERs are fractions; the supplied routine selects the ROC point with the smallest false-positive/false-negative rate difference and averages those rates, without interpolation.Source: §3 · pp. 2–3

The estimand

• The additive prediction is EER(A)+EER(B)−EER(clean), and IE is the joint EER minus that prediction.Positive IE is error above the additive prediction on the EER scale; negative IE is below it.Source: §3 · p. 3

Requirements for interpretation

• Interpreting contrasts requires comparable populations and trials, measured marginals and reference, and checks of metric range and ratio-denominator stability.A shared corpus alone does not control speaker, session, or content composition; substituting the reference for a missing marginal changes the estimand.Source: §3 · p. 3

Experiment families

• E13, E14, and E17 contain same-corpus terms for recomputation, but aggregate CSVs lack trial identities needed to independently verify matching.E13 contrasts an age stratum with an overlapping all-age population; it does not measure within-speaker ageing. E17 also lacks the recording-level dates needed by the bundled legacy loader.Source: §3 · p. 3
• E15 and E18 cannot isolate pathology or accent interactions because their reference populations belong to different corpora.E16 is analysed through subgroup gaps, and temporal and age proxies limit interpretation; recomputable contrasts are distinguished from verified controlled experiments.Source: §3 · p. 3

Chance performance and mathematical coupling

• EER 0.5 is a chance reference, not a hard upper bound, and a negative correlation between IE and the prediction does not establish saturation.The supplied routine does not reverse nor clip scores, and the artefact contains EERs above 0.5; the prediction is subtracted in IE, so correlation is mathematically coupled.Source: §3 · pp. 3–4

Corpora and encoder configurations

• The roster includes seven labels assigned to speaker encoders and five SSL configurations, but recorded labels do not verify distinct resolved checkpoints or controlled training-paradigm comparisons.The SSL representations use temporal means of final hidden layers without trained speaker heads; some speaker-encoder wrappers have unrecorded fallback paths.Source: §4 · p. 4

Operators and provenance

• Configured operators include telephony encode–decode, noise, packet-loss masks, time-scale perturbations, codecs, and speech conversion, but some retained runs are not described by legacy proxy paths.Noise sources may fall back to Gaussian noise, and codec or synthesis failures can return unchanged audio; not every experiment has augmentation and decode-failure counts.Source: §4 · p. 4

Trials, splits, and leakage

• A fixed initial sampler seed does not give every model or condition the same trial list, and speaker-level overlap with pre-training has not been audited.Scoring failures can change class balance, while repeated calls advance the random generator; several checkpoints use VoxCeleb and some experiments reference VoxCeleb1.Source: §4 · pp. 4–5

Coverage and availability

• The submission contains 4,068 scored records from 17 experiments and 12 model labels, totalling 11,975,026 trial evaluations.The six named corpora are RAVDESS, TORGO, EARS, Common Voice, L2-ARCTIC, and VoxCeleb1; the trial total counts scoring operations, not distinct utterances or independent speaker pairs.Source: §4 · pp. 4–5
• The retained roster covers the same 12 models across experiments, but E21 is removed, E09, E12, and E19 are absent, and no missing measurements are imputed.E13 and E14 use replacement cohorts; cross-corpus contrasts from restored E15 and E18 are excluded from interaction analysis.Source: §4 · p. 5
• The supplement supports recomputation of aggregate summaries, but the legacy harness is not established as the updated-run implementation and corpus audio is not included.Intended uses include condition-level comparisons and subgroup analysis; low interaction alone is not a robustness score, and absolute error, coverage, uncertainty, and held-out conditions are needed alongside interaction summaries.Source: §4 · p. 5

Encoder summaries

• CAM++ has macro EER 0.098, ECAPA-TDNN 0.101, and ECAPA-TDNN-Large 0.102; the ordering remains descriptive given sampling and provenance limits.All 12 labels cover the same 17 experiments, but common coverage alone does not establish matched trials or comparable sampling.Source: §5 · p. 5

Emotion and enrolment–test mismatch

• RAVDESS neutral→fearful mean EER is 0.256 versus 0.083 for neutral/neutral; matched fearful speech averages 0.191.Matched and mismatched trials have different budgets, and the pattern does not isolate a causal mechanism.Source: §5 · p. 6

Same-corpus additive contrasts

• The E13, E14, and E17 contrasts are reconstructed from CSV settings with all four finite terms for the same model label, verifying arithmetic but not sampling or provenance.Table 2 retains all such settings without a post-hoc family-level validity filter.Source: §5 · p. 6
• +0.0026, +0.0088, and +0.0024 are the E13, E14, and E17 mean contrasts for nominal speaker-trained labels; their SDs are 0.009–0.030.The means are descriptive and do not establish equivalence to additivity; Table 2 reports means and SDs across model–setting combinations.Source: §5 · p. 6, Table 2 · p. 6

Near-chance predictions

• For E14 mean-pooled SSL labels, additive predictions average 0.438 (maximum 0.670), with IE correlation r = −0.706; this is not evidence of saturation.The prediction and contrast share a term, and chance EER is not an upper bound.Source: §5 · p. 6, Table 3 · p. 6, Figure 3 · p. 7
• Across five E14 SSL labels, mean predictions range from 0.397 to 0.457, observed means from 0.350 to 0.414, and mean contrasts from −0.073 to −0.028.These within-family differences are descriptive and do not separate explanations for negative contrasts.Source: §5 · p. 7

Demographic gaps under stress

• All nine noisy mean absolute gender EER gaps exceed the clean mean of 0.0086, but none of the three noise sequences rises monotonically as SNR falls.Figure 4 shows means by labelled SNR and SDs across 12 model labels.Source: §5 · p. 7, Figure 4 · p. 7
• Noisy signed-gap ratios range from −157.6 to +193.7 because clean signed gaps can be as small as 0.0004 in magnitude.Absolute gaps avoid that denominator but still require sampling uncertainty; E16 uses 5,000 clean versus 555 noisy trials per subgroup.Source: §5 · p. 7

Validation of additive predictions

• A useful additive-prediction validation would predefine acceptable error, test held-out factor settings, and retain per-model prediction-error distributions.The benchmark should report where predictions fail, not only whether positive and negative deviations cancel.Source: §6 · p. 8

Metric interpretation

• EER uses each condition’s own equal-error operating point, while demographic EER gaps compare separately optimised operating points.Neither directly measures risk at a fixed production threshold.Source: §6 · p. 8
• Deployment-oriented analysis needs subgroup false-accept and false-reject rates at a common threshold selected on separate development data.These additions would test practical meaning without treating benchmark averages as deployment guarantees.Source: §6 · p. 8

Result provenance

• Only 1,878 of 4,068 records have checkpoint hashes; trial lists, per-trial scores, hardware, and full software environments are not supplied.Aggregate agreement alone cannot authenticate an encoder.Source: §7 · p. 8

Code–artefact discrepancies

• Exact scored-pair manifests and full checkpoint binding remain unavailable, while publication-year and age proxies and the nominal zero-gap reference limit temporal interpretation.The run audit supersedes failures inferred solely from old code paths, but these limitations remain.Source: §7 · p. 8

Uncertainty and comparability

• Each retained record has one aggregate measurement and no linked score list; dependent contrasts, varying trial budgets, and between-model SDs leave sampling uncertainty unquantified.Corpus membership alone does not establish matched population composition.Source: §7 · p. 8

Scope

• Incomplete execution documentation, mixed source populations, proxy conditions, and unaudited metadata prevent broad claims about real-world compound robustness or demographic fairness.Pre-training overlap, corpus versions, full subgroup sizes, and metadata provenance remain unaudited.Source: §7 · p. 8

8 CONCLUSION

• Three same-corpus experiment families support recomputation of additive contrasts, but trial matching and experimental provenance remain unverified.The artefact supports descriptive condition summaries, not validated controlled-interaction claims.Source: §8 · p. 8, §A.1 · p. 12

ETHICS STATEMENT

• Demographic comparisons do not establish fairness for specific populations, and the results should not certify fairness or justify biometric surveillance or access-control deployment.Coarse categories, incomplete metadata, and unresolved label provenance constrain interpretation.Source: ETHICS STATEMENT · p. 9

REPRODUCIBILITY STATEMENT

• The supplementary CSV and scripts reconstruct aggregate tables and plots, but missing provenance and code–artefact discrepancies prevent full experimental reproduction.The paper distinguishes summary-level reproducibility from reproducing the underlying runs.Source: REPRODUCIBILITY STATEMENT · p. 9

AI USE STATEMENT

• AI assistance covered implementation, analysis scripting, interpretation, literature retrieval, and manuscript work; retained observations were not altered and no new benchmark experiments were run.The authors remain responsible for verifying scientific content and disclosure accuracy.Source: AI USE STATEMENT · p. 9

A.1 MOTIVATION, COMPOSITION AND INTENDED USE

• The artefact supports descriptive summaries and recomputation, while controlled interaction, demographic, spoofing, and temporal-drift claims require further provenance and experimental checks.There is no benchmark-specific training split or fine-tuning, and evaluation-speaker overlap in pre-training remains unresolved.Source: §A.1 · p. 12

A.2 FULL EXPERIMENT INVENTORY WITH AGGREGATE SCORES

• Aggregate scores pool heterogeneous conditions and encoders, so they describe condition-set difficulty rather than a controlled ranking of stressors.The inventory records corpus, encoder and condition counts, scored records, trial counts, and each experiment’s descriptive use.Source: §A.2 · p. 13, Table 5 · p. 13, Table 6 · p. 14
• E15 and E18 provide within-corpus clean/codec summaries, but their cross-corpus populations cannot isolate pathology or accent effects.Their historical rows enter descriptive summaries, not the E13/E14/E17 interaction table.Source: §A.2 · p. 13, Table 5 · p. 13
• Table 6 reports EER as the unweighted mean over scored records with standard deviation across those records; cross-corpus and temporal/age-proxy results remain qualified.Table 5 defines each experiment’s admitted descriptive use and marks cross-corpus, proxy, and subgroup-gap cases.Source: Table 5 · p. 13, Table 6 · p. 14

B ENCODER RESULTS

• Table 7 macro-averages conditions within each experiment and weights the contributing experiments equally, with all seventeen contributing for each retained encoder.Condition coverage is complete, but trial matching and provenance keep the ordering descriptive.Source: B ENCODER RESULTS · p. 15, Table 7 · p. 15

C SINGLE-FACTOR RESULTS: E01 EMOTIONAL SPEECH

• Table 8 reports mean EER across available encoders and ratios to the neutral/neutral reference, with matched conditions using the same emotion for enrolment and test.Its standard deviation measures between-encoder variability, not a confidence interval.Source: Table 8 · p. 15

D.1 ABSOLUTE GENDER GAP BY CONDITION

• Table 9 summarizes recorded F/M subgroup EERs using mean absolute per-encoder gaps and the maximum absolute gap across encoders.The available description specifies the table’s summaries, not a result establishing fairness for specific populations.Source: Table 9 · p. 16

D.2 INSTABILITY OF RATIO-FORM BIAS AMPLIFICATION

• The clean gender gap is the denominator of the ratio, and the reported ratio range illustrates sensitivity to a near-zero denominator rather than reliable amplification.Mean absolute gap summarizes the nine noise conditions.Source: Table 10 · p. 16

E.1 PER-ENCODER INTERACTION EFFECTS

• In E14, all five SSL means are negative (−0.073 to −0.028 EER), while six of seven speaker-trained means are positive and RawNet3 is negative (−0.003).The table includes finite four-term contrasts, but NLI means use only positive additive predictions and no family-level chance screen is applied.Source: §E.1 · p. 16, Table 11 · p. 16

E.2 SETTING-LEVEL INTERACTION LISTING

• The per-setting listing shows pooled observed joint EER and additive predictions alongside separate family-mean contrasts for settings with four available terms.Its dispersion is what the pooled means average over.Source: §E.2 · p. 17, Table 12 · p. 17

F ENCODER-AGREEMENT AUDIT

• No encoder pair agrees exactly on more than 20.9% of compared conditions, but differing sampled trials and augmentation realizations prevent agreement rates from authenticating checkpoints.Hashes provide additional provenance only for the subset of records containing them.Source: F ENCODER-AGREEMENT AUDIT · p. 20, Table 13 · p. 20

G EXTENDED LIMITATIONS

• A small signed mean cannot establish equivalence to additivity; uncertainty requires per-trial scores, dependence-aware resampling, and a predefined practical margin.Independent speaker/trial and augmentation seeds would separately test construction sensitivity; treating model–setting combinations as independent replicates would understate dependence.Source: G EXTENDED LIMITATIONS · p. 20
• Run identifiers and checkpoint hashes cover only subsets, and no score manifests bind all observations to the bundled legacy implementation.The audit also finds the old loader is not the producing revision; temporal labels and nominal zero-gap references do not verify same-session or same-year baselines.Source: G EXTENDED LIMITATIONS · pp. 20–21
• The noise and attack audit resolves legacy-branch contradictions but lacks exact scored-pair lists and complete operator-to-checkpoint binding.It does not establish real-time latency.Source: G EXTENDED LIMITATIONS · p. 21
• E13 compares an age stratum with an overlapping all-age reference, while representative-integer age bins and incomplete trial manifests limit demographic matching checks.The retained E01 and replacement E14 rows use RAVDESS; all twelve retained labels cover all seventeen experiments, but coverage alone does not establish matched sampling.Source: G EXTENDED LIMITATIONS · p. 21
• E21, E09, E12, and E19 lack usable updated or replacement coverage; ERes2Net lacks four retained experiments, while E15 and E18 remain historical descriptive results.E20 is retained descriptively rather than used as a three-way interaction estimate.Source: G EXTENDED LIMITATIONS · p. 21
• The release lacks complete corpus versions, speaker-count coverage, audio hours, hardware, library versions, and execution costs; its trial total counts scoring operations, not independent data.Some records lack usable corpus identifiers, and reuse also requires licensing, attribution, acquisition instructions, and stable protocol manifests.Source: G EXTENDED LIMITATIONS · p. 21

H CONDITION-LEVEL RESULTS FOR EVERY EXPERIMENT

• Tables 14–19 report condition-level means for E01–E06 across emotional speech, dysarthric speech, speaking rate, recorded voice style, codec round trips, and packet loss; SD is across contributing encoder rows, not a confidence interval.Source: Table 14 · p. 22, Table 15 · p. 22, Table 16 · p. 22
• Tables 20–25 report condition-level means for E07 (temporal drift), E08 (L2 accent), E10 (TTS spoofing), E11 (voice conversion), E13 (age × codec), and E14 (emotion × noise), over contributing encoder rows.The reported SD is over contributing encoder rows, not a confidence interval.Source: Table 20 · p. 24, Table 21 · p. 25, Table 22 · p. 25