PaperlayerView exact revision on arXiv

Source-linked AI summary

Layer-wise Cross-Lingual Depression Detection from Speech: Analysis with Contrastive Alignment

Anisha Pattanayak, Hanie Kang, Huang-Cheng Chou, Shrikanth Narayanan, Sudarsana Reddy Kadiri

arXiv:2607.02920v1eess.AS

TL;DR

Cross-lingual generalization remains challenging for speech-based depression detection across linguistic populations. CLeaD aligns English and Mandarin WavLM embeddings in a shared clinical space without parallel data or target-language fine-tuning, yielding modest F1 gains under speaker-independent evaluation.

  • Problem

    Cross-lingual generalization remains an open challenge for speech-based depression detection across linguistic populations.

  • Method

    CLeaD uses supervised contrastive alignment to map English and Mandarin WavLM embeddings into a shared clinical space without parallel data or target-language fine-tuning.

  • Results

    F1 improves from 0.622 to 0.640 over the cross-entropy baseline on 52 MODMA speakers, with improved depressed-class recall at Base-Plus Layers 7–8.

  • Takeaways & Limitations

    Cross-lingual performance degrades with WavLM-Large, while speaker-identity leakage inflates Mandarin F1 by 0.23 under segment-level splits.

  • Takeaways & Limitations

    Generalizability is limited by the small test set, single training seed, and trailing linear baselines.

Abstract

from arXiv · show

Significant disparities exist in the diagnosis and clinical presentation of depression across different linguistic populations. Speech-based depression detection performs well monolingually, but cross-lingual generalization remains an open challenge. A key reason is that prior work uses segment-level random splits without speaker grouping, leading to identity leakage that inflates reported metrics. We propose CLeaD, a supervised contrastive alignment framework that maps WavLM embeddings from English and Mandarin into a shared clinical space, without parallel data or target-language fine-tuning. Evaluating 52 Mandarin speakers, contrastive alignment modestly outperforms the baseline (F1: 0.640 vs. 0.622) under leave-one-speaker-out evaluation. It also improves depressed-class recall at intermediate layers (7-8), though the small test set limits generalizability. Two findings remain robust: model scaling degrades cross-lingual performance while improving monolingual English, and speaker identity leakage artificially inflated previously reported Mandarin F1 scores to 0.954, an artifact we reproduce and quantify.

I. INTRODUCTION · II. RELATED WORK · A. Depression Detection from Speech

The paper introduces CLeaD for cross-lingual depression detection and emphasizes rigorous speaker-level evaluation. It also addresses prior methodological limitations, including speaker-identity leakage and the evolution from handcrafted speech features to self-supervised learning models.

  • I. INTRODUCTION: Clinical depression diagnosis relies on trained professionals administering structured interviews and validated rating scales, limiting deployment in low-resource healthcare settings.Depression affects over 280 million people globally and increases the risk of disability and suicide.
  • I. INTRODUCTION: CLeaD maps English and Mandarin WavLM embeddings into a shared clinical space without parallel data or target-language fine-tuning.It is presented as a supervised contrastive alignment framework.
  • I. INTRODUCTION: Layer-wise analysis compares WavLM-Base-Plus and WavLM-Large across six transfer conditions using 95% bootstrap confidence intervals with 2000 resamples.The comparison is systematic and layer-focused.
  • I. INTRODUCTION: Segment-level random splits artificially inflate Mandarin F1 scores to 0.954 by allowing speaker-identity leakage.The paper identifies and corrects this evaluation artifact.
  • I. INTRODUCTION: Evaluation uses leave-one-speaker-out testing on 52 MODMA speakers, ablations, and hyperparameter sensitivity analysis.The study also prioritizes depressed-class speaker recall alongside standard LOSO speaker-level F1 metrics.
  • A. Depression Detection from Speech: Depression detection progressed from MFCCs, pitch, and shimmer with SVMs to recurrent architectures and self-supervised models such as HuBERT and wav2vec 2.0.The passage states that self-supervised models outperformed handcrafted pipelines and offered better generalization.

B. Layer-Wise Analysis of SSL Models · C. Cross-Lingual Depression Detection · III. METHODOLOGY

The paper analyzes SSL layers for cross-lingual depression detection, identifies intermediate-layer advantages, and demonstrates that speaker leakage—not scale leakage—drives inflated Mandarin performance. It validates this artifact within its own WavLM pipeline and frames cross-lingual clinical transfer through embedding-space reshaping.

  • B. Layer-Wise Analysis of SSL Models: Intermediate layers yield the best depression-detection results in both monolingual and cross-lingual settings, consistent with findings in cross-lingual emotion recognition.These observations derive from prior systematic layer-wise analyses of HuBERT.
  • B. Layer-Wise Analysis of SSL Models: WavLM extends HuBERT with a masked denoising objective that improves robustness for variable-quality clinical recordings.
  • B. Layer-Wise Analysis of SSL Models: Mandarin F1 increases by +0.23 from speaker leakage, whereas scale leakage contributes <0.001.The comparison comes from progressively relaxed within-pipeline leakage protocols.
  • B. Layer-Wise Analysis of SSL Models: The study exposes and quantifies speaker leakage as an artifact inflating previously reported cross-lingual scores.It presents this artifact alongside an objective that reshapes the embedding space for cross-lingual clinical transfer.
  • C. Cross-Lingual Depression Detection: Chen’s segment-level random splits without speaker grouping inflated reported Mandarin F1 to 0.954.The authors confirmed the issue by examining Chen’s public codebase.
  • III. METHODOLOGY: The authors validate leakage within their own pipeline using the same WavLM Base-Plus backbone and MODMA dataset.This validation directly tests the leakage contribution in a comparable cross-lingual depression-detection setting.

A. Datasets … 2) Mixed-language training:

The study harmonizes English and Mandarin depression datasets under a shared screening criterion, standardizes speech preprocessing, prevents speaker identity leakage through grouped splits, and balances languages during mixed-language training.

  • A. Datasets: E-DAIC contains 219 semi-structured English clinical interviews, with depression defined by PHQ-8 ≥10 and participant speech separated from interviewer turns.The official test labels are withheld, so publicly labeled data are re-partitioned using speaker-independent splits.
  • A. Datasets: Both corpora use a total questionnaire score ≥10 to label depression, while demographic and recording differences remain potential cross-lingual confounds.The threshold corresponds to the standard screening cut-off for probable major depression on both instruments.
  • B. Data Preprocessing: All audio is converted to mono at 16 kHz and segmented into 3-second windows with 50% overlap, balancing prosodic context and data augmentation.Each segment inherits its speaker’s binary depression label, and E-DAIC interviewer turns are excluded using transcript-aligned timestamps.
  • B. Data Preprocessing: The preprocessing pipeline applies consistent mono, sampling-rate, windowing, and overlap choices across both corpora before model training.These choices standardize the inputs while retaining short-term prosodic context and augmenting limited clinical data.
  • 1) Speaker-independent splitting:: Speaker identity leakage is controlled by assigning every speaker’s segments to exactly one stratified partition using participant ID as the grouping variable.Without grouping, WavLM embeddings can enable recognition of individuals rather than depression-relevant patterns.
  • 2) Mixed-language training:: English segments are randomly subsampled to match the MODMA segment count, preventing the larger corpus from dominating each training batch.This balancing ensures that every CLeaD batch contains cross-lingual positive pairs sharing the same clinical label.

C. WavLM Feature Extraction · 1) Layer selection rationale:

CLeaD uses frozen WavLM-Base-Plus and WavLM-Large extractors, selecting proportionally corresponding middle layers to study representational depth. Mean-pooled utterance embeddings feed alignment and classification, while Layer 6 supports prior-work comparison and ablation identifies Layer 8 as the peak.

  • C. WavLM Feature Extraction: WavLM-Base-Plus has 12 layers and d=768, while WavLM-Large has 24 layers and d=1024; both are pretrained on 94K hours of English speech.Hidden states are extracted from layers 6–9 of Base-Plus and layers 12, 14, 16, and 18 of Large.
  • C. WavLM Feature Extraction: The selected layer ranges occupy roughly the same relative depth in each model, enabling comparisons of depth-in-proportion rather than absolute layer index.Large has twice as many layers as Base-Plus.
  • C. WavLM Feature Extraction: Mean pooling across F frames produces an utterance-level embedding, with embedding dimension d equal to 768 for Base-Plus and 1024 for Large.The resulting representation is used as input to CLeaD.
  • C. WavLM Feature Extraction: Freezing WavLM ensures that performance differences reflect layer representational properties rather than fine-tuning artifacts.The full processing pipeline is shown in Fig. 1.
  • C. WavLM Feature Extraction: Both languages share a frozen WavLM extractor, while a projection head aligns same-class cross-lingual embeddings using SupCon loss.The classification head uses class-weighted CE.
  • C. WavLM Feature Extraction: Setting λ=0 produces the CLeaD w/o SupCon variant.This isolates the contribution of supervised contrastive alignment in the pipeline.
  • 1) Layer selection rationale:: Layer 6 is used in comparative Tables I and II because prior work reports Base-Plus Layer 6 as the strongest monolingual layer.This choice anchors cross-paper comparison rather than asserting that Layer 6 is optimal across the full ablation.
  • 1) Layer selection rationale:: CLeaD’s peak F1 occurs at Layer 8 (0.561), while best recall occurs at Layers 7–8 (4/5 Dep-Rec).The comparative Layer 6 choice and the ablation-derived peak are complementary rather than contradictory.

D. Contrastive Learning for Depression Detection (CLeaD)

CLeaD trains a two-head network on frozen WavLM embeddings, projecting English and Mandarin speech into a normalized shared space while jointly optimizing classification and supervised contrastive objectives. Same-label examples across languages are pulled together, including depressed samples, without augmentation-based views.

  • Architecture: CLeaD uses frozen WavLM embeddings with a projection head that maps pooled representations into a normalized 128-dimensional space.The projection uses learnable matrices W1∈R256×d and W2∈R128×256, ReLU, batch normalization, and dropout rate 0.3 before W2.
  • Architecture: The classification head maps the 128-dimensional projection through a two-layer MLP, R128 → R64 → R2, with ReLU and dropout rate 0.2 between layers.The output consists of class logits for depression detection.
  • Training objective: CLeaD combines class-weighted cross-entropy and supervised contrastive loss with λ=0.5 balancing the two objectives.The contrastive term operates on projection embeddings, while cross-entropy is computed from classifier logits.
  • Cross-lingual alignment: The supervised contrastive loss pulls same-label English and Mandarin embeddings together, including depressed samples from both languages.For each anchor, positives are other batch samples sharing its clinical label, regardless of language.
  • Training objective: CLeaD uses single-view contrastive learning, excluding pitch-shifting and time-stretching augmentations.The supplied passage explicitly states that these augmentation-based views are not used.

E. Baselines and Evaluation · IV. EXPERIMENTAL SETUP · A. Training Configurations

The evaluation compares standard classifiers on identical frozen embeddings across monolingual, zero-shot, and mixed-language conditions. CLeaD’s alignment objective operates only in mixed training, while speaker-level Dep-Rec measures depressed-speaker identification in the held-out MODMA test set.

  • E. Baselines and Evaluation: Baselines use identical frozen embeddings with Logistic Regression, SVM-Linear, and a bidirectional GRU sequence classifier.LR and SVM use balanced weighting, while the GRU retains its default architecture.
  • E. Baselines and Evaluation: No additional baseline hyperparameter search was performed, potentially understating their performance ceilings.The paper acknowledges this as a limitation.
  • E. Baselines and Evaluation: Dep-Rec counts depressed speakers correctly identified among five depressed speakers in the ten-speaker held-out MODMA test set.The metric is used instead of the standard MDD abbreviation to distinguish it from the clinical term Major Depression Disorder.
  • A. Training Configurations: Monolingual conditions train and test on the same language, providing an upper bound for each self-supervised learning model.These conditions assess performance without cross-lingual transfer.
  • IV. EXPERIMENTAL SETUP: The monolingual performance table evaluates base-plus L6 and large L12 models using depressed-class F1.It lists standard LR and SVM configurations and sets CLeaD’s λ to 0.5.
  • A. Training Configurations: Zero-shot conditions train on one language and test on the other without adaptation or target-language labeled data.This represents the hardest practical cross-lingual setting.
  • A. Training Configurations: Mixed conditions combine both languages in each training batch, enabling CLeaD’s cross-lingual alignment objective through supervised contrastive positive pairs.The SupCon loss requires cross-lingual positive pairs.
  • A. Training Configurations: CLeaD’s Dep-Rec gains are specific to mixed-to-Mandarin evaluation conditions.The supplied passage identifies this setting as the one in which the alignment objective can engage.

B. Hyperparameters … A. Monolingual Performance

The study fixes a frozen-WavLM CLeaD configuration, evaluates multiple intermediate and large-model layers, and reports corrected monolingual results alongside cross-lingual evaluation definitions. Large outperforms Base-Plus in English, while Mandarin performance is lower and a Large GRU condition is degenerate.

  • B. Hyperparameters: CLeaD uses AdamW with lr=10−3, weight decay 10−4, 100 epochs, batch size 32, τ=0.1, λ=0.5, and projection dimension 256.WavLM weights remain frozen, with equal alignment and classification loss weighting and a fixed random seed.
  • C. Layer-Wise Ablation Design: 240 evaluation runs cover six conditions across eight WavLM layers, including Base-Plus layers 6–9 and Large layers 12–18.The selected ranges target intermediate representations associated with transferable paralinguistic information.
  • A. Monolingual Performance: Large significantly outperforms Base-Plus on EN→EN at Layer 6/12, with non-overlapping CLeaD AUC confidence intervals of [0.763,0.787] versus [0.825,0.846].The reported comparison is based on Table II within-language results.
  • A. Monolingual Performance: English LR achieves F1 0.638, compared with Chen’s 0.74 under segment-level splits.This comparison contextualizes the corrected within-language evaluation against prior work.
  • A. Monolingual Performance: Mandarin monolingual performance is substantially lower under corrected evaluation using MODMA rather than CMDC.The passage attributes the difference to the evaluation correction and dataset choice.
  • V. EXPERIMENTAL RESULTS AND ANALYSES: Cross-lingual Table III reports depressed-class F1 and DEP-REC, defined as depressed speakers correctly identified out of 5 in the held-out MODMA test set.The table compares BP=Base-Plus L6 with LG=Large L12.
  • A. Monolingual Performance: Large GRU predicts every speaker as depressed, correctly identifying 0/5 healthy controls in the held-out condition.This is reported as degenerate recall.

B. Cross-Lingual Transfer · C. Layer Ablation

Cross-lingual transfer favors the Base-Plus model over Large, whose Mandarin performance collapses and whose GRU can degenerate into predicting nearly every speaker as depressed. Layer ablation shows that CLeaD’s strongest depressed-class sensitivity occurs at Layer 7, while peak F1 occurs at Layer 8.

  • B. Cross-Lingual Transfer: Large achieves near-zero F1 across all Mandarin conditions, with non-overlapping confidence intervals versus Base-Plus at every layer pair.For MIX→ZH CLeaD, Base-Plus has CI [0.462,0.513] versus Large [0.184,0.236].
  • B. Cross-Lingual Transfer: Large’s GRU degenerates in EN→ZH and MIX→ZH, producing 5/5 depressed recall but only 0/5 or 1/5 healthy-control recall.This pattern means it predicts every speaker as depressed.
  • C. Layer Ablation: Base-Plus CLeaD peaks at Layer 8 with F1 0.561 and Dep-Rec 4/5 on MIX→ZH.The table reports depressed-class F1 as the F1 metric.
  • C. Layer Ablation: At Layer 7, CLeaD reaches Dep-Rec 4/5, exceeding CLeaD w/o SupCon at 1/5 and SVM-Linear at 2/5.The SupCon gap is most pronounced at this layer.
  • C. Layer Ablation: Contrastive alignment adds depressed-speaker sensitivity beyond CLeaD w/o SupCon and SVM-Linear at Layer 7.The comparison is based on depressed-class recall under MIX→ZH layer ablation.
  • C. Layer Ablation: Base-Plus outperforms Large at all MIX→ZH layers, with non-overlapping 95% confidence intervals.This pattern is reported for F1 across layer pairs.

D. LOSO Evaluation (Primary Mandarin Metric) · E. Hyperparameter Sensitivity · F. Projection Space Analysis

Under leave-one-speaker-out evaluation on 52 Mandarin speakers, CLeaD modestly improves over its no-SupCon variant, while sensitivity analysis favors τ=0.1 and λ=0.5. Projection analysis shows reduced language separability and stronger clinical clustering after alignment.

  • D. LOSO Evaluation (Primary Mandarin Metric): Spk F1 0.762: SVM-Linear leads LOSO evaluation on all 52 MODMA speakers.Spk F1 denotes speaker-level depressed-class F1.
  • D. LOSO Evaluation (Primary Mandarin Metric): CLeaD 0.640 outperforms CLeaD w/o SupCon 0.622 at Base-Plus Layer 7, a small 0.018 margin.The result indicates a positive speaker-level contribution from contrastive alignment beyond the MLP architecture alone, although CLeaD trails SVM-Linear and LR at this layer.
  • E. Hyperparameter Sensitivity: τ=0.1 and λ=0.5 are near-optimal at Layer 7, the primary MIX→ZH layer.The chosen configuration balances contrastive sharpness and SupCon–CE loss weighting.
  • E. Hyperparameter Sensitivity: τ=0.05, τ=0.2, and λ=0.7 reduce peak Dep-Rec, showing that neither alignment nor classification alone is sufficient.Both colder and warmer temperatures hurt performance, as does strong down-weighting of CE.
  • F. Projection Space Analysis: Language-classifier accuracy drops from 89.2% to 84.0% after MIX→ZH training, indicating reduced English–Mandarin domain separability.The classifier is logistic regression evaluated with 5-fold cross-validation on the 128-dimensional CLeaD embeddings at Layer 6.
  • F. Projection Space Analysis: Clinical-label silhouette scores rise from 0.010 to 0.066 after training, while language silhouette drops from 0.020 to 0.018.These changes support reduced language separation and improved clinical cluster structure in the CLeaD projection space.

VI. DISCUSSION … C. Zero-Shot Failure and Future Directions

The discussion attributes weak cross-lingual Mandarin transfer to English-specialized representations in larger models and identifies intermediate layers as the strongest transfer point. In EN→ZH zero-shot, CLeaD fails to benefit from cross-lingual alignment without Mandarin training samples, whereas the GRU generalizes better.

  • A. Why Model Scale Hurts Cross-Lingual Transfer: WavLM-Large outperforms Base-Plus on monolingual English but fails under cross-lingual Mandarin conditions at every layer pair.The reported confidence intervals do not overlap across the cross-lingual layer-pair comparisons.
  • A. Why Model Scale Hurts Cross-Lingual Transfer: Larger WavLM models may encode deeper English-specific phonetic and prosodic structure that interferes with Mandarin transfer.The passage identifies stress patterns and vowel contrasts as examples of English-specific structure lacking depression-relevant signal for Mandarin.
  • B. Why Intermediate Layers Transfer Best: Intermediate Base-Plus layers 6–8 consistently outperform shallow and deep layers for cross-lingual transfer.Shallow layers retain site-dependent acoustic artifacts, while deep layers specialize toward English phonetics.
  • B. Why Intermediate Layers Transfer Best: Intermediate layers retain language-agnostic prosodic and temporal cues, including speaking rate, energy, pause duration, and turn dynamics, that correlate with depression across languages.The passage contrasts these cues with shallow acoustic surface features and deep English-phonetic specialization.
  • B. Why Intermediate Layers Transfer Best: 0.561 is CLeaD’s peak segment F1 at Layer 8, while best Dep-Rec is 4/5 at Layers 7–8.Layer 6 anchors the cross-paper comparison in Tables II and III.
  • C. Zero-Shot Failure and Future Directions: 1/5 is CLeaD’s EN→ZH zero-shot Dep-Rec, compared with 3/5 for the GRU.English-only training provides no Mandarin samples in any batch, making the SupCon cross-lingual alignment gradient identically zero and reducing CLeaD to standard supervised contrastive learning.

D. Clinical Implications · VII. CONCLUSION

CLeaD supports cross-lingual depression screening by modestly improving Mandarin F1 and depressed-class recall, while emphasizing recall and speaker-grouped evaluation for clinical use. The analysis also identifies model scaling and speaker identity leakage as robust threats, with corpus mismatch and limited evaluation constraining conclusions.

  • D. Clinical Implications: CLeaD targets first-stage depression screening in multilingual healthcare, emphasizing depressed-class recall while retaining LOSO Spk F1 as the primary statistical metric.The clinical rationale is that missed patients cost more than false positives that clinician review can catch.
  • VII. CONCLUSION: 0.640 vs 0.622: CLeaD yields modest F1 gains over a cross-entropy baseline on 52 MODMA speakers.The framework uses supervised contrastive alignment for cross-lingual depression detection.
  • VII. CONCLUSION: CLeaD improves depressed-class recall at Base-Plus Layers 7–8, but the small test set, single training seed, and trailing linear baselines limit the strength of its claims.The study characterizes where the SupCon objective helps rather than claiming state-of-the-art detection.
  • VII. CONCLUSION: WavLM-Large degrades cross-lingual performance despite strong English results, with the finding confirmed by non-overlapping confidence intervals.This scaling effect is identified as robust in the conclusion.
  • VII. CONCLUSION: 0.23: speaker identity leakage inflates Mandarin F1, showing that segment-level splits inadvertently reward speaker recognition.This motivates speaker-grouped evaluation for reliable cross-lingual assessment.
  • VII. CONCLUSION: Corpus mismatches, including age and clinical instruments, indicate that current transfer effects reflect broad domain shifts.Future work will isolate language from corpus effects using multilingual backbones, multi-seed testing, and adaptation baselines; code and data splits are released via an anonymized link.
Loading 2607.02920v1…