PaperlayerView exact revision on arXiv

Source-linked AI summary

Automatic Detection of Depression in Speech Using Ensemble Convolutional Neural Networks

Adrián Vázquez-Romero, Ascensión Gallardo-Antolín

arXiv:2402.02830v1eess.AS

TL;DR

Depression detection from speech may support assessment of a serious disorder, and the authors note that ensemble CNNs had not previously been applied to speech-based depression detection. They combine differently initialized 1d-CNNs for speaker-level classification and report relative F1-score improvements over three comparison systems. For real-world use with reversible log-spectrograms, they identify privacy protections as a consideration.

  • Problem

    Ensemble CNN models had not previously been used for automatic depression detection from speech, a task the paper addresses using voice alone.

    Source: §2 · p. 3

  • Method

    The system classifies depression from speech log-spectrograms by fusing predictions from multiple 1d-CNNs trained with different initializations.

    Source: §1 · p. 2, §3.2.2 · p. 8

  • Results

    The ensemble achieved relative F1-score improvements of 58.5% over the baseline, 30.0% over DepAudionet, and 10.2% over a single 1d-CNN.

    Source: §5 · p. 13

  • Takeaways & Limitations

    On the DAIC-WOZ dataset in the AVEC-2016 Depression Sub-Challenge, the proposed ensemble outperformed the cited baseline, DepAudionet, and single 1d-CNN comparisons in F1-score.

    Source: §5 · p. 13

    For real-world use, reversible log-spectrogram inputs require local feature extraction and encryption before transmission to a secure server; this issue is beyond the paper’s scope.

    Source: §2 · p. 4

Abstract

from arXiv · show

This paper proposes a speech-based method for automatic depression classification. The system is based on ensemble learning for Convolutional Neural Networks (CNNs) and is evaluated using the data and the experimental protocol provided in the Depression Classification Sub-Challenge (DCC) at the 2016 Audio-Visual Emotion Challenge (AVEC-2016). In the pre-processing phase, speech files are represented as a sequence of log-spectrograms and randomly sampled to balance positive and negative samples. For the classification task itself, first, a more suitable architecture for this task, based on One-Dimensional Convolutional Neural Networks, is built. Secondly, several of these CNN-based models are trained with different initializations and then the corresponding individual predictions are fused by using an Ensemble Averaging algorithm and combined per speaker to get an appropriate final decision. The proposed ensemble system achieves satisfactory results on the DCC at the AVEC-2016 in comparison with a reference system based on Support Vector Machines and hand-crafted features, with a CNN+LSTM-based system called DepAudionet, and with the case of a single CNN-based classifier.

1. Introduction

Depression has serious personal and societal consequences, while traditional reports can be time-consuming and burdensome for patients. This work focuses on audio-based depression classification using an ensemble of CNNs and evaluates it on the AVEC-2016 DCC data.

  • Depression is associated with low mood, low self-esteem, loss of interest, low energy, and persistent unexplained pain, with impacts on family, work, sleep, and eating.The passage reports that over 300 million people are estimated to suffer from depression.Source: §1 · p. 1
  • Traditional reports including BDI-II, GDS, HRSD, and PHQ-8 can take substantial clinician time and may be burdensome or bias-prone for patients.Source: §1 · p. 1
  • Automatic depression detection is intended to assist, not replace, medical experts’ judgments in clinical decision support.Source: §1 · p. 1
  • The study classifies whether a speaker has been diagnosed with depression from speech log-spectrograms using an ensemble of CNNs, evaluated on DAIC-WOZ under the AVEC-2016 DCC guidelines.The DCC task allows audio, visual, and/or text data; this work focuses on audio.Source: §1 · p. 2

2. Related Work

Prior depression-detection work spans visual and audio modalities, conventional acoustic features, and deep-learning systems. The paper refines a 1d-CNN and applies an ensemble to speech-based depression detection, while noting privacy requirements relevant to real-world deployment.

  • Visual depression-detection systems model facial expressions, head movements, eye gazes, or blinks, and multimodal systems combining visual and other cues usually outperform individual modalities.Examples include AVEC-2013 audio-visual fusion and the AVEC-2016 challenge winner.Source: §2 · p. 2
  • Speech carries paralinguistic cues about speakers’ emotional, neurological, and mental traits, and has been studied for depression and other health conditions.Source: §2 · p. 2
  • Conventional speech-health systems use hand-crafted acoustic features, while depression-detection work also uses deep learning, including the DepAudionet 1d-CNN, LSTM, and fully connected architecture.Reported acoustic features include pitch, formants, intensity, jitter, shimmer, harmonic-to-noise ratio, and speech rate.Source: §2 · p. 3
  • Ensemble methods combine multiple models, usually with the same base learner, and have been used in applications including image categorization, sentiment analysis, and acoustic environment classification.Source: §2 · p. 3
  • The paper proposes a refined 1d-CNN based on DepAudionet and applies ensemble learning to speech-based depression detection, which the authors say had not previously used ensemble CNN models.The authors describe these as the paper’s two main contributions.Source: §2 · p. 3
  • For real-world applications using reversible log-spectrograms, the paper says acoustic features should be extracted and encrypted locally, then sent to a secure server for analysis.The DAIC-WOZ recordings were shared for research with participant consent; the real-world privacy issue is outside the paper’s scope.Source: §2 · pp. 3–4

3. Materials and Methods

The method converts DAIC-WOZ speech into balanced 4-second log-spectrogram samples, classifies them with a one-dimensional CNN, and combines predictions from differently initialized models to assign speaker-level labels.

3.1.1. Database Description

  • DAIC-WOZ contains 189 English-speaking subjects, split into 142 training and 47 test participants; PHQ-8 scores at least 10 define depression.The resulting training set has 100 non-depressed and 42 depressed recordings; the test set has 33 depressed and 14 non-depressed participants.Source: §3.1.1 · pp. 4–5, Figure 1 · p. 4

3.1.2. Pre-Processing

  • Diarization removes the interviewer and long silences; 4-second crops are sampled to balance classes and select the same number per speaker.Training uses 89 crops for each of 62 randomly selected speakers—31 per class—for 5,518 samples; four shorter test recordings contribute fewer crops.Source: §3.1.2 · p. 5

3.1.3. Feature extraction

  • Each 4-second speech crop becomes a 513 × 125 log-spectrogram computed from the STFT using 64-ms Hamming windows, 32-ms steps, and 1,024 FFT points.The log-spectrogram is the model input.Source: §3.1.3 · pp. 5–6, Figure 2 · p. 6

3.2. Depression Detection System

  • The depression detector has two stages: CNN-based classifiers first produce predictions, then an ensemble fuses them.Figure 5 summarizes the three proposed ensemble methods.Source: §3.2 · p. 6, Figure 5 · p. 9

3.2.1. 1d-CNN Model

  • Because log-spectrogram time and frequency axes differ, the model uses frequency-wide 1D convolutions followed by time-axis max-pooling and fully connected layers.The convolution captures short-term frequency correlations, while pooling captures mid-term temporal dynamics; the output sigmoid gives each sample’s depression probability.Source: §3.2.1 · p. 7, Figure 3 · p. 7
  • Speaker predictions are derived either by taking the mode of sample labels or by thresholding their mean probability at 0.5; the two approaches usually, but not always, agree.Figure 4 illustrates sample- and speaker-level label computation.Source: §3.2.1 · p. 8, Figure 4 · p. 8

3.2.2. Ensemble Method

  • The ensemble combines differently initialized 1d-CNNs using three strategies: averaging sample probabilities, taking the mode of all sample labels, or taking the mode of model-level speaker labels.For even-sized ensembles, tied mode decisions are resolved randomly.Source: §3.2.2 · pp. 8–9, Figure 5 · p. 9

4. Results and Discussion

The experiments select a 50-model probability-averaging ensemble and report relative F1-score improvements over the AVEC baseline, DepAudionet, and a single 1d-CNN. Evaluation and architecture sweeps identify class-specific trade-offs and the selected pooling and layer settings.

4.1. Experimental Protocol

  • Evaluation uses speaker-level F1-score as the main metric and five-fold cross-validation, comparing the proposal with the AVEC SVM baseline and DepAudionet.The baseline uses hand-crafted features; DepAudionet uses 1d-CNN, LSTM, and fully connected layers.Source: §4.1 · p. 10

4.2.1. Results with Different Ensemble Methods

  • F1-score generally increases with ensemble size across the three methods, although Method 3 worsens for depressed speakers at large ensemble sizes; Method 1 with M = 50 is selected.Figure 6 plots F1-score against ensemble size for depressed and non-depressed speakers and the three methods.Source: Figure 6 · p. 11, §4.2.1 · p. 11
  • Prediction variance generally decreases as ensemble size grows, except for Method 3; sample-level ensembles are more robust to individual prediction changes than speaker-level aggregation.The paper attributes this robustness to having considerably more samples than speakers.Source: §4.2.1 · p. 11

4.2.2. Number of Filters in the Layer 1 and Size of the Layer 4

  • Our systems achieve better F1-scores than the baseline and single network, and outperform DepAudionet across accuracy, F1-score, precision, and recall for both classes.Against the baseline, the depressed-class precision is higher despite lower recall; for non-depressed speakers, recall is higher while precision is slightly lower.Source: §4.2.2 · pp. 11–12, Table 1 · p. 12
  • Several filter and neuron combinations reach F1-scores of 0.65 for depressed and 0.80 for non-depressed samples; 128 filters and 128 Layer 4 neurons are chosen for requiring fewer parameters.Figure 7 shows F1-score variation across filter counts and Layer 4 neuron counts.Source: §4.2.2 · p. 12, Figure 7 · p. 12

4.3. Size of the Max-Pooling Window

  • 58.5%, 30.0%, and 10.2% relative F1-score improvements over the baseline, DepAudionet, and single 1d-CNN, respectively, are reported for the selected ensemble.The selected configuration uses 128 filters, a (1, 5) pooling kernel with (1, 4) stride, 128 Layer 4 neurons, and 50 Method 1 machines.Source: §4.3 · p. 13

5. Conclusions

The paper presents voice-based depression detection using an ensemble of 1D-CNNs and reports higher F1-score than the three compared systems. It identifies alternative ensembles, network designs, additional inputs, and visual-audio fusion as future work.

5. Conclusions

  • The system analyzes voice with an ensemble of 50 1d-CNNs using log-spectrogram inputs, and random sampling balances and augments training data.Each network has one input layer, four hidden layers, and one output layer.Source: §5 · p. 13
  • Relative F1-score improvements were 58.5% over the baseline, 30.0% over DepAudionet, and 10.2% over a single 1d-CNN.The evaluation used DAIC-WOZ in the AVEC-2016 Depression Sub-Challenge.Source: §5 · p. 13
  • Future work includes studying bagging or stacking, deeper and narrower networks, metadata inputs, and strategies for combining audio with visual information.Source: §5 · p. 13

Abbreviations

This section lists abbreviations used in the manuscript, spanning depression-related terms, challenge and corpus names, model types, and feature representations.

Abbreviations

  • ADD means Automatic Depression Detection; MDD, Major Depressive Disorder; BDI-II, Beck’s Depression Inventory; GDS, Geriatric Depression Scale; HRSD, Hamilton Rating Scale for Depression; PHQ-8, Patient Health Questionnaire.Source: Abbreviations · p. 14
  • AVEC means Audio–Visual Emotion Challenge; DCC, Depression Classification Sub-Challenge; DAIC-WOZ, Distress Analysis Interview Corpus - Wizard of Oz; WHO, World Health Organization.Source: Abbreviations · p. 14
  • ANN means Artificial Neural Network; CNN, Convolutional Neural Network; 1d-CNN, One-Dimensional Convolutional Neural Network; LSTM, Long Short-Term Memory.Source: Abbreviations · p. 14
  • AAM means Active Appearance Model; FACS, Facial Action Coding System; LPQ-TOP, Local Phase Quantization at Three Orthogonal Planes.Source: Abbreviations · p. 14
  • eGeMAPS means Geneva Minimalistic Acoustic Parameter Set; MFCC, Mel-Frequency Cepstral Coefficients; STFT, Short-Time Fourier Transform; ReLU, Rectified Linear Unit.Source: Abbreviations · p. 14
  • SVM means Support Vector Machine; SVR, Support Vector Regression.Source: Abbreviations · p. 14
Loading 2402.02830v1…