Source-linked AI summary
Automatic Depression Detection: An Emotional Audio-Textual Corpus and a GRU/BiLSTM-based Model
Ying Shen, Huiyu Yang, Lin Lin
TL;DR
Depression detection needs accessible approaches because depression is widespread and conventional diagnosis can be hindered by treatment burdens and concealed mental states. The paper releases a Chinese audio-text corpus and proposes GRU/BiLSTM-based multimodal detection using interview speech and linguistic content. Across DAIC-WoZ and EATD-Corpus, the method reports strong performance and generalization, including a fused F1 score of 0.85 on DAIC-WoZ.
Problem
Depression is widespread and diagnosis can be limited by costly conventional treatment and patients hiding their mental states during interviews.
Method
The paper releases EATD-Corpus and uses GRU and attention-based BiLSTM models to summarize audio and text representations for multimodal fusion.
Results
The method achieves the best reported fused F1 score of 0.85 on DAIC-WoZ and the best single-modality results on EATD-Corpus, with F1 scores of 0.66 for audio and 0.65 for text.
Takeaways & Limitations
Results from DAIC-WoZ and EATD-Corpus indicate that the method is effective and generalizes across different depression datasets.
Abstract
from arXiv · showhide
Depression is a global mental health problem, the worst case of which can lead to suicide. An automatic depression detection system provides great help in facilitating depression self-assessment and improving diagnostic accuracy. In this work, we propose a novel depression detection approach utilizing speech characteristics and linguistic contents from participants' interviews. In addition, we establish an Emotional Audio-Textual Depression Corpus (EATD-Corpus) which contains audios and extracted transcripts of responses from depressed and non-depressed volunteers. To the best of our knowledge, EATD-Corpus is the first and only public depression dataset that contains audio and text data in Chinese. Evaluated on two depression datasets, the proposed method achieves the state-of-the-art performances. The outperforming results demonstrate the effectiveness and generalization ability of the proposed method. The source code and EATD-Corpus are available at https://github.com/speechandlanguageprocessing/ICASSP2022-Depression.
1. INTRODUCTION
Depression is widespread and can have severe consequences, while conventional diagnosis and treatment can be costly, time-consuming, ineffective, and hindered by concealed mental states. Automatic detection may support private self-assessment and assist psychologists when patients hide their states.
- Depression affects about 264 million people worldwide and can lead to suicide in the worst case.
- Traditional depression treatment is time-consuming, costly, and sometimes ineffective, creating burdens for people with financial difficulties.
- Patients may conceal their real mental states during clinical interviews because of fear of prejudice or discrimination, hindering correct diagnosis.
- Automatic depression detection can help individuals assess depressive states privately and assist psychologists when patients hide their mental states.
2. RELATED WORK AND OUR CONTRIBUTIONS
Existing depression datasets and methods are limited, motivating a public Chinese audio-text dataset and a multimodal model that summarizes and fuses speech and linguistic representations.
- Related Work: Earlier detection studies manually selected depression-related questions or topics before extracting text features for classification.
- Related Work: Deep-learning approaches began integrating multimodal features, including audio, video, and text, for depression detection.
- Our Contributions: EATD-Corpus is established as a publicly available Chinese depression dataset containing audios and text transcripts from interviews with 162 volunteers.
- Our Contributions: The proposed method uses GRU and attention-based BiLSTM models to summarize audio and text features, then applies multimodal fusion for depression detection.
3. EATD-CORPUS
EATD-Corpus addresses scarce public depression data by providing Chinese interview audios and transcripts from 162 student volunteers, collected with SDS assessments and manually corrected transcription preprocessing.
- Public depression datasets are scarce, with DAIC-WoZ providing recordings and transcripts and AViD-Corpus providing audio and video without author-provided transcripts.
- EATD-Corpus contains interviews from 162 Tongji University student volunteers, including three randomly selected questions and an SDS questionnaire.
- The corpus includes 30 depressed and 132 non-depressed volunteers according to the Chinese index SDS criterion of at least 53.
- The collected response audios have an overall duration of about 2.26 hours.
- Corpus construction comprises data collection and preprocessing, including virtual-interviewer recordings, audio cleaning, noise removal, transcript extraction, and manual transcript correction.
4. A MULTI-MODAL DEPRESSION DETECTION METHOD
The method combines text and audio representations using attention-based BiLSTM and GRU models, then performs weighted multimodal fusion for binary depression prediction.
- Feature Extraction: Text features are projected into sentence embeddings, while variable-length audio Mel spectrograms are converted into fixed-length embeddings with NetVLAD.ELMo produces the text embeddings; NetVLAD addresses audio-duration variation before recurrent processing.
- Text Modeling: A two-layer BiLSTM with an attention layer summarizes text features by emphasizing sentences contributing most to depression detection.The resulting representation is passed to a two-layer fully connected network for binary prediction.
- Audio Modeling: A two-layer GRU summarizes audio embeddings and feeds them to a two-layer fully connected network that predicts depression presence.The GRU processes the audio representations produced from the fixed-length audio embeddings.
- Multimodal Fusion: The final representations from the GRU and BiLSTM are concatenated, weighted by modal attention, and passed through a fully connected network to generate binary labels.Modal attention learns the relative importance of the audio and text modalities.
5. EXPERIMENTS AND RESULTS
The experiments address class imbalance and evaluate audio, text, and fused representations on DAIC-WoZ and EATD-Corpus. The proposed models achieve strong single-modality and fusion results, with fusion improving performance on both datasets.
- Data Imbalance: Data imbalance is addressed through resampling before training, including group resampling for DAIC-WoZ and response rearrangement for EATD-Corpus.For EATD-Corpus, rearranging three responses provides six training-sample orderings per volunteer.
- Feature Extraction and Model: The system extracts ELMo-based text embeddings and NetVLAD-based audio embeddings before GRU/BiLSTM modeling and modal-attention fusion.Text representations are 128-dimensional and audio representations are 256-dimensional before concatenation.
- Performance on DAIC-WoZ: 0.85 F1 score is achieved by the proposed multi-modal fusion method on DAIC-WoZ, outperforming the reported audio-only and text-only comparisons.The proposed GRU reaches 0.77 F1 with audio features, while the proposed BiLSTM reaches 0.83 F1 with text features.
- Performance on EATD-Corpus: 0.71 F1 score is achieved by the fusion model on EATD-Corpus, compared with 0.66 for the proposed audio-only model and 0.65 for its text-only model.The fusion model's Recall reaches 0.84, and the evaluation uses 3-fold cross validation.
- Generalization: Results on DAIC-WoZ and EATD-Corpus are reported as indicating powerful generalization ability across different depression datasets.The authors state that the method can be applied to different depression datasets.
6. CONCLUSION
The paper releases a public Chinese audio-text depression corpus and proposes a detection method based on participants' audio signals and linguistic contents. Evaluation on two depression datasets demonstrates the method's effectiveness.
- Dataset: EATD-Corpus is released as a public Chinese depression dataset containing audio responses and manually corrected text transcripts from 162 volunteers.The corpus is presented as valuable data for depression research in psychology and computer science.
- Detection Method: The proposed method detects depression by analyzing participants' audio signals and linguistic contents without relying on interview-question content.The method is evaluated on DAIC-WoZ and EATD-Corpus, and the authors intend to build an app for self-detection in future work.