Source-linked AI summary
MODMA dataset: a Multi-modal Open Dataset for Mental-disorder Analysis
Hanshu Cai, Yiwen Gao, Shuting Sun, Na Li, Fuze Tian, Han Xiao, Jianxiu Li, Zhengwu Yang, Xiaowei Li, Qinglin Zhao, Zhenyu Liu, Zhijun Yao, Minqiang Yang, Hong Peng, Jing Zhu, Xiaowei Zhang, Guoping Gao, Fang Zheng, Rui Li, Zhihua Guo, Rong Ma, Jing Yang, Lan Zhang, Xiping Hu, Yumin Li, Bin Hu
TL;DR
High-quality, clinically diagnosed physiological data for mental-disorder analysis are difficult to acquire. This paper presents the MODMA open dataset of EEG and audio recordings from depressed patients and controls, with validation showing significant alpha-rhythm differences between depression and control groups.
Problem
High-quality EEG and audio data from clinically diagnosed patients are difficult to acquire, limiting available evidence for mental-disorder analysis.
Method
The paper constructs a multimodal open dataset of clinically diagnosed depressed patients and controls using 128-electrode and wearable 3-electrode EEG plus audio recordings.
Results
Statistical analysis found significant differences in relative alpha-rhythm power between depression and control groups for the 3-electrode EEG signals.
Takeaways & Limitations
The publicly accessible MODMA repository provides EEG, audio, demographic, and psychological-assessment data for testing mental-disorder analysis methods.
Takeaways & Limitations
Participants were subject to restrictive eligibility criteria, including no psychotropic-drug use during the first two weeks and no other mental illnesses or brain damage.
Abstract
from arXiv · showhide
According to the World Health Organization, the number of mental disorder patients, especially depression patients, has grown rapidly and become a leading contributor to the global burden of disease. However, the present common practice of depression diagnosis is based on interviews and clinical scales carried out by doctors, which is not only labor-consuming but also time-consuming. One important reason is due to the lack of physiological indicators for mental disorders. With the rising of tools such as data mining and artificial intelligence, using physiological data to explore new possible physiological indicators of mental disorder and creating new applications for mental disorder diagnosis has become a new research hot topic. However, good quality physiological data for mental disorder patients are hard to acquire. We present a multi-modal open dataset for mental-disorder analysis. The dataset includes EEG and audio data from clinically depressed patients and matching normal controls. All our patients were carefully diagnosed and selected by professional psychiatrists in hospitals. The EEG dataset includes not only data collected using traditional 128-electrodes mounted elastic cap, but also a novel wearable 3-electrode EEG collector for pervasive applications. The 128-electrodes EEG signals of 53 subjects were recorded as both in resting state and under stimulation; the 3-electrode EEG signals of 55 subjects were recorded in resting state; the audio data of 52 subjects were recorded during interviewing, reading, and picture description. We encourage other researchers in the field to use it for testing their methods of mental-disorder analysis.
Background & Summary · Methods
The paper motivates EEG and audio as accessible physiological modalities for mental-disorder analysis but emphasizes the scarcity of high-quality data from clinically diagnosed patients. It presents a publicly available multimodal dataset of clinically depressed patients and matched controls, with ethically approved participant procedures.
- Background & Summary: EEG is motivated as a non-invasive physiological measure with millisecond temporal resolution that directly reflects postsynaptic potentials.Because depression involves complex brain cognition, EEG is presented as a naturally favored data modality.
- Background & Summary: Audio is motivated as another accessible, non-invasive physiological modality whose characteristics can differ between patients with mental disorders and healthy controls.Prior work cited in the passage used emotion-based and gender-dependent speech features for depression-related analysis.
- Background & Summary: The dataset addresses the difficulty of acquiring high-quality EEG and audio data from patients diagnosed by professional doctors rather than self-rating scales.The paper notes that self-rating scales are less comprehensive than clinical diagnosis for establishing patient status.
- Background & Summary: The dataset contains publicly available EEG and audio recordings from clinically depressed patients and matching normal controls selected by hospital psychiatrists.The contribution is framed as a multimodal open dataset for mental-disorder analysis.
- Background & Summary: The EEG component includes recordings from both a traditional 128-electrode elastic cap and a novel wearable three-electrode collector.The two EEG collection formats support both conventional recording and pervasive applications.
- Methods: All participants provided written informed consent before experimentation, and the study received approval from the Lanzhou University Second Hospital biomedical ethics committee under the Declaration of Helsinki.The methods are described as expanded versions of related work.
Participants · Experimental material
The study recruited clinically diagnosed depression outpatients and healthy controls for 128-electrode EEG, 3-electrode resting-state EEG, and audio experiments. Materials included affective facial stimuli, Chinese speech tasks, standardized word lists, and interview questions, with ethical approval and informed consent.
- Participants: The 128-electrode EEG experiment included 24 depressed outpatients and 29 healthy controls, totaling 53 participants aged 16–56 and 18–55 years, respectively.Depressed participants included 13 males and 11 females; controls included 20 males and 9 females.
- Participants: The 3-electrode EEG experiment included 26 depressed outpatients and 29 healthy controls, totaling 55 participants aged 16–56 and 18–55 years, respectively.Depressed participants included 15 males and 11 females; controls included 19 males and 10 females.
- Participants: The audio experiment included 23 depressed outpatients and 29 healthy controls, totaling 52 participants aged 16–56 and 18–55 years, respectively.Depressed participants included 16 males and 7 females; controls included 20 males and 9 females.
- Participants: Patients were clinically diagnosed at Lanzhou University Second Hospital, controls were recruited by posters, and the study obtained ethics approval and written informed consent.All participants had normal or corrected-to-normal vision.
- Experimental material: The 128-electrode dot-probe task used Chinese Facial Affective Picture System facial images classified as fear, sad, happy, or neutral, with emotional and neutral valences selected.MATLAB standardized mean luminance, contrast, and centro-spatial frequency, then converted images to 8-bit grayscale.
- Experimental material: The 3-electrode EEG experiment recorded only resting-state activity and therefore used no experimental material.
- Experimental material: The audio experiment varied speaking style and emotional valence across interview, word reading, and picture description, using positive, neutral, and negative speech in randomized order.The interview used 18 DSM-IV, HRSD, and other-scale questions; reading used affective word sources [19, 20]; picture description used three CFAPS images.
Experimental equipment
The experiments used standardized EEG and audio-recording equipment under controlled acquisition conditions. Full-brain EEG employed 128 electrodes, pervasive EEG used a three-electrode device, and audio was recorded in a soundproof environment.
- Full-brain 128-electrodes EEG experiment: Full-brain EEG used a 128-channel HydroCel Geodesic Sensor Net with Net Station software, 250 Hz sampling, and Cz referencing.Electrode-net size was selected based on head circumference, and electrode impedance was checked before recording.
- Pervasive 3-electrodes EEG experiment: Pervasive EEG used a three-electrode collection device with a 24-bit A/D converter and 250 Hz sampling.The device is shown in Figure 1.
- Audio experiment: Audio was recorded with Neumann TLM102 microphones and an RME FIREFACE UCX audio card at 44.1 kHz and 24-bit depth, saved as uncompressed WAV files.Recordings occurred in a quiet, clean, soundproof room without electromagnetic interference, with ambient noise below 60 dB.
- Recording procedure: Each participant completed three tasks during an approximately 25-minute session while seated comfortably, kept about 20 cm from the microphone, and refrained from touching equipment.Ambient noise was required to remain below 60 dB to reduce interference with audio signals.
Experimental paradigm
The experimental paradigm comprised a controlled 128-electrode EEG study using resting-state and emotional dot-probe tasks, alongside a 3-electrode resting-state recording protocol. The dot-probe task presented emotional-neutral face pairs and required rapid spatial responses to randomly positioned targets.
- Full brain 128-electrodes EEG experiment: The 128-electrode EEG experiment was conducted in a quiet, sound-proof room, with participants completing resting-state and dot-probe tasks under monitored conditions.Data acquisition began after electrode placement was completed and impedance met requirements.
- Full brain 128-electrodes EEG experiment: The dot-probe task used Fear-Neutral, Sad-Neutral, and Happy-Neutral blocks, each containing 160 trials, with a central fixation cross preceding each trial.The paradigm was programmed in E-prime v2.0, and participants completed 10 practice trials before the formal experiment.
- Full brain 128-electrodes EEG experiment: Participants identified whether the dot appeared left or right of fixation by pressing buttons 1 or 4, with responses accepted for up to 2000 ms.The dot target appeared randomly on either side, followed by a 600-ms black screen between trials.
- Pervasive 3-electrodes EEG experiment: The 3-electrode EEG protocol recorded 90 seconds of resting-state data after participants closed their eyes until their EEG signals stabilized.Recordings were conducted in a room without loud noise or strong magnetic interference.
Audio experiment:
The audio experiment used three fixed-order tasks—interview, reading, and picture description—with materials designed to elicit speech across emotional and clinically relevant contexts. Participants followed on-screen instructions to complete each task.
- Audio experiment:: The experiment comprised fixed-order interview, reading, and picture-description tasks completed according to on-screen instructions.The three-part sequence was used for all participants.
- Audio experiment:: The interview included 18 positive-, neutral-, and negative-meaning questions drawn from DSM-IV topics and commonly used depression scales.Questions addressed experiences, feelings, relationships, self-evaluation, and desired activities.
- Audio experiment:: The reading task combined “The North Wind and the Sun” with positive, neutral, and negative emotion words for acoustic analysis.The story came from the International Phonetic Association booklet, while affective words included examples such as “happy,” “center,” and “depression.”
- Audio experiment:: The picture-description task presented four images, including positive, neutral, and negative faces plus a crying woman, which participants described freely.The face images came from CFAPS, and the crying-woman image came from TAT.
Data Records · 1. Data recording and storage
The dataset records multimodal EEG and audio experiments using standardized acquisition settings, task-based organization, and explicit file formats and labels. Its 128-electrode, three-electrode, and audio recordings were manually organized for subject-level analysis.
- 1. Data recording and storage: Five minutes of eyes-closed resting-state EEG were recorded, saved as .mff files, and converted into analysis files using Net Station Waveform Tools.The passages report conversion to .mat files for one recording description and .raw files for another, while retaining trigger timestamps where specified.
- 1. Data recording and storage: The 128-electrode EEG system used 128 Ag/AgCl electrodes, impedance below 50 kΩ, 250 Hz digitization, and Net Station 4.5.4 acquisition.Signals were collected with a wired HydroCel Geodesic Sensor Net and Electrical Geodesics amplifiers.
- Data Records: The dataset stores recordings by task, with one file per subject in “128-channel_RestingState” and “128-channel_DotProbe” folders.The 128-electrode EEG files distinguish MDD patients with the “0201” prefix and normal controls with the “0203” prefix.
- 1. Data recording and storage: The pervasive EEG device placed three electrodes on the prefrontal lobe at Fp1, Fpz, and Fp2.The electrode locations and device are shown in Fig. 3.
- 1. Data recording and storage: Three-electrode EEG files use a referential-montage TXT format containing an M × N array, with M = 8 channels and the first three channels representing Fp1, Fpz, and Fp2.The remaining five channels are alternate channels whose default values are used when they are not active.
- 1. Data recording and storage: Audio was recorded in a quiet, soundproof, electromagnetically interference-free room with ambient noise below 60 dB using Neumann TLM102 microphones and an RME FIREFACE UCX audio card.Recordings used a 44.1 kHz sampling rate, 24-bit depth, and uncompressed WAV format.
- 1. Data recording and storage: Audio recordings were manually segmented and labeled, retaining only participant speech, with 29 recordings per subject across interviews, reading, and picture description.The 29 recordings comprised 18 interviews, 1 passage reading, 6 word readings, and 4 picture descriptions.
2. 3-electrodes EEG signals
The 3-electrode EEG signals were converted, filtered, and denoised to support real-time waveform display. Preprocessing used FIR band-pass filtering and adaptive cancellation of eyeblink artifacts.
- 2. 3-electrodes EEG signals: Raw hex data from the first three columns were converted to decimal before EEG preprocessing.
- 2. 3-electrodes EEG signals: Signals were filtered with a 1Hz high-pass and 45Hz low-pass FIR filter, then processed using an adaptive noise canceller to remove eyeblink artifacts.
- 2. 3-electrodes EEG signals: Figure 4 presents a real-time graphic display of the EEG waveforms after preprocessing.
3. Whole-brain EEG signals
The dataset’s whole-brain EEG recordings include 128-channel data with event triggers for task epochs and a pervasive three-electrode protocol using forehead placements. Participant screening and preparation included depression assessments, psychotropic-medication restrictions, and standardized electrode procedures.
- Whole-brain EEG acquisition: The 128-electrode recordings used Ag/AgCl electrodes in an elastic cap, conductive gel, repeated impedance calibration below 50 kΩ, and Electrical Geodesics amplification.The supplied passage states that the signals were recorded at a sampling rate, but the value is truncated.
- Pervasive 3-electrodes EEG experiment: Participants in the pervasive EEG experiment underwent MMSE screening, followed by PHQ-9 assessment when they were judged at high risk of depression.Basic participant information was collected alongside these assessments.
- Pervasive 3-electrodes EEG experiment: Candidates were selected against experimental criteria, washed their hair before recording, and were required to avoid psychotropic drugs and other mental illnesses during the first two weeks.Participants wore the experimental equipment in a controlled experimental environment.
- Pervasive 3-electrodes EEG experiment: The pervasive protocol placed three electrodes on the forehead according to international 10-20 system standards.The three selected positions were on the forehead.
2. 3-electrodes EEG signals validation
The validation used filtered FP1 and FP2 EEG signals to compare relative power across four rhythms in depression and control groups. Alpha-rhythm relative power differed significantly in the depression group but not in controls.
- Validation method: FP1 and FP2 EEG leads from 11 depressed and 11 control participants were filtered at 0.5–30 Hz, and relative power was calculated for δ, θ, α, and β rhythms.The bands were δ (0.5–4 Hz), θ (4–8 Hz), α (8–14 Hz), and β (14–30 Hz).
- Validation results: The depression group showed statistically significant differences in left- and right-brain α-rhythm relative power.Paired-sample t-tests were conducted using SPSS 19.0.
- Validation results: The control group’s α-rhythm relative-power difference between left and right brain signals was not statistically significant.
3. Whole-brain EEG signals validation
The MODMA dataset stores raw whole-brain EEG data from both resting-state and dot-probe tasks.
- 3. Whole-brain EEG signals validation: Raw EEG data were collected for both the resting-state and dot-probe tasks.
- 3. Whole-brain EEG signals validation: The resting-state EEG recordings were saved in the MODMA dataset.
- 3. Whole-brain EEG signals validation: The dot-probe EEG recordings were also saved in the MODMA dataset.
4. Audio validation · Usage Notes
Audio recordings were collected under controlled conditions with noise rejection and sound quality characterized by an SNR of 20–30. The raw EEG and Audio datasets are freely downloadable after EULA acceptance, with accompanying demographic and psychological assessment files.
- 4. Audio validation: Audio experiments took place in a quiet, clean, soundproof room without electromagnetic interference.
- 4. Audio validation: Each audio recording followed one minute of rest, a computer-displayed task, and another one-minute break.
- 4. Audio validation: Recordings were not collected when environmental noise exceeded 60 dB.
- 4. Audio validation: The recordings’ informal sound-quality indication corresponds roughly to the signal-to-noise ratio.The SNR ranges from 20 to 30 and can be calculated normally.
- Usage Notes: Raw experimental data are freely downloadable from the publicly accessible MODMA repository after users sign an End User License Agreement.
- Usage Notes: The raw datasets are packaged separately under “EEG” and “Audio” categories.Each package includes an Excel file containing demographic data and psychological assessment scores for its subjects.