Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition

1. INTRODUCTION

• Vocal-sound recognition can help automatic speech recognition transcribe non-speech vocalizations and support monitoring of health-related sounds.Coughing and sneezing density and intensity could serve as indicators of group health.Source: §1 · p. 1
• Existing datasets have relatively few human vocal-sound samples or potentially noisy labels, and state-of-the-art models achieve only around 0.5 average precision on AudioSet cough and sneeze classes.The cited potential causes include limited vocal-sound samples and annotation challenges with large sound vocabularies.Source: §1 · p. 1
• VocalSound contains over 21,000 crowdsourced recordings of six vocal sounds from 3,365 speakers across 60 countries.The classes are laughter, sighs, coughs, throat clearing, sneezes, and sniffs.Source: §1 · p. 1
• Adding VocalSound to an existing dataset significantly improved recognition performance on real vocal sounds by over 41.9%.The dataset includes speaker age, gender, native language, country, and health-condition metadata.Source: §1 · p. 1

2. RELATED WORK

• AudioSet and FSD50K have relatively few vocal-sound samples, imbalanced classes, and no speaker information such as age, gender, or native language.AudioSet labels may also be noisy because annotating YouTube videos across a large sound vocabulary is difficult [10].Source: §2 · p. 1
• Figure 1 presents the speaker age, country, and native-language distributions.The supplied caption identifies age on the left, country in the center, and native language on the right.Source: Figure 1 · p. 2
• Table 1 compares the data volumes of VocalSound and existing datasets.The supplied caption identifies the comparison but does not provide the table values.Source: Table 1 · p. 2
• Existing imitated vocal-sound datasets are much smaller than VocalSound.Source: §2 · p. 2
• VocalSound is class-balanced, has more vocal-sound samples from many speakers, and includes gender, age, language, country, and health metadata.These metadata may support analyses of group differences and research on health classification or speaker re-identification.Source: §2 · p. 2

3. VOCALSOUND DATA COLLECTION

• Each Human Intelligence Task collects gender, age, country, native language, and health information, then asks the subject to record six vocalizations.The six recordings are laughter, sighing, coughing, throat clearing, sneezing, and sniffing; participation is anonymous and no personally identifiable information is collected.Source: §3 · p. 2
• Recordings must exceed two seconds, contain no speech, and match the assigned class under the collection checks.Google Speech API helps screen for speech, and a model from is used with a low threshold to exclude obvious mismatches; manual review judged about 96% of 600 samples high quality.Source: §3 · p. 2

4. DATA DISTRIBUTION

• The 3,365 subjects are 45% female and 55% male, range from 18 to 80 years old, and come from 60 countries.Most are aged 20–40, but 321 are older than 50; 4.0% report health-related symptoms that might affect speech during collection.Source: §4 · p. 2
• The speaker-independent split contains 15,570 training samples, 1,860 validation samples, and 3,594 evaluation samples.One sample per evaluation-set speaker was manually checked, and low-quality recordings were removed.Source: §4 · p. 2
• The classifier converts waveforms to 128-dimensional log Mel filterbank features, processes them with EfficientNet-B0, and pools the outputs into class predictions.The features use a 25 ms Hanning window every 10 ms; frequency mean pooling precedes 1×1 convolutional filters, sigmoid activation, and temporal mean pooling.Source: Figure 2 · p. 3

5. BASELINE EXPERIMENTS

• The experiments use an EfficientNet-B0 audio classifier, with the two training sets compared under the same model setting.The experiments use Adam, cross-entropy loss, SpecAugment, and three runs, reporting mean and standard deviation.Source: §5 · p. 3

5.1. Six-class Vocal Sound Classification

• 90.5±0.2% evaluation accuracy shows that VocalSound can train a six-class vocal sound classifier.Accuracy varies across age groups (91.5±0.3%, 90.1±0.2%, and 90.9±1.6%) and male and female subjects (89.2±0.5% and 91.9±0.1%), motivating future research on model bias.Source: §5.1 · p. 3, Table 2 · p. 3

5.2. Vocal Sound Recognition from Background Sounds

• For background-sound recognition, FSD50K samples outside laughter, sigh, cough, and sneeze are relabeled as a single background class.The resulting task has four vocal-sound classes plus background; the FSD50K multi-label samples with multiple vocal labels are assigned one randomly selected label.Source: §5.2 · p. 4
• 31.8% relative F1-score improvement and 41.9% average precision improvement result from adding VocalSound to FSD50K training, with statistically significant gains on the FSD50K evaluation set.The comparison uses real sounds independent of VocalSound; the confusion matrices also show improved precision for vocal sound classes.Source: §5.2 · p. 4, Table 3 · p. 4, Figure 3 · p. 4

• VocalSound contains over 21,000 recordings of laughter, sighs, coughs, throat clearing, sneezes, and sniffs.Source: §6 · p. 4
• Compared with existing generic audio event datasets, VocalSound has more vocal sound samples and richer speaker information.Source: §6 · p. 4
• Experiments show that VocalSound can noticeably improve vocal sound recognition performance.Source: §6 · p. 4