PaperlayerView exact revision on arXiv

Source-linked AI summary

VAANI Noise Event Dataset: A curated spontaneous speech dataset annotated with timestamps for noise events

Pavan Kumar J, Agneedh Basu, Pranav Bhat, Sujith Pulikodan, Suryansh Shukla, Nihar Desai Prasanta K. Ghosh

arXiv:2609.02474v1eess.AS

TL;DR

Public corpora provide limited combinations of realistic speech, diverse Indian acoustic conditions, and timestamped overlapping noise events, motivating VAANI. The paper introduces a derived annotation layer over spontaneous Project VAANI recordings, with broad release coverage and staged quality control. VAANI therefore offers an auditable resource for noise-robust ASR, sound event detection, and speech enhancement research.

  • Problem

    Existing resources rarely combine naturally co-occurring speech and noise, overlapping span-level noise timestamps, and spontaneous multilingual speech.

    Source: §I · p. 1, E. Positioning · p. 2

  • Method

    VAANI adds seven-class, overlapping start/end noise-event annotations to spontaneous Project VAANI speech recordings and applies staged sanity checks, re-timestamping, and independent auditing.

    Source: §I · p. 1, §IV · p. 4, §V · p. 5

  • Results

    72,756 release-quality segments totaling 122.17 hours cover 38,541 speakers, 58 languages, 30 states, and 162 districts.

    Source: §III · p. 4

  • Takeaways & Limitations

    VAANI provides an auditable resource for noise-robust ASR, sound event detection, and speech enhancement in authentic Indian acoustic conditions.

    Source: §V · p. 5

Abstract

from arXiv · show

Most public sound-event corpora are optimized either for general audio tagging or for clean speech separation, and comparatively few provide strong timestamped noise annotations layered directly on top of spontaneous, real-world speech. We present the VAANI Noise Event Timestamp Dataset, a derived annotation layer built on Project VAANI field recordings of spontaneous speech collected across 165 Indian districts in 105 languages. Unlike synthetically mixed corpora, VAANI captures speech and ambient noise in situ and simultaneously, and annotates each recording with fine-grained start/end timestamps for overlapping background noise events organized into a compact seven-class semantic taxonomy: animal, traffic, baby/child, music, signal/alarm, appliance, and non-speech human. This combination of spontaneous multilingual Indic speech, authentic regional soundscapes, and span-level noise tags that may overlap with speech targets tasks that existing datasets address only partially: noise-robust Automatic Speech Recognition (ASR), sound event detection (SED), and speech enhancement. We position VAANI against nine widely used corpora and benchmarks, including WHAM!, AVA-Speech, MUSAN, FSD50K, CHiME-6, AudioSet, DESED, the India-specific iNoise noise database, and the Kathbath-Noisy noisy-ASR benchmarks, and describe the annotation protocol and quality-control procedure used to produce the timestamped tags.

I. INTRODUCTION

Real-world Indian speech contains diverse, non-stationary background events whose timing and overlap affect recognition and enhancement. VAANI addresses this need with timestamped noise annotations over spontaneous speech for noise-robust ASR, sound event detection, and speech enhancement.

  • Indian speech recordings commonly contain traffic, animals, children, appliances, music, alarms, and human non-speech sounds.Source: §I · p. 1
  • Noise-event onset, duration, and overlap with speech influence recognition errors and perceived enhancement quality.Source: §I · p. 1
  • Existing corpora cover different combinations of synthetic mixing, clip-level labels, isolated noise, domain mismatch, or India-specific noisy speech.Source: §I · p. 1
  • VAANI adds exact start/end timestamps for background events to spontaneous Project VAANI recordings alongside transcripts.Source: §I · p. 1

II. RELATED DATASETS

The reviewed resources address separate aspects of noisy speech and sound-event research, but differ in recording realism, speech content, annotation granularity, and construction. VAANI is positioned against these complementary design choices.

  • Table I compares nine resources by recording method, speech content, noise annotation type, and scale.Source: §II · p. 1
  • In-situ audio records speech and background noise together in the field rather than mixing separately recorded signals afterward.Source: §II · p. 2
  • Span-level annotation records each noise event’s exact start and end timestamps instead of only clip-level or frame-level labels.Source: §II · p. 2
  • WHAM! uses clean read speech with separately recorded urban noise mixed at controlled signal-to-noise ratios.Source: A. Synthetically mixed corpora · p. 2

B. Frame- and clip-level general-audio corpora

General-audio and spontaneous-speech corpora provide useful labels or realistic recordings, but they do not jointly offer VAANI’s event-level timing and target setting. Their limitations differ by corpus.

D. Indian noise and noisy-ASR resources

India-specific resources address environmental or noisy-speech conditions, but their labels do not provide timestamped noise events co-occurring with spontaneous speech. VAANI targets that remaining annotation gap.

E. Positioning

Across nine corpora and benchmarks, no single resource combines in-situ speech–noise co-occurrence, overlapping span-level noise timestamps, and spontaneous multilingual speech from Indian mobile-device recordings.

  • No single compared resource combines real in-situ speech–noise co-occurrence, overlapping event timestamps, and spontaneous multilingual Indian field speech.Source: E. Positioning · pp. 2, 4
  • The comparison includes nine resources spanning synthetic mixtures, clip-level labels, noise-only recordings, and noisy-ASR benchmarks.Source: E. Positioning · p. 2, Table I · p. 3
  • VAANI addresses this combination while Table I positions it against representative noise and sound-event corpora.Source: Table I · p. 3, E. Positioning · p. 4

III. DATASET DESCRIPTION

The release-quality dataset contains 72,756 short speech segments with transcripts and timestamped background-noise events, totaling 122.17 hours from 38,541 speakers across India.

  • 72,756 speech segments total 122.17 hours and come from 38,541 distinct speakers.Source: §III · p. 4
  • Each short utterance carries a speech transcript and one or more timestamped background-noise events.Segment durations range from 0.79 to 23.49 seconds, with a 6.05-second mean.Source: §III · p. 4
  • The release-quality subset excludes structural-issue, held-out evaluation, and synthetic-data segments, retaining verified_timestamps and unverified_timestamps tiers.Source: §III · p. 4
  • Coverage spans 58 languages, 30 states, and 162 districts of India.Source: §III · p. 4

A. Linguistic and geographic coverage

The corpus provides broad linguistic and geographic coverage, combining Hindi-dominant data with a long tail of Indian languages and recordings distributed across diverse regional environments.

B. Noise taxonomy and annotation format

VAANI represents noise at both segment and event levels, using seven top-level categories and exact timestamps for overlapping events within spontaneous speech recordings.

IV. ANNOTATION AND QUALITY CONTROL

VAANI uses staged annotation and quality control to produce timestamped noise-event labels, separating a broad unverified tier from a smaller, independently audited verified tier.

Annotation pipeline

  • Roughly 150+ hours of spontaneous-speech segments are sampled for noise-event annotation by trained freelancers marking each audible event’s start, end, and category.Source: §IV · p. 4
  • A structural sanity check rejects failed freelancer batches and releases about 100 hours as unverified_timestamps while forwarding at least 20 hours for verification.Source: §IV · p. 4

Quality control

  • An internal team re-timestamps the candidate subset using the unverified annotations as reference before independent auditing.Source: §IV · p. 4
  • A second reviewer audits a random 10% sample, and any disagreement sends the batch back for rework before verified_timestamps release.Source: §IV · p. 4

Quality distinction

  • Both tiers contain fully timestamped events, but the staged sanity gate, internal re-timestamping, and audit give verified_timestamps higher trust.Source: §IV · p. 4

V. CONCLUSION

VAANI concludes by presenting a large, geographically and linguistically broad release-quality dataset that pairs authentic multilingual speech with overlapping timestamped noise events and auditable quality tiers.

Conclusion

  • VAANI contributes a span-level annotation layer over in-situ spontaneous multilingual Indic speech, unlike synthetic mixtures or clip-level noise-label corpora.Source: §V · pp. 4–5
  • The release-quality subset contains 72,756 segments totaling 122.17 hours from 38,541 speakers across 58 languages, 30 states, and 162 districts.Source: §V · p. 5
  • The dataset organizes naturally co-occurring background noise into seven classes with exact, overlapping start/end timestamps.Source: §V · pp. 4–5
  • Non-speech human, animal, and vehicle/traffic events dominate segment coverage, while animal and vehicle/traffic events contribute most annotated noise duration.Source: §V · p. 5
  • The explicit distinction between verified_timestamps and unverified_timestamps provides downstream users with an auditable quality choice for ASR, sound event detection, and speech enhancement research.Source: §V · p. 5
Loading 2609.02474v1…