PaperlayerView exact revision on arXiv

Source-linked AI summary

Building and Evaluation of a Real Room Impulse Response Dataset

Igor Szoke, Miroslav Skacel, Ladislav Mosner, Jakub Paliesek, Jan "Honza" Cernocky

arXiv:1811.06795v2eess.AS

TL;DR

The paper addresses the difficulty of obtaining varied real acoustic data for robust far-field ASR and introduces BUT ReverbDB as a public resource. It describes how the dataset is collected and evaluates real and artificial RIR augmentation, finding that carefully selected real RIRs perform comparably to many artificial RIRs and that combining both is beneficial.

  • Problem

    Far-field ASR requires data spanning diverse rooms, microphones, placements, and noise conditions, but collecting sufficient transcribed target-domain data and real RIRs is difficult.

  • Method

    The paper constructs and documents BUT ReverbDB, including RIRs, environmental noises, retransmitted speech, metadata, and ASR experiments comparing real and artificial RIR augmentation.

  • Results

    Carefully selected real RIRs achieved better augmentation results than many randomly selected RIRs, while real and artificial RIRs were complementary; artificial retransmission could also match real retransmitted-data WAC.

  • Takeaways & Limitations

    BUT ReverbDB provides a free, non-restrictively licensed resource with data from 8 rooms and unusual microphone positions for ASR and SRE development and testing.

Abstract

from arXiv · show

This paper presents BUT ReverbDB - a dataset of real room impulse responses (RIR), background noises and re-transmitted speech data. The retransmitted data includes LibriSpeech test-clean, 2000 HUB5 English evaluation and part of 2010 NIST Speaker Recognition Evaluation datasets. We provide a detailed description of RIR collection (hardware, software, post-processing) that can serve as a "cook-book" for similar efforts. We also validate BUT ReverbDB in two sets of automatic speech recognition (ASR) experiments and draw conclusions for augmenting ASR training data with real and artificially generated RIRs. We show that a limited number of real RIRs, carefully selected to match the target environment, provide results comparable to a large number of artificially generated RIRs, and that both sets can be combined to achieve the best ASR results. The dataset is distributed for free under a non-restrictive license and it currently contains data from 8 rooms, which is growing. The distribution package also contains a Kaldi-based recipe for augmenting publicly available AMI close-talk meeting data and test the results on an AMI single distant microphone set, allowing it to reproduce our experiments.

I. INTRODUCTION

Far-field ASR remains less robust than close-talk ASR because target-domain transcription is impractical across diverse rooms, microphones, placements, and noise conditions. The paper introduces BUT ReverbDB to support augmentation and evaluation with real acoustic data.

  • Far-field ASR is substantially less advanced than close-talk ASR, motivating research on performance and robustness.
  • Target-domain transcription is impractical for far-field ASR because acoustic conditions vary across rooms, microphones, placements, and noise environments.Data augmentation therefore uses estimated or artificially generated RIRs together with real noises.
  • Real RIR collection is technically difficult and time-consuming, while artificial RIRs are scalable but less supported for partly or fully hidden microphones.For such microphone configurations, estimating real impulse responses is described as the only available approach.
  • Existing datasets lack parallel clean, reverberated, and noised speech across environments, limiting reuse for enhancement, denoising, dereverberation, and beamforming.
  • The paper evaluates augmentation with real versus artificial RIRs when no target training data are available for ASR development.
  • BUT ReverbDB was created to provide RIRs, environmental noises, retransmitted speech, and metadata for ASR and SRE development and testing.The dataset includes data from 8 rooms, 31 microphones per room, and multiple loudspeaker positions, with retransmitted LibriSpeech, HUB5, and NIST SRE material.

II. RELATED WORK

Prior work shows that distant-microphone ASR is sensitive to environmental mismatch and that existing datasets often lack publicly available RIRs, retransmitted public speech, metadata, or non-cooperative microphone placements. These gaps motivate a broader RIR resource for varied acoustic environments.

  • Distant-microphone ASR performance degrades under mismatches in rooms, microphones, recording conditions, noise, and speaker characteristics.Collecting and transcribing data for every new condition is costly or infeasible.
  • Room and microphone variability can strongly affect robustness, and speaker-microphone orientation matters beyond source-to-microphone distance.
  • Prior studies report that real RIR augmentation can outperform artificial RIR augmentation, while adding point-source noises may reduce the gap between them.
  • Existing datasets cover speech enhancement, ASR, sound scenes, and smart-home scenarios, but differ in rooms, microphone configurations, tasks, and public availability.
  • Many smart-home and related datasets provide limited or no public RIR access, including DIRHA and VoiceHome.
  • Across the surveyed datasets, publicly retransmitted speech, non-cooperative microphone placements, and precise placement metadata remain important gaps.

III. OBTAINING ROOM IMPULSE RESPONSES

Room impulse responses can be obtained either by measuring the physical environment or by generating them through simulation.

  • An RIR is obtained either by measuring the environment to produce a real response or by generating one artificially through simulation.

A. Real room impulse responses

The paper reviews MLS, IRS, TSP, and ESS methods for measuring real RIRs, emphasizing their trade-offs in distortion, noise immunity, synchronization, and loudspeaker-level robustness. It selects ESS for most experiments while compensating MLS clock mismatch when needed.

  • MLS uses a periodic pseudo-random excitation and circular cross-correlation to estimate the RIR, with immunity to uncorrelated disturbances but sensitivity to time aliasing and nonlinear distortion.Its sequence length must exceed the expected RIR duration, and averaging can reduce distortions.
  • IRS reduces MLS distortion peaks, while TSP reduces them through pulse expansion and compression but cannot be used in occupied rooms because it is not noise-immune.
  • ESS uses an exponential frequency sweep, avoids the MLS linear-time-invariant assumption, and separates the linear response from harmonic distortion.
  • ESS remains robust to changing loudspeaker output levels, whereas MLS and LSS can degrade ASR WER at higher volumes.The cited comparison also reports only 0.5% WER deterioration when switching from studio monitors to cheap PC loudspeakers.
  • The study chose ESS for subsequent experiments because it is insensitive to output calibration, while MLS was retained as an optional recording method after clock compensation.
  • Compensated MLS and ESS produced an average word-accuracy difference of only 0.37% absolute across RIRs estimated for 31 microphones in two rooms.

C. Artificial room impulse responses

Artificial RIR generation uses computational room-acoustics methods, with ray-based approaches favored for efficiently simulating many conditions. The paper uses an established generator while documenting the dataset’s room coverage and collection scope.

  • Wave-based methods model the wave equation but are computationally costly when numerous room conditions must be simulated.
  • Ray-based methods model sound as reflected rays whose attenuated energies are used to construct the RIR.
  • The Image Source Method unfolds room geometry into virtual sources and sums delayed, filtered, attenuated impulses.
  • The study uses Habets’ artificial RIR generator, which supports wall reflection coefficients and source orientation and directionality settings.
  • 8 rooms had been measured, with RIR counts determined by microphones multiplied by loudspeaker positions; speech retransmission covered only selected positions because it was time consuming.
  • The dataset collection was planned to expand toward about 50 indoor environments, including cars, with additional microphone and device types.

V. ASR TEST DATA EXPERIMENTS

The experiments evaluate artificial retransmission against real retransmission using ASR test data, examining RIR estimation, background noise, and microphone occlusion. Artificial retransmission can match real-data word accuracy when the acoustic conditions are modeled appropriately.

  • Experimental setup: Artificial retransmission by convolving test data with estimated or generated RIRs can achieve the same word accuracy as real retransmission.The comparison uses ESS-estimated and ISM-generated RIRs, with clean speech as the retransmission source.
  • Experimental setup: Adding background noise is helpful for ASR test-data augmentation.The noise is added using room- and microphone-matched conditions.
  • Experimental setup: Microphone occlusion makes real RIRs clearly superior to ISM-generated RIRs.ISM is adequate for non-occluded microphone placement, but hidden microphones remain difficult to simulate.
  • Experimental setup: The ASR recognizer is a pretrained Czech stacked-bottleneck system trained on 3900 hours without test-data adaptation.The training data combines telephone, close-talk, distant-microphone, and augmented speech.
  • Experimental setup: The test data uses clean close-talk speech from 39 balanced speakers and has 75.9% baseline word accuracy.The source consists of 92 minutes of prompted and phonetically balanced speech.

A. Simulated (ISM) vs. real (ESS) RIRs

The comparison shows that ESS-estimated real-room RIRs better reproduce real retransmission than ISM simulation, especially when microphones occupy unusual or occluded positions. Matching background noise nearly closes the real-versus-ESS gap.

  • RIR comparison: ESS produces word accuracies closer to real retransmission than ISM does.The comparison covers RIRs from rooms L207 and Q301.
  • Noise influence: Adding matching room-and-microphone noise almost eliminates the gap between real retransmission and ESS artificial retransmission.A remaining gap between ESS and ISM indicates that artificial RIR estimation is less accurate for uncommon microphone positions.
  • Microphone occlusion: ISM degrades more strongly than ESS for occluded microphones.The analysis measures WAC differences between real-retransmitted and artificial-retransmitted test sets.
  • RIR comparison: The experiments compare artificial and real RIRs using selected rooms, microphone-distance tags, visibility, and face-to-face orientation.The real-RIR set includes four rooms selected for similarity to AMI meeting rooms, while artificial RIRs are generated for comparison.
  • Training augmentation: Training augmentation reverberates IHM audio with selected RIRs and adds stationary noise at uniformly distributed SNRs from 10 to 20dB.RIRs can be fixed for a whole file or changed during convolution.

A. Baseline system description

The AMI baseline uses a standard Kaldi acoustic-model pipeline, with fixed baseline segmentation for fair comparisons and repeated runs to address unstable WER. Per-segment RIR changes improve distant-microphone recognition.

  • Baseline pipeline: The baseline extracts MFCC, delta, and double-delta features, applies CMVN, and trains GMM-HMM and context-dependent triphone models.The pipeline follows the standard AMI Kaldi recipe.
  • Baseline pipeline: Baseline segmentation is frozen across experiments, preserving the same 210 hours of training data.Using reverberated data for cleaning and segmentation reduced accuracy and changed the retained audio amount.
  • Evaluation protocol: AMI WER fluctuates across runs, so reported results are averages over 5 ASR-training runs.The instability persists despite adding more neural-network training iterations.
  • Per segment reverberation: Changing RIRs only during silences longer than 3 seconds further reduces SDM dev/eval WER to 41.70%/46.74%.This setup stabilizes the channel during i-Vector extraction and avoids convolution artifacts.

E. Room impulse response passivation and delay compensation

The paper evaluates passivation and delay compensation for real and artificial RIRs, then compares their use in ASR augmentation for an unseen target environment. Results favor careful RIR selection and combining real with artificial RIRs rather than simply increasing dataset size.

  • RIR passivation: Passivation significantly helps artificial RIRs in the IHM condition but does not significantly improve real RIRs.The authors attribute this difference to over-excitation and clipping in ISM-generated RIRs, whereas ReverbDB RIRs are well estimated.
  • Delay compensation: Artificial RIR delays cluster at 0 seconds, whereas real-RIR delays have a broader distribution extending to extreme values near 0.14 seconds.The real-RIR distribution is described as Gaussian with tails toward ±0.05 seconds; artificial RIRs include a roughly 0–0.02-second component.
  • Delay compensation: Delay compensation has little effect for artificial RIRs and mixed effects for real RIRs, so subsequent experiments use passivation but not delay compensation.Small distance-measurement errors may add useful variability to augmented data.
  • RIR set size and selection: Increasing artificial RIRs from AR306 to AR2k brings no gain or deterioration, while increasing real RIRs from RR30 to RR306 significantly deteriorates results.The authors conclude that RIRs should be selected to cover the target scenario rather than increased indiscriminately.
  • Real versus artificial RIRs: Artificial and real RIRs have no clear overall winner, but their combination achieves the best WER on the SDM dataset.RR30 + AR2k uses equal amounts of real- and artificial-RIR-augmented data and slightly worsens IHM performance relative to the best single systems.

VII. CONCLUSIONS AND FUTURE WORK

The paper releases BUT ReverbDB as a public resource and uses experiments to validate its collection and augmentation procedures. The authors find that real and artificial RIRs can complement each other, while identifying dataset growth and real-speech recordings as future work.

  • Dataset contribution: BUT ReverbDB is a free CC-BY dataset of RIRs, noise, and retransmitted data for ASR and SRE development and testing.It contains data from 8 rooms and includes non-standard microphone positions.
  • Validation findings: Cross-correlation-based clock-ratio estimation fixes MLS clock asynchronicity and yields WER comparable to the ESS technique.The experiments also confirm the importance of adding real noise when preparing ASR test data.
  • ASR augmentation: Real and artificial RIRs are complementary, and the experiments identify passivation, long-file reverberation, and delay compensation as important technical issues.The conclusions concern ASR training-data augmentation with either real or artificial RIRs.
  • Future work: Future work will enlarge the dataset, add real speech data, and study using only one or two impulse responses per room.Additional planned studies concern i-Vector adaptation and changing noise within speaker-adaptation segments.

APPENDIX A MEASURING RIRS IN BUT REVERBDB

The appendix describes a custom multichannel recording setup for measuring RIRs, recording environmental sounds, and creating parallel retransmitted speech corpora. It combines synchronized recording, controlled loudspeaker placement, varied microphone configurations, and metadata collection.

  • Hardware and synchronization: The recording system was custom-designed to provide many high-quality channels with sample-to-sample synchronization at reasonable cost.It uses an Analog Devices SC589 board with two 16-channel boards and 96 kHz, 24-bit converters.
  • Playback and recording: A stereo soundcard plays source audio and a control signal, which detects playback problems and segments long recordings into retransmitted parallel-corpus data.The control signal is recorded as channel 32, while the left channel feeds the loudspeaker.
  • Loudspeaker placement: Loudspeakers are placed to represent sitting people, standing people, noise sources, and non-standard source positions.Examples include ceiling- or floor-directed placement and sources simulating radios, air conditioners, or fans.
  • Noise sources: The recordings include environmental sounds such as air conditioning, ventilation, and street noise, with artificial noise sources used in selected sessions.A Tecsun PL-680 radio tuned to a random local FM station provides an additional noise source.
  • Microphones: The setup uses omnidirectional microphone capsules, including standard capsules and Sennheiser MKE 2 microphones.The microphones use symmetrical wiring and phantom power.

1) Spherical array mounting:

The dataset includes a spherical 8-channel array alongside many flexible microphone mountings designed to model ordinary, hidden, and elevated placements. Detailed positional metadata links these configurations to the loudspeaker and room environment.

  • Spherical array mounting: The spherical array contains 8 standard capsules arranged on two parallel planes inside an 8 cm diameter sphere.The microphones occupy square vertices, with two vertices rotated by 45°, and point outward from the sphere center.
  • Alternative mountings: Other microphones are mounted in magnetic wall boxes, on adjustable stands, or on objects such as monitors and lamps.These configurations model wall-mounted and tabletop microphone placements.
  • Occluded placements: Some microphones are laid on furniture or placed in chairs, cupboards, and shelves, usually oriented approximately toward the sound source.The collection also includes microphones partly or fully hidden in shelves, drawers, waste bins, flowers, vents, and behind paintings or boards.
  • Elevated placements: About 5 microphones are placed near ceilings, including in upper corners, near sensors or lights, and hanging freely away from obstacles.Fishing rods are used to position microphones in elevated locations.
  • Metadata and positioning: Metadata records room, loudspeaker, and microphone properties using Cartesian and spherical coordinates relative to the loudspeaker.It also stores room photos, descriptions, materials, furniture, temperature, and background-noise level.
  • Room configurations: Each room can contain several microphone setups, while loudspeakers are usually placed at several positions, with at least five distinct positions targeted.The first loudspeaker position is used to measure microphone coordinates and metadata.
Loading 1811.06795v2…