PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Neural Compression for RIR-Characterized Acoustic Environments with Structure-Aware Constraints

Chen-Yuan Ning, Yang Ai, Hui-Peng Du, Xiao-Hang Jiang, Zhen-Hua Ling

arXiv:2609.04085v1eess.AS

TL;DR

RIR storage grows with spatial sampling, while neural audio codecs are primarily trained for speech or general audio rather than RIR characteristics. The paper adds RIR-structure-aware constraints to an EnCodec-based compressor, which at 375 bps achieves lower RIR error and better reverberant-speech consistency than audio-oriented codecs.

  • Problem

    RIR storage grows with spatial sampling, and existing neural audio codecs are mainly tailored to speech or general audio rather than RIR structure.

    Source: §I · p. 1

  • Method

    The EnCodec-based compressor constrains global RIR decay and local energy, and supervises consistency of reverberant speech generated from reference and decoded RIRs.

    Source: §I · pp. 1–2, 1) RIR-Level Constraints · p. 2

  • Results

    375 bps yielded the lowest T60 error and highest ViSQOL score (4.11) among the compared audio-oriented codec baselines.

    Source: §I · p. 2, A. Main Results · p. 4

  • Takeaways & Limitations

    At low bitrates, the method preserves key RIR acoustic properties more accurately than audio-oriented codecs.

    Source: §V · p. 6

Abstract

from arXiv · show

Room impulse responses (RIRs) characterize the acoustic environment of a room by capturing how sound propagates and decays within an enclosed space. In applications such as immersive audio rendering, accurate acoustic reconstruction often relies on spatially densely sampled RIRs. This consequently gives rise to a large volume of RIR data, imposing a substantial burden on storage. Although recent neural audio codecs provide an effective framework for low-bitrate compression, their training objectives are mainly tailored to speech and general audio, and are therefore not well aligned with the acoustic characteristics of RIRs. Therefore, we propose an EnCodec-based neural RIR compression method, which incorporates RIR structure-aware constraints at two levels. Specifically, at the RIR level, structure-aware constraints are imposed on the global decay behavior and local energy distribution of RIRs through energy decay curve (EDC) regularization and a short-time window energy constraint, while at the reverberant-speech level, reverberant-speech supervision is further introduced to constrain the consistency of the reverberant speech generated by the reconstructed RIRs. Experimental results show that, at a low bitrate of 375 bps, the proposed method achieves lower RIR reconstruction error and better reverberant-speech perceptual consistency than audio-oriented codecs.

I. INTRODUCTION

The paper targets storage-intensive RIR reuse and the mismatch between RIR structure and audio codecs designed for speech or general audio. It proposes EnCodec-based compression with structure-aware constraints and reports improved RIR and reverberant-speech results at 375 bps.

I. INTRODUCTION

  • As spatial sampling grows, storing high-rate, temporally extended RIRs becomes burdensome, despite their repeated use in applications including immersive audio rendering.Prior RIR compression approaches include GLRAM, truncation, and thresholding, but jointly achieving compression efficiency and faithful preservation of acoustically important characteristics remains challenging.Source: §I · p. 1
  • Existing neural audio codecs emphasize waveform, spectral, or perceptual fidelity for speech and general audio, making direct application to RIR compression suboptimal.The passage notes that pretrained EnCodec can compress RIRs at low bitrates to some extent, but overall performance remains limited.Source: §I · p. 1
  • The EnCodec-based method constrains RIR global decay and local energy, then adds reverberant-speech supervision in time and mel domains.The speech supervision compares signals generated using reference and decoded RIRs.Source: §I · pp. 1–2
  • At 375 bps, the method achieved the lowest T60 error and a ViSQOL score of 4.11, the highest among compared audio-oriented codec baselines.The comparison used the public real-world Motus dataset at 24 kHz.Source: §I · p. 2

II. PROPOSED METHOD

The method encodes an RIR sequence as discrete tokens for compact storage and reconstructs it with an EnCodec-style encoder–quantizer–decoder. Its compressor uses a single-codebook vector quantizer for low-bitrate compression.

II. PROPOSED METHOD

  • The compressor maps an input RIR sequence to discrete tokens for storage, then decodes those tokens into a reconstructed RIR sequence.Training uses structure-aware constraints at both the RIR and reverberant-speech levels.Source: §II · p. 2

A. Model Details of Neural RIR Compressor

  • The EnCodec-style architecture uses a convolutional encoder, quantizer, and convolutional decoder, with four downsampling strides of 4, 4, 5, and 8.The encoder also uses a two-layer LSTM; the decoder mirrors the encoder with transposed convolutions and reversed stride order.Source: A. Model Details of Neural RIR… · p. 2
  • Unlike EnCodec’s residual vector quantization, the compressor uses a single-codebook vector quantizer for deep, low-bitrate compression.The bitrate depends on the sampling rate, waveform-to-token downsampling ratio, and codebook size.Source: A. Model Details of Neural RIR… · p. 2

B. Structure-Aware Multi-Level Constraints

Training combines RIR-level and reverberant-speech-level structure-aware constraints. At the RIR level, the losses address global decay, local temporal energy, quantization, and adversarial reconstruction characteristics.

1) RIR-Level Constraints:

  • The EDC loss uses mean squared error between reference and reconstructed log-EDC curves over the effective decay interval.That interval ends at the earlier of the reference and reconstructed curves’ -35 dB crossings.Source: 1) RIR-Level Constraints · pp. 2–3
  • The local energy loss compares energy within short-time windows to preserve the distinct patterns of direct sound, early reflections, and late reverberation.The windows are non-overlapping and 50 ms long.Source: 1) RIR-Level Constraints · p. 3
  • The RIR-level objectives also include latent-to-codeword MSE quantization and adversarial and feature-matching losses based on discriminator features.The quantization term encourages the encoder latent representation to remain close to its selected codeword.Source: 1) RIR-Level Constraints · p. 3

2) Reverberant-Speech-Level Constraints:

Reverberant-speech-level constraints compare speech generated using reference and decoded RIRs. They combine waveform and multi-scale mel-domain losses to encourage consistency.

2) Reverberant-Speech-Level Constraints:

3) Overall Training Objective:

The overall training objective combines RIR-level and reverberant-speech-level constraints. The compressor and discriminators are optimized alternately using adversarial training.

3) Overall Training Objective:

A. Datasets

Experiments use the Motus real-world RIR dataset and VCTK clean speech. The RIRs are converted to mono samples and split into training, validation, and test sets.

A. Datasets

  • Motus comprises 3,320 raw 32-channel RIR recordings from 830 furniture configurations and four fixed source–receiver pairs per configuration in one room.Each recording has an effective duration of approximately 2.5 s.Source: A. Datasets · p. 3
  • Table I reports objective and subjective evaluation results for the proposed method and baselines, with MOS reported as mean ± 95% confidence interval.Source: Table I · p. 4
  • The 32-channel recordings were decomposed into 106,240 mono RIR samples, randomly split approximately 80%/10%/10% and downsampled to 24 kHz.VCTK speech was also downsampled to 24 kHz.Source: A. Datasets · p. 4

B. Implementation Details

The compressor uses an EnCodec configuration yielding a bitrate of 375 bps. Training combines several loss terms and uses AdamW for 240k iterations.

B. Implementation Details

  • A codebook size of 1024 and waveform-to-token downsampling ratio of 640 yield a bitrate of 375 bps.Source: B. Implementation Details · p. 4
  • The configuration uses five short-time windows for local energy loss and six 64-bin mel-spectrogram configurations for the mel-domain loss.Loss weights are λEDC = 1, λeng = 10, λadv = 1, λfm = 1, λq = 1000, and λwav = 100.Source: B. Implementation Details · p. 4
  • Training uses AdamW with β1 = 0.5, β2 = 0.9, batch size 10, and 240k iterations.The learning rate starts at 0.0003 and decays by a factor of 0.999 after each epoch.Source: B. Implementation Details · p. 4

C. Baselines and Evaluation Metrics

The proposed method outperformed two EnCodec-based baselines on RIR and reverberant-speech evaluations. The experiments assessed both reconstructed RIR properties and the acoustic effect of using those RIRs to generate reverberant speech.

C. Baselines and Evaluation Metrics

  • The comparison used Speech-trained EnCodec and RIR-trained EnCodec baselines with the same architectural configuration and bitrate as the proposed method.Speech-trained EnCodec used VCTK data; RIR-trained EnCodec used Motus RIR data, and both used the original EnCodec objective.Source: C. Baselines and Evaluation Metrics · p. 4
  • Evaluation covered RIR-level T60, SFM, and DRR errors, plus reverberant-speech LSD, SegSNR, and ViSQOL and subjective MOS naturalness ratings.MOS tests used 20 utterances per method and at least 25 native English listeners, rating naturalness from 1 to 5 in 0.5-point intervals.Source: C. Baselines and Evaluation Metrics · p. 4

A. Main Results

  • The proposed method achieved the best performance on all three RIR-level metrics and outperformed two baselines on objective and subjective reverberant-speech metrics; at 375 bps, generated reverberant speech scored 4.11/5 on ViSQOL.Source: A. Main Results · p. 4
  • 4.11 ViSQOL (full score 5) was reported for reverberant speech generated using decoded RIRs at 375 bps.The corresponding speech waveform more closely matched the reference in the time domain.Source: A. Main Results · p. 4, Figure 2 · p. 5

B. Ablation Studies

Ablations indicate that the EDC and local-energy losses contribute directly to preserving RIR structure, while reverberant-speech constraints have a smaller, indirect contribution to RIR reconstruction.

B. Ablation Studies

  • Removing EDC loss substantially increased T60 error, while removing local-energy loss increased both T60 and DRR errors.The local-energy constraint helps preserve the balance among direct sound, early reflections, and late reverberation.Source: B. Ablation Studies · p. 5
  • Removing reverberant-speech-level constraints slightly degraded some RIR metrics but caused no significant deterioration in reverberant-speech quality.The reported RIR metrics included T60 and DRR errors.Source: B. Ablation Studies · p. 5
  • Reverberant-speech constraints indirectly support high-fidelity RIR reconstruction, but contribute less than losses imposed directly on RIRs.Source: B. Ablation Studies · p. 5

C. Lower-Bitrate Limit Analysis

The bitrate tests place the likely practical lower limit for RIR compression between 46.875 and 93.75 bps. The conclusion describes the method's joint RIR- and reverberant-speech-level constraints and identifies spatial and multichannel RIRs as future work.

C. Lower-Bitrate Limit Analysis

  • To investigate the practical lower bitrate limit for RIR compression, the study reduced bitrate from 375 bps to 187.5, 93.75, and 46.875 bps by changing downsampling-layer strides.Source: C. Lower-Bitrate Limit Analysis · p. 5
  • The likely practical lower bitrate limit lies between 46.875 and 93.75 bps: speech quality was largely maintained at 93.75 bps, but both RIR- and speech-level metrics deteriorated markedly at 46.875 bps.Source: C. Lower-Bitrate Limit Analysis · p. 5

V. CONCLUSION

  • The conclusion describes an EnCodec-based method that constrains decoded-RIR EDC and local energy while promoting consistency of downstream reverberant speech.The stated constraints aim to capture RIR decay and temporal structure and preserve acoustic rendering effects.Source: §V · pp. 5–6
  • Extending the framework to spatial and multichannel RIR scenarios is identified as future work.Source: §V · p. 6
Loading 2609.04085v1…