A Comparison of deep learning methods for environmental sound

1. INTRODUCTION

• The study addresses environmental sounds, which span a wide range of frequencies and are often less well defined than speech.The introduction notes that conventional classifiers lack the feature-abstraction capability of deeper models and do not model temporal dynamics.Source: §1 · p. 1
• Acoustic scene identification selects a semantic label that characterizes the environment of an audio stream.The study compares DNN, RNN, and CNN architectures, combinations of these models, GMMs, and i-vectors.Source: §1 · p. 1
• The comparison uses the DCASE dataset of 15 indoor and outdoor scene classes, totaling 13 hours of audio recordings.The paper presents the work as a systematic comparison of successful and complementary approaches to sound event detection on DCASE.Source: §1 · p. 1

2.1. Dataset

• The training set contains 15 indoor and outdoor scene labels and 9.75 hours of recording across 1,170 files.The audio is dual-channel, 44.1 kHz, 24-bit WAV, with 30-second files.Source: §2.1 · p. 2
• The evaluation set contains 390 files and 3.25 hours of recordings from the same classes as the training set.The evaluation recordings use the same WAV format as the training data.Source: §2.1 · p. 2
• Models are selected and tuned using 4-fold cross-validation on the training dataset.The dataset comes from the IEEE DCASE challenge, and the experiments use its evaluation setup.Source: §2.1 · pp. 1–2

2.2. Features

• Monaural and binaural MFCC features use 23 Mel-frequency cepstral coefficients with first- and second-order differences, yielding 61 dimensions.Binaural MFCC is computed for the right, left, and channel-difference signals.Source: §2.2 · p. 2
• OpenSMILE features include Smile983 and Smile6k, with the latter drawn from a 6,573-feature set and using 100 ms time windows.The 983-dimensional set contains features recommended by domain experts.Source: §2.2 · p. 2
• LogMel features use 60 or 200 Mel frequencies, and all feature sets are standardized using training-set means and variances.The same standardization is applied during validation and testing.Source: §2.2 · p. 2

2.3.1. Guassian Mixture Models (GMMs)

• For GMMs, each class is modeled with a Gaussian mixture trained on acoustic features from that class’s audio clips.Each clip is represented as a bag of acoustic features extracted from audio segments.Source: §2.3.1 · p. 2

2.3.2. i-vector Pipeline

• The i-vector pipeline represents each scene with a 400-dimensional latent vector, estimated by maximum a posteriori probability from a scene-dependent subspace.The UBM is a 256-component GMM trained on the development dataset using BiMFCC features.Source: §2.3.2 · p. 2

2.3.3. Deep Neural Networks (DNNs)

• DNN hyperparameters span 2–10 layers, 256–1,024 hidden units, dropout rates of 0–0.4, regularization, and several optimization methods.Deep models are tuned by cross-validation to achieve their best performance.Source: §2.3.3 · p. 2

2.3.4. Recurrent Neural Networks (RNNs)

• The RNN experiments use bidirectional LSTM and GRU layers, with two layers chosen because deeper models had convergence-time costs and limited improvement.Bidirectional architectures generally perform better than their unidirectional counterparts.Source: §2.3.4 · p. 2

2.3.5. Convolutional Neural Networks (CNNs)

• The CNNs use VGG-like architectures with ReLU nonlinearities and omit most bottom dense layers to reduce computation.Dropout improves performance, while convolutional layers make CNNs slower to train than RNNs and DNNs.Source: §2.3.5 · p. 2

2.4. Pipeline & System Configuration

• Feature transforms are applied to each training and test audio clip to extract representations.Source: §2.4 · pp. 2–3
• Non-temporal models treat each feature as a training example, whereas temporal models use a sequence of features as one example.Source: §2.4 · p. 3
• At test time, each clip is split into segments, predicted class likelihoods are summed, and the class with the highest total is selected.Deep learning models were trained using Keras on Theano and TensorFlow with four Titan X GPUs.Source: §2.4 · p. 3

2.5. Late Fusion

• Thirty models spanning five architectures are ranked, and only models passing a predefined accuracy threshold enter fusion.Source: §2.5 · p. 3
• The fusion process also constructs ensembles of ensembles to further stabilize the model.Source: §2.5 · p. 3
• Fusion methods tested include random forests, extremely randomized trees, Adaboost, gradient tree boosting, and weighted-average probabilities.The baseline GMM is excluded because of its poor performance.Source: §2.5 · p. 3

3. RESULTS

• 84.2% mean CV accuracy and 84.1% test accuracy were achieved by the DNN with Smile6k features.Source: §3 · p. 3
• 88.1% mean CV accuracy and 88.2% test accuracy for late fusion, versus 72.5% mean CV accuracy for the GMM/MFCC baseline.The fusion result is reported as competitive with the DCASE challenge winner.Source: Figure 1 · p. 3, §3 · p. 3
• Figure 1 compares cross-validation accuracy for five classifiers across six features, with 60-dim and 200-dim LogMel shown in one column.Source: Figure 1 · p. 3, §3 · p. 3
• Table 2 reports class-wise accuracy for the best CV-average models, with colored rows marking classes identified as most challenging in [6].Source: Table 2 · p. 3

4. DISCUSSION

• With Smile6k, DNNs outperform RNNs and CNNs, while RNNs and CNNs outperform DNNs on MFCC, BiMFCC, and Smile983; late fusion compensates for model-specific errors.The authors note that park and residential area, and train and tram, are difficult to distinguish; they suggest environmental sounds lack the coherent context found in speech.Source: §4 · p. 3

4.1. GMM & I-Vectors

• I-vectors perform best among BiMFCC models and outperform deep-learning models with low-dimensional features, while Factorial Analysis keeps their dimensionality low.The authors contrast this with GMMs, whose performance suffers in high-dimensional spaces as available data remain constant.Source: §4.1 · pp. 3–4

4.2. DNN

• DNN first-layer neurons respond most to MFCC dimensions 0–23 and less to delta and double-delta dimensions; their chaotic responses indicate they do not capture temporal information.After Savitzky-Golay smoothing, the weights resemble a filter bank.Source: §4.2 · p. 4

4.3. RNN

• For train audio, RNN neurons remain stable when features do not vary over time, consistent with stronger RNN performance on monotonous train and tram scenes than event-rich scenes.The authors suggest attention-based RNNs as a potential way to address event-rich scenes.Source: Figure 5 · p. 4

4.4. CNN

• CNN filters resemble a bandpass filter bank, with a sharp transition near Mel band 40 attributed to weak input energy beyond that band.The learned filters are wider than those learned in speech.Source: §4.4 · p. 4

• 77.2% test accuracy for the MFCC GMM baseline compares with 88.2% test accuracy for the hierarchical DNN with Smile6k features.Source: §5 · p. 4
• Fusing temporal-specialized CNNs or RNNs with resolution-specialized DNNs or i-vectors significantly improves overall performance.The classifiers are trained independently first to maximize model diversity before fusion.Source: §5 · p. 4
• RNNs and CNNs generally achieve 73–82% performance, and no single model outperforms all others across feature sets.Model performance can vary significantly with feature representation.Source: §5 · p. 4
• The best-performing single model is non-temporal, which the authors take as evidence that scene sounds do not necessarily exhibit strong temporal dynamics.Source: §5 · p. 4