Deep Convolutional Neural Network with Mixup for Environmental Sound Classification

1 Introduction

• Environmental sound events are more diverse and less well defined than speech and music, making ESC more difficult than ASR and MIR.The paper identifies performance and accuracy improvement as continuing ESC design issues.Source: §1 · p. 1
• Traditional ESC classifiers such as SVM, GMM, and KNN lack feature-extraction ability, limiting their performance gains.The paper notes that discriminative representations such as Mel filterbank, Gammatone, and wavelet-based features have achieved state-of-the-art performance.Source: §1 · pp. 1–2
• Because existing ESC CNNs mostly use shallow architectures, the paper proposes a deeper CNN based on VGG Net.The paper uses 1-D filters to learn local patterns across frequency and time and reports better performance than 3×3 filters at the same network depth.Source: §1 · p. 2
• Mixup creates training samples by mixing two randomly selected examples and uses their mixing ratio as the training target.The paper explores mixup's effects on classification performance and feature distribution.Source: §1 · p. 2
• The ESC system achieved state-of-the-art performance (83.7%) on UrbanSound8K and competitive performance on ESC-50 and ESC-10.Source: §1 · p. 2

2 Related Work

• CNN studies used log Mel spectrogram features, sometimes with delta information, while another used gammatone spectrograms; one reported result was excluded as not comparable under official cross-validation.The paper says the cited gammatone results used different training and validation data than the main published papers.Source: §2 · pp. 2–3
• Raw-waveform ESC approaches included deep 1-D CNNs and EnvNet models, with EnvNet-v2 reported to perform better than EnvNet.EnvNet was described as extracting features complementary to log Mel features.Source: §2 · p. 3
• External-data approaches used transferred DNN parameters or representations learned from unlabeled sound and video data for target sound recognition.The described methods adapted pretrained models or applied a linear-SVM classifier to learned sound representations.Source: §2 · p. 3

3.1 Convolutional Neural Network

• The proposed VGG-inspired CNN has eight convolutional layers and two fully connected layers, using 3×1 and 1×5 filters for frequency and time patterns before 3×3 filters learn joint patterns.The architecture begins with two convolutional layers using large filter kernels; batch normalization is applied to convolutional layers.Source: §3.1 · pp. 3–4, Table 1 · p. 4

3.2 Mixup

• Mixup forms each mini-batch example and label by linearly mixing two randomly selected training samples and their one-hot labels.The mix factor λ is drawn from Beta(α, α), and mixup is not used during testing.Source: §3.2 · pp. 4–5, Figure 1 · p. 5

4.1 Dataset

• ESC-10 has 400 samples in 10 classes, ESC-50 has 2,000 recordings in 50 classes, and UrbanSound8K has 8,732 clips across 10 folds and 10 classes.ESC-10 is a subset of ESC-50.Source: §4.1 · p. 5

4.2 Preprocessing

• Audio was sampled at 44.1 kHz, normalized to −1 to 1, augmented with time stretch and pitch shift, and represented as overlapping spectrogram segments with temporal derivatives.The log-mel and gammatone features use 128 bands and 128-frame segments with 50% overlap; silence regions are dropped before segmentation.Source: §4.2 · pp. 5–6

4.3 Training settings

• Training used mini-batch SGD with Nesterov momentum, while evaluation averaged segment probabilities and used 5-fold validation for ESC datasets and 10-fold validation for UrbanSound8K.Models ran for 200 epochs on UrbanSound8K and 300 epochs on ESC-10 and ESC-50, with batches of 200 samples.Source: §4.3 · p. 6

5 Results and Analysis

• 83.7% on UrbanSound8K, 91.7% on ESC-10, and 83.9% on ESC-50 are the reported results for the proposed system.The authors describe its UrbanSound8K result as state of the art and its ESC-10 and ESC-50 results as competitive.Source: §5 · p. 6, Table 3 · p. 7
• Mixup improved Mels accuracy by 1.5%, 2.4%, and 2.6% on ESC-10, ESC-50, and UrbanSound8K, respectively.Mels outperformed PiczakCNN by 10.8%, 17.6%, and 9.9% on those datasets; data augmentation improved results by 1.1%, 3.3%, and 5.3%, and GTs outperformed Mels by 0.4%, 1.4%, and 1.1%.Source: §5 · p. 7

5.1 Comparison of network architecture

• The proposed CNN performs better than VGG10 on all three datasets.The comparison uses a VGG10 network with the same depth and parameters, but 3x3 convolution filters and 2x2 stride pooling.Source: §5.1 · pp. 7–8, Table 4 · p. 8

5.2 Effects of Mixup

• Mixup improves results for most UrbanSound8K classes, especially air conditioner, drilling, jackhammer, and siren, but increases confusion for some class pairs.It reduces confusion between jackhammer and engine idling while increasing confusion between jackhammer and siren.Source: §5.2 · p. 8, Figure 3 · p. 9
• PCA visualizations of FC1 features show different class distributions with and without mixup, including variation in within-class variance and between-class distances.The authors report that most classes in Fig. 4(b) occupy a small space and have relatively small within-class variance and clear boundaries.Source: §5.2 · pp. 8, 10, Figure 4 · p. 9
• Across ESC-10, ESC-50, and UrbanSound8K, the best accuracy is achieved with mixup α = 0.2 in the tested range of 0.1 to 0.5.The accuracy-versus-α curves are presented for all three datasets.Source: Figure 5 · p. 10, §5.2 · p. 10

• State-of-the-art performance was achieved on UrbanSound8K, with competitive performance on ESC-10 and ESC-50.Source: §6 · p. 10
• The proposed CNN always performed better than VGG10, and mixup was applied to further improve classification accuracy.Source: §6 · p. 10
• On UrbanSound8K, the study explored mixup's impacts on classification accuracy and feature-space distributions across classes.Source: §6 · p. 10
• Future work will focus on network design and exploring mixup for specific classes.Source: §6 · p. 10