Music Genre Classification using Machine Learning Techniques

1 Introduction

• Genre-based tagging is presented as a way to organize songs in growing online music libraries and support audio streaming services.Genre is identified through characteristics such as rhythmic structure, harmonic content, and instrumentation.Source: §1 · p. 1
• The study compares a CNN trained on audio MEL spectrograms with four traditional classifiers trained on time- and frequency-domain features.The traditional classifiers are Logistic Regression, Random Forests, Gradient Boosting, and Support Vector Machines.Source: §1 · p. 1
• The models are evaluated on the Audio Set dataset, with comparisons of model performance and feature importance.Source: §1 · p. 1

2 Literature Review

• Music-genre classification research has used supervised methods including Gaussian Mixture models, k-nearest neighbours, Hidden Markov Models, and support vector machines.Tzanetakis and Cook (2002) grouped features into timbral structure, rhythmic content, and pitch content.Source: §2 · p. 1
• Prior work examined psycho-acoustic features, Bark-scale STFT, MFCCs, spectral contrast, spectral roll-off, and combinations of visual and acoustic features.The cited studies used methods including SVM and AdaBoost classifiers.Source: §2 · p. 2
• Deep-learning studies have used spectrograms as image-like CNN inputs, alongside raw MFCC matrices and constant Q-transform spectrograms for genre classification.Source: §2 · p. 2
• This work compares spectrogram-input deep models with traditional classifiers trained on hand-crafted features and investigates the relative importance of those features.Source: §2 · p. 2

3 Dataset

• Audio Set is described as a large-scale human-annotated database built from 10-second clips drawn from 2.1 million YouTube videos and covering 527 sound classes.Source: §3 · p. 2
• The study selects Audio Set audio clips in the music category that have one of the seven genre tags listed in Table 1.Source: §3 · p. 2
• Table 1 reports the number of instances in each genre class.Source: Table 1 · p. 2
• The dataset release provides YouTube IDs and clip start and end times rather than the raw audio, so the researchers downloaded videos with youtube-dl and converted the MP4 files to WAV with ffmpeg.Source: §3 · p. 2
• The paper reports approximately 34 GB of audio data for the study.Each WAV file is reported to be about 880 KB.Source: §3 · p. 2

4 Methodology

• The methodology section introduces preprocessing steps and details for the two proposed classification approaches.Source: §4 · p. 2
• Figure 1 presents a sample spectrogram for one audio signal from each music genre.Source: Figure 1 · p. 3
• Figure 2 depicts a convolutional neural network architecture.The caption credits Hvass Tensorflow Tutorials as the image source.Source: Figure 2 · p. 3

4.1 Data Pre-processing

• Pre-emphasis filtering is applied to the original audio to improve signal-to-noise ratio and boost high-frequency amplitudes.The filter uses α = 0.97.Source: §4.1 · p. 3

4.2.1 Spectrogram Generation

• In this study, each audio signal is converted into a MEL spectrogram, treated as an image for a CNN to predict one of seven genre classes.The spectrogram uses 96 MEL bins, a Hann window, and a hop size of 512, resulting in 75% overlap.Source: §4.2.1 · pp. 3–4, §4.2 · p. 3

4.2.2 Convolutional Neural Networks

• VGG-16’s convolutional base extracts spectrogram features for a new feed-forward network, with convolution, pooling, and nonlinear activation forming CNN blocks.The model is evaluated using transfer learning with fixed convolutional weights and fine-tuning with all weights trainable.Source: §4.2.2 · p. 4

4.2.3 Implementation Details

• Training uses cross-entropy on seven-class probabilities, and backpropagation updates network weights to minimize the loss.The study uses L2 regularization with λ = 0.001 and dropout at rate 0.3.Source: §4.2.2 · p. 4, §4.2.3 · p. 5
• The dataset is split into 90% training, 5% validation, and 5% test sets, and epoch 4 is selected for evaluation by validation accuracy and loss.The same split is used for all experiments; epoch 4 has the highest validation accuracy and lowest validation loss.Source: §4.2.3 · p. 5, Figure 3 · p. 5

4.2.4 Baseline Feed-forward Neural Network

• A two-layer feed-forward baseline receives flattened spectrogram pixels, with hidden layers of 512 and 32 units before the output layer.It uses ReLU and the same regularization techniques as the CNN.Source: §4.2.4 · p. 5

4.3.1 Time Domain Features

• Time-domain features include amplitude mean, standard deviation, skewness, and kurtosis, alongside frame-based zero-crossing statistics.For zero-crossing rate, the 10-second signal is divided into 2048-point frames with a 512-point hop, and the mean and standard deviation across frames are retained.Source: §4.3.1 · p. 6
• Root mean square energy is calculated frame by frame, then summarized by its mean and standard deviation across frames.The section also includes tempo, estimated from a tempogram and aggregated across frames by the mean.Source: §4.3.1 · p. 6

4.3.2 Frequency Domain Features

• Frequency-domain features include MFCCs derived from the log MEL filterbank energies after a discrete cosine transform, using 20 MEL filters.The STFT uses n_fft = 2048, hop size 512, and a Hann window.Source: §4.3.2 · p. 6
• Chroma features represent energy across 12 pitch classes, with mean and standard deviation across frames used as representative values.The frequency features are summarized across frames using their mean and standard deviation.Source: §4.3.2 · pp. 6–7
• Other spectral features describe the energy-centered frequency, bandwidth around the centroid, within-band contrast, and the frequency below which 85% of spectral energy lies.Spectral contrast is the difference between maximum and minimum magnitudes within each frequency band; the roll-off threshold can be user-defined.Source: §4.3.2 · pp. 6–7

4.4 Classifiers

• Logistic regression uses seven one-vs-rest binary classifiers and predicts the class with the highest probability.Source: §4.4 · p. 7
• Random forest combines decision trees trained on bootstrap samples and random feature subsets, with the final class chosen by majority vote.Source: §4.4 · p. 7
• XGBoost trains weak learners sequentially to focus on earlier errors, then combines their outputs as a weighted linear prediction.The study describes XGBoost as supporting fast, parallelized training.Source: §4.4 · p. 7
• The SVM uses an RBF kernel for the nonlinear problem and, like logistic regression, is implemented one-vs-rest.Source: §4.4 · p. 7

5.1 Metrics

• The evaluation uses accuracy, F-score, and AUC; AUC measures area under the ROC curve, whose axes are true- and false-positive rates.A model that predicts each class label randomly with equal probability has an AUC of 0.5.Source: §5.1 · pp. 7–8

5.2 Results and Discussion

• The VGG-16 spectrogram-only CNN performs best across all reported metrics; transfer learning and fine-tuning show no significant difference.The baseline feed-forward network using unrolled spectrogram pixels performs poorly on the test set.Source: §5.2 · p. 8

5.2.1 Most Important Features

• MFCCs appear most often among XGBoost’s important features, alongside spectral-contrast statistics and tempo.Feature importance is ranked by how often a feature is used as a decision node in the model’s trees.Source: §5.2.1 · p. 8, Figure 4 · p. 9
• Using the top 30 features instead of all 97 yields performance only 2 points lower on AUC and 4 points lower on accuracy.The authors also report surprisingly good performance using only the top 10 features.Source: §5.2.1 · pp. 8–9, Table 3 · p. 9
• XGBoost models using frequency-domain features perform better than those using time-domain features on AUC and accuracy.The comparison trains separate models using only time-domain or only frequency-domain features.Source: §5.2.1 · p. 9, Table 4 · p. 9

5.2.2 Confusion Matrix

• Both the best CNN and XGBoost predict Rock well, while Hip Hop and Pop are often confused with each other.The authors note that some songs may fit multiple genres, making exact genre assignment difficult even for people.Source: §5.2.2 · p. 9, Figure 5 · p. 10

5.2.3 Ensemble Classifier

• Averaging predicted probabilities from VGG-16 transfer learning and XGBoost outperforms all individual classifiers.The ensemble’s ROC curve is also above those of VGG-16 fine-tuning and XGBoost.Source: §5.2.3 · p. 10

• CNN models outperform feature-engineered models, and ensembling CNN with XGBoost is beneficial.Source: §6 · p. 11, Figure 6 · p. 11
• The comparison pairs VGG-16 on spectrogram images with traditional classifiers on time- and frequency-domain features; XGBoost is the best feature-based classifier.The study also reports the most important features.Source: §6 · p. 11
• Because the AudioSet clips came from YouTube and are generally noisy, future work could preprocess them before model input.Source: §6 · pp. 10–11