Source-linked AI summary
Representation Learning of Music Using Artist Labels
Jiyoung Park, Jongpil Lee, Jangyeon Park, Jung-Woo Ha, Juhan Nam
TL;DR
Semantic-label supervision can be noisy or costly, and unsupervised feature discovery is challenging. This paper trains plain and Siamese DCNNs to identify artists, then uses their features for music classification and retrieval; the artist-label models are comparable to or outperform tag-label models, with the Siamese model strongest in retrieval.
Problem
Unsupervised feature discovery is challenging, while semantic labels such as genre and mood can be ambiguous or expensive to annotate.
Source: §1 · p. 1Method
The paper trains plain and Siamese DCNNs to identify artists and uses their learned features for music classification and retrieval.
Source: §1 · p. 1Results
Artist-label models are comparable to or outperform the tag-label model, and the Siamese model performs best on song retrieval across the tested datasets.
Source: §7 · p. 6Takeaways & Limitations
Artist labels can provide cost-free supervision that is as effective as expensive and possibly noisy tag labels.
Source: §7 · p. 6The basic model’s output layer can become excessively large when the dataset contains numerous artists.
Source: §2.2 · p. 2
Abstract
from arXiv · showhide
In music domain, feature learning has been conducted mainly in two ways: unsupervised learning based on sparse representations or supervised learning by semantic labels such as music genre. However, finding discriminative features in an unsupervised way is challenging and supervised feature learning using semantic labels may involve noisy or expensive annotation. In this paper, we present a supervised feature learning approach using artist labels annotated in every single track as objective meta data. We propose two deep convolutional neural networks (DCNN) to learn the deep artist features. One is a plain DCNN trained with the whole artist labels simultaneously, and the other is a Siamese DCNN trained with a subset of the artist labels based on the artist identity. We apply the trained models to music classification and retrieval tasks in transfer learning settings. The results show that our approach is comparable to previous state-of-the-art methods, indicating that the proposed approach captures general music audio features as much as the models learned with semantic labels. Also, we discuss the advantages and disadvantages of the two models.
1. INTRODUCTION
Semantic-label supervision can require noisy or costly annotation, while artist labels offer objective metadata associated with diverse musical styles. The paper trains artist-identification DCNNs and evaluates their features for music tasks, reporting performance comparable to models trained with semantic labels.
1. INTRODUCTION
- Unsupervised feature learning exploits unlabeled audio but many approaches use only one or two layers, which the authors consider insufficient for music’s complicated feature hierarchy.Prior approaches include sparse coding, K-means, and restricted Boltzmann machines.Source: §1 · p. 1
- Semantic labels such as genre and mood can be ambiguous or crowd-tagged, while expert annotation is time-consuming and expensive.The paper contrasts these labels with artist metadata annotated from album releases, described as objective and free of disagreement.Source: §1 · p. 1
- Two DCNNs identify artists from audio: a plain model predicts among all artists, while a Siamese model uses a subset of artist labels based on identity.The models are used as feature extractors for classification and retrieval tasks.Source: §1 · p. 1, Figure 1 · p. 2
- The proposed artist-label DCNN features capture useful information for unseen audio datasets and perform comparably to models trained with semantic labels.The paper applies the trained models to music classification and retrieval.Source: §1 · pp. 1–2
2. LEARNING MODELS
The paper compares a plain artist-classification DCNN with a Siamese model that learns from pairwise artist identity, alongside a tag-label baseline. The learned representations are extracted from the models for downstream use.
2.1 Basic Model
- The basic DCNN classifies artists, then uses the last hidden layer’s 256-dimensional vector as the learned audio feature.The feature is the representation from which the linear softmax classifier predicts artist identity.Source: §2.1 · p. 2
2.2 Siamese Model
- The Siamese DCNN addresses the basic model’s large-output-layer and full-retraining concerns using artist-based positive and negative song examples.The basic model’s last weight matrix can reach one million parameters for 10,000 artists and a 100-unit hidden layer; adding artists requires retraining it entirely.Source: §2.2 · p. 2
- With negative sampling, the Siamese model was successfully trained in preliminary experiments using max-margin hinge loss rather than softmax cross-entropy.The tested setup used four negative samples and a margin of 0.4.Source: §2.2 · pp. 2–3
2.3 Compared Model
- The tag-label comparison model shares the basic DCNN architecture but predicts semantic tags instead of artists.Its output layer size corresponds to the number of tag labels.Source: §2.3 · p. 3
3. EXPERIMENTS
The experiments train artist-label and tag-label models on source music data, then evaluate their extracted features on three genre datasets. The paper also specifies source splits, preprocessing, and training settings.
3. EXPERIMENTS
- The experiment compares two artist-label models and one tag-label model on source training data and target datasets.The section also describes model-training details.Source: §3 · p. 3
3.1 Source Tasks
- All models use Million Song Dataset tracks with 30-second 7digital preview clips; artist metadata supplies artist labels, and Last.fm tags matched to MSD supply tag labels.The Last.fm dataset is augmented on MSD.Source: §3.1 · p. 3
3.1.1 Artist-label Model
- Artist-label source sets contain 500 to 10,000 artists, with 20 songs per artist and artist-wise training, validation, and test splits.The testing split was not used in these experiments because the source data trained feature extractors; the authors describe the splits as intended for future work.Source: §3.1.1 · p. 3
3.1.2 Tag-label Model
- The tag-label baseline uses a 90,000-song set matched in size to the 5,000-artist baseline, sampled from songs with filtered top-50 tags.The songs were split into 75,000 training and 15,000 validation examples for the artist baseline.Source: §3.1.2 · p. 3
3.2 Target Tasks
- Genre classification uses fault-filtered GTZAN, FMA small, and NAVER Music 3, each described as a dataset of 930 or 8,000 songs.GTZAN has 10 genres; FMA small and NAVER Music 3 each have eight balanced genres.Source: §3.2 · p. 3
3.3 Training Details
- Inputs are log-compressed 128-bin mel-spectrograms computed at 22,050 Hz, and the DCNN receives randomly selected three-second contexts from 30-second clips.Training uses stochastic gradient descent with Nesterov momentum, dropout, and validation-loss-based learning-rate reduction.Source: §3.3 · p. 3
4. FEATURE EVALUATION
The section evaluates learned audio features for genre classification through similarity-based retrieval and transfer learning. It describes song-level feature construction and the evaluation measures and classifiers used.
4.1 Feature Extraction Using the DCNN Models
- Each 30-second clip is divided into 10 segments, and the 256-dimensional last-hidden-layer features are averaged into one song-level vector.Cosine similarity is used for song-to-song distances to match the Siamese model’s relevance score.Source: §4.1 · p. 4
4.2 Feature Similarity-based Song Retrieval
- Retrieval ranks songs by cosine similarity and evaluates genre-relevant results with mean average precision (MAP).The experiment tests how similar feature vectors of the same genre are in the learned feature space.Source: §4.2 · p. 4, Table 1 · p. 4
4.3 Transfer Learning
- Transfer learning evaluates genre classification with k-NN and linear softmax accuracy, testing similarity-based classification and linear separability on unseen datasets.The k-NN setting uses k = 20.Source: §4.3 · p. 4, Table 2 · p. 4, Table 3 · p. 4
5. RESULTS AND DISCUSSION
Artist-label models are competitive with semantic-tag models, with the Siamese model strongest for similarity-based retrieval and the basic model more robust across genres. Increasing the number of training artists generally improves performance, and artist-label models are comparable to prior state-of-the-art methods.
5.1 Tag-label Model vs. Artist-label Model
- On 90,000-song training sets, the artist-based Siamese model leads retrieval across target datasets, while tag-label models are slightly better on some genre-classification datasets.The authors suggest tag-label models may benefit from overlap between Last.fm tags and target genre labels.Source: §5.1 · p. 4
5.3 Effect of the Number of Artists
- As training artist counts increase, performance is generally proportional, though trends differ: Siamese retrieval MAP significantly exceeds basic above 1,000 artists, and its MAP rises as basic MAP saturates at 2,000 or 5,000.For classification, the basic model is consistently better on NAVER; GTZAN favors basic at 500 and 1,000 artists but Siamese at 2,000 or more, while FMA small is mixed across classifiers.Source: §5.3 · p. 5
5.2 Basic Model vs. Siamese Model
- On NAVER k-NN classification, the basic model outperforms the Siamese model for Trot, K-pop Ballad, and Kids, genres absent from training.The authors interpret this as greater robustness to unseen genres; the Siamese model slightly overfits to training data.Source: §5.2 · pp. 4–5, Figure 5 · p. 5
5.4 Comparison with State-of-the-arts
- The proposed artist-label models are comparable to previous state-of-the-art methods on classification accuracy.The comparison uses linear softmax features from models trained with 10,000 artists.Source: Table 4 · p. 6, §5.4 · p. 6
6. VISUALIZATION
Visualizations of artist-trained features show organization by musical attributes, including genre, vocal style, and gender. GTZAN features cluster by genre, with some genres dividing into multiple groups that may reflect subgenres.
6. VISUALIZATION
- The artist-feature visualization uses two-dimensional t-SNE embeddings of features from a DCNN trained to classify 5,000 artists.Source: §6 · p. 6
- Artist songs in the visualization cluster by genre, vocal style, and gender, including female pop singers; Michael Jackson tracks lie near female vocals.The authors attribute the latter pattern to his distinctive high-pitched tone.Source: §6 · p. 6
- GTZAN features cluster by genre, while disco, rock, and hip-hop each split into multiple groups that may correspond to subgenres.Source: §6 · p. 6
7. CONCLUSION AND FUTURE WORK
The paper compares artist-label feature learners with a tag-label model and reports strong retrieval and classification results for artist-label models, particularly Siamese. It also outlines further evaluations of the Siamese model.
7. CONCLUSION AND FUTURE WORK
- The comparison includes an all-artist softmax DCNN, an artist-identity subset Siamese model, and a tag-label model using the first model’s architecture.Source: §7 · p. 6
- Cost-free artist-label models, especially Siamese, matched or outperformed expensive, possibly noisy tag-label models; Siamese led retrieval on all tested datasets, and more artists increased performance.Source: §7 · p. 6
- Future work will investigate advanced architectures and pairwise losses, then evaluate retraining with added artists on cross-cultural data and one-shot learning.Source: §7 · p. 6