Source-linked AI summary
Representation Learning of Music Using Artist, Album, and Track Information
Jongpil Lee, Jiyoung Park, Juhan Nam
TL;DR
Semantic-label annotation for music representation learning takes time and can be ambiguous. This paper learns representations from artist, album, and track metadata, separately and jointly. The results show distinct concept characteristics, good joint performance across concepts, and transfer benefits that vary across datasets.
Problem
Music representation learning has mainly used semantic labels such as genres, but annotating them requires time and cost.
Source: §1 · p. 1Method
The paper trains separate and joint Siamese-style CNN models using similarity-based artist, album, and track metadata.
Source: §2 · p. 1Results
The joint model performs well across concepts, slightly outperforms the artist model in transfer learning, and transfer is effective on GTZAN and FMA but not NAVER.
Source: §4 · p. 2Takeaways & Limitations
Artist, album, and track metadata have individual concept characteristics, and using them jointly improves overall performance.
Source: §4 · p. 2
Abstract
from arXiv · showhide
Supervised music representation learning has been performed mainly using semantic labels such as music genres. However, annotating music with semantic labels requires time and cost. In this work, we investigate the use of factual metadata such as artist, album, and track information, which are naturally annotated to songs, for supervised music representation learning. The results show that each of the metadata has individual concept characteristics, and using them jointly improves overall performance.
1. Introduction
Music representation learning has relied on semantic labels that are costly to annotate and can be ambiguous. This work investigates naturally annotated artist, album, and track metadata as alternatives.
1. Introduction
- Semantic labels such as genres, moods, and instruments require significant annotation time and cost and can produce disagreement because they are ambiguous.Source: §1 · p. 1
- Artist metadata is factual and requires no annotation cost, and prior work found artist-based representations comparable to those learned from semantic labels.Source: §1 · p. 1
- This work extends metadata-based learning from artist information to the more specific album and track levels, using similarity-based learning and examining negative-sample and training-sample counts.Source: §1 · p. 1
2. Models
The models learn artist, album, and track similarity from audio segments using a Siamese-style CNN. The joint model combines the three objectives while sharing parameters, and is compared with separate concept models.
2. Models
- The joint model adds the artist, album, and track loss functions and shares model parameters across the three concepts.Source: §2 · p. 1
- Each single-concept model uses anchor, positive, and negative audio segments selected according to whether they share the target concept with the anchor.For track similarity, segments are selected from the same or different tracks; the model processes audio segments, such as three-second excerpts, rather than whole songs.Source: §2 · p. 1
- The comparison includes separate artist, album, and track models alongside the joint model; Table 1 reports hold-out positive/negative prediction and Table 2 reports transfer learning.Source: §2 · p. 1, Table 1 · p. 2, Table 2 · p. 2
- All layers use twice the number of filters used in the previous work.Source: §2 · p. 1
3. Experiments and Evaluations
The experiments train four models on Million Song Dataset data and artist and album metadata. They assess concept discrimination and genre-transfer generalization, with training splits and settings defined for the comparisons.
3. Experiments and Evaluations
- The four models are trained using Million Song Dataset songs and artist and album metadata, with artist and album splits constructed separately.Source: §3 · p. 2
- For artist splits, each artist contributes 20 songs; album splits use 10 songs per album, divided into 8 training, 1 validation, and 1 test song.Twice as many albums are used to match the artist split's number of training samples; track validation positives come from another part of the anchor song.Source: §3 · p. 2
- Evaluation measures both hold-out positive/negative prediction for concept discrimination and transfer learning on GTZAN, FMA small, and NAVER Korean for generalization.Transfer representations are fed to a linear softmax classifier; the hold-out evaluation uses four negatives, making random guessing 20%.Source: §3 · p. 2
- The reported negative-sample study trains on 1,000 artists and 2,000 albums plus related track concepts; the training-sample study uses four negatives and twice as many albums as artists.Source: Table 3 · p. 2, Table 4 · p. 2
- The main evaluations use 5,000 artists and/or 10,000 albums with four negative samples, and set loss margins to 0.4, 0.25, and 0.1 for artist, album, and track.Source: §3 · p. 2
4. Results
The joint model performs well across concepts in hold-out prediction and slightly outperforms the artist model in transfer learning. Transfer benefits vary by dataset, while increasing negative samples and training songs improves performance.
4. Results
- Each model performs best when its training and test concepts match, while the joint model achieves good performance across all concepts.Source: §4 · p. 2
- The jointly learned model slightly outperforms the artist model in transfer learning; transfer is highly effective on GTZAN and FMA but not NAVER.Among single-concept models, artist performs best, followed by album; the authors attribute the NAVER result to cross-cultural differences between source and target datasets.Source: §4 · p. 2
- Increasing the number of negative samples and the number of training songs improves model performance.Source: §4 · p. 2