PaperlayerView exact revision on arXiv

Source-linked AI summary

musicnn: Pre-trained convolutional neural networks for music audio tagging

Jordi Pons, Xavier Serra

arXiv:1909.06654v1cs.SDcs.CLeess.AS

TL;DR

The paper presents pre-trained models and training tools for music audio tagging and related music tasks. Its models report state-of-the-art results on MagnaTagATune and Million Song Dataset, including an attention-based variant that replaces temporal pooling.

  • Problem

    The paper addresses the need for pre-trained models and training tools for music audio tagging and related music tasks.

    Source: EXTENDED ABSTRACT · pp. 1–2

  • Method

    The authors provide pre-trained musically motivated CNNs and VGG-like models, feature-extraction and transfer-learning tools, and a framework for implementing attention-based models.

    Source: EXTENDED ABSTRACT · pp. 1–2

  • Results

    90.77 ROC-AUC / 38.61 PR-AUC on MagnaTagATune and 88.81 ROC-AUC / 31.51 PR-AUC on Million Song Dataset are reported for a musicnn-like model with attention instead of temporal pooling.

    Source: EXTENDED ABSTRACT · p. 2

  • Takeaways & Limitations

    The pre-trained models can be used as music taggers, feature extractors, or starting points for transfer learning.

    Source: EXTENDED ABSTRACT · pp. 1–2

Abstract

from arXiv · show

Pronounced as "musician", the musicnn library contains a set of pre-trained musically motivated convolutional neural networks for music audio tagging: https://github.com/jordipons/musicnn. This repository also includes some pre-trained vgg-like baselines. These models can be used as out-of-the-box music audio taggers, as music feature extractors, or as pre-trained models for transfer learning. We also provide the code to train the aforementioned models: https://github.com/jordipons/musicnn-training. This framework also allows implementing novel models. For example, a musically motivated convolutional neural network with an attention-based output layer (instead of the temporal pooling layer) can achieve state-of-the-art results for music audio tagging: 90.77 ROC-AUC / 38.61 PR-AUC on the MagnaTagATune dataset --- and 88.81 ROC-AUC / 31.51 PR-AUC on the Million Song Dataset.

EXTENDED ABSTRACT

The extended abstract introduces pre-trained musicnn and VGG-like models for music audio tagging, feature extraction, and transfer learning. It reports benchmark results on MagnaTagATune, Million Song Dataset, and GTZAN, and describes a training framework that also supports attention-based models.

EXTENDED ABSTRACT

  • The pre-trained models support out-of-the-box tagging, feature extraction, and transfer learning; musicnn extraction returns tags, a taggram, and intermediate representations such as timbral, temporal, CNN, and pooling features.Transfer learning can fine-tune the pre-trained models with a classifier or use extracted features with SVM classifiers.Source: EXTENDED ABSTRACT · pp. 1–2
  • The library provides pre-trained musically motivated CNNs and VGG-like baselines trained on MagnaTagATune or Million Song Dataset; MSD_musicnn_big is included because MSD has more training data.The listed models are MTT_musicnn, MSD_musicnn, MSD_musicnn_big, MTT_vgg, and MSD_vgg.Source: EXTENDED ABSTRACT · p. 1
  • On fault-filtered GTZAN, feature extraction plus 128 PCA and SVM yields 77.58% accuracy with VGGish, 74.65% with OpenL3, 71.37% with MTT_musicnn, 72.75% with MTT_vgg, and 77.24% with MSD_musicnn.The abstract says MSD pre-trained models outperform MTT models, and MSD_musicnn achieves results similar to VGGish, which was trained with 2M audios.Source: EXTENDED ABSTRACT · p. 2
  • 90.69 ROC-AUC / 38.44 PR-AUC for MTT_musicnn and 90.26 ROC-AUC / 38.19 PR-AUC for MTT_vgg are reported as state-of-the-art on MagnaTagATune.The reported state-of-the-art results also include models evaluated on the Million Song Dataset.Source: EXTENDED ABSTRACT · p. 2
  • 88.01 ROC-AUC / 28.90 PR-AUC for MSD_musicnn, 88.41 ROC-AUC / 30.02 PR-AUC for MSD_musicnn_big, and 87.67 ROC-AUC / 28.19 PR-AUC for MSD_vgg are reported on Million Song Dataset.The abstract reports these results alongside MagnaTagATune results as state-of-the-art performance.Source: EXTENDED ABSTRACT · p. 2
  • 90.77 ROC-AUC / 38.61 PR-AUC on MagnaTagATune and 88.81 ROC-AUC / 31.51 PR-AUC on Million Song Dataset are reported for a musicnn-like model with attention instead of temporal pooling.The training framework also supports implementing other models.Source: EXTENDED ABSTRACT · p. 2
Loading 1909.06654v1…