PaperlayerView exact revision on arXiv

Source-linked AI summary

APEX: Large-scale Multi-task Aesthetic-Informed Popularity Prediction for AI-Generated Music

Jaavid Aktar Husain, Dorien Herremans

arXiv:2605.03395v2cs.SDcs.AIcs.LGcs.MM

TL;DR

AI-generated music popularity lacks the conventional signals used for commercially released music, and its relationship with aesthetic quality remains underexplored. APEX jointly models popularity and perceptual aesthetics from audio, finding that aesthetic features consistently improve human-preference prediction across eleven unseen generative music systems despite not improving engagement-based popularity over a popularity-only baseline.

  • Problem

    Popularity prediction for AI-generated music lacks dedicated models, while the relationship between aesthetic quality and downstream popularity remains largely unexplored.

  • Method

    APEX jointly predicts streams, likes, and five perceptual aesthetic dimensions from frozen MERT audio representations, trained on over 211k Suno and Udio songs.

  • Results

    Including aesthetic features consistently improves preference prediction across eleven unseen generative music systems, while joint modeling matches popularity-only performance and achieves Pearson correlations up to 0.75 for aesthetics.

  • Takeaways & Limitations

    Aesthetic and popularity representations capture complementary aspects of AI-generated music that support preference prediction beyond engagement-based popularity alone.

Abstract

from arXiv · show

Music popularity prediction has attracted growing research interest, with relevance to artists, platforms, and recommendation systems. However, the explosive rise of AI-generated music platforms has created an entirely new and largely unexplored landscape, where a surge of songs is produced and consumed daily without the traditional markers of artist reputation or label backing. Key, yet unexplored in this pursuit is aesthetic quality. We propose APEX, the first large-scale multi-task learning framework for AI-generated music, trained on over 211k songs (10k hours of audio) from Suno and Udio, that jointly predicts engagement-based popularity signals - streams and likes scores - alongside five perceptual aesthetic quality dimensions from frozen audio embeddings extracted from MERT, a self-supervised music understanding model. Aesthetic quality and popularity capture complementary aspects of music that together prove valuable: in an out-of-distribution evaluation on the Music Arena dataset, comprising pairwise human preference battles across eleven generative music systems unseen during training, including aesthetic features consistently improves preference prediction, demonstrating strong generalisation of the learned representations across generative architectures.

1. INTRODUCTION

APEX addresses popularity prediction for AI-generated music, where conventional artist, marketing, and listener-history signals are often absent, by jointly modeling engagement-based popularity and perceptual aesthetic quality from audio representations. Trained on over 211k songs, it shows that aesthetic quality and popularity are complementary, while aesthetic-informed predictions generalize to human preferences across unseen generative systems.

  • Motivation: AI-generated music removes many conventional popularity signals, increasing the importance of modeling intrinsic audio properties.Commercial popularity prediction commonly relies on artist identity, marketing exposure, and historical listener behavior.
  • Method: APEX is the first large-scale multi-task framework to jointly predict streams, likes, and five perceptual quality dimensions from MERT audio representations.The quality dimensions are derived from SongEval.
  • Data and evaluation: 211k+ AI-generated songs from Udio and Suno train APEX, which is evaluated on Music Arena pairwise preferences across eleven unseen generative music systems.The evaluation tests generalisation beyond the generative architectures represented in training.
  • Findings: Aesthetic quality and popularity capture complementary but distinct signals that are learnable from audio representations alone.The full multi-task configuration performs comparably to the popularity-only baseline, while aesthetic dimensions are predicted with considerably higher accuracy.
  • Findings: APEX predictions act as meaningful proxies for human preference in a fully out-of-distribution setting, demonstrating strong generalisation across unseen generative architectures.The study also includes a systematic ablation across 24 conditions covering loss strategy, shared layer depth, input mode, and task configuration.

2. RELATED WORK

Related work has advanced from acoustic-feature-based hit prediction toward deep learning and multimodal approaches, while AI-music evaluation has developed specialized aesthetic and automated quality metrics. However, predicting the popularity of AI-generated music remains largely unaddressed, and objective metrics may poorly reflect human musical preferences.

  • Music popularity prediction evolved from supervised learning on acoustic characteristics toward deep learning models that learn features directly from audio.
  • Specialized AI-generated-music evaluation methods include expert aesthetic ratings, perceptual quality metrics, and automated measures such as Fréchet Audio Distance and MuQ-Eval.
  • Objective metrics for AI-generated music often align poorly with human musical preferences, despite advances from audio-only features to sophisticated multimodal approaches.
  • Virtually no prior work predicts the popularity of AI-generated music, leaving a significant gap that motivates dedicated models for this domain.

3. PROPOSED APEX MODEL

APEX uses MERT representations to jointly model popularity and perceptual aesthetics. Popularity scores are normalized from percentile-ranked streams and likes, while five aesthetic tasks provide human-aligned auxiliary supervision and are combined through alternative loss-weighting strategies.

  • Audio representation: APEX adopts MERT, whose dual-teacher pretraining captures low-level acoustic features and higher-level musical structure for popularity prediction.The teachers are based on RVQ-VAE and the Constant-Q Transform (CQT).
  • Popularity scoring: The same percentile-based procedure derives likes scores by substituting like counts for stream counts.Percentile normalization reduces dependence on absolute count magnitude across tracks.
  • Popularity scoring: α = ln 0.5 ln 0.8 ≈3.106 transforms percentile-ranked stream counts into scores s ∈[0, 100], with the 80th percentile mapped to 50.The transformation compresses the upper tail and rewards tracks with strong percentile standing.
  • Aesthetic supervision: Five SongEval auxiliary tasks predict coherence, musicality, memorability, clarity, and naturalness, each rated from 1 to 5.A model trained on SongEval generates labels for all songs, providing multi-dimensional, human-aligned aesthetic supervision.
  • Multi-task optimization: The model combines task-specific MSE losses using equal weighting, manual weighting, or uncertainty-based learned weighting.Manual weighting sets wi = 5.0 for popularity tasks and wi = 1.0 for aesthetic tasks, while learned uncertainty parameters σi balance contributions automatically.

4. EXPERIMENTAL SETUP · 4.1 Dataset

The dataset combines AI-generated music from Udio and Suno, retaining approximately 124k songs per platform after filtering and stratified sampling. Each song includes streams, likes, and other metadata used to support popularity analysis.

  • 4.1 Dataset: The dataset combines subsets of two large-scale AI-generated music repositories: Udio-126k and Suno-307k.The repositories source music from Udio and Suno, respectively.
  • 4.1 Dataset: Each song is accompanied by streams counts, likes counts, and other metadata.
  • 4.1 Dataset: Songs with zero streams are removed from the dataset.
  • 4.1 Dataset: Duplicated songs and corrupted audio files are excluded during dataset construction.
  • 4.1 Dataset: Songs released within two weeks of the dataset release are removed to avoid recency bias.
  • 4.1 Dataset: Approximately 124k songs per platform are retained after filtering.
  • 4.1 Dataset: Stratified sampling is applied to the larger raw Suno subset to match the size of Udio.

4.2 Embedding extraction

The framework extracts audio embeddings with MERT-v1-95M after standardized mono conversion, 24 kHz resampling, and 30-second segmentation. Representations are drawn from four transformer layers to capture features across acoustic and higher-level musical processing.

  • Preprocessing: MERT-v1-95M processes each song after conversion to mono and resampling to 24 kHz.These preprocessing steps match the model’s expected sampling rate.
  • Segmentation: Audio is divided into non-overlapping 30-second windows, with shorter final segments zero-padded to a consistent length.This ensures every segment has the same duration before embedding extraction.
  • Layer selection: Hidden states are extracted from MERT transformer layers 3, 6, 9, and the final layer to span the network’s depth.Earlier layers capture acoustic-level features, whereas deeper layers model higher-level musical abstractions.

4.3 Training

The training study varies loss strategy, shared-layer configuration, input representation, and task configuration across 24 experimental conditions. Models use AdamW-based optimization and compare segment-level versus holistic song-level representations and popularity-only versus seven-branch multi-task learning.

  • Optimization: Models use AdamW with learning rate 1 × 10−4, weight decay 1 × 10−4, cosine annealing, mixed precision, and validation-loss early stopping.Training uses batch size 512 per GPU across 4 NVIDIA Tesla V100 GPUs with Distributed Data Parallel.
  • Input modes: Segment mode treats each 30-second segment as an independent sample, whereas song mode averages all segment embeddings into one song-level vector.These modes differ in whether training uses fine-grained temporal windows or holistic song representations.
  • Task configurations: Popularity-only training uses streams and likes branches, while full training jointly adds coherence, musicality, memorability, clarity, and naturalness branches.The full configuration contains all seven branches and enables multi-task learning across engagement and perceptual quality signals.
  • Network architecture: The two shared-layer configurations use dimensions 768 →512 →256 and 768 →512 →384 →256, respectively.Shared layers use linear transformation, batch normalization, GELU, and dropout rate 0.3; task-specific branches use 256 →128 →64 →1 and dropout rate 0.1.
  • Experimental design: 24 experimental conditions combine three loss strategies, two shared-layer configurations, two input modes, and two task configurations.The loss strategies are equal-weight sum, manually weighted, and uncertainty-based learned weighting.

4.4 Pairwise human preference experiment

The experiment tests whether APEX-predicted popularity and aesthetic scores generalize to human pairwise music preferences on an out-of-distribution Music Arena dataset. It compares multiple predictive approaches across battles generated by eight state-of-the-art music systems.

  • Dataset and evaluation: The Music Arena evaluation uses pairwise human preference battles between tracks generated by eight open and commercial music systems unseen in training.The systems include Sonauto, ACEStep, ElevenLabs, MusicGen, Riffusion, and Lyria.
  • Dataset and evaluation: 1,259 battles remain after removing ties, both-bad votes, missing audio files, and other invalid preferences.Each battle contains two tracks generated from the same prompt and a binary human preference label; the dataset includes 780 instrumental and 479 vocal tracks.
  • Feature construction: For each APEX-predicted score, the experiment derives difference scores and ratio scores to represent absolute and relative track advantages.Difference scores use ∆f = fa −fb, while ratio scores use rf = fa/(fb + ϵ) with ϵ = 10−9 to prevent division by zero.
  • Models and baselines: The comparison includes several classifiers and a naive rule that selects the track with the higher total predicted score for the chosen feature set.Class imbalance between tracks A and B is addressed using inverse-frequency weighting, with 674 versus 585 instances.

5. RESULTS

APEX performs consistently on popularity and aesthetic prediction, with song-level representations and balanced multi-task configurations generally strongest. Adding aesthetic features improves preference prediction on Music Arena, including across unseen generative systems, though performance is higher for instrumental than vocal tracks.

  • Popularity Prediction: Popularity prediction remains stable across 24 conditions, with streams MSE 699–714 and likes MSE 659–677.Streams MAE ranges from 21.0–22.3, while likes MAE ranges from 19.97–21.68.
  • Popularity Prediction: Song mode consistently outperforms segment mode, while three shared layers provide only marginal MSE gains over two layers.The results suggest holistic song-level embeddings are more effective, with limited benefit from additional shared-layer capacity.
  • Aesthetic Prediction: Model C achieves the strongest aesthetic performance, reaching MSE 0.166 for coherence and naturalness and Pearson correlations of 0.734–0.751.Across models and SongEval dimensions, MSE ranges from 0.166 to 0.289 and Pearson correlations from 0.59 to 0.75; weighted configurations perform least well.
  • Music Arena Preference Prediction: Aesthetic features improve preference prediction beyond engagement signals, with the all-score rule reaching AUC = 0.535 versus AUC = 0.518 for likes alone.Models with aesthetic features consistently outperform corresponding models without them across all classifiers; the SVM achieves AUC of 0.642.
  • Music Arena Preference Prediction: Preference prediction is higher for instrumental tracks, with the SVM using aesthetic features achieving AUC of 0.686 versus 0.560 for vocal tracks.The passage attributes this gap to vocal artefacts in AI-generated singing that introduce difficult-to-capture perceptual inconsistencies.
  • Music Arena Preference Prediction: APEX achieves above-chance preference prediction on Music Arena systems entirely unseen during training, indicating generalisation beyond the Suno and Udio distribution.The results support transfer of MERT-based audio representations across generative architectures.

6. CONCLUSION

APEX is a large-scale multi-task framework for jointly predicting popularity and aesthetic quality in AI-generated music using frozen MERT embeddings. Its evaluations show strong aesthetic representation, complementary objectives, and downstream value despite no popularity gain over a popularity-only baseline.

  • Framework: APEX jointly predicts popularity and aesthetic quality for AI-generated music, training on over 211k Suno and Udio songs with frozen MERT audio embeddings.The framework is described as the first large-scale multi-task approach for this setting.
  • Ablation study: Uncertainty-based loss weighting and song-level embedding aggregation produce the best overall performance across 24 experimental conditions.The full multi-task configuration captures both popularity and aesthetic dimensions.
  • Evaluation: MERT representations capture aesthetic quality with Pearson correlations up to 0.75, while joint modelling matches but does not exceed popularity-only prediction.The two objectives capture complementary aspects of music that prove valuable downstream, including an out-of-distribution pairwise human-preference experiment on Music Arena.
Loading 2605.03395v2…