How Well Do Generative Music Models Follow Emotion Conditioning?

1. INTRODUCTION

• Emotion is represented through valence and arousal, describing positive–negative and calm–energetic dimensions, respectively.Source: §1 · p. 1
• Existing music-generation evaluations measure realism, distributional similarity, prompt relevance, or overall quality, but not direct matching to intended emotion.Surveys also identify limited emotion-labeled data, listener disagreement, and inconsistent evaluation protocols.Source: §1 · p. 1
• The proposed pipeline analyzes all 1,000 GTZAN tracks, combines DashengLM descriptions with Music2Emotion emotion tags in text prompts, and compares generated and source valence–arousal estimates.Source: §1 · p. 1
• The paper contributes a reproducible continuous valence–arousal evaluation pipeline for measuring emotion-following in music generation.Source: §1 · p. 1
• Text conditioning with explicit emotion cues significantly outperforms audio conditioning across five generation configurations.The study also analyzes emotion preservation by dimension and genre.Source: §1 · p. 1
• Valence is preserved more reliably than arousal, and emotion-following varies substantially across genres.Source: §1 · p. 1

2.1 Music Generation and Conditioning

• Audio-domain music generation spans autoregressive and diffusion approaches, with modern systems supporting text, melody, or audio conditioning [3] [6].Earlier work also includes symbolic neural generation.Source: §2.1 · pp. 1–2

2.2 Emotion in Music Generation and Recognition

• Emotion-aware symbolic systems include lyric–melody generation conditioned on positive or negative emotion and hierarchical emotional guidance across time scales.Prior emotion-generation work discussed here focuses on symbolic rather than modern audio-domain systems.Source: §2.2 · p. 2
• Music emotion recognition uses categorical or dimensional representations; this paper adopts valence–arousal as operationalized by Music2Emotion [15].Recognition remains challenged by annotation and emotion representation.Source: §2.2 · p. 2

2.3 Evaluation of Generative Music

• Common evaluations measure realism, distributional similarity, prompt adherence, or human-rated quality, but not direct matching to an intended emotional condition.The paper motivates post-hoc music-emotion-recognition analysis to compare generated and source valence–arousal profiles.Source: §2.3 · p. 2

3. METHOD

• The pipeline extracts conditioning information from real music, generates audio with multiple models, and compares generated and source emotional profiles.Source: §3 · p. 2

3.1 Dataset

• All 1,000 GTZAN tracks are used, covering 10 genres, with genre labels retained for genre-wise analysis.Source: §3.1 · p. 2

3.2 Audio Description and Emotion Extraction

• DashengLM supplies semantic descriptions, while Music2Emotion provides continuous valence–arousal estimates and categorical emotion tags with confidence scores.Music2Emotion is the sole emotion source, ensuring a consistent valence–arousal extraction procedure.Source: §3.2 · p. 2

3.3 Prompt Construction

• Each prompt combines a DashengLM description with the three highest-confidence Music2Emotion tags, rewritten into a fluent instruction emphasizing target affect.All text-conditioned systems receive the same prompt for each source track.Source: §3.3 · p. 2

3.4 Generation Models and Settings

• Three systems use text conditioning and two use audio conditioning; each generates one 30-second output per track with default settings.MusicGen uses different checkpoints for text and audio modes; outputs retain each model’s native sample rate without resampling or normalization.Source: §3.4 · pp. 2–3, Figure 1 · p. 3

3.5 Emotion-Following Metrics

• Generated and source valence–arousal estimates are compared using dimension-wise mean absolute errors, per-track Euclidean distance, and Pearson correlations.Lower errors and distances indicate better emotion-following, while higher correlations indicate stronger correspondence.Source: §3.5 · p. 3

4.1 Overall Emotion-Following Performance

• Mean VA distances were 1.362 for MusicGen (text) and 1.367 for InspireMusic (text), versus 1.714 for Stable Audio Open (text), 1.804 for MusicGen (audio), and 2.101 for InspireMusic (audio).The two text-conditioned systems had the best overall values; the table reports results across models and conditioning types.Source: §4.1 · p. 3, Table 1 · p. 4
• Across all five systems MAE-V is consistently lower than MAE-A; MusicGen (text) has the lowest valence error (0.744), while InspireMusic (text) has the lowest arousal error (0.953).The authors suggest dynamic intensity and activation level are harder to match than positive–negative emotional character.Source: §4.1 · p. 3

4.4 Source–Generated Correlation

• MusicGen (text) had the strongest Corr-V (0.519) and Corr-A (0.666); text-conditioned systems also reached r=0.44–0.52 for valence and r=0.61–0.67 for arousal.Figure 3 plots source versus generated values, with points per track and a diagonal for perfect emotion preservation.Source: §4.1 · p. 3, §4.4 · p. 5, Figure 3 · p. 5

4.2 Genre-Wise Analysis

• Text-conditioned MusicGen and InspireMusic have lower VA-distance medians and tighter interquartile ranges than their audio-conditioned counterparts in nearly every genre.Figure 2 shows VA-distance distributions across the 10 GTZAN genres; lower distances indicate better alignment.Source: §4.2 · p. 3, Figure 2 · p. 4
• Classical and country yield lower distances and tighter spreads, whereas disco, metal, and pop show larger distances and broader distributions.The authors suggest high-energy, rhythmically driven profiles may be harder to match, particularly when arousal dominates.Source: §4.2 · pp. 3–4

4.3 Text vs. Audio Conditioning

• Within-family tests favor text conditioning for InspireMusic (win rate 67.3%, p < 0.001, |d| = 0.502) and MusicGen (win rate 64.5%, p < 0.001, |d| = 0.411).The authors attribute the advantage to explicit prompts combining semantic descriptions with emotion tags rather than source-audio conditioning alone.Source: §4.3 · p. 4, Table 2 · p. 4
• MusicGen’s text/audio comparison confounds conditioning mode with model variant because it uses different checkpoints; InspireMusic provides the cleaner controlled comparison.InspireMusic uses the same checkpoint for both conditioning types.Source: §4.3 · pp. 4–5

• Text conditioning outperforms audio conditioning for emotion-following, contrary to the expectation that source audio would better preserve emotional character.Source: §5 · p. 5
• The authors hypothesize that explicit emotion labels are easier for models to use than affective attributes that must be inferred from audio.They suggest audio conditioning may require preserving emotion alongside features such as timbre and rhythm.Source: §5 · p. 5
• Valence is preserved more reliably than arousal, which the authors relate to differences in the musical attributes associated with each dimension.They associate valence with mode and harmony, and arousal with tempo, dynamics, and energy.Source: §5 · p. 5
• Emotion-following varies by genre, with disco, metal, and pop posing greater challenges than classical and country.The authors suggest models may default toward moderate affective ranges, making extreme or genre-specific profiles harder to reproduce.Source: §5 · p. 5
• For developers, explicit emotion labels in text prompts are currently a more reliable strategy than audio-to-audio conditioning.Source: §5 · p. 5
• The findings are bounded by the emotion model, dataset, generation settings, and conditioning comparisons used.Music2Emotion supplies both source emotion estimates and generated-audio evaluation, so the metrics reflect one model’s representation rather than perceptual ground truth; other caveats include GTZAN’s limitations, single outputs with default settings, MusicGen’s different checkpoints, and GPT-4 prompt rewriting.Source: §5 · pp. 5–6

• Across 1,000 tracks, the pipeline compares three models under five conditioning configurations to evaluate emotion-following in generative music.Source: §6 · p. 6
• Text conditioning with explicit emotion cues significantly outperforms audio conditioning (p < 0.001).Source: §6 · p. 6
• The valence-arousal preservation gap and genre-dependent variation point to future work on additional models, larger datasets, human validation, and musical attributes mediating emotion preservation.Source: §6 · p. 6