Source-linked AI summary
DiffRhythm+: Controllable and Flexible Full-Length Song Generation with Preference Optimization
Huakang Chen, Yuepeng Jiang, Guobin Ma, Chunbo Hao, Shuai Wang, Jixun Yao, Ziqian Ning, Meng Meng, Jian Luan, Lei Xie
TL;DR
Full-length song generation still faces data imbalance, limited style control, and inconsistent musical quality. DiffRhythm+ combines expanded balanced training data, multimodal style conditioning, and preference optimization; it reports improvements over DiffRhythm and a narrower gap with state-of-the-art open-source models.
Problem
Full-length song generation remains challenged by imbalanced training data, limited style controllability, and inconsistent musical quality.
Source: §I · p. 2Method
DiffRhythm+ combines expanded balanced training data, text-and-audio style conditioning, and preference optimization for full-length song generation.
Source: §I · p. 2Results
DiffRhythm+ improves intelligibility, musicality, and generation efficiency over its predecessor while substantially closing the gap with state-of-the-art open-source models.
Source: §VI · p. 7Takeaways & Limitations
The framework offers controllable full-length lyric-to-song generation with improved alignment to listener preferences.
Source: §I · p. 2Future work targets reducing reliance on imprecise timestamps and adding richer structural cues to improve coherence and expressiveness.
Source: §VI · p. 7
Abstract
from arXiv · showhide
Songs, as a central form of musical art, exemplify the richness of human intelligence and creativity. While recent advances in generative modeling have enabled notable progress in long-form song generation, current systems for full-length song synthesis still face major challenges, including data imbalance, insufficient controllability, and inconsistent musical quality. DiffRhythm, a pioneering diffusion-based model, advanced the field by generating full-length songs with expressive vocals and accompaniment. However, its performance was constrained by an unbalanced model training dataset and limited controllability over musical style, resulting in noticeable quality disparities and restricted creative flexibility. To address these limitations, we propose DiffRhythm+, an enhanced diffusion-based framework for controllable and flexible full-length song generation. DiffRhythm+ leverages a substantially expanded and balanced training dataset to mitigate issues such as repetition and omission of lyrics, while also fostering the emergence of richer musical skills and expressiveness. The framework introduces a multi-modal style conditioning strategy, enabling users to precisely specify musical styles through both descriptive text and reference audio, thereby significantly enhancing creative control and diversity. We further introduce direct performance optimization aligned with user preferences, guiding the model toward consistently preferred outputs across evaluation metrics. Extensive experiments demonstrate that DiffRhythm+ achieves significant improvements in naturalness, arrangement complexity, and listener satisfaction over previous systems.
I. INTRODUCTION
Full-length song generation combines vocals and accompaniment, but DiffRhythm is limited by imbalanced, relatively small training data and audio-only style control. DiffRhythm+ addresses these gaps with balanced data, multimodal style conditioning, and preference optimization.
I. INTRODUCTION
- Song generation synthesizes vocals and instrumental accompaniment together, unlike SVS and text-to-music approaches used independently.SVS typically produces vocals without accompaniment, while text-to-music systems produce instrumentals without vocal melodies and lyrics.Source: §I · p. 1
- DiffRhythm’s imbalanced dataset, limited scale, audio-only style prompts, and occasional instability constrain lyric consistency, arrangement richness, and stylistic flexibility.The dataset’s approximate Chinese, English, and instrumental ratio is 3:6:1; Chinese-song performance is poorer, with repetitive or missing lyrics.Source: §I · p. 2
- DiffRhythm+ integrates large-scale balanced training data, multimodal style control, and listener-preference optimization for full-length song generation.The framework aims to improve quality and creative versatility.Source: §I · p. 2
- Preference optimization guided by music-aesthetic evaluation models aligns generated outputs more closely with human preferences and targets greater expressiveness, arrangement complexity, and listener satisfaction.Source: §I · p. 2
- MuLan enables fine-grained style conditioning through both textual descriptions and audio prompts, increasing flexibility and creative control.Source: §I · p. 2
II. RELATED WORK
Prior song-generation work includes autoregressive language-model and diffusion approaches. The former can be slow, while diffusion systems support long-form generation and flexible conditioning; hybrid systems may retain autoregressive inference costs.
A. Language Model-Based Song Generation
- Language-model song generators use sequential musical-token modeling and multimodal conditioning, but face challenges with long-form, high-fidelity audio and coherence across sections.Examples include Jukebox, MelodyLM, SongCreator, and SongGen.Source: A. Language Model-Based Song Generation · p. 2
- YuE scales model capacity and training data for long-form lyric-to-song generation, but its large-scale autoregressive modeling brings slow inference and high computational demands.Source: A. Language Model-Based Song Generation · p. 2
B. Diffusion-Based Song Generation
- Latent diffusion supports efficient synthesis of extended music, multimodal conditioning, and editing, while control systems target attributes such as genre, instrumentation, and melody.Source: B. Diffusion-Based Song Generation · p. 2
- Hybrid systems combine diffusion audio decoding with autoregressive symbolic generation, but often have complex pipelines and inherit slow inference from autoregressive components.Source: B. Diffusion-Based Song Generation · p. 2
III. DIFFRHYTHM+
DiffRhythm+ combines a VAE with non-autoregressive DiT-based flow matching, with lyrics and style prompts conditioning generation. Its architecture uses MuLan for unified text-and-audio style control and refines lyric encoding and alignment.
A. Overview
- DiffRhythm+ pairs a VAE that encodes and reconstructs audio latents with a DiT-based flow-matching module that models musical latents non-autoregressively.Style conditioning and lyric-to-latent alignment are also refined, and a preference-optimization stage is added.Source: A. Overview · pp. 2–3
B. Detailed Architecture
- The DiT receives style, timestep, and lyric features; MuLan maps audio and text into a shared embedding space, while a G2P tokenizer represents lyrics as phoneme sequences.Style and lyric embeddings, the timestep embedding, and noisy latents are concatenated along the feature dimension before entering DiT blocks.Source: B. Detailed Architecture · p. 3
- MuLan style embeddings replace DiffRhythm’s LSTM-based style extractor, allowing text or audio prompts to control song style through a unified representation.Source: B. Detailed Architecture · p. 3
C. Training Objective
DiffRhythm+ learns conditional flow matching in latent space to generate songs from lyrics and style features, then adapts diffusion generation using metric-derived preference pairs. Its preference procedure filters pairs by score gap and extends optimization across the full diffusion trajectory.
C. Training Objective
- Conditional flow matching trains a time-dependent vector field to transform prior samples into data samples, conditioned on lyric and style features.The coupling independently samples prior and data latents, with a straight conditional path between them.Source: C. Training Objective · p. 3
D. Preference Optimization
- DiffRhythm+ adapts direct preference optimization to diffusion models, whose optimization target spans the entire diffusion trajectory.The diffusion-DPO loss is derived as an L2 noise-estimation loss using Jensen’s inequality and standard latent-diffusion approximations.Source: D. Preference Optimization · pp. 3–4
- The framework’s preference objective rewards outputs under a reward model while penalizing divergence from a reference model.The prompt, generated output, reward model, and reference model are denoted by τ, x, rϕ, and πref, respectively.Source: D. Preference Optimization · p. 3
- SongEval scores vocal songs while Audiobox-aesthetic scores instrumentals, with Audiobox scores mapped from 1–10 to SongEval’s 1–5 scale.The method primarily relies on SongEval for song-level aesthetic evaluation because Audiobox scores may not always align with human ratings and SongEval cannot score instrumentals.Source: D. Preference Optimization · p. 4
- The preference dataset retains generated best–worst pairs only when their score gap exceeds a preset threshold.Candidate pairs are selected from each batch using automated aesthetic scores.Source: D. Preference Optimization · p. 4
A. Dataset
DiffRhythm+ is pre-trained on a large balanced corpus, then fine-tuned on a smaller quality-filtered subset.
A. Dataset
- 300,000 hours of raw recordings yielded ∼120,000 hours balanced 2:2:1 across Chinese songs, English songs, and instrumentals, then 25,000 hours for supervised fine-tuning.Audiobox-aesthetic and SongEval scores set quality thresholds using a manually selected high-quality subset; Audiobox alone is used for instrumentals and a weighted combination for vocal tracks.Source: A. Dataset · p. 4
B. Baseline Systems
The evaluation includes autoregressive and diffusion-based song-generation baselines, including SongLM, YuE, and DiffRhythm.
B. Baseline Systems
- SongLM combines an autoregressive language model for lyric-conditioned semantic tokens with a diffusion generator for music reconstruction.The model uses lyrics and short acoustic prompts as conditioning.Source: B. Baseline Systems · pp. 4–5
- YuE [21] uses a two-stage dual-language-model framework, separately modeling vocal and accompaniment tokens before residual audio reconstruction.Its first stage uses track-decoupled next-token prediction.Source: B. Baseline Systems · p. 5
- DiffRhythm [25] uses latent diffusion with a DiT conditioned on lyrics and style prompts, plus sentence-level lyric alignment.The passage describes it as a fast, non-autoregressive full-length song-generation model.Source: B. Baseline Systems · p. 5
C. Evaluation Metrics
Evaluation combines objective audio, alignment, and content-quality metrics with listener ratings of generated songs.
C. Evaluation Metrics
- Objective evaluation covers audio-distribution similarity, phoneme error, runtime, style alignment, and aesthetic or content qualities.Metrics include FAD, KL divergence, PER, RTF, CLaMP 3, Audiobox-aesthetic, and SongEval; RTF is measured on an Nvidia RTX 4090.Source: C. Evaluation Metrics · p. 5
- Thirty listeners—15 music experts and 15 without formal musical backgrounds—rate generated samples for musicality, audio quality, and intelligibility on a 1–5 scale.The subjective evaluation uses mean opinion scores.Source: C. Evaluation Metrics · p. 5
D. Model Configuration
DiffRhythm+ uses a roughly 1.1B-parameter model, trained in stages for longer sequences and configured with specific diffusion, guidance, and preference-training settings.
D. Model Configuration
- The model has approximately 1.1B parameters and reuses DiffRhythm’s VAE architecture and pre-trained weights.Its DiT backbone has 16 LLaMA decoder layers, a 2048-dimensional hidden size, and 32-head self-attention.Source: D. Model Configuration · p. 5
- Training proceeds from pre-training on sequences up to 2048 tokens to fine-tuning on sequences up to 6144 tokens.SFT runs for 10 epochs and DPO for 8 epochs.Source: D. Model Configuration · p. 5
- Inference uses a 32-step Euler ODE solver with classifier-free guidance at scale 4, while lyrics and style prompts each receive independent 20% dropout.Source: D. Model Configuration · p. 5
- DPO uses β = 2000, a 0.4 win-lose score-gap threshold, winners scoring above 3, and a per-GPU batch size of 8 across eight GPUs.AdamW is used throughout; learning rates are 1×10−4 for pre-training, 1×10−5 for SFT, and 1×10−6 for DPO.Source: D. Model Configuration · p. 5
V. EXPERIMENTAL RESULTS
DiffRhythm+ improves on DiffRhythm across objective metrics and subjective ratings, while narrowing the musical-quality gap with YuE; YuE nevertheless has the highest MOS.
A. Subjective Evaluation
- YuE achieves the highest MOS across intelligibility, musicality, and quality, while both DiffRhythm+ variants exceed earlier versions in median and upper-quartile scores.DiffRhythm+ improves particularly in intelligibility and perceived audio quality.Source: A. Subjective Evaluation · pp. 5–6
B. Objective Evaluation
- DiffRhythm+ improves over DiffRhythm across objective metrics and subjective scores, narrowing the musical-quality gap with YuE.Source: B. Objective Evaluation · p. 6, A. Subjective Evaluation · p. 6
- 14.85% and 14.96% PER for DiffRhythm+ (base/full) are the lowest among baseline systems, compared with YuE at 15.14%; RTF is 0.036–0.039 versus YuE at 10.385.DiffRhythm+ also improves KL divergence and FAD over DiffRhythm, with base/full scores of 0.488/1.835 and 0.512/1.872, respectively, approaching YuE at 0.372/1.624.Source: B. Objective Evaluation · p. 6
- On SongEval, DiffRhythm+ performs on par with YuE, while AudioBox-aesthetic favors DiffRhythm+ but shows discrepancies with human evaluations.SongEval aligns more closely with subjective quality, and the results indicate strong musicality, structure, and naturalness.Source: B. Objective Evaluation · p. 6
C. Ablation Studies
Ablations favor expanded, balanced data, mixed MuLan style conditioning, and DPO after SFT; the conclusion also identifies future needs in control and structural cues.
C. Ablation Studies
- DPO after SFT yields substantial quality gains, with the best improvements when generated outputs serve as winners; moderate settings work best, while aggressive settings risk instability or overfitting.Source: C. Ablation Studies · p. 6
- Training with both audio and text MuLan embeddings achieves the best style-conditioning results and generalizes better to text-prompt inference than audio-only models.MuLan features outperform T5 embeddings with LSTM, and expanding data and balancing Chinese and English further improve quality, consistency, and multilingual performance.Source: C. Ablation Studies · p. 6
VI. CONCLUSION AND FUTURE WORK
- DiffRhythm+ combines expanded, balanced training data, multimodal style conditioning, and DPO to improve intelligibility, musicality, and generation efficiency over DiffRhythm while closing the gap with leading open-source models.Source: §VI · p. 7
- Future work targets greater model capacity, stronger preference alignment, finer-grained controls, less reliance on imprecise timestamps, and richer structural cues.The authors aim to use the structural cues to further improve coherence and expressiveness.Source: §VI · p. 7