Source-linked AI summary
YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality
Ruibin Yuan, Jiahao Pan, Junyan Jiang, Zhiyue Wu, Ziya Zhou, Jiankai Sun, Yizhi Li, Ge Zhang, Yicheng Gu, Zeyue Tian, Junyu Dai, Hanfeng Lin, Kai Li, Shangda Wu, Xuanjie Liu, Jiaming Wang, Zihan Liu, Yue Wang, Yinghao Ma, Hanzhi Yin, Kangrui Chen, Xinyue Zhang, Ziyang Ma, Mengqi Liao, Hejia Zhao, Guowei Huang, Chao Yan, Lei Ke, Jianwei Yu, Bei Liu, Joe Guo, Liumeng Xue, Gus Xia, Wei Xue, Yike Guo
TL;DR
Symbolic generators make composition readable but stop short of finished recordings, while audio generators produce songs without explicit scores. YuE2 unifies score-first composition and full-song audio generation; experts favor symbolic planning, and it leads evaluated public baselines on SongBench Global Avg.
Problem
The gap is whether one generator can make composition readable and editable while producing finished recordings, since symbolic and audio models typically expose only one side.
Source: §1 · p. 2Method
YuE2 uses one model to draft readable symbolic compositions, expand them into semantic tokens, and render full-song audio, with MERT2 and SheetSage2 supplying supervision from recordings.
Source: §1 · p. 2Results
Experts favor symbolic planning, and YuE2 leads evaluated public baselines on SongBench Global Avg (6.7316).
Source: §1 · p. 2, §8 · p. 21Takeaways & Limitations
Explicit scores let users inspect and revise musical decisions before rendering, with local edits changing targeted content while preserving unedited content.
Source: §1 · p. 2, §8 · p. 21
Abstract
from arXiv · showhide
Symbolic models make melody, harmony, rhythm, and form explicit but typically stop before a finished recording; audio models produce complete songs while leaving composition implicit. We introduce YuE2, which unifies symbolic and audio music generation at frontier quality through symbolic planning. A single AR-NAR Mixture-of-Transformers (MoT) first writes a readable score specifying melody and harmony, expands it into semantic music tokens, and realizes it as full-song audio. In comparisons using the same checkpoint, experts prefer symbolic planning for overall quality and musicality, with 49.3% of overall preferences versus 34.6% without planning. Experts also favor the unified model over a separate language model and diffusion Transformer. On WildSongBench, YuE2 scores 6.73 on SongBench Global Avg, exceeding all evaluated public baselines. Selecting from eight candidates (best-of-8), YuE2 reaches 6.96, the highest observed mean among all evaluated systems. Expert listening further establishes its competitiveness with proprietary song generators, favoring best-of-8 over Suno v4.5 and yielding nearly balanced preferences against Suno v5. To learn this generation process from recordings without aligned scores, we introduce MERT2 and SheetSage2 to supply semantic and symbolic supervision. MERT2 sets a new state of the art in music representation learning, surpassing previous best results on 14 of 15 MARBLE metrics; SheetSage2 leads 12 of 15 benchmark-metric pairs in our lead-sheet transcription comparison. The same checkpoint follows score edits while largely preserving unedited musical content and generates zero-shot covers without cover-specific training. Its readable score also enables agentic music editing, with external language models translating user feedback into revisions of the composition.
1 INTRODUCTION
YuE2 uses an explicit, editable score as the first stage of a single model that generates complete songs, bridging symbolic composition and audio performance. The paper also introduces MERT2 and SheetSage2 to derive semantic and symbolic supervision from recordings.
- YuE2 frames song generation as progressive musical commitment: text states intent, scores specify melody, harmony, rhythm, and form, and performance and audio resolve further details.Symbolic generators make composition readable but stop before a finished recording, while audio generators produce recordings but usually leave composition implicit.Source: §1 · p. 2
- YuE2’s single AR–NAR Mixture-of-Transformers writes a score specifying melody, chords, key, meter, tempo, and form, then generates semantic tokens and acoustic latents for stereo audio.It predicts symbolic and semantic sequences causally and realizes the acoustic sequence through bidirectional flow matching; symbolic planning supports frontier full-song quality competitive with evaluated proprietary systems.Source: §1 · p. 2
- 49.3% of expert judgments favor symbolic planning for overall quality versus 34.6% favoring generation without planning, with the same checkpoint, prompts, sampling budget, and decoder.With melody-and-chord planning in both systems, experts also prefer YuE2’s unified MoT to a separate LM+DiT for overall quality, musicality, audio quality, vocals, and accompaniment.Source: §1 · p. 2
- On WildSongBench, YuE2 scores 6.7316 on SongBench Global Avg, leading evaluated public systems, while best-of-8 scores 6.9632, the highest observed mean among evaluated systems.In expert listening, best-of-8 is preferred over Suno v4.5 and receives nearly balanced overall preferences against Suno v5 and Mureka 9; its mean tie-adjusted audio-quality preference across six proprietary systems is 58.9%.Source: §1 · p. 2
- The same checkpoint follows score edits while largely preserving unedited musical content, generates zero-shot covers without cover-specific training, and supports agentic revisions from user feedback.With full scores, it surpasses both evaluated cover systems on all eight work-identity retrieval measures, AudioBox production quality, and SongBench Musicality across 948 unseen works.Source: §1 · p. 2
- MERT2 establishes a new state of the art on MARBLE, surpassing previous best results on 14 of 15 metrics across nine tasks; SheetSage2-AR leads 12 of 15 benchmark–metric pairs in unified full-song transcription.Trained entirely on SheetSage2-Prober labels, SheetSage2-AR also surpasses its label generator on 10 of 15 benchmark–metric pairs under the same evaluation protocol.Source: §1 · p. 3
2 YUE2: COMPOSING IN SYMBOLS, PERFORMING IN AUDIO
YuE2 makes a readable symbolic score the composition state, then expands it into semantic tokens and full-song audio. The same checkpoint uses scores for creation, editing, and cover generation.
2.1 GENERATION OVERVIEW
- YuE2 progressively realizes a symbolic composition as semantic tokens and acoustic latents, which a separately trained 48-kHz stereo decoder turns into audio.The symbolic score is readable and editable; the semantic sequence runs at 25 Hz, and the acoustic latents retain detail for audio decoding.Source: §2.1 · p. 3
2.2 SYMBOLIC COMPOSITION AS A GENERATIVE STATE
- YuE2 serializes scores as ABC notation, with headers, barlines, section labels, chord symbols, and voices, then autoregressively emits the score and semantic tokens before acoustic rendering.The acoustic stream conditions on both completed sequences.Source: §2.2 · p. 3
2.3 AR–NAR MIXTURE-OF-TRANSFORMERS
- YuE2 uses a 28-layer AR–NAR backbone with separate stream-specific experts around shared attention: discrete score and semantic tokens are causal, while acoustic frames use bidirectional context.The NAR stream predicts acoustic flow velocity, updating all frames in parallel at each evaluation.Source: §2.3 · pp. 3–4, Figure 3 · p. 4
2.4 TRAINING
- Training combines next-token cross-entropy for symbolic and semantic spans with conditional flow matching for acoustic latents; four task settings support generation with different available inputs.The model trains on approximately 346,000 hours of music and has approximately 3.58B parameters.Source: §2.4 · pp. 4–5
2.5 CREATION, EDITING, AND COVER GENERATION
- The same checkpoint creates songs from generated scores, renders revised scores for editing, and renders transcribed reference scores for covers without original–cover training pairs.For editing and covers, the supplied score is a fixed prefix; cover generation uses SheetSage2 to transcribe the reference and a new style description for rendering.Source: §2.5 · p. 5
3 FROM RECORDINGS TO MUSICAL SUPERVISION
The section builds aligned symbolic, semantic, and acoustic supervision from recordings, using SheetSage2 lead sheets and MERT2-derived semantic codes to learn the complete s → c → z trajectory.
3.1 MERT2: MULTI-VIEW TARGETS AND FOUNDATION PRETRAINING
- MERT2 learns four cached code streams jointly constrained by aligned MuQ and Qwen2-Audio-Instruct views, then branches into full-song and causal adaptations.The Stage 1 foundation predicts the codes from masked audio; the full-song branch supports SheetSage2, while the causal branch continues toward the semantic tokenizer.Source: §3.1 · pp. 5–6
3.2 SHEETSAGE2: FULL-SONG TRANSCRIPTION
- SheetSage2-AR2 converts full recordings into editable lead sheets containing melody, chords, key, beat, downbeat, meter, and section structure.A full-context MERT2-FS encoder and autoregressive decoder produce chronological events, which a deterministic builder converts to ABC notation; training annotations are assembled under a shared vocabulary and timing convention.Source: §3.2 · pp. 6–7
3.3 MERT2 TOKENIZER: COMPACT SEMANTIC TARGETS
- Causal MERT2 is supervised on lyrics, mel, and chroma before a 32,768-entry quantizer emits one semantic ID every 40 ms.The deployed stream runs at 25 Hz and 375 bits/s; the lyric CTC and mel/chroma objectives continue supervising the discrete bottleneck.Source: §3.3 · pp. 7–8
3.4 CONSTRUCTING ALIGNED TRAINING SEQUENCES
- Training examples concatenate text and lyrics with symbolic and semantic targets and boundary-aligned acoustic latents, enabling matched s → c → z and omission ablations.A separate Oobleck VAE compresses 48-kHz stereo audio into a 64-dimensional acoustic bottleneck at 25 Hz.Source: §3.4 · p. 8
4 FRONTIER SONG QUALITY WITH EXPLICIT COMPOSITION
YuE2 achieves leading public-system SongBench results with symbolic planning, which expert comparisons favor for perceived song quality; experts also prefer its unified MoT over a separate LM+DiT when both use the same planning method.
4.1 FRONTIER FULL-SONG QUALITY ON WILDSONGBENCH
- YuE2 leads public systems on SongBench Musicality (5.9075) and Global Avg (6.7316); best-of-8 has the highest observed means across evaluated systems at 6.2666 and 6.9632, respectively.YuE2 and best-of-8 also score 8.2598 and 8.2714 on AudioBox PQ, exceeding every evaluated proprietary system.Source: §4.1 · p. 9, Table 2 · p. 10, Table 1 · p. 10
- YuE2 best-of-8 ranks second among public systems on SongEval Musicality (4.2506) and Avg (4.2960), behind HeartMuLa; YuE2 leads public systems on MuLan (0.5068) and AllMusicCaps (0.4054).YuE2's PER is 0.0844, compared with the lowest public-system PER of 0.0627 for MiniMax Music 3.Source: §4.1 · pp. 9–10, Table 2 · p. 10
- Experts favor YuE2 best-of-8 over Suno v4.5 for overall quality (57.3% versus 30.5%), while preferences against Suno v5 are nearly balanced (40.4% versus 39.9%).Suno v6 receives more overall preferences (59.3% versus 31.6%) despite YuE2 best-of-8’s higher SongBench Avg; best-of-8 also has a 58.9% average tie-adjusted audio-quality preference across six proprietary baselines.Source: Figure 7 · p. 11, §4.1 · p. 11
4.2 SYMBOLIC PLANNING IMPROVES PERCEIVED SONG QUALITY
- Expert comparisons favor symbolic planning for perceived song quality, melody, and chord progression.In a paired example, planning varies a low–low–high–high motif in the verse and recalls it an octave higher in the chorus, while the unplanned verse repeats B–D–E–E; its Em–D♯aug–Bm transition also interrupts the descending bass motion. The controlled comparison holds prompts, lyrics, candidate budget, and decoder fixed.Source: §4.2 · pp. 11–12, Figure 8 · p. 12, Figure 9 · p. 12
4.3 UNIFIED MOT IMPROVES SONG QUALITY OVER SEPARATE LM+DIT
- With symbolic planning in both systems, experts prefer YuE2’s unified MoT over separate LM+DiT for overall quality (53.4% versus 35.6%) and audio quality (48.5% versus 29.6%).Text-alignment preferences are nearly balanced at 34.0% versus 33.5%.Source: Figure 10 · p. 13, §4.3 · p. 13
5 THE SCORE AS A GENERATIVE INTERFACE
YuE2’s generated scores correspond closely to their audio realizations, and editing those scores changes requested musical content while largely preserving unedited material. The section evaluates score–audio agreement and controlled edits, including adherence and preservation.
5.1 THE GENERATED PLAN IS REALIZED IN AUDIO
- 0.9464 melody and 0.9246 chord sequence similarity for corresponding recordings far exceed 0.2160 and 0.2473 for mismatched recordings.Exact key agreement is 0.9307; tempo is within 8% for 0.9869 of prompts, vocal onset-interval F1 is 0.9396, form similarity is 0.7799, and section-boundary F1 is 0.8724. These are prompt means for 378–380 evaluable scores from 191 prompts.Source: Table 3 · p. 14, §5.1 · pp. 14–15
- 100% assignment accuracy shows melody and chord similarity identify the correct score–audio pairing in every evaluable two-candidate case.The advantage over controls persists when comparisons allow one song-wide pitch shift, so it extends beyond global key and register differences.Source: §5.1 · p. 15
5.2 CONTROLLED SCORE EDITING
- 79.54% target-chord agreement, 84.17% exact target-pitch accuracy, and 73.43% relative onset accuracy show adherence to harmony, melody, and rhythm edits.Across these local edits, unedited melody and harmony agreement ranges from 90.16% to 94.34%; harmony edits retain 93.43% full-song vocal melody agreement.Source: §5.2 · p. 15, Table 4 · p. 16
- 90.58/100 weighted key agreement and 95.68% tempo Acc2 show adherence to global edits, while SongBench quality stays close to unedited controls.Edited-song quality is 6.64–6.73, versus 6.70 before melody, harmony, and key edits and 6.73 before rhythm edits.Source: §5.2 · p. 15, Table 4 · p. 16
Appendix D.2 gives the protocol and metric definitions.
YuE2 transfers its score-to-audio interface to unseen works without cover-specific training, and its readable score also enables iterative, agentic composition edits. Full-score conditioning best preserves work identity, while relaxing it increases style alignment and quality scores.
5.3 ZERO-SHOT COVER GENERATION PRESERVES WORK IDENTITY
- The zero-shot evaluation uses 948 unseen SHS100K works; YuE2 generates four recordings per work from a source score, lyrics, and target-style descriptions, with no candidate selection.Each query retrieves other performances from a gallery of 10,546 recordings; the exact source recording is excluded, and higher retrieval scores indicate stronger work-identity preservation.Source: §5.3 · p. 16
- Removing chords lowers CLEWS and Discogs-VINet mAP to 0.598 and 0.179, while removing the score lowers them to 0.006 and 0.004; Musicality rises from 5.104 to 5.490 and 5.691, respectively.Experts link this tradeoff to harmony: retaining source harmony favors fidelity to the original work, while relaxing it allows greater adaptation to target style.Source: §5.3 · pp. 16–17
- Without cover-specific training, YuE2 with the full score leads both evaluated cover systems on all eight retrieval measures, production quality, and Musicality; CLEWS and Discogs-VINet mAP are 0.647 and 0.288 versus SongEcho’s 0.419 and 0.122.Source: §5.3 · p. 16, Table 5 · p. 17
5.4 AGENTIC MUSIC EDITING: A CASE STUDY
- YuE2’s readable ABC score lets an external language-model agent revise melody, harmony, form, style, and lyrics, while YuE2 renders each version as a complete song.A researcher and agent reworked The Last Train from Mandarin pop into English jazz across nine curated steps and fourteen rendered versions; the researcher selected the final recording by listening.Source: §5.4 · p. 17
6 MUSIC UNDERSTANDING AND FULL-SONG TRANSCRIPTION … A.3 SELECTION AND EVALUATOR REUSE
MERT2 and SheetSage2-AR provide strong music-understanding and full-song transcription results, while the appendix specifies WildSongBench’s evaluators and candidate-selection procedures. The conclusion connects these capabilities to YuE2’s score-based generation and editing.
6.1 MERT2 LEADS MUSIC-UNDERSTANDING BENCHMARKS
- MERT2-30s exceeds external baselines on 14 of 15 MARBLE metrics, while MERT2-FS does so on 13 of 15 and stays within 1.87 points across all metrics.MERT2-FS achieves the highest scores on GiantSteps key, EmoMusic valence, and MTG-Jamendo Instrument AP, and ties MERT2-30s on Instrument ROC.Source: Table 6 · p. 18, §6.1 · p. 18
6.2 MERT2 RETAINS STRONG MUSICAL STRUCTURE AT 375 BITS/S
- At 375 bits/s, the MERT2 tokenizer leads five of six quantized-representation metrics, with post-VQ probe scores at 78.3–98.3% of pre-VQ scores.Beat F1 is 86.38, versus 79.72 for LeVo 2 and 76.50 for EnCodec; the tokenizer supplies YuE2’s semantic tokens.Source: §6 · p. 17, Table 7 · p. 19, §6.2 · p. 19
6.3 SHEETSAGE2-AR LEAD-SHEET TRANSCRIPTION
- SheetSage2-AR achieves the highest scores on 12 of 15 benchmark–metric pairs in the comparison; trained exclusively on SheetSage2-Prober-generated labels, it surpasses its label generator on 10 of 15 pairs under the same evaluation protocol.Source: Table 8 · p. 20, §6.3 · p. 20
7 RELATED WORK
- Prior systems either model structure through audio representations or generate symbolic sequences that require separate song realization; YuE2 jointly models readable scores, semantic tokens, and acoustic latents.The related work also distinguishes YuE2’s full-song score planning from methods that follow supplied controls or plan without jointly specifying melody, harmony, rhythm, and form in readable notation.Source: §7 · pp. 20–21
8 CONCLUSION
- The conclusion reports that YuE2 leads public baselines on standard WildSongBench SongBench Global Avg, while best-of-8 achieves the highest observed mean across evaluated systems.It also states that the same checkpoint supports score edits and zero-shot covers without cover-specific training, while largely preserving unedited musical content.Source: §8 · p. 21
A.1 QWEN3-OMNI PROMPT ALIGNMENT (Q3O)
- Q3O scores how well generated songs meet request requirements across seven dimensions, using the raw prompt and only dimensions specified by the request.Its two-pass process assigns dimension-importance weights from text, then audio-match scores; the prompt template supplies the title, language, genre, style tags, and raw user prompt.Source: §A.1 · p. 29, SYSTEM MESSAGE · p. 29, USER MESSAGE TEMPLATE · AUDIO ATTACHED · p. 30
A.2 PRODUCTION QUALITY AND AUDIO–TEXT ALIGNMENT
- AudioBox estimates production quality, while MuQ-MuLan and AllMusicCaps measure alignment with standardized style descriptions rather than the original user request.The standard comparison selects the lower-PER output from two candidates; best-of-8 filters by Musicality and Q3O before choosing the lowest PER, and HeartMuLa reuses SongEval and AudioBox in training-data and preference construction.Source: §A.2 · p. 30, §A.3 · p. 30
A.4 CALCULATION OF THE QUALITY AND ALIGNMENT INDICES · A.5 MUSICALITY AND MODEL SIZE · B REPRESENTATION DETAILS
YuE2’s two settings exceed every public baseline on both quality and alignment indices, while the appendix defines how these composites are standardized and displayed. It also reports YuE2’s Musicality and parameter-count comparison with public baselines.
A.4 CALCULATION OF THE QUALITY AND ALIGNMENT INDICES
- The quality composite weights standardized SongBench Global Avg twice SongEval Global Avg; alignment equally weights standardized MuLan, AllMusicCaps, and Q3O.SongBench Global Avg averages seven dimensions and SongEval Global Avg averages five; standardization uses the 15 external systems as the reference set, excluding both YuE2 settings.Source: §A.4 · p. 31
- Display indices map observed extrema across all 17 settings to 10 and 90, preserving rankings and pairwise dominance; AudioBox production quality is separately encoded by bubble area and excluded from Pareto calculations.The composites summarize standardized metric means for visualization; original metrics remain the primary reported measurements.Source: §A.4 · p. 31
- YuE2 and YuE2 (Bo8) score 82.82 and 88.34 on quality and 84.59 and 83.89 on alignment, exceeding every public baseline on both indices.Mureka 9 has the highest quality index (90), while Suno v5 has the highest alignment index (90).Source: §A.4 · p. 31
- YuE2 (Bo8), Suno v5, and Mureka 9 form the empirical Pareto front across quality and alignment indices.A setting dominates another when it is at least as high on both indices and strictly higher on at least one.Source: §A.4 · p. 31
- YuE2 exceeds every public baseline throughout SongBench quality weights of 0.6–0.8, whether quality uses global averages or Musicality scores.At the selected 2:1 weighting, YuE2 (Bo8) has the highest Musicality composite, while Mureka 9 has the highest global-average composite.Source: §A.4 · p. 31
A.5 MUSICALITY AND MODEL SIZE
- 81.63 is YuE2’s highest observed Musicality index among public generators in two-candidate settings, with 3.58B parameters versus HeartMuLa’s 78.74 and 5.43B.Six of eight baselines have both more parameters and a lower index; DiffRhythm 2 and SongBloom are smaller and have lower indices.Source: Figure 11 · p. 32, §A.5 · p. 32
- The Musicality index combines SongBench and SongEval Musicality dimensions using Appendix A.4’s prompt aggregation and 15-system standardization reference, then maps extrema across all 17 WSB settings to 10 and 90.It summarizes observed metric means, not a percentage.Source: §A.5 · p. 32
- Parameter counts include evaluated language models, generative diffusion networks, and conditioning encoders, while excluding audio tokenizers, VAEs, vocoders, and fixed buffers.Counts derive from checkpoint tensor shapes and model configurations; Best-of-8 uses the same model with a larger candidate budget and a different selection rule.Source: §A.5 · p. 32
B.1 MERT2 TARGET SYNTHESIS AND TRAINING STAGES · B.2 SEMANTIC TOKENIZER RETENTION PROTOCOL
The appendices specify how MERT2’s synthesized multi-view targets and staged tokenizer training are built, then define a controlled probe protocol for measuring semantic-tokenizer retention against other representations.
B.1 MERT2 TARGET SYNTHESIS AND TRAINING STAGES
- Offline targets combine frozen MuQ layer 7 [91] and Qwen2-Audio-Instruct layer 32 [9] features, fuse them, and quantize the shared representation with a four-level residual vector quantizer.Separate decoders reconstruct both source representation spaces; this shared target quantizer is distinct from YuE2’s single-codebook semantic tokenizer.Source: §B.1 · p. 33
- MERT2 processes 24-kHz mono audio through a 25-Hz ConvNeXt [48] frontend and a 24-layer, width-1,024 Conformer.The Conformer uses feed-forward width 4,096, 16 attention heads, and convolution kernel 31.Source: §B.1 · p. 33
- The curriculum moves from masked prediction of synthesized codes to causal adaptation, supervised fine-tuning, and Stage 4 semantic quantization; a separate full-song adaptation retains bidirectional context.Stage 3 adds lyric CTC and mel/chroma objectives; full-song adaptation produces MERT2-FS on 30–360-second recordings.Source: §B.1 · p. 33
- Stage 4 adapts CVQ-VAE [90] with a 32,768-entry cosine quantizer, gradually ramps quantized contributions over 5,000 steps, and freezes the frontend and layers 0–13 through step 20,000.Online codebook updates use EMA-decayed assignment frequencies and cosine-similarity-sampled feature anchors; the loss combines codebook, 0.5-weighted commitment, and temperature-0.07 contrastive terms.Source: §B.1 · p. 33
B.2 SEMANTIC TOKENIZER RETENTION PROTOCOL
- Retention is the post-VQ to pre-VQ task-probe score ratio, comparing 1,024-dimensional, 25-Hz layer-14 outputs with layer-12 outputs at a 375-bits/s discrete rate.The tokenizer uses a 32-dimensional codebook with 32,768 entries.Source: §B.2 · p. 34
- The retention comparison also includes LeVo 2 [44] at 350 bits/s and EnCodec [13] at its 1.5-kbit/s setting, using their specified post-quantization representations.LeVo 2 applies generation-time one-codebook RVQ to MusicFM hidden-state index 6; EnCodec uses its quantizer-decoded 128-dimensional latent with two 1,024-entry codebooks at 75 Hz.Source: §B.2 · p. 34
- Frozen encoders and separate probes share task splits and optimization settings; tasks use macro AUROC and average precision for MagnaTagATune, mean R2 for emotion, and beat F1 for GTZAN.MagnaTagATune averages frame representations over 30-second excerpts and 50 tags; GTZAN uses 10-second excerpts and a 70-ms tolerance, interpolating representations to a shared 100-Hz label grid. Probes use seed 1234 and at most 50 epochs; MTT and beat tracking checkpoints are selected by validation AUROC and beat F1, respectively.Source: §B.2 · p. 34
B.3 SHEETSAGE2-AR AND SHEETSAGE2-PROBER
SheetSage2-AR transcribes audio into a shared chronological score and is trained on labels generated by SheetSage2-Prober. It scores higher than Prober on 10 of 15 benchmark–metric pairs and exceeds ChordFormer on JAAH maj/min recall.
Autoregressive transcriber
- SheetSage2-AR uses a full-context MERT2-FS encoder and six-layer RoFormer decoder to predict beat, downbeat, meter, key, chord, structure, and vocal or instrumental melody.The shared event vocabulary encodes chronological timing, score attributes, and melody events.Source: §B.3 · pp. 34–35
Generating aligned annotations
- SheetSage2-Prober supplies AR training labels: its MERT2-based task heads use partial-annotation masking and CRF decoding, and AR trains on 441,094 recordings totaling 28,434 hours.The generated labels use a common vocabulary and timing convention; AR training uses next-token cross-entropy.Source: §B.3 · p. 35
Comparison with SheetSage2-Prober
- SheetSage2-AR scores higher than SheetSage2-Prober on 10 of 15 benchmark–metric pairs, generating attributes in one chronological event sequence instead of using task-specific CRF decoders.The comparison uses the same evaluation protocol and benchmark cohorts.Source: §B.3 · p. 35, Table 11 · p. 36
JAAH chord evaluation
- On JAAH, SheetSage2-AR exceeds ChordFormer by 5.06 percentage points and Prober by 1.56 percentage points in maj/min recall.The evaluation covers all 113 JAAH recordings with fixed checkpoints and inference settings.Source: §B.3 · p. 37
Rock Corpus vocal melody
- On the 194 Rock Corpus recordings with nonempty reference note lists, vocal-melody F1 is 50.71 for SheetSage1, 68.02 for Prober, and 69.15 for AR.The six recordings with empty reference lists are excluded from these reported restricted-average values; the full 200-song average assigns them zero F1.Source: §B.3 · p. 37
C ARCHITECTURE AND TRAINING DETAILS · D EVALUATION AND STATISTICAL DETAILS
YuE2 is trained through a staged AR–NAR process that mixes score-conditioned and score-free tasks in one model. The section also specifies its annotation pipeline, training setup, and interfaces for creation, editing, and covers.
C ARCHITECTURE AND TRAINING DETAILS
- YuE2, MERT2, and SheetSage2-AR are trained primarily on CC0 music and synthetic data, with most synthetic data supplied under license by Tokenwave.AI.Table 12 summarizes the training corpus sizes in rounded audio hours.Source: C ARCHITECTURE AND TRAINING DETAILS · p. 38, Table 12 · p. 38
- YuE2 adapts YuE’s annotation pipeline: Qwen3-Omni labels musical and vocal attributes, SongFormer supplies lyric-section structure, and Qwen3-ASR transcribes sung lyrics.These annotations provide text and lyric conditions; corresponding symbolic scores, semantic tokens, and acoustic latents are constructed as described in Section 3.Source: C ARCHITECTURE AND TRAINING DETAILS · p. 38
- Four balanced-ratio tasks train one YuE2 model; the first two retain semantic tokens while toggling score input, enabling same-checkpoint planning comparisons.All tasks train acoustic generation with flow matching, and next-token loss supervises included score and semantic tokens.Source: C ARCHITECTURE AND TRAINING DETAILS · p. 38, Table 14 · p. 38
- Training supplies target-derived score and semantic-token sequences as conditioning prefixes, while text and lyrics may be dropped separately or together for classifier-free guidance.Source: C ARCHITECTURE AND TRAINING DETAILS · p. 38
- A 60,000-update AR checkpoint initializes joint AR–NAR training; NAR experts initialize from AR counterparts, followed by 40,000 joint updates and 6,000 annealing updates.The optimizer is reset for joint training and its state is retained for annealing; the three phases have specified learning-rate schedules and context lengths.Source: C ARCHITECTURE AND TRAINING DETAILS · p. 39, Table 15 · p. 39
- All phases use bfloat16, batch size 256, gradient clipping at 1.0, and 64 H800 GPUs; songs are packed without temporal splitting, with attention and positional indices reset at document boundaries.Adam uses β = (0.9, 0.98) and weight decay 0.05 during AR pretraining, then β = (0.9, 0.95) and weight decay 0.1 during joint training and annealing.Source: C ARCHITECTURE AND TRAINING DETAILS · p. 39
- Cover training uses no original–cover pair supervision; evaluation uses the general song-generation checkpoint without cover-specific fine-tuning on SHS100K works absent from training.The conditioning y′ combines source lyrics with the target style description.Source: C ARCHITECTURE AND TRAINING DETAILS · p. 39
- Creation generates a score, whereas editing and cover generation supply a score as a fixed prefix.Table 16 summarizes score conditioning for the three interfaces.Source: C ARCHITECTURE AND TRAINING DETAILS · p. 39, Table 16 · p. 39
D.1 GENERATED-SCORE AGREEMENT WITH AUDIO · D.2 CONTROLLED SCORE EDITING ON WILDSONGBENCH
D.1 evaluates how well generated scores match their corresponding audio, including whether scores identify the correct recording among two candidates. D.2 tests whether YuE2’s recordings follow controlled score edits, with onset timing moving toward edited rhythm targets.
D.1 GENERATED-SCORE AGREEMENT WITH AUDIO
- D.1’s score–audio comparisons assess ordered melody and chord content, rhythm, key, tempo, musical form, and section boundaries as separate dimensions.Rhythm interval F1 compares ordered onset intervals with a tolerance of min(0.125, 0.25d) beats, independently of pitch; form compares section roles and order using melody and harmony content, while boundaries are evaluated separately.Source: §D.1 · pp. 40–41
- Corresponding-audio section-boundary F1 was 0.8724 with translation and 0.5358 at the original recovered beat origin.Translation uses a single global shift estimated from exact melody-pitch matches; the two scores reflect relative section placement and, without translation, start-position and analyzer beat-origin discrepancies.Source: §D.1 · p. 41
- With global transposition allowed, corresponding-audio melody and chord scores were 0.9465 and 0.9245, versus 0.3152 and 0.4658 for the other same-prompt recording.The content advantage persists after removing global key and register differences; half/double-tempo tolerance raises corresponding-audio tempo accuracy from 0.9869 to 1.0000, while specified-tempo accuracy remains primary.Source: Table 17 · p. 41, §D.1 · p. 41
- Under a lower-bound convention that assigns zero to undefined bounded scores, melody and harmony similarities were 0.9343 and 0.9122 across all 192 prompts.The same convention gives corresponding-audio weighted key agreement of 0.9414, tempo accuracy of 0.9766, form of 0.7696, rhythm of 0.9277, and boundary F1 of 0.8628.Source: §D.1 · p. 41
- The correct score-to-recording assignment had a higher similarity sum in all 188 cases for each metric, without ties.The melody and chord tasks each covered 188 prompts, with 187 prompts in common; assignment accuracy identifies each score’s corresponding recording from two candidates.Source: §D.1 · pp. 41–42
D.2 CONTROLLED SCORE EDITING ON WILDSONGBENCH
- D.2 applies key, tempo, melody, harmony, and rhythm edits while holding lyrics, style, checkpoint, sampling settings, and the main evaluation acoustic decoder fixed.Key edits transpose all voices and declarations; tempo edits scale notated BPM; melody and rhythm edits affect up to four bars of the first chorus, while harmony edits alter chord spans and preserve chord quality and timing.Source: §D.2 · p. 42
- 96.79% of rhythm-edit pairs observable in both conditions moved closer to the target, and mean target-onset error fell from 0.501 to 0.037 quarter-note beats.Onset accuracy counts unresolved correspondences as misses; coverage was 80.36%, or 74.95% for pairs observed in both conditions. Instrumental target controls reached 94.08% accuracy, while unchanged and opposite-shift controls scored zero.Source: §D.2 · p. 43, Table 18 · p. 43
- Across 764 recordings, tempo Acc2 was 95.68% and Acc1 was 75.92%.Acc2 accepts estimates within 4% of the target at beat levels {1/3, 1/2, 1, 2, 3}; Acc1 accepts only the target beat level. The section also reports quality across seven SongBench dimensions and lyric retention using PER and CER after melody, harmony, rhythm, and key edits.Source: §D.2 · pp. 43–44, Table 19 · p. 44
D.3 COVER GENERATION AND VERSION RETRIEVAL · D.4 EXPERT LISTENING EVALUATION · D.4.1 COMPLETE FRONTIER PREFERENCES
Across the evaluated proprietary generators, experts favor YuE2 for audio quality, while best-of-8 wins overall against Suno v4.5 and is nearly balanced against v5. In fixed-source cover generation, retaining the source score favors work-identity retrieval, while relaxing score constraints yields higher audio quality and target-style alignment.
D.3 COVER GENERATION AND VERSION RETRIEVAL
- The cover task is zero-shot on unseen SHS100K works and uses YuE2’s general checkpoint without cover-specific training or paired original-to-cover supervision.The source score is supplied only at inference through the existing conditioning interface; SheetSage2 is an external analyzer.Source: §D.3 · p. 44
- The cover evaluation uses 948 eligible source recordings from distinct works, with two same-work alternatives selected for contrasting annotated style descriptions rather than waveform reconstruction.Source annotations include lyrics and score information; Qwen3-Omni supplies genre, instrumentation, voice, tempo, and mood descriptions.Source: §D.3 · p. 44
- Cover conditions provide the full melody-and-chord score, the score without chords, or an empty score prefix; all retain lyrics and target style, and YuE2 renders without the source waveform.All conditions use the same generator checkpoint and 48-kHz stereo decoder.Source: §D.3 · pp. 44–45
- Without candidate selection, removing source chords raises Musicality over the full-score condition for 798 of 948 works (84.2%), and supplying no score does so for 864 (91.1%).These are descriptive comparisons of work means after averaging four outputs per work; experts associate retaining source harmony with work fidelity and relaxing it with greater target-style adaptation.Source: §D.3 · pp. 45–46
D.4 EXPERT LISTENING EVALUATION
- The expert study reports six separate judgments, including direct overall quality; 62 trained or audio-AI expert annotators contributed to the panel, with identity-hidden randomized comparisons and listening and consistency checks.Overall quality is not an average of the other five responses.Source: §D.4 · p. 46, Table 20 · p. 46
- The planning-versus-architecture comparison holds prompts and candidate budget fixed, comparing melody-and-chord-planned MoT with an LM+DiT baseline trained on the same data volume.The comparison uses 192 prompts and two candidates per prompt; candidates are transcribed four times and selected by lowest lyric PER.Source: §D.4 · pp. 46–47
D.4.1 COMPLETE FRONTIER PREFERENCES
- Experts prefer YuE2 over all six proprietary baselines for audio quality, while overall preferences favor YuE2 in two comparisons.Preferences for musicality, text alignment, vocals, and accompaniment vary by baseline.Source: Figure 12 · p. 47, §D.4.1 · p. 47
- Best-of-8 wins overall preferences over Suno v4.5, is nearly balanced against Suno v5, and loses to Suno v6; experts favor it for audio quality in five of six comparisons (58.9% average).The six-baseline average audio-quality preference is 58.9%.Source: Figure 13 · p. 48, §D.4.1 · p. 48
D.4.2 COMPLETE PLANNING AND ARCHITECTURE COMPARISONS … YuE2 vs Separate LM+DiT
Expert comparisons favor complete symbolic planning over generation without planning, and favor YuE2’s unified MoT over a separate LM+DiT on several criteria. The evaluation also reports melody and chord judgments and its statistical procedures.
D.4.2 COMPLETE PLANNING AND ARCHITECTURE COMPARISONS
- Full melody-and-chord planning receives more preferences than no planning on all six criteria, with the largest margins in overall quality and musicality.Figure 14 compares planning conditions within YuE2; each panel contains five direct comparisons.Source: Figure 14 · p. 49, §D.4.2 · p. 49
- Melody-only planning also beats no planning across six dimensions; adding chords improves four preference criteria, while audio quality and vocals are nearly balanced.The four improved criteria are overall quality, musicality, text alignment, and accompaniment.Source: §D.4.2 · p. 49
YuE2 vs Separate LM+DiT
- The unified MoT is preferred over separate LM+DiT for overall quality, musicality, audio quality, and vocals both with and without symbolic planning.With melody-and-chord planning, accompaniment favors the MoT and text alignment is nearly balanced; without planning, text alignment favors the MoT and accompaniment is nearly balanced.Source: §D.4.2 · p. 49, Table 22 · p. 54
D.4.3 MELODY AND CHORD EVALUATION
- Four experts evaluated 100 song pairs with and without planning, holding prompts, checkpoint, candidate budget, and decoder fixed; each criterion received 200 judgments.Listeners assessed melody, chord progression, and overall quality separately; each pair received two evaluations from different listeners, and presentation order was randomized with system identities hidden.Source: §D.4.3 · p. 50
D.4.4 STATISTICAL PROTOCOL
- Expert standard errors use two-way CR1 clustering by evaluator and song; pointwise 95% t intervals and unadjusted two-sided tests compare preference with 50%.Tables 21 and 22 provide counts, CR1 intervals, and p-values for expert direct and average comparisons.Source: §D.4.4 · p. 50, Table 21 · p. 50, Table 22 · p. 54
- 58.9% was the best-of-8 average audio-quality preference (CR1 95% CI 55.6–62.2%; p = 1.70×10−6).Average comparisons weight model pairs equally and responses equally within each pair.Source: §D.4.4 · p. 50
E FULL SCORES FOR THE PLANNING COMPARISON
The full-score comparison shows that symbolic planning produces more varied instrumental responses, livelier rhythmic phrasing, and a more settled ending than generation without planning.
E FULL SCORES FOR THE PLANNING COMPARISON
- The planned score starts from its generated plan; SheetSage2 transcribes the unplanned audio, and expert score corrections were made only for display, not regeneration.Source: E FULL SCORES FOR THE PLANNING… · p. 55
- Planned-song fills connect vocal phrases and smooth the chorus entry, while comparable responses are sparse in the unplanned song after its introduction.The full-song comparison uses songs generated from the same prompt and lyrics.Source: E FULL SCORES FOR THE PLANNING… · pp. 55–56
- Planned passages use syncopation and varied note lengths to animate vocal phrasing, whereas the unplanned song’s continuous sixteenth-note texture sounds more monotonous.Source: E FULL SCORES FOR THE PLANNING… · p. 56
- The planned song reprises its closing line so the ending settles, while the unplanned song’s single statement feels abrupt.Source: E FULL SCORES FOR THE PLANNING… · p. 56