Source-linked AI summary
Piano2Pop: Expanding Piano Demos into Pop Productions Without Aligned Supervision
Yun-Chen Cheng, Chih-Pin Tan, Tzu-Hung Huang, Yi-Hsuan Yang
TL;DR
Piano2Pop tackles how to turn piano MIDI demos into full pop productions when aligned piano-pop supervision is scarce and mismatched. It trains a pretrained flow-matching model on synthetic pairs derived from pop recordings, using alternative pseudo-demo routes. Pair-supervised pseudo-demos achieve the best objective results on real-piano inputs, and the findings suggest task-specific modeling is needed.
Problem
Turning piano demos into full productions requires inferring musical intent, while aligned piano–audio supervision is scarce and recordings may differ in duration, tempo, and structure.
Source: §1 · p. 1Method
The approach fine-tunes a piano-conditioned generation model on time-aligned synthetic pairs, comparing pair-free transcription with pair-supervised piano-cover pseudo-demos.
Source: §1 · p. 1Results
Takeaways & Limitations
The findings suggest Piano2Pop should be treated as a task of its own and needs task-specific modeling.
Source: §6 · p. 4No user study was conducted; the research used publicly available YouTube music and machine-generated audio.
Source: §7 · pp. 4–5
Abstract
from arXiv · showhide
We introduce Piano2Pop, a music-expansion task that transforms piano MIDI into instrumental pop productions. Enabled by recent advances in music generation and polyphonic music transcription, our pseudo-supervised approach conditions a pretrained flow-matching model on piano rolls without aligned piano-pop supervision. We train on synthetic piano-instrumental pairs derived from pop recordings, comparing pair-free transcription, including function-selective (FST) and register-density (RDA) augmentations, with a pair-supervised piano-cover generator as pseudo-demo sources. On 476 testing real-world piano-cover/pop pairs, the pair-supervised approach achieves the best Frechet Audio Distance, Accompaniment Prompt Adherence, and spectral shape on real-piano inputs, while FST mitigates piano-heavy outputs in pair-free training and remains a promising alternative. All fine-tuned systems significantly outperform the zero-shot baselines, which suggests a role for task-specific modeling.
1. INTRODUCTION
Piano2Pop expands a piano MIDI demo into a full instrumental pop production while preserving its musical intent. The paper addresses scarce, mismatched piano-pop supervision with pseudo-supervised training and evaluates alternative pseudo-demo sources.
1. INTRODUCTION
- Piano2Pop expands a piano sketch into an instrumental pop production, developing its structure, melody, harmony, rhythm, and instrumentation rather than merely accompanying an unchanged piano part.The reverse task, piano cover generation, maps pop audio to piano and performs music reduction.Source: §1 · p. 1
- Aligned MIDI–audio supervision is scarce and messy for piano-to-pop because covers and corresponding pop recordings can differ in duration, tempo, and overall structure.Existing polyphonic MIDI-to-audio systems focus on synthesis and require aligned pairs [4].Source: §1 · p. 1
- The proposed pseudo-supervised strategy derives piano MIDI pseudo-demos from unpaired pop instrumentals to create time-aligned training pairs, using pair-free transcription or a pair-supervised piano arrangement model.The routes may preserve different musical information and may differ substantially from real piano demos.Source: §1 · p. 1
- The study asks whether task-specific adaptation improves piano–production adherence and spectral-shape matchness, and how pseudo-demo construction affects transfer to real piano covers.The latter is measured by closeness to real-pop audio distributions.Source: §1 · p. 1
- The paper contributes a piano-conditioned generation framework and an objective evaluation protocol with tailored behavioral metrics.Source: §1 · p. 1
2. RELATED WORK
Related work situates Piano2Pop among symbolic piano orchestration and temporally controlled or condition-complementing music generation. Unlike symbolic orchestration, Piano2Pop generates instrumental audio directly, including its sonic realization.
Orchestration in the Symbolic Domain
- Symbolic piano-to-ensemble orchestration also expands a piano sketch into a multi-instrument arrangement, using paired scores [7] or alternatives motivated by pair scarcity.Those methods output symbolic scores, whereas Piano2Pop combines arrangement with audio realization, including timbre and production texture.Source: §2 · p. 1
Temporally Controlled Music Generation
- MuseControlLite and JASCO steer generation with time-aligned controls [10], while SingSong and StemGen generate complementary accompaniments or stems [11].ACE-Step supports audio-based cover generation and editing [6]; Piano2Pop instead uses polyphonic MIDI as a sketch, preserving entangled interactions [13].Source: §2 · p. 2
3. METHODOLOGY
The methodology creates synthetic piano–instrumental training pairs from pop recordings, then fine-tunes a pretrained generator conditioned on piano rolls. It compares two pseudo-demo routes and adds pair-free augmentations.
3. METHODOLOGY
- The study focuses on fixed-length 30-second audio segments, which often span a musical section, and uses synthetic pairs to fine-tune a pretrained generator with a piano-roll encoder.Source: §3 · p. 2
3.1. Pair-Free Pseudo-Demo Generation
- Pair-free demos come from transcribing the full instrumental as piano by restricting MuScriptor to Grand Piano, then assigning velocities from onset-energy rank quantiles.The quantile mapping uses fixed quantile–velocity anchors to preserve relative dynamics rather than absolute loudness.Source: §3.1 · p. 2
- RDA tests register and density alignment through octave folding and function-aware note thinning; Table 1 reports 30 ms local pitch spread and mean note density across 476 matched songs.Source: §3.1 · p. 2, Table 1 · p. 2
- FST transcribes four HTDemucs stem mixtures as piano, adding Harmonic, No-bass, and Piano+bass variants alongside the repeated Full-mix demo.The mixtures exclude vocals; Harmonic excludes drums, No-bass excludes bass, and Piano+bass retains only piano and bass.Source: §3.1 · p. 2
3.2. Pair-Supervised Pseudo-Demo Generation
- The pair-supervised route uses PiCoGen2 to generate piano covers from instrumental features, then stretches each generated bar to beat-tracked source boundaries.These covers provide a consistent, model-derived arrangement style beyond the local edits of register–density transformations.Source: §3.2 · p. 2
3.3. Model Architecture
- The model fine-tunes ACE-Step v1.5 with a new piano-roll encoder whose frame-aligned features condition the pretrained rectified-flow transformer.The encoder uses a local receptive field of approximately 0.4 seconds per side, leaving global structure modeling to the transformer backbone.Source: §3.3 · p. 2
4. EXPERIMENTS
The experiments compare pseudo-demo training routes and zero-shot baselines, evaluate on held-out real piano–pop pairs, and assess generated audio with distributional, adherence, spectral-shape, and stem-share metrics.
4.1. Training Setup and Zero-Shot Baselines
- Fine-tuned ACE-Step v1.5 systems use pair-free MuScriptor demos (Base, FST, or RDA) or PiCoGen2-generated covers (PiCo), with two external zero-shot baselines.Training runs for 500,000 steps; the piano-roll encoder is trained alone for 2,000 steps before joint fine-tuning with the DiT.Source: §4.1 · pp. 2–3
4.2. Evaluation Dataset
- Evaluation uses 476 held-out real piano–pop pairs, with five matched 30-second windows per track and real-piano MIDI inputs transcribed using Kong et al..A manually aligned 33-pair oracle provides the frozen APA reference; training and evaluation datasets are disjoint.Source: §4.2 · p. 3
4.3. Objective Evaluation Metrics
- Metrics include FAD, APA, level-normalized Mel-Shape RMSE, and bass-plus-drums and piano stem shares; FAD, APA, and shape estimates use 1,000 song-cluster draws.APA uses PCA-32; shape summarizes frequency-normalized spectral profiles, while stem-share closeness to ground truth is assessed for real-piano inputs.Source: §4.3 · pp. 3–4, Table 2 · p. 4, Figure 2 · p. 4
5. EXPERIMENTAL RESULTS
Pair-supervised PiCo performs best on real-piano inputs across the reported objective metrics. FST is a promising pair-free alternative, while all fine-tuned systems outperform zero-shot baselines on key real-piano measures.
Benefits of Pair-Supervised Pseudo Demos
- PiCo achieves the best real-piano APA (0.964), FAD (0.231), and spectral-shape RMSE (1.044), and has the least difference from ground truth in instrument-energy shares.These results indicate the closest adherence, distributional, and spectral match to real productions in this evaluation.Source: §5 · p. 4
Augmentations for Pair-Free Pseudo Demos
- FST and Base have similar training-distribution FAD, but FST is stronger on real piano; Base and RDA under-produce bass and drums, whereas FST brings rhythm-section energy closer to ground truth.RDA offers no consistent objective gain over Base; FST remains a promising pair-free alternative, though slightly weaker than PiCo on real-piano objectives.Source: §5 · p. 4
Baselines versus Fine-Tuned Piano2Pop Models
- All fine-tuned systems significantly outperform both zero-shot baselines on real-piano APA and spectral-shape RMSE, and improve FAD over MuseControlLite.ACE-Step cover has near-floor APA, while MuseControlLite has the largest real-piano FAD and spectral-shape error and remains largely piano.Source: §5 · p. 4
6. CONCLUSION
The conclusion presents Piano2Pop as an expansion of piano MIDI into instrumental pop, using pseudo-supervised fine-tuning on synthetic pairs. Pair-supervised demos lead on real piano covers, while FST remains a promising pair-free alternative.
6. CONCLUSION
- Piano2Pop expands piano MIDI into instrumental pop through pseudo-supervised fine-tuning on synthetic pairs; pair-supervised demos achieve the best objective results on real piano covers, while FST remains a promising pair-free alternative.Both zero-shot baselines fail, suggesting Piano2Pop needs task-specific modeling.Source: §6 · p. 4
7. COMPLIANCE WITH ETHICAL STANDARDS
The study reports no human participants, personal data, or animals, and no user study; it used publicly available YouTube music and machine-generated audio for research only, without redistributing copyrighted audio.
7. COMPLIANCE WITH ETHICAL STANDARDS
- The study involved no human participants, personal data, or animals, conducted no user study, and used publicly available YouTube music and machine-generated audio for research only without redistributing copyrighted audio.Institutional ethics approval was not required.Source: §7 · pp. 4–5