Source-linked AI summary
Does Notation Matter? MusicXML Tokenisation for Symbolic Music Generation and Understanding
Andrea Poltronieri, Matteo Spanio, Simone Chieppa, Xavier Serra, Martín Rocamora
TL;DR
Whether notational richness helps deep learning on symbolic music remains an open question, especially for underexplored MusicXML. The paper introduces MxlTok and compares MusicXML with MIDI and ABC; MusicXML-tokenised models outperform counterparts on understanding and generation evaluations.
Problem
The impact of representational richness on deep learning remains poorly understood, and MusicXML tokenisation is largely unexplored despite the format's rich notation.
Source: §1 · p. 1, §2.1 · p. 2Method
The authors introduce MxlTok and compare MusicXML, MIDI, and ABC tokenisations across 26 autoregressive and masked language models and multiple tasks.
Source: §1 · p. 2Results
MusicXML models achieve the best results on composer classification, difficulty estimation, and accompaniment suggestion, and are preferred over MIDI and ABC counterparts in generative evaluation.
Source: §6 · p. 6, §5.1 · p. 6, §5.2 · p. 6Takeaways & Limitations
The study concludes that MusicXML tokenisation matches or outperforms the baselines, with notational expressiveness beneficial on the evaluated understanding tasks.
Source: §6 · p. 6The study relies on PDMX license metadata, which may not reflect underlying composition status, so some scores likely arrange still-copyrighted works.
Source: §8 · p. 7
Abstract
from arXiv · showhide
Symbolic music representations encode musical content at varying levels of completeness, from performance-oriented formats such as MIDI to fully notated scores such as MusicXML. Yet the impact of this representational richness on deep learning models remains poorly understood. While tokenisation has been extensively studied for MIDI, score-based formats have received considerably less attention, and MusicXML in particular remains largely unexplored despite being the de-facto standard for score interchange. In this paper, we introduce the first general-purpose tokenisation framework for MusicXML, and conduct a systematic empirical study to determine whether notational richness confers a measurable advantage for deep learning on symbolic music. We train 26 autoregressive and masked language models, comparing MusicXML tokenisation against MIDI and ABC-based representations, probing the effect of different tokenisation strategies within MusicXML, and testing whether music-grammar-aware tokenisation outperforms treating scores as plain text. We evaluate music understanding tasks using linear probing across composer classification, difficulty estimation, and accompaniment suggestion, and generative tasks using objective statistical metrics and a human pairwise preference study. Our results show that MusicXML-tokenised models outperform MIDI and text-based counterparts on understanding and generative tasks.
1. INTRODUCTION
MusicXML preserves notational information that performance-oriented MIDI loses, yet its potential for deep learning has been little studied. This paper introduces MxlTok and finds that MusicXML-tokenised models outperform MIDI and text-based counterparts on understanding tasks and perform competitively on generation.
1. INTRODUCTION
- MusicXML captures enharmonic spelling, multi-voice polyphony, articulations, dynamics, and hierarchical parts and measures, whereas MIDI flattens score structure and loses some notation.MIDI preserves performance attributes such as pitch, velocity, and timing, but enharmonic equivalents collapse into a single pitch class.Source: §1 · p. 1
- MusicXML-tokenised models outperform MIDI and text-based counterparts across understanding benchmarks and perform competitively on generation, including in human preference evaluations.The study evaluates 26 autoregressive and masked language models across understanding and generative tasks.Source: §1 · p. 2
- MxlTok is introduced as the first general-purpose MusicXML tokenisation framework, with comparisons against MIDI and ABC and across MusicXML tokenisation strategies.The study also tests music-informed tokenisation against plain-text encoding across autoregressive and masked language models.Source: §1 · p. 2
- The paper releases seven pretrained MusicXML models covering generative and understanding tasks and all MxlTok tokenisation configurations.The authors describe them as the first general-purpose, publicly available language models trained on MusicXML data.Source: §1 · p. 2
2. RELATED WORK
Prior symbolic-music tokenisation research has largely focused on MIDI, leaving format choice and tokenisation granularity unresolved. Score-based work has used ABC plain text and a small number of task-specific or incomplete MusicXML approaches.
2.1 MIDI Tokenisers
- MIDI tokenisation spans direct event encodings, structured bar and duration tokens, and pooled embeddings; REMI+ and Octuple serve as widely adopted baselines.REMI+ adds program tokens for multitrack settings, while Octuple pools note attributes to shorten sequences at the cost of multiple output heads in its original formulation.Source: §2.1 · p. 2
- BPE merges frequent token pairs into vocabulary entries, and prior symbolic-music work reports benefits across generation and understanding tasks.A separate MIDI study found that explicit temporal encoding has task-dependent benefits, but does not generalise beyond MIDI.Source: §2.1 · p. 2
- Earlier studies restricted to MIDI leave open whether symbolic format or tokenisation granularity primarily drives model performance.Comparisons across formats had not shown a consistent advantage for notationally richer representations, while a survey attributed this to tooling rather than information.Source: §2.1 · p. 2
2.2 Score Tokenisers
- ABC score models use plain-text tokenisation, while its limited and inconsistently implemented multivoice support has led to incompatible extensions.Prior MusicXML tokenisers are task-specific and incomplete; MxlTok is presented as a general-purpose framework encoding the format's full range of notational elements.Source: §2.2 · pp. 2–3
3. METHODOLOGY
The methodology introduces a configurable MusicXML tokeniser and compares representation, serialisation, and vocabulary choices on a filtered public-domain score corpus. Models share a controlled architecture so that the study can assess tokenisation differences across generation and understanding tasks.
3.1 The MxlTok Framework
- MxlTok supports MusicXML notation and configurable tokenisation strategies, including lossless part-first and measure-first serialisation with different ordering of score material.Part-first traverses each instrument's measures before moving to the next; measure-first emits all parts within a measure before advancing. The framework supports BPE at arbitrary vocabulary sizes, with 5k and 10k tested.Source: §3.1 · p. 3
3.2 Dataset
- Models are trained on PDMX's Rated and Deduplicated subset, chosen as a quality-filtered corpus with diverse genres and pervasive notational content.In this subset, 53% of scores carry dynamics, 50% articulations, and 61% include a note with spelling unrecoverable from pitch class.Source: §3.2 · pp. 3–4
3.3 Baselines and Experimental Conditions
- The comparison prepares the corpus in MusicXML, MIDI, and ABC, then evaluates MxlTok, established MIDI tokenisers, and plain-text ABC tokenisation under matched conditions.For autoregressive generation, all configurations receive the number of parts and instrument names after the beginning-of-sequence token; Octuple is excluded from generative evaluation.Source: §3.3 · p. 4
3.4 Models
- The 26 models use GPT-2 for generation and BERT for understanding, with a shared approximately 20M-parameter configuration to isolate tokenisation differences.Training runs for up to 100k steps with early stopping, accounting for different convergence rates across BPE and no-BPE sequence lengths.Source: §3.4 · p. 4
4. EVALUATION
The evaluation uses linear probing for understanding and combines objective metrics with listener preferences for generation. It specifies datasets and protocols for composer recognition, difficulty estimation, accompaniment retrieval, and generative assessment.
4. EVALUATION
- Understanding uses BERT embeddings as fixed features for a two-layer linear probing classifier with hidden dimension 512.The evaluation assesses understanding separately from generation; generative quality is measured with objective metrics and a human preference study.Source: §4.1 · p. 4, §4 · p. 4
4.1.1 Composer Recognition
- Composer classification uses 1,968 KernScores spanning 20 composers, balanced splits, and 5-fold cross-validation.The scores are converted from Humdrum **kern to MusicXML; an ablation derives MIDI and ABC from the same MIDI-converted source.Source: §4.1.1 · p. 4
4.1.2 Difficulty Estimation
- Difficulty estimation uses 652 CIPI piano compositions across 9 difficulty levels, with predefined 5-fold splits and 9-class and balanced ±1 accuracy.Source: §4.1.2 · p. 4
4.1.3 Accompaniment Suggestion
- Accompaniment suggestion treats 5,000 Wikifonia lead sheets as retrieval: melody tracks pair with matching accompaniments, while nonmatching pairs serve as negatives.Performance is measured with MAP and HITS@k for k = 1, 5; Table 3 reports downstream-task results and omits standard deviations below 0.05.Source: §4.1.3 · pp. 4–5, Table 3 · p. 5
4.2.1 Objective Analysis
- Objective generation evaluation uses PR, SC, and GC in a JS score comparing generated and reference distributions.The reference statistics come from the PDMX R∩D training split, and all models share musical-attribute conditions; each generates 200 pieces over 20 prompts.Source: §4.2.1 · p. 5
4.2.2 Human Listening Test
- The listening study compares continuations on fidelity, correctness, diversity, and overall preference using a balanced pairwise design.Sixteen participants made 960 judgments; Table 4 reports Bradley–Terry preference scores alongside statistical metrics, with JS measuring similarity to the target corpus.Source: §4.2.2 · p. 5, Table 4 · p. 6
5. RESULTS
MusicXML models lead the understanding comparisons and outperform the baselines on objective generative evaluation, while listener preferences differ by serialization order. Ablations and vocabulary results further qualify these comparisons.
5.1 Understanding Tasks
- MxlTok models outperform both baselines on composer classification, difficulty estimation, and accompaniment suggestion; BPE helps the first two tasks but not retrieval.For MusicXML accompaniment models, 5k BPE outperforms 10k; Octuple underperforms other tokenisers, which the authors attribute possibly to pooled embeddings not accounting for expressive tokens.Source: §5.1 · p. 6
- Converting scores to MIDI before tokenisation makes MIDI-derived MxlTok match MIDI counterparts, locating the understanding advantage in notational content lost during conversion.Source: §5.1 · p. 6
5.2 Generative Tasks
- 89.2 JS score is the best objective generative result, achieved by part-first MusicXML at 5k BPE; measure-first scores 88.3.REMI+ is intermediate, while ABC word-level tokenisation performs worst across configurations.Source: §5.2 · p. 6
- The 5k BPE vocabulary consistently outperforms 10k in generative evaluation, which the authors suggest may reflect vocabulary over-parameterisation for the training-set size.Source: §5.2 · p. 6
- Part-first MusicXML leads overall preference (0.81) and diversity (0.89), while measure-first leads fidelity (1.19) and correctness (0.63).The authors interpret the two serialization orders as capturing complementary generative strengths; BPE does not improve perceptual quality in this study.Source: §5.2 · p. 6
6. CONCLUSIONS
The paper presents MxlTok and finds measurable differences between MusicXML and the baseline representations across understanding and generation. Its results also distinguish the strengths of measure-first and part-first serialization.
6. CONCLUSIONS
- MxlTok is presented as the first general-purpose tokenisation framework for MusicXML.Source: §6 · p. 6
- MusicXML models achieve the best results across the three understanding tasks and produce generations that better match the target distribution and are preferred by listeners.Source: §6 · p. 6
- Measure-first performs better on understanding, whereas part-first is preferred in generative evaluation, suggesting complementary inductive biases.The authors conclude that score semantics carry information not captured by performance-oriented and plain-text representations.Source: §6 · p. 6
8. ETHICS STATEMENT
The ethics statement identifies uncertainty in licensing metadata for the training corpus. The authors rely on that metadata while noting that some underlying compositions may remain under copyright.
8. ETHICS STATEMENT
- Models use PDMX scores labeled CC-0 or Public Domain, but some likely contain arrangements of works still under copyright because license metadata may not reflect underlying composition status.The authors rely on license metadata as the best available signal and identify this as a limitation of large-scale symbolic corpora.Source: §8 · p. 7