Source-linked AI summary
BMdataset: A Musicologically Curated LilyPond Dataset
Matteo Spanio, Ilay Guler, Antonio Rodà
TL;DR
Symbolic music research has largely relied on MIDI, leaving text-based engraving formats such as LilyPond underexplored. The paper constructs BMdataset and LilyBERT for LilyPond representation learning, finding that curated data outperforms much larger automatically converted data and that combining both regimes works best.
Problem
MIDI-centered symbolic music resources omit notational structure that LilyPond can encode, leaving LilyPond underexplored for music understanding.
Method
The paper builds a manuscript-based LilyPond dataset and adapts CodeBERT with LilyPond-specific vocabulary and masked language-model pre-training.
Results
Combining broad pre-training with BMdataset fine-tuning yields the best overall performance, reaching 84.3% composer accuracy on out-of-domain Mutopia evaluation.
Takeaways & Limitations
Expert-curated LilyPond data and broad automatically converted pre-training are complementary regimes for symbolic music representation learning.
Takeaways & Limitations
BMdataset is distributionally skewed toward Vivaldi, string instruments, and Late Baroque works, while PDMX-LilyPond conversion artifacts remain unquantified.
Abstract
from arXiv · showhide
Symbolic music research has relied almost exclusively on MIDI-based datasets; text-based engraving formats such as LilyPond remain unexplored for music understanding. We present BMdataset, a musicologically curated dataset of 393 LilyPond scores (2,646 movements) transcribed by experts directly from original Baroque manuscripts, with metadata covering composer, musical form, instrumentation, and sectional attributes. Building on this resource, we introduce LilyBERT (weights can be found at https://huggingface.co/csc-unipd/lilybert), a CodeBERT-based encoder adapted to symbolic music through vocabulary extension with 115 LilyPond-specific tokens and masked language model pre-training. Linear probing on the out-of-domain Mutopia corpus shows that, despite its modest size (~90M tokens), fine-tuning on BMdataset alone outperforms continuous pre-training on the full PDMX corpus (~15B tokens) for both composer and style classification, demonstrating that small, expertly curated datasets can be more effective than large, noisy corpora for music understanding. Combining broad pre-training with domain-specific fine-tuning yields the best results overall (84.3% composer accuracy), confirming that the two data regimes are complementary. We release the dataset, tokenizer, and model to establish a baseline for representation learning on LilyPond.
1. INTRODUCTION
The paper addresses the lack of LilyPond resources for symbolic music understanding by introducing a curated dataset, tokenizer, and CodeBERT-based encoder with a probing evaluation.
- MIDI datasets dominate symbolic music research but discard engraving directives, section boundaries, and multi-movement hierarchy.
- LilyPond encodes musical content and typographic structure in human-readable, version-controllable text.
- BMdataset contains 347 compilable LilyPond scores spanning 2,646 movements, transcribed from original manuscripts with structured metadata.
- LilyBERT adapts CodeBERT to symbolic music through masked language-model pre-training on LilyPond corpora.
- The study probes four model variants to isolate pre-training data source, corpus size, and domain-specific fine-tuning effects on composer and style classification.
2. RELATED WORK
Prior symbolic-music datasets and language models largely target MIDI or other formats, while LilyPond offers richer structured notation and motivates code-pre-trained modeling.
- Most large-scale symbolic music datasets use MIDI or MusicXML, with MIDI sacrificing notational detail and MusicXML tied to XML tooling.
- Existing symbolic pre-training work includes MusicBERT and MidiBERT-Piano for MIDI classification, alongside MIDI-focused tokenization and generation systems.
- LilyPond represents layout directives, reusable macros, and hierarchical score structure as structured plaintext.
- LilyPond had not yet been used for pre-training despite its richer structural and engraving expressivity than ABC notation.
- CodeBERT is selected because LilyPond behaves like a compiled formal language with strict grammar, nested blocks, commands, bindings, and macros.
3. BMDATASET DATASET
BMdataset is built from musicologist-produced, manuscript-traceable LilyPond projects, validated through compilation and richly annotated, while retaining substantial Baroque-era distributional imbalance.
- 3.1 Source and Compilation: BMdataset originates from 383 LilyPond projects transcribed by musicologists directly from original manuscripts.
- 3.1 Source and Compilation: Each project is assembled from macro, header, movement, and score files into a self-contained modeling file.
- 3.1 Source and Compilation: Compilation filtering produced 347 files, 2,646 movements, 1,679 tempo-marked sections, and approximately 90M tokens.
- 3.2 Metadata: Metadata covers composers, musical forms, instruments, section-level key, scale, tempo, time signature, and historical period.
- 3.3 Dataset Statistics: The corpus is skewed toward Vivaldi, strings, Late Baroque works, and concertos, despite spanning 71 composers, 16 forms, and 25 instruments.
- 3.1 Source and Compilation: Manuscript references and catalogue numbers provide provenance connecting each transcription to its historical source.
- 3.4 Parser Validation: Parser validation found a perfect note-count match in 332 of 347 files, while the remaining files had a mean error of 0.26%.
4. LILYBERT MODEL AND TOKENIZER
LilyBERT adapts CodeBERT to LilyPond by preserving domain-specific commands as atomic tokens and training with masked language modeling, alongside broad and focused corpus stages.
- 4.1 Tokenizer: Standard BPE fragments semantically meaningful LilyPond commands such as \relative into meaningless subwords.
- 4.1 Tokenizer: The tokenizer adds 115 LilyPond-specific tokens to CodeBERT so musical commands remain unsplittable units.
- 4.1 Tokenizer: The resulting vocabulary contains 50,380 tokens after extending CodeBERT’s 50,265-token vocabulary.
- 4.2 Model Architecture: LilyBERT uses CodeBERT’s RoBERTa-based encoder with 12 layers, 768 hidden dimensions, 12 attention heads, and approximately 125M parameters.
- Dataset Context: BMdataset statistics show dominance by Late Baroque works, string-centric instrumentation, and concerto forms, with Vivaldi at 30.5% of compositions.
- 4.3 Pre-training Procedure: Training uses masked language modeling, randomly masking 15% of input tokens and omitting replaced-token detection.
- 4.3 Pre-training Procedure: The two-stage regime performs one-epoch continuous pre-training on approximately 15B PDMX-LilyPond tokens followed by 10-epoch BMdataset fine-tuning on approximately 90M tokens.
5. EXPERIMENTS
The experiments probe LilyBERT representations on an out-of-domain Mutopia corpus using composer and style classification, comparing pre-training sources, corpus sizes, and fine-tuning strategies. Curated BMdataset data improves both tasks over size-matched converted data, while combining broad pre-training with BMdataset fine-tuning produces the strongest overall composer performance.
- Probing Setup: Mutopia provides an out-of-domain evaluation with 2,123 LilyPond files spanning 320 composers and 13 musical styles.The evaluation uses filtered subsets of 30 composer classes and 10 style classes.
- Probing Setup: The probing setup strips composer headers, mean-pools frozen model representations from 512-token chunks, and evaluates linear classifiers at layers 3, 6, 9, and 12.Results use StandardScaler preprocessing and 5-fold stratified cross-validation.
- Models Compared: Four variants isolate the effects of full PDMX pre-training, BMdataset fine-tuning, size-matched PDMX fine-tuning, and two-stage PDMX-to-BMdataset training.The size-matched comparison controls for corpus size, while the two-stage variant tests complementarity between broad and domain-specific training.
- Results: BMdataset improves accuracy over size-matched PDMX90M by 1.2 percentage points for composer classification and 1.4 percentage points for style classification.Both models are trained on approximately 90M tokens, isolating domain specificity from corpus size.
- Results: 0.843 composer accuracy is achieved by the two-stage CB + PDMX →BM model, which also obtains the best recall on both tasks.For style classification, CB + BMdataset remains best by top-1 accuracy at 0.837.
- Results: The learned embeddings separate Baroque, Classical, and Romantic styles while forming localized composer clusters within styles.Most composer errors occur between stylistically proximate pairs such as Haydn–Mozart and Grieg–Chopin.
- Results: Layer-wise probing peaks at middle layers for composer information, whereas style features are strongest earlier and degrade toward the output layer.Composer classification is strongest at layers 6–9; style classification is best at layers 3–6.
6. CONCLUSION
The paper introduces BMdataset and LilyBERT as resources for representation learning on text-based LilyPond scores. Its results favor expert-curated data over much larger automatically converted data, while combining broad pre-training with domain-specific fine-tuning performs best overall.
- Findings: 90M tokens of expert-curated data outperform 15B tokens of automatically converted data on both composer and style classification in out-of-domain Mutopia evaluation.The comparison establishes a performance advantage for curated data despite the large corpus-size difference.
- Limitation: BMdataset is distributionally skewed toward Vivaldi, string instruments, and Late Baroque works.This scope limits how broadly the dataset distribution represents composers, instrumentation, and historical periods.
- Findings: 84.3% composer accuracy is obtained by combining broad pre-training with domain-specific fine-tuning, indicating complementary data regimes.The paper releases the dataset, tokenizer, and model as a baseline for LilyPond representation learning.
- Availability: The released resources are publicly available for reproducibility, including code, model weights, and the dataset.The paper identifies these resources as supporting reproducibility.