PaperlayerView exact revision on arXiv

Source-linked AI summary

MusPyExpress: Extending MusPy with Enhanced Expression Text Support

Phillip Long, Hao-Wen Dong, Julian McAuley, Zachary Novack

arXiv:2608.21678v1cs.SDcs.LGeess.AS

TL;DR

Symbolic-music modeling commonly uses MIDI, which omits much expression text present in western sheet music. The paper introduces MusPyExpress to extract this information, analyzes its prevalence in PDMX, and demonstrates expression-aware generation and tagging tasks. PDMX contains expression text in 95.33% of files, and the experiments show that expression tokens can provide useful information for music generation while prefix-conditioned models perform better for tagging.

  • Problem

    MIDI-based symbolic-music modeling represents note sequences but omits much western-sheet-music expression text, limiting access to annotations such as tempo text, dynamics, section markings, and articulations.

  • Method

    MusPyExpress extends MusPy to process expression text into discrete tensor data, analyzes PDMX, and supports joint generation, expression-conditioned note generation, and expression tagging.

  • Results

    PDMX contains expression text in 212,406 of 222,820 files (95.33%); joint models achieve best overall perplexity with prefix-conditioned metrical time, while prefix-conditioned tagging models dominate anticipatory models.

  • Takeaways & Limitations

    MusPyExpress enables downstream symbolic-music modeling that incorporates expression-text sequences alongside notes.

Abstract

from arXiv · show

Current work in modeling symbolic music primarily relies on representations extracted from MIDI-like data. While such formats allow for modeling symbolic music as sequences of notes, they omit the large space of symbolic annotations common in western sheet music broadly known as expression text, such as tempo or dynamics, which specify time- and velocity-dependent controls on the musical composition and performance. To alleviate this gap, we present MusPyExpress, an extension to the popular symbolic music processing library MusPy that enables the extraction of expression text along with symbolic music for downstream modeling. Utilizing this extension, we parse the PDMX dataset to illustrate the wealth of expression text available in MusicXML datasets. Additionally, we introduce multiple generative tasks, including joint expression-note generation, expression-conditioned music generation, and expression tagging, that take advantage of this additional notational information.

1 Introduction

Symbolic-music modeling has largely relied on MIDI, which represents note sequences but omits much expression text found in western sheet music. MusPyExpress addresses this gap by extracting expression text for downstream modeling.

  • MIDI dominates symbolic-music modeling but omits most expression text, including tempo words, dynamic crescendos, section markings, and articulations.MusicXML often contains more of these annotations because it is designed for rendering sheet music rather than listening.
  • Expression text provides performance guidance by indicating how note sequences should be played.
  • MusPyExpress processes expression text into discrete tensor data for expression-aware sequence modeling.The extension preserves MusPy’s base Music object while expanding its annotation functionality.
  • The extension is evaluated on PDMX and used for joint expression-note generation, expression-conditioned music generation, and expression tagging.

2 Related Works

Existing MusicXML libraries differ in how directly they support expression-text extraction for downstream symbolic-music modeling. music21 parses these annotations extensively but embeds them deeply, while symusic prioritizes efficient MIDI parsing.

  • music21 offers extensive expression-text parsing, but its annotations are often embedded deep in the object hierarchy.This makes them cumbersome to extract for downstream modeling tasks.
  • symusic supports MusicXML but primarily targets efficient MIDI parsing rather than broad symbolic-music representation.

3 MusPyExpress

MusPyExpress extends MusPy’s universal Music object to represent expression text alongside musical structure, metadata, parts, notes, and timing information. It organizes annotations at score-wide or track-specific levels.

  • MusPyExpress extends MusPy to store expression types such as dynamics, slurs, and articulations that MusPy generally fails to parse.
  • Annotations are stored either score-wide for temporally related expression text and rehearsal marks or per track for dynamics and individual instructions.
  • The extension represents different expression-text categories as Python objects stored within annotations.A spanner denotes expression text that spans an explicit duration.

4 Dataset Analysis

PDMX contains expression text in most of its MusicXML files, with substantial variation in density, annotation usage, and composer practice. The analysis quantifies its prevalence and characterizes these distributions.

  • 212,406 of 222,820 PDMX files (95.33%) contain expression text, totaling more than 3.5M markings and averaging 20.1 per song.
  • Figure 2a shows relative density distributions, while Figure 2b shows per-song expression-text usage for selected composers.Special barlines exclude ordinary single barlines, and key and time-signature changes provide reference densities.
  • 17.21 beats is the average absolute expression-text density in PDMX.This measures the metrical distance between consecutive expression markings regardless of type.
  • Tempo, dynamics, and text features can be concentrated, while rehearsal marks are sparser and structural controls tend to avoid overuse.
  • Romantic composers use expression text more frequently than Baroque and Classical composers in the analyzed PDMX metadata.

5 Experiments

The experiments evaluate expression-aware joint generation, conditional note generation, and expression tagging across timing and conditioning schemes. Results show useful expression information for generation, while prefix conditioning performs best for tagging.

  • Generation Evaluation: Table 1 compares timing and conditioning schemes against a note-only MMT baseline using PCE, SC, GC, and note perplexity.PCE, SC, and GC are judged by closeness to ground truth, while lower note perplexity is better.
  • Experimental Tasks: The study evaluates joint note-expression generation, expression-conditioned note generation, and expression tagging using annotated PDMX tracks.The conditional experiments use both metrical and real-time domains.
  • Generation Results: The joint prefix-conditioned metrical-time model achieves the best overall perplexity, suggesting expression tokens provide useful local information for realistic music generation.No configuration clearly dominates across all metrics, possibly because of the small model size.
  • Expression Tagging Results: Prefix-conditioned expression-tagging models, especially in real time, clearly outperform anticipatory models on accuracy.The authors attribute this pattern to the simpler prefix setup for predicting sparse expression-token sequences.

6 Conclusion

MusPyExpress extends MusPy to represent, process, render, and export expression text across MusicXML and related formats. The paper demonstrates expression-aware modeling with PDMX and identifies future expansion toward MIDI annotation and fine-grained text-to-music generation.

  • Conclusion: MusPyExpress extends MusPy with expression-text processing for downstream joint generation, conditional generation, and tagging tasks.The conclusion frames these tasks as demonstrations of expression-aware downstream modeling.
  • Type of Expression Text: The extension defines 28 Python classes for expression types spanning text, structural controls, note-level annotations, and general-purpose abstractions.Examples include rehearsal marks, dynamics, articulations, slurs, pedals, and spanning annotations.
  • Type of Expression Text: MusPyExpress adds score-level support for tempo-marking text, special barline types, and grace-note attributes.These extensions capture details such as appoggiaturas and acciaccaturas.
  • Timing: Its timing utilities convert between tempo-agnostic metrical time and real time while accounting for temporally related expression text.This supports alignment and interpretation of markings such as tempo changes and fermatas.
  • File I/O: Audio and symbolic outputs now reflect expression markings, including slurs, accents, ritardandos, and crescendos through duration, velocity, and tempo changes.Native MusPy renderings omit dynamic markings and local tempo variations such as ritardando and accelerando.
  • File I/O: MusPyExpress parses standard MusicXML and structurally compatible MSCZ files, and supports lossless JSON import and export.The MSCZ demonstration uses the original files from which PDMX was derived.

B.2 Expression Text Usage in PDMX

PDMX contains extensive expression text, with frequencies and durations varying substantially across annotation types. Tempo and dynamics are prevalent, while structural markings span longer sections and pedal or text spanners are brief.

  • >3.5M expression text annotations occur in PDMX, with tempo and dynamics most prevalent and pedal and text spanners comparatively infrequent.Every song requires at least one tempo and dynamic marking, whereas pedal and text spanners occur less often.
  • Implied duration is measured to the next expression of the same type, or to the song’s end when none follows.This enables duration analysis for types without explicit duration attributes, including time and key signatures.
  • Key signatures, rehearsal marks, and time signatures tend to span entire sections, whereas pedal and text spanners usually cover short note sequences.
  • Expression types with implied durations generally last longer, while explicitly delimited pedal and text markings are shorter.For implied-duration types, relative density and implied duration are identical; they differ for explicit-duration types.

C.1 Data Representation

MusPyExpress adapts the MMT event representation to include expression tokens alongside notes. It supports both metrical and real-time encodings and adds velocity for dynamic-related expressions.

  • Songs are represented as event sequences in which each event is a note or expression token with type, timing, pitch or value, duration, and instrument fields.The underlying MMT-style representation uses six fields: type, beat, position, pitch, duration, and instrument.
  • Expression tokens add a type-field value, while the pitch field becomes a value field combining 128 MIDI pitches with nearly 700 expression values.
  • An alternative real-time encoding replaces beat and position with real time and expresses temporal fields in seconds.This addresses expression types whose temporal behavior is not captured at note level by a metrical system.
  • A velocity field is added to support learning dynamic-related expressions.

C.2 Interleaving Methods

The representation interleaves notes and expression text into one autoregressive sequence, and the interleaving strategy determines how controls relate to generated events.

  • The model tests prefix and anticipation interleaving methods for combining note and expression sequences.
  • Prefix conditioning: Prefix conditioning places controls before the main event sequence, requiring long-term attention to connect controls with nearby events.
  • Anticipation: Anticipation places a control near the events it affects to reduce the sequence distance between them.

C.3 Experimental Setup and Metrics

Experiments use a decoder-only MMT-style transformer and evaluate joint and expression-conditioned generation with musical-structure metrics and note perplexity.

  • Model and training: Each model uses a 6-layer, 8-head decoder-only transformer with hidden dimension 512 and about 20M parameters.Training runs for 80K steps with batch size 8 and learning rate 5e-4.
  • Model and training: Models are trained with absolute positional embeddings, a maximum sequence length of 1024, Adam optimization, and one NVIDIA GeForce RTX 3090 GPU.
  • Metrics: Joint and expression-conditioned generation are evaluated across 256 generations per configuration using pitch class entropy, scale consistency, and groove consistency.These metrics assess how well generated music captures underlying musical patterns.
  • Metrics: Note perplexity is measured on the held-out test set while ignoring expression text.Expression sequences for expression-conditioned generation are randomly drawn from the test set.
Loading 2608.21678v1…