PaperlayerView exact revision on arXiv

Source-linked AI summary

Unifying Score and Performance for Fine-Grained Music Understanding in Audio-Language Models

Milan Liessens Dujardin, Song-Ze Yu, Kevin Miao

arXiv:2609.10351v1cs.SDeess.AS

TL;DR

Fine-grained music understanding requires attending to both what is played and how it is realized, while existing music-caption datasets often offer coarse or weakly time-grounded descriptions. This paper introduces MuNo-SP, a representation and data-generation pipeline that unifies score and performance information, and reports stronger aggregate benchmark results than conventional baselines.

  • Problem

    Existing music-caption datasets often provide coarse or weakly time-grounded descriptions, leaving fine-grained nuances such as tempo variation, articulation, expressive gestures, and phrasing insufficiently represented.

    Source: §1 · p. 2, §2.3 · p. 4

  • Method

    MuNo-SP unifies aligned score and performance information in a language-model-accessible text format, which a pipeline uses to generate long-form analyses and temporally grounded question–answer pairs.

    Source: §1 · p. 2, §4 · p. 8, §9 · p. 16

  • Results

    80.8% with MuNo versus 55.0% for conventional baselines on GPT-5.6 Sol’s aggregate over 442 questions, with MuNo also higher for Muse Spark 1.2 and Qwen3.6-Plus.

    Source: §3.3.1 · p. 8

  • Takeaways & Limitations

    The findings indicate that representation design substantially influences language models’ music reasoning, and that score-and-performance representations provide richer understanding than alternative formats.

    Source: §7 · p. 14

    Because MuNo-SP is grounded in scores and aligned MIDI, generated analyses cannot characterize timbre, acoustics, or other properties absent from MIDI.

    Source: §8 · p. 15

Abstract

from arXiv · show

Large audio language models (LALMs) have shown promising progress in broad music-understanding tasks such as tagging, retrieval, and captioning. Music understanding that requires finer hearing over both the content and how it is realized within a performance through dynamics, phrasing, articulation, time, and other performance techniques, however, remains at an earlier stage. Existing audio-language model (ALM) training pipelines typically rely on coarse, weakly grounded captions and therefore provide little support for learning these subtle nuances in music, limiting their ability to serve real-world applications in education or artistic practice. We therefore introduce MuNo-SP (Music Notation unifying Score and Performance), a text-based representation that jointly encodes score content and performance information. Building on MuNo-SP, we develop an automatic training-data generation pipeline that uses aligned scores and performances to produce long-form auditory analyses and musically informed question-answer pairs. We use this pipeline to construct MAESTROCaps, a classical piano dataset comprising 148 long-form performance analyses and 31,080 question-answer pairs derived from 148 aligned score-performance pairs. In a human evaluation, MuNo-SP analyses were preferred by majority vote over MIDI-only analyses for eight of nine excerpts. MuNo-SP also performed strongly on a benchmark of score-performance understanding, suggesting that integrating score and performance information enables more reliable and musically informative LALM supervision than a MIDI-only baseline.

DATASET ARTICLE · 1 INTRODUCTION

Fine-grained music understanding requires modeling both musical content and how it is realized in sound and time, but existing LALM supervision often lacks detailed, event-level grounding. The introduction presents MuNo-SP and an automatic pipeline for generating score- and performance-grounded analyses and question–answer pairs, and introduces the MAESTROCaps dataset.

1 INTRODUCTION

  • Musician-level understanding involves hearing interplay, motifs, harmonic nuance, phrase boundaries, climaxes, and transitions, as well as how timing, dynamics, and articulation shape performance.The introduction identifies these capabilities as relevant to learners and musicians and to fine-grained control in music generation.Source: §1 · p. 2
  • Existing music-caption datasets often lack coarse descriptions’ event-level alignment, omitting tempo variation, articulation, expressive gestures, and phrasing.The introduction argues that these omissions limit supervision for deeper music understanding.Source: §1 · p. 2
  • Scores encode musical content and intent and provide context for performers’ musical imagination (Neuhaus (1973)).The introduction presents scores as a resource for connecting audible content with performance choices.Source: §1 · p. 2
  • MuNo-SP unifies detailed score content with temporally aligned performance information in a text-based format interpretable by language models.The representation is intended to support fine-grained, musically grounded training data at scale.Source: §1 · p. 2
  • The automatic pipeline uses aligned scores and performance MIDI to generate long-form auditory analyses and temporally grounded question–answer pairs.Score grounding supports reasoning about musical content, while performance alignment conditions analyses on how that content is realized in sound and time.Source: §1 · p. 2
  • MAESTROCaps contains 148 long-form analyses and 31,080 question–answer pairs across 148 aligned score–performance pairs.The authors also release 148 updated ASAP-derived scores and available (n)ASAP alignment data after correcting over 2,000 errors.Source: §1 · p. 2

2 RELATED WORK

Prior work develops music-understanding benchmarks and symbolic, audio-captioning, and performance-analysis resources, but these remain unreliable or limited in representing precise, time-grounded musical events and performance realization.

2.1 MUSIC UNDERSTANDING

  • Score and audio benchmarks report persistent difficulty with advanced score reasoning and fine-grained musical understanding, including event identification, form, and pitch hearing.MuSP-Bench spans scores, performances, and their interaction, while audio benchmarks expose limitations in interpreting specific events in broader context.Source: §2.1 · pp. 2–3

2.2 SYMBOLIC MUSIC REPRESENTATIONS

  • MusicXML is semantically rich but verbose, ABC is compact but difficult for reasoning, and MIDI captures performance events while omitting explicit score structure.Together these formats trade off expressiveness, compactness, and performance detail, without unifying score instructions and their realization in time.Source: §2.2 · p. 3
  • Sequence-model representations such as MIDI-like tokenizations, REMI, TSD, Structured, CPWord, and Octuple improve event serialization but retain MIDI’s implicit musical intent and missing score-performance relationship.Their designs differ in temporal encoding, token organization, sequence length, and modeling efficiency.Source: §2.2 · p. 3

2.3 AUTOMATED CAPTIONING

  • Automated captions scale beyond manual annotation, but remain bounded by available metadata or audio-feature reliability and generally lack dense, human-verified, event-level descriptions across whole works.Existing systems offer global attributes or coarse structural and timestamped windows, rather than precise temporal accounts of musical events and performance realization.Source: §2.3 · p. 4

2.4 MUSIC PERFORMANCE ANALYSIS

  • Performance-analysis studies link notation and sound, but loudness prediction across different pieces is poor, suggesting piece-specific score context matters more than individual style.Prior work examines dynamics and performed attributes in jazz and classical music, including correspondence between notated dynamics and loudness in Chopin mazurkas.Source: §2.4 · p. 4, Dataset Example caption · p. 5

Dataset Example caption

  • Expert performance datasets provide perceptual ratings or annotations for restricted excerpts and exercises, but neither supplies long-form explanations of complete performances.PercePiano covers excerpts of at most 16 bars, while NeuroPiano focuses on short technical exercises.Source: Dataset Example caption · p. 5

3 UNIFYING SCORE AND PERFORMANCE

MuNo-SP combines notated musical content with aligned performance information in a structured representation designed to be efficient, interpretable, and expressive. In benchmark comparisons, its strongest representation outperforms conventional baselines, while its text formats are substantially smaller than timed MusicXML.

3 UNIFYING SCORE AND PERFORMANCE

  • MuNo-SP unifies score content—including pitch, rhythm, dynamics, articulation, phrasing, and written voices—with performance timing, duration, velocity, and pedal activity.Its design criteria are efficiency relative to expressive formats such as MusicXML, interpretability through musical hierarchy and LLM readability, and expressiveness for detailed score and performance information.Source: §3 · p. 5

3.1 SYNTAX

  • MuNo-SP is line-oriented, bar-ordered, and beat-organized, with nested instrument and voice structure and note lines that encode written and performed event information.Beat-level markings include dynamics, tempo, expression, and pedal events; MuNo-S retains the same structure without performance information.Source: §3.1 · p. 6

3.2 EVALUATION OF SCORE AND PERFORMANCE UNDERSTANDING

  • MuSP-Bench contains 490 open-ended questions across 24 classical works, categorized by whether answers require score, performance, both, or either independently.The evaluation uses three model families and compares MuNo with ABC notation, score images, MIDI-as-text, and ABC+MIDI on matched sets; its primary comparison excludes 48 identification questions, leaving 442 questions.Source: §3.2 · p. 7

3.3.1 MuNo enables greater understanding

  • Across 442 questions, best-per-subset MuNo scores were 80.8% vs 55.0% (GPT-5.6 Sol), 62.0% vs 36.0% (Muse Spark 1.2), and 25.3% vs 15.8% (Qwen3.6-Plus), beating conventional baselines.The strongest MuNo representation outperformed ABC, MIDI-as-text, score images, and ABC+MIDI across all three model families and every question subset.Source: §3.3.1 · pp. 7–8, Table 2 · p. 7

3.3.2 GPT-5.6 Sol leads

  • For GPT-5.6 Sol, MuNo-SP reached 80.6% on joint S&P questions versus 45.8% for ABC+MIDI, a 34.8-point gain.Across 264 score-input questions, MuNo-S and MuNo-SP achieved 81.8% and 82.2%, versus ABC's 54.9% and score images' 33.7%. On performance-only questions, MuNo-SP scored 76.4%, 17.0 points above MIDI-as-text (ABC); on the matched 190-question set, it scored 78.4% versus 62.1%, while General (S/P) scores were 81.0% for MuNo-SP, 78.6% for MuNo-S, and 65.5% for MIDI-as-text.Source: §3.3.2 · p. 8
  • GPT-5.6 Sol's MuNo advantages were largest for short-horizon questions.Table 3 reports per-horizon results for score-only and performance-inclusive questions.Source: §3.3.2 · p. 8, Table 3 · p. 8

3.3.3 Token Efficiency

  • MuNo-S averaged 33,973 tokens (7.0% of timed MusicXML), while MuNo-SP averaged 75,779 tokens (15.6%); compact representations were 84.4%–97.6% smaller.The token-count gap between MuNo-S and ABC was 22,428 tokens, versus 12,463 between MuNo-SP and ABC+MIDI; despite ABC+MIDI's greater compactness, MuNo-SP offers an interpretability advantage.Source: §3.3.3 · p. 8, Table 4 · p. 8

4 CAPTION GENERATION PIPELINE

The pipeline converts aligned scores and performances into MuNo-SP, then uses score and performance information to generate structured auditory analyses and question–answer pairs. Its stages cover data collection, score cleaning, alignment, representation conversion, and automated captioning.

4.1 DATA COLLECTION

  • ASAP links 222 scores with 1,068 performances, and 179 MAESTRO-linked works provide notation, performance MIDI, and audio; BSED extends the pipeline to orchestral excerpts.The sources include ASAP (Foscarin et al. (2020)), MAESTRO (Hawthorne et al. (2018)), and BSED (Berendes et al. (2026)); BSED contains 20 excerpts with four performances each, and score dynamics are used to approximate missing velocities.Source: §4.1 · p. 9

4.2 DATA CLEANING

  • Scores are cleaned using evidence aggregated across aligned performances, supplemented by external symbolic scores, with threshold-based corrections reviewed manually.For each of 148 catalog entries, the release includes source-preserving, annotated, and cleaned MusicXML, plus records of assessed candidates and accepted changes.Source: §4.2 · pp. 9–10

4.3 SCORE–PERFORMANCE ALIGNMENT

  • For MAESTRO-linked performances, the pipeline retains original MIDI and propagates refined PianoCoRe correspondences through score and MIDI edits; BSED uses its supplied note-level annotations.This avoids score-informed MIDI cleaning that could alter captured performances to match errors in a reference score.Source: §4.3 · p. 10

4.4 MUNO-SP CONVERSION

  • Each aligned pair is converted by adding aligned performance attributes to MusicXML, creating a timing-enriched intermediate representation, and then converting it into MuNo-SP.Source: §4.4 · p. 10

4.5 AUTOMATED CAPTIONING

  • Automated captioning segments scores, generates performance-informed auditory analyses, and derives question–answer pairs from those analyses.GPT-5.6 Sol performs the first two stages, and a separate LLM generates the question–answer pairs.Source: §4.5 · p. 10

4.5.1 Stage 1: Structural Segmentation

  • GPT-5.6 Sol segments MuNo-S scores into sections, thematic presentations and returns, and phrases, assigning bar ranges, musical functions, and supporting evidence.An explicit formal scaffold is included because validation found downstream analyses more coherent and reliably structured with it.Source: §4.5.1 · p. 10

4.5.2 Stage 2: Long-form Auditory Analysis

  • Auditory analyses are prompted to be time-grounded, precise, musician-like, and informative, addressing melody, harmony, texture, dynamics, phrasing, structure, rhythm, and performance decisions.The format requires discussion of each phrase and synthesis at theme, section, and complete-piece levels.Source: §4.5.2 · p. 11

4.5.3 Stage 3: Conversion into QA pairs

  • An LLM converts analyses into standalone, musically grounded, audible, unambiguous, informative question–answer pairs intended to reflect human inquiry.Gemini 3.7 Flash was selected for the quality and variety of its outputs.Source: §4.5.3 · p. 11

5 THE MAESTROCAPS DATASET

MAESTROCaps contains aligned classical piano performances and scores, with MuNo-SP representations, long-form auditory analyses, and question–answer pairs. The section also reports scientific-pitch mentions and describes how auditory-analysis characteristics are categorized.

5 THE MAESTROCAPS DATASET

  • 31,080 question–answer pairs accompany 148 classical piano performances aligned one-to-one with finalized score pieces, each with a segmentation, MuNo-S and MuNo-SP representations, and a long-form analysis.Each recording has 210 question–answer pairs; Table 5 breaks down pieces and audio duration by composer.Source: §5 · p. 11, Table 5 · p. 11

Dimension Share of mentions (%)

  • Auditory analyses are categorized across seven dimensions, with coverage reported using word matching and dimension shares among non-exclusive assignments.Sequences count toward melody and chords toward harmony; Table 6 also reports rounded per-file means for general characteristics.Source: §5 · p. 11, Table 6 · p. 12

Scientific-pitch mentions 1429

  • 197 pitch sequences and 149 explicit pitch collections are counted among the scientific-pitch mentions in the auditory analyses.The sequence example is F3-G3-Ab3; the collection example is A4+C5.Source: Scientific-pitch mentions 1429 · p. 12

6 EVALUATION

Evaluation combines human judgments, an ALM judge, a score-versus-MIDI case study, and rule-based validation. Across these evaluations, MuNo-SP was preferred over MIDI-only descriptions, and the generated analyses showed strong correspondence to musical evidence.

6.1.1 Caption quality derived from MuNo and MIDI

  • MuNo-SP received the largest human response share on all four dimensions, led insightfulness 62.79% to MIDI’s 30.23%, and won eight of nine comparisons.MIDI descriptions had a slightly greater average sentence count, a potential nuisance variable.Source: §6.1.1 · p. 12, Table 7 · p. 13

6.1.2 Auditory Analysis Quality

  • Mean ratings were 4.78 for temporal correctness, 4.89 for pitch correctness, 4.42 for high-level descriptive correctness, 4.64 for musical intelligence, and 4.39 for musician-likeness.Across all five dimensions, 83.33%–97.22% of ratings were 4 or 5, across 36 ratings per dimension.Source: §6.1.2 · p. 12

6.2 ALM-AS-A-JUDGE EVALUATION

  • MuNo-SP was preferred in 23 of 27 ALM-judge comparisons (85.2%); Gemini and the human majority agreed on all 9/9 shared outcomes.The judge preferred MuNo-SP in 68 of 81 trials (84.0%) and made the same overall decision across all three trials for 22 of 27 pairs.Source: §6.2 · p. 13, Table 7 · p. 13

6.3 CASE STUDY: MIDI AND MUNO COMPARISON

  • In the Rachmaninoff case, MuNo-SP distinguished the melody, upper response, and inner line, and related the bass and E# pitches to F-sharp-minor dominant pressure rather than MIDI-only’s C-sharp minor.MuNo-SP also identified the A5 at 03:18.40 as a grace note.Source: §6.3 · pp. 13–14, Figure 3 · p. 14

6.4 PROGRAMMATIC VALIDATION

  • 0 of 17,143 single pitches, 10 of 6,943 named harmonies, 14 of 17,527 pitch sequences, and 208 of 13,033 simultaneous pitch groups or chords were unsupported.Timestamp mismatches were 63 of 56,600 occurrences, or 12 of 56,600 with ±100 ms tolerance; some unsupported groups combined separately played bass notes and chords.Source: §6.4 · p. 14

7 DISCUSSION

The discussion argues that MuNo-S and MuNo-SP improve models’ understanding of score and performance, enabling detailed, more reliable automated musical analysis. It highlights fine-grained performance analysis and comparison as capabilities supported by aligning performance information with score structure.

7 DISCUSSION

  • MuNo-S and MuNo-SP provide richer score-and-performance understanding than alternative formats, and human evaluation indicates MuNo-SP enables more reliable automated captions.Combined with reasoning models’ in-context learning and instruction-following, the representations support detailed descriptions for fine-grained performance analysis and comparison.Source: §7 · pp. 14–15

7.1 FINE-GRAINED PERFORMANCE ANALYSIS

  • By aligning performance information with score structure, MuNo-SP supports event-specific descriptions of timing, dynamics, articulation, duration, and pedal use.For high-confidence alignments, the pipeline can flag potential performance errors while expressing uncertainty; an example identifies a likely omitted Db5 after Eb5.Source: §7.1 · p. 15

7.2 FINE-GRAINED PERFORMANCE COMPARISON

  • Aligning performances to the same score lets models compare corresponding events and distinguish performers’ timing, duration, velocity, articulation, and pedal use, supporting comparisons of phrasing, pacing, balance, and emphasis.These comparisons could help separate compositional properties from interpretative choices and support music education and controllable expressive-performance generation.Source: §7.2 · p. 15

8 LIMITATIONS AND FUTURE WORK

The limitations concern repertoire coverage, source-representation constraints, possible supervision errors, and uncertainty about how prior exposure and listener background shape generated captions. Future work includes expanding the pipeline’s data and model inputs and enabling researchers to adapt it.

Limitations and future work

  • The corpus focuses on Western classical piano, while future work could extend the approach to broader repertoires and ensembles.The authors note a proof of concept with Beethoven orchestral works and propose unifying performance data with other LLM-interpretable information.Source: §8 · p. 15
  • Performer A makes the largest tempo change near the beginning, whereas Performer B spends more time in the later quiet passage and transition.Source: §8 · p. 15
  • Score- and MIDI-grounded analyses characterize timing, velocity, duration, and pedal use, but not timbre or acoustics; audio-derived features and audio-capable models could address this gap.Undetected score or alignment errors may propagate into supervision despite the verification mechanism; combining verifiable evidence with directly perceived acoustic information could support more comprehensive captions.Source: §8 · p. 15
  • Automatic transcription such as MuScriptor (Rouard et al. (2026)) could approximate the performance-MIDI layer when aligned MIDI is unavailable, while future work could assess locally deployable open-weight models.This could extend the pipeline to a broader range of audio–score datasets.Source: §8 · p. 15
  • The influence of prior exposure to well-known works on captions remains unknown, and captions may reflect listeners’ cultural backgrounds, histories, and training rather than a single definitive way of listening.The pipeline is available for researchers to use different models and adapt instructions to their own inquiries.Source: §8 · p. 15

9 CONCLUSION

MuNo-SP unifies written score content with performance realization in a language-model-accessible format. Its data-generation pipeline produces MAESTROCaps, whose human evaluation supports the reliability and musician-likeness of its generated captions.

9 CONCLUSION

  • MuNo-SP jointly represents score content and performance realization, including musical features and aligned timing, duration, velocity, and pedal data.The joint format makes score and performance information more accessible to language models.Source: §9 · p. 16
  • The pipeline generates time-grounded auditory analyses and question–answer pairs from aligned scores and performances, forming the MAESTROCaps dataset.Its supervision gives concrete accounts of musical events, structural roles, and interpretive choices.Source: §9 · p. 16
  • Human evaluation supports the reliability and musician-likeness of the generated captions, and MuNo-SP with MAESTROCaps provides a foundation for audio-language models with musician-level music understanding.Source: §9 · p. 16

COPYRIGHT: … SEGMENTED TIMED MUSICAL DATA

This section specifies procedures for score correction and structural segmentation, alongside prompts and constraints for producing concise, evidence-grounded musical analyses that connect score evidence, musical structure, and performance. It addresses timing, pitch notation, and output structure, with examples of timestamped analysis and segmented representation.

A.3 SCORE-CORRECTION ALGORITHM

  • Score corrections are proposed from alignment and external-score disagreements, tested against performance evidence, and manually checked against public-domain editions before documentation.The procedure uses Parangonar alignments to locate insertion candidates and PianoCoRe support to validate proposals; disagreement with another score prompts validation rather than automatic correction.Source: §A.2 · p. 21, §A.3 · pp. 21–22

A.4 STRUCTURAL-SEGMENTATION PROMPT

  • The segmentation prompt maps the complete piece into hierarchical sections, theme presentations, and phrases, grounding boundaries in MuNo while using prior musical knowledge only as interpretive aid.It distinguishes large-scale formal regions, thematic occurrences or transformations, and local phrase functions, with musically informative names rather than generic numbering where possible.Source: MUNO REPRESENTATION: [MUNO-S… · p. 23, §I · p. 23

II. Boundary Rules

  • Segments use inclusive bar ranges, may overlap, and express hierarchy through names and roles; boundaries and identities require concrete musical evidence and concise explanations.Top-level sections must cover the requested scope, while phrase divisions need not force a boundary into every bar; the prescribed output is strict JSON with required fields and final checks.Source: §II · p. 23, §III · p. 23, §IV · p. 24

A.5 LONG-FORM AUDITORY-ANALYSIS PROMPT

  • The segmentation-aware MuNo-SP prompt is populated with verified piece context, a calculated phrase-sentence plan, and a segmented representation for each recording.The supplied notation guide explains timestamps, voices, sounding duration, rests, expressive markings, and pedal events to support interpretation of the score and performance together.Source: §A.5 · p. 26

SCOPE AND LENGTH

  • Analyses cover every declared phrase, describe its unfolding, and begin with a two-sentence overview; theme and section syntheses each contain two sentences on thematic change and harmonic tension.Phrase sentence counts are calculated from inclusive bar counts, capped at eight sentences, with analytical space allocated to melody, harmony, texture, dynamics, phrasing, rhythm, and performance decisions.Source: SCOPE AND LENGTH · pp. 26–27

MELODY IDENTIFICATION AND GROUPING

  • Melody identification favors continuous, supported lines over isolated loud notes, tests secondary strands across multiple cues, and treats velocity as contextual attack-prominence evidence rather than a universal loudness measure.A transfer requires sustained thematic or phrase continuity and prominence in the new voice; unsupported lines should not be manufactured.Source: §1 · p. 27, §2 · p. 27, §3 · p. 27

4. ATTACKS, GROUPING, AND TRANSFERS

  • Within selected lines, the prompt preserves genuine attacks and slurred gestures, distinguishes larger groupings from individual slurs, and fully transcribes first presentations while abbreviating later recurrences.In melodic chords, it selects the tone continuing the line; later versions identify timestamps and report only changed, omitted, added, reharmonized, redistributed, or newly emphasized pitches.Source: §4 · p. 27, §5 · p. 27

ANALYTICAL METHOD

  • The analytical method separately examines melody, harmony and tension, texture, dynamics and phrase shaping, function, and rhythm and time.It traces sounding harmony and bass motion, chromatic color, dissonance, and resolution while distinguishing active tones from passing notes or residual resonance; cadence claims require a named type, tonal goal, and supporting harmonies.Source: ANALYTICAL METHOD · p. 27
  • Ground interpretive choices in local timing, articulation, tension, and repeated patterns, and report a missed note only when surrounding evidence strongly supports that inference.The guidance calls for cautious treatment of possible missed notes, names audible cues for tension and its course, and advises summarizing repeated accompaniment patterns without cataloguing every event.Source: ANALYTICAL METHOD · p. 28

TIME AND EVIDENCE GROUNDING

  • Use numeric MM:SS.cc timestamps for observations and include octave numbers and supplied pitch spellings for every written note.Each sentence must contain a timestamp; observations should be timestamped where they occur, and sequential notes and simultaneous chord tones use different separators.Source: TIME AND EVIDENCE GROUNDING · p. 28

AUDITORY INTERPRETATION

  • Describe only performance qualities supported by what is audible, using literal musical language and avoiding unsupported affect or bodily-technique inferences.The guidance permits terms such as legato or staccato only when the audible performance supports them, and treats dynamics, tempo, articulation, and resonance in terms of their audible effects.Source: AUDITORY INTERPRETATION · p. 28

PROHIBITED OUTPUT

  • Exclude notation and encoding details, MIDI velocity numbers, written note values, and claims about unobservable context or performer intent.The guidance also prohibits bar references and artificial contrasts such as “rather than.”Source: PROHIBITED OUTPUT · p. 28

OUTPUT EXAMPLE

  • The output example uses timestamped pitch and harmony evidence to describe articulation, pacing, intensity, tension, and phrase-level change.It also specifies a Markdown hierarchy for whole-piece, section, theme, and phrase analyses, with phrases outside themes placed under their enclosing section and each declaration covered once.Source: OUTPUT EXAMPLE · p. 29

SEGMENTED TIMED MUSICAL DATA

A.6 PAIRED AUDITORY-ANALYSIS EXAMPLES · A.6.1 MUNO-SP ANALYSIS

The paired examples compare MuNo-SP- and MIDI-event-text-grounded analyses of the same complete performance, organized with a shared structural hierarchy.

A.6 PAIRED AUDITORY-ANALYSIS EXAMPLES

  • For Recording 4, both analyses cover the complete Rachmaninoff prelude with the same scope, while Figure A.1 shows their shared section, theme, and phrase hierarchy.The first analysis is grounded in segmented MuNo-SP; the second in segmented MIDI performance-event text.Source: §A.6 · p. 30

Whole-piece analysis · Developmental middle section · Scalar-link idea

The piece moves from a quiet D-major arpeggiation through two presentations of a cantabile theme into an agitated developmental middle and compressed closing recollection. Its scalar link builds dominant-directed pressure through contrary motion above a descending bass.

Whole-piece analysis

  • Across the piece, a quiet arpeggiated introduction leads to two thematic presentations, an agitated sequential middle, and a compressed closing recollection.The principal climax at 02:42.36–02:45.84 grows from registral expansion, faster harmonic turnover, and massive chordal attacks; a long D-major sonority at 04:11.93–04:24.49 provides the deepest release.Source: Whole-piece analysis · pp. 30–31
  • From 01:02.90–01:54.80, the second presentation retains the melody in an inner register while a new upper counterline grows more chromatic, expansive, and forceful.Its arc moves from D-major stability through B-minor and chromatic pressure to a forte D-major apex, then dissolves through E-dominant and A7-directed tension.Source: Whole-piece analysis · p. 31

Developmental middle section

  • The developmental middle introduces a contrasting scalar link, develops a chordal sequential idea, and drives it to a central fortissimo culmination.Source: Developmental middle section · p. 31

Scalar-link idea

  • From 01:56.77–02:01.70, a brief scalar link transforms an F#4–E4–F#4 recollection into contrary-motion ascent above a descending accompaniment, intensifying toward dominant-directed pressure.The bass reaches C#2 while the upper line reaches C#5 at the link’s boundary.Source: Scalar-link idea · p. 31

Sequential chordal episode … A.6.2 MIDI ANALYSIS

The analysis follows a chordal sequence through transposition, chromatic intensification, and a central climax, then traces an expanded thematic return and a coda that moves from suspension to a final D-major close.

Sequential chordal episode

  • From 02:02.26 to 02:30.50, the model shifts through B, A, A minor, E minor, and G while preserving paired upper attacks and rocking bass.The ritardando at 02:16.90-02:18.82 temporarily releases momentum; chromatic E4-Eb4-D4 motion and the G-centered continuation restore it.Source: Sequential chordal episode · p. 32
  • The sequential episode fragments and expands its chordal model into a registral climax at 02:42.36, then reverses into a descending liquidation and immediate diminuendo.The sequence’s bass progression G-F-E-Eb-E and increasingly chromatic upper structures drive tension before descending B-minor- and E-minor-related sonorities release it.Source: Sequential chordal episode · p. 32

Grand varied return and aftersong

  • The grand return restates the principal theme in expanded chords with a high countermelody, reaches its forte apex at 03:12.44, and resolves from A7 to D major at 03:39.27.Its tensions pass through B-minor, E-minor, F#-minor, and altered-dominant regions before the final authentic arrival.Source: Grand varied return and aftersong · p. 33
  • After the D-major arrival, two quiet extensions move through B-minor, E-dominant, and G-minor colors before ending on sustained A7.The extensions reduce the texture and leave the dominant color suspended in a long diminuendo.Source: Grand varied return and aftersong · p. 33

Scalar-link idea

  • The scalar-link idea returns quietly in D major, then expands upward toward an unresolved high point through added-note and dominant tension.The rising line reaches A4, C#5, D5, and E5 before the suspended arrival at 04:01.79-04:02.04.Source: Scalar-link idea · p. 33
  • Before the final tonic, a sustained B-minor-derived sonority gives way to A7 and a soft, short-lived D4 resolution.The B-minor-derived sonority suspends motion for more than three seconds before release.Source: Scalar-link idea · p. 33

Coda

  • The coda compresses the scalar recollection into a rising gesture, suspends motion on a B-minor-derived sonority, and then resolves definitively to D major.The final tonic begins at 04:11.93 and remains resonant until 04:24.49.Source: Scalar-link idea · p. 33

Whole-piece analysis · Opening cantabile exposition · First developmental expansion

The analysis traces a cantabile theme from quiet arpeggiation through increasingly dense, sequential and chordal development to forceful arrivals, then toward quieter release. It describes how bass foundations, chromatic shifts, register, pedal and resonance shape this changing arc.

Whole-piece analysis

  • The piece grows from widely spaced arpeggiation into dense sequential and chordal waves, peaks at 02:39.48–02:45.89, then returns toward a quieter lyrical texture.Recurrent D2 and A2 foundations, chromatic bass shifts, upper-register transformations, pedal clearances and longer final resonances shape the arc from emergence through culmination to dissolution.Source: Whole-piece analysis · p. 34

Opening cantabile exposition

  • The initial F#4 idea expands through answering and sequential phrases before closing on an open D2+G3+B3 sonority that moderates, but does not resolve, the accumulated tension.The melody reaches G4 twice and a registrally enlarged B4; dominant and minor inflections progressively heighten tension before the open close.Source: Opening cantabile exposition · p. 34
  • Across the opening exposition, an isolated F#4 grows into answering, sequential, ornamented and chordal forms, while D-centered arpeggiation gives way to altered and minor inflections.The expanded melody reaches D5 at 00:51.35, and its strongest tension leads into a sustained D-centered culmination at 00:50.59–00:52.47.Source: Opening cantabile exposition · p. 35

First developmental expansion

  • After the opening culmination, falling C#5–B4 tones become a flowing E4–F#4 sequence, dissolve through descent and renew with an independent upper ornament.The dense C2-based sonority relaxes through A-dominant and descending-bass fields, then regains D-centered stability while upper chromatic neighbors preserve forward tension.Source: First developmental expansion · p. 35
  • The repeated C#4–D4 motive rises through F#4 and G4 with increasingly rapid upper decorations, culminating in a sharp G#4+F5 dissonance at 01:28.17.A7-related suspension, D- and G-centered shifts, and chromatic D#4 and G#4 heighten tension without a settled tonic arrival.Source: First developmental expansion · p. 36
  • The lyrical ascent becomes a forceful chordal wave, withdraws through descending figures, then rebuilds tension with low G1 and C2 foundations toward a final D5 thrust.Broad D-major blocks alternate with altered-Bb and B-minor colors; density and attack intensity peak at 01:44.48–01:45.95.Source: First developmental expansion · p. 36

Central development and first grand climax

The passage moves from a broad proclamation through quieter liquidation and sequential rebuilding into three increasingly compressed climactic waves, ending on an unresolved B6 apex.

Opening statement and liquidation

  • The opening C-based proclamation expands to A5, then descends through a dominant-resolution figure into a quieter A7-related field and relaxed dominant resonance.The liquidation includes a mostly stepwise Bb4-G4-E4-D4-E4-D4-C#4 melody, with wider onset gaps and softer attacks.Source: Central development and first grand… · pp. 36–37

Long gathering transition

  • A modest D4-F#4 idea rebuilds through contrary inner motion, accelerating registral ascent, and a renewed E5 rise that restores high tension.A chromatically descending bass and increasingly close, stronger attacks turn the quiet transition into a cumulative surge.Source: Central development and first grand… · p. 37

First broad sequential period

  • A broad A-dominant antecedent rises toward D5-E5, answered by a slightly lower C5-E5 consequent before a brief, unresolved reduction to E5-F#5.The consequent’s more measured spacing briefly relaxes intensity, while suspended seconds and fourths recede without tonal resolution.Source: Central development and first grand… · p. 37

Cumulative build-up and launch

  • After the reduction, isolated tones expand into a denser ascent and crescendo, then a compressed chromatic D#4-D4 collision launches the central climax.The build includes a remote B5 that sharply expands the register, followed by an altered dominant cluster with forceful D5 attack.Source: Central development and first grand… · p. 37

Central culmination

  • Three climactic waves transform repeated chordal surges into sequential rises to F5 and A5, then an octave-doubled ascent to C6 under sustained maximum tension.The waves progress from a G-major field through G-minor and F-based colors to semitone-rich dominant coloring.Source: Central development and first grand… · pp. 37–38

Central high-register culmination

  • The final culmination ascends to B6 through chromatic and whole-step motion, octave doubling, added seconds, and ascending bass supports, leaving the sonority suspended.The ascent spans C#6-D6-D#6-F#6-G6-A6-B6 and peaks at 02:41.64-02:42.39.Source: Central development and first grand… · p. 38

Climactic continuation and retransition … TIME-STAMPED AUDITORY ANALYSIS: [TIME-STAMPED AUDITORY ANALYSIS]

This section traces the passage from an extreme-register climax through a softer reprise to a resonant D-major close. It also specifies how to generate clear, musically meaningful listening questions grounded in the time-stamped analysis, including questions about performance dimensions such as articulation, pacing, pitch emphasis, duration, and rubato.

Climactic continuation and retransition

  • The extreme-register apex descends through D6-E5, reanimates in six surges to an altered F6 peak, then gives way to fragmented thematic return and a D-centered boundary.The surges rise from F#5-D6 to G5-E6, while the principal theme reappears in fragments before the octave-D arrival at 03:28.96.Source: Climactic continuation and retransition · p. 39

Varied reprise and subsiding close

  • The elevated reprise compresses the theme, subsides through softer sequential echoes, then makes a final rise to F#5 as prolonged A7 tension redirects into B-minor/F#.The reprise moves from altered C, G-major, and A-dominant colors through prolonged A7-related harmony to a sustained B-minor/F# sonority.Source: Varied reprise and subsiding close · pp. 40–41

Coda and resonant postlude

  • The coda dissolves the theme into sustained tonic-like resonance; a final E4-D4 echo over A7-related support leaves D4 suspended against C#4 before pedal release.The repeated D4 is quieter and held into the pedal clearance, exposing the unresolved suspension before the terminal sonority.Source: Coda and resonant postlude · p. 41
  • The A7-related suspension resolves into open D major, and the sustained D2+A2+F#3+D4 chord provides the final release without further melodic attacks.Its resonance continues through natural decay until the final pedal release at 04:24.49.Source: Coda and resonant postlude · p. 41

A.7 GENERAL QUESTION–ANSWER GENERATION PROMPT

  • The general QA variant requests seven questions, one for each dimension, using piece metadata and the time-stamped auditory analysis for each recording.The dimensions are melody; harmony, color, and tension; texture; dynamics and phrase shaping; function; rhythm and time; and particular performance decisions.Source: §A.7 · p. 42

SYSTEM PROMPT

  • Questions should sound like genuine inquiries from one trained musician to another and use only the supplied time-stamped auditory analysis as their source.The prompt rejects generic quiz-like or fact-extraction wording and forbids introducing claims absent from the analysis.Source: SYSTEM PROMPT · p. 42

OUTPUT FORMAT

  • The required output is a JSON array of seven objects with category, question, answer format, answer, reasoning, and verbatim evidence fields.The evidence field contains a contiguous excerpt from the auditory analysis supporting the question and answer.Source: OUTPUT FORMAT · p. 42

CORE REQUIREMENTS

  • Core requirements make questions source-faithful, answer-first, precisely specific, audible without hidden data, and anchored only as needed to identify an event.The answer must be concise and gradable; timestamp questions must avoid revealing their answer time, and evidence must reproduce the source wording contiguously.Source: CORE REQUIREMENTS · pp. 42–44

MANDATORY SILENT AUDIT

  • The silent audit checks that each question identifies one defensible, audible target, avoids answer leakage, and reflects genuine musical inquiry.It also rejects ambiguous context timestamps and hidden-mechanism questions.Source: MANDATORY SILENT AUDIT · p. 44

CATEGORY DISTRIBUTION

  • The prompt requires exactly one question for each of the seven axes and asks for a varied set.Each question is silently checked for genuine inquiry, a single clear task, answer leakage, and competing defensible answers.Source: CATEGORY DISTRIBUTION · p. 44

USER PROMPT

  • The user prompt requests exactly seven listening-comprehension questions, directly supported by the analysis and voiced as a trained musician listening analytically.It explicitly disallows prior knowledge, reconstruction of missing facts, and invented details.Source: USER PROMPT · p. 45
  • Questions must address a genuine musical inquiry, ask about one of seven axes, and use only enough context to identify the target without revealing its answer.Generate exactly one question for each axis; timestamp questions must not include the answer time or an equivalent rounded form.Source: USER PROMPT · p. 45

MELODY

  • Melody questions should target an explicit, audible melodic claim and request only the pitch, register, contour, recurrence, or transformation stated in the analysis.For timestamp answers, add only enough context to distinguish the target and never include its timestamp or a rounded equivalent.Source: MELODY · pp. 45–46

HARMONY, COLOR, AND TENSION

  • Harmony questions should ask only about a chord, color, tonal center, dissonance, cadence, or tension cause explicitly supplied by the analysis, without performing new harmonic analysis.When the analysis gives color but no chord name, ask about that stated color using finite choices rather than inventing a chord label.Source: HARMONY, COLOR, AND TENSION · p. 46

TEXTURE

  • Texture questions should focus on stated layer count, balance, register, spacing, interaction, or role changes, without inferring unnamed voices or instrumentation.Ask for an exact layer count only when the analysis states it clearly.Source: TEXTURE · p. 46

DYNAMICS AND PHRASE SHAPING

  • Dynamics and phrase-shaping questions should match the analysis’s stated intensity, emphasis, connection, breath, arrival, taper, release, or phrase shape without combining dimensions.Avoid stems that disclose the requested contour or make its answer predictable by naming a climax.Source: DYNAMICS AND PHRASE SHAPING · p. 47

FUNCTION

  • Function questions should ask about a stated structural role or uniquely located event, without deriving conventional form or naming the requested role in the stem.Timestamp questions should describe the event without including its answer time.Source: FUNCTION · p. 47

RHYTHM AND TIME

  • Rhythm-and-time questions should use only analysis-supported pulse, subdivision, recurrence, displacement, pause, or restart claims, without inferring meter, note values, or numerical tempo.Define recurring events clearly enough for a single answer and avoid counts in dense or vaguely bounded passages.Source: RHYTHM AND TIME · pp. 47–48

PARTICULAR PERFORMANCE DECISIONS

  • Performance-decision questions should target one explicitly attributed audible choice or event, with its defining moment or comparison stated and no inference about technique, intention, emotion, or causation.A likely missed note may be included only when the analysis grounds it in audible context, and the performer’s intention must not be presented as certain.Source: PARTICULAR PERFORMANCE DECISIONS · p. 48
  • Avoid cadence-tempo questions that invite the conventional answer of slowing down unless the analysis explicitly reports acceleration or unchanged pacing.Keep the question neutral; do not signal that a contrary result is surprising or unexpected.Source: PARTICULAR PERFORMANCE DECISIONS · p. 48
  • For timestamp questions, use only enough musical context to distinguish the target and never include its timestamp or a rounded equivalent.Add context such as thematic occurrence, affected voice, comparison, nearby event, or a broad search interval only when multiple decisions could match or the comparison scope is unclear.Source: PARTICULAR PERFORMANCE DECISIONS · p. 48
  • Example questions target comparative articulation, gesture pacing, agogic emphasis, shortest sounding duration, and the timestamp of greatest rubato.The examples specify musical context such as a theme statement, foreground gesture, upper line, or turn into a closing cadence.Source: PARTICULAR PERFORMANCE DECISIONS · p. 48

A.8 HUMAN-EVALUATION RECORDINGS … STRUCTURED OUTPUT CONSTRAINT

The appendices specify blinded human and LLM-judge comparisons of MuNo-SP and MIDI descriptions, including the recordings, evaluation criteria, and response format. The judge protocol uses complete recordings and supplies detailed Bach-description examples.

A.8 HUMAN-EVALUATION RECORDINGS

  • Three evaluators assessed the same three solo-piano recordings, selected to span contrasting forms, historical styles, and interpretive demands.The pieces were Beethoven’s Appassionata, Schubert’s Moment musical No. 3, and Debussy’s Reflets dans l’eau.Source: §A.8 · p. 49

A.9 HUMAN-EVALUATION PROMPT

  • Human evaluators compared source-hidden descriptions, judging musical content rather than length or layout over approximately matching recording intervals, and chose A, B, or tie.The participant-facing prompt concealed whether each description was MuNo-SP or MIDI-only.Source: §A.9 · p. 50, STUDY INSTRUCTIONS · p. 50

1. More/less correct

  • The four comparison dimensions were correctness, musician-likeness or perceived fit, insightfulness, and overall preference.The prompts asked which description better described the recording, resembled the listener’s perception, offered more useful insight, and was preferred overall.Source: §1 · p. 50, §2 · p. 50, §3 · p. 50

4. Overall

  • For the overall human-evaluation response, participants selected A, B, or exactly same for each criterion and explained their choices, giving an example of any error.The LLM-judge overall prompt likewise requested a brief explanation and an example if something was incorrect.Source: §4 · pp. 50, 52

A.10 LLM-AS-A-JUDGE PIECES

  • The LLM-as-a-Judge evaluation used complete recordings of nine piano pieces.The listed repertoire spans Bach, Beethoven, Chopin, Debussy, Liszt, Mozart, Rachmaninoff, Schubert, and Scriabin.Source: §A.10 · p. 51

A.11 LLM-AS-A-JUDGE PROMPT

  • The judge received the complete recording and two descriptions in randomized, blinded order, and was instructed to judge only from those materials.The Bach Fugue in D minor example identifies Description A as MuNo-SP and Description B as MIDI.Source: §A.11 · p. 52, SYSTEM INSTRUCTION · p. 52

USER PROMPT

  • The judge’s fixed listening evidence was the complete recording; the Bach example contrasts a subject’s triplet opening and chromatic descent with an exposed line’s registral apex and descent.The prompt also requested the four comparison dimensions and said to ignore possible differences in analysis time windows.Source: AUDIO CONTEXT · p. 52, USER PROMPT · p. 52

STRUCTURED OUTPUT CONSTRAINT

  • The required structured response contained correctnessChoice, musicianChoice, insightChoice, choice, and comment; each choice had to be a, b, or tie, with no extra fields.The comment had to be a string.Source: STRUCTURED OUTPUT CONSTRAINT · p. 52
Loading 2609.10351v1…