Getting Motif-ated: Controllable AI Compositions from Injected Motif Prompts

1 Introduction

• Few models offer motif-level control, while existing approaches provide either coarse controls or require fully formed musical material.Motifs are transposition-invariant abstractions, so conditioning on precise notes can limit generation; evaluating motif incorporation also requires specialized methods.Source: §1 · pp. 1–2
• MotiGen represents motifs as transposition-invariant interval-class sequences and injects them as structured prompt lines reinforced by attention bias.Its two-phase curriculum moves from focused crops around motif occurrences to full scores annotated with extracted motifs.Source: §1 · p. 2
• The recipe combines motif prompt lines and train- and inference-time attention bias, focused-crop training, and a transposition-invariant, enharmonic-aware containment evaluator.Source: §1 · p. 2

Generative Symbolic Music Models

• NotaGen scales symbolic music generation with LLM-style pretraining and preference optimization, yet the discussed models do not control thematic material.Earlier systems include event-token, text-based, and hierarchical ABC-notation models.Source: Generative Symbolic Music Models · p. 2

Attention and Activation Steering

• Unlike frozen-model steering methods, MotiGen applies its attention bias during training and inference, while focused crops make motif-content associations explicit before full scores.Theme Transformer conditions on concrete pitches and rhythms and is trained from scratch; MotiGen uses abstract, transposition-invariant motifs with a pretrained LLM.Source: Attention and Activation Steering · p. 3, Generative Symbolic Music Models · p. 3

Motif Discovery in Music Information Retrieval (MIR)

• MotiGen’s contribution is using a sliding-window repeated-pattern extractor as a control signal, rather than proposing a new motif-discovery algorithm.Repeated-pattern discovery is an established MIR problem with geometric, interval-based, and closed-pattern approaches.Source: Motif Discovery in Music Information… · p. 3

3.1 Motif Representation and Extraction

• Motifs are sequences of signed diatonic interval classes: 0 marks the first note, while steps, skips, and leaps encode contour and interval category.The representation collapses immediate repetitions and is transposition-invariant by construction; extraction scans windows of 4–10 notes, selects the most common sequence weighted by length, and uses four-note motifs in the experiments.Source: §3.1 · p. 3

3.2 Conditioning Interface

• MotiGen places an abstract motif in a structured ABC header prompt and adds a scalar bias to motif-key attention logits during both training and inference.The reported optimal bias is β=4; β=2 has little effect, while β≥8 collapses generation.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

3.3 Curriculum training

• Training first uses real excerpts cropped ±5 measures around motif occurrences, then fine-tunes on full scores annotated with extracted motifs.The first phase makes the prompt-content association more direct; the second teaches how motifs can be present and varied throughout a piece.Source: §3.3 · p. 4

3.4 Training Datasets and Motivic Analysis

• The monophonic training pool combines OpenScore Lieder vocal lines and Irishman folk tunes, augmented across 12 keys; motif frequencies are strongly skewed.The pool contains 210 distinct length-4 patterns, and evaluation is stratified by motif frequency.Source: §3.4 · pp. 4–5, Figure 3 · p. 5

4 Experiments

• The evaluation generates 50 pieces per model–motif pair under fixed decoding settings and tests 24 motifs sampled across three frequency bands.The metrics distinguish whether a generated melody contains the motif from whether the model’s emitted motif line realizes it.Source: §4 · p. 5

4.1 Ablations

• 79.4% containment from plain fine-tuning exceeds the 35.9% base rate, while the full recipe leads on every reported metric.Removing attention bias costs the most; removing cropped training has a milder effect on mid-distribution motifs but matters more for tail motifs and small corpora.Source: §4.1 · p. 5, Table 1 · p. 6

4.2 Case Study

• In the case study, MotiGen varies rhythm and motif realization and adds connecting material, whereas Fable 5 produces a simpler sequence with repeated sixth-up motifs.The comparison uses the motif described as a leap up and two steps down.Source: §4.2 · p. 6, Figure 4 · p. 6

5 Conclusion

• MotiGen represents motifs as transposition-invariant interval-class patterns, inserts them as structured prompt lines, reinforces them with attention bias, and uses focused-crop training.The resulting model composes music with motivic material integrated into the texture rather than simply appended at the beginning.Source: §5 · p. 6
• The recipe could also carry instructions for harmonic progressions, bass lines, rhythmic cells, or formal plans.Source: §5 · p. 6
• The reported scope is motif control in a pretrained symbolic music model, with other instruction types presented as possibilities.Source: §5 · p. 6

A Implementation Details

• Each curriculum phase is evaluated on data from its own distribution and with its own bias configuration to avoid misleading cross-phase comparisons.Evaluating a bias-trained model with bias off understates its quality; phase matching avoids an apparent phase-two jump that would suggest phase one was undone.Source: A Implementation Details · p. 8
• Fine-tuning starts from the public NotaGen-large checkpoint and each run fits on one NVIDIA RTX A6000 with 48 GB of memory.The reported ablation, scaling, and evaluation runs total roughly under a GPU-month on that hardware.Source: A Implementation Details · p. 9
• The setup uses AdamW with a learning rate of 10^-5 and batch size 1 in NotaGen’s patchstreaming regime.Source: A Implementation Details · p. 9

B Data scaling improves motif adoption

• The scaling study compares 10k and 1k training subsets with the 200k+ full dataset.The subsets are each an order of magnitude smaller than the next larger dataset.Source: B Data scaling improves motif adoption · p. 9
• Motifs are stratified into bands A, B, and C, sampling eight each from the top 10, ranks 11–40, and ranks 41 and rarer.Each motif is sampled 50 times, and scores are averaged over generated pieces.Source: B Data scaling improves motif adoption · p. 9
• The study trains both ablated and full-model variants on the subsets, while the Full Training block reports the component results underlying Table 1.Source: B Data scaling improves motif adoption · p. 9

B.1 Full per-motif results

• The full pipeline’s advantage is greatest with few data points, smaller with abundant data, and weakest when severe scarcity hinders generalization across configurations.Source: §B.1 · p. 9
• For tail motifs, the full recipe yields almost consistently 10% higher containment and faithfulness than the next-best ablation on both the 10k and full datasets.The reported advantage is especially prominent for rank-C motifs.Source: §B.1 · p. 9

Pitch representation

• Each note is tracked by chromatic pitch and diatonic staff position, allowing the classifier to distinguish enharmonically equivalent intervals by spelling.Source: Pitch representation · p. 9

Repeat collapse

• Immediate equal-pitch repetitions are collapsed to one note, so a restruck tone within a motif span still counts as part of its contour.Source: Repeat collapse · p. 9

Interval classification

• Interval classes encode signed movement as steps, skips, or leaps, using letter-step distance while treating chromatic alterations of the same letter as steps and honoring enharmonic spelling.For example, C–D♯ is a step whereas C–E♭ is a skip; plain semitone input uses a nearest-interval approximation.Source: Interval classification · pp. 9–10
• Across 2,353 pieces × 12 key transpositions, extracted motifs were identical except in 29 cases traced to source-encoding inconsistencies.Source: Interval classification · p. 10

Extraction and containment

• At evaluation, a generated melody contains a target when a collapsed-note window maps to that motif under the same parsing and classification procedure.Source: Extraction and containment · p. 10

• Body containment counts a generation when any window in the parsed, collapsed melody matches the target motif, regardless of key or location, excluding prompt lines.The reported rate is the fraction of 50 generations per motif that pass.Source: Evaluation metrics · p. 10
• Realization faithfulness checks whether the model’s opening %motif:abc: header line parses to the requested abstract pattern.A missing or malformed line counts as unfaithful.Source: Evaluation metrics · pp. 10–11
• Because faithfulness examines only the header and containment only the tune body, their cross-tabulation distinguishes instruction echoing from motif use in the composition.Source: Evaluation metrics · p. 11

• The 210 distinct length-4 motifs across 217k annotated pieces are highly skewed: two stepwise patterns account for 48% of annotations, motivating head-, mid-, and tail-frequency evaluation strata.Figure 5 plots motif rank frequency on log-log axes and marks the sampling regimes with shaded bands.Source: D Training Data Motivic Analysis · p. 11, Figure 5 · p. 11

NeurIPS Paper Checklist

• The checklist says the paper provides the information and artifacts needed to reproduce its main results, with data and foundational model publicly available and code in Appendix A.It also reports that training and test details are provided in Appendix A.Source: NeurIPS Paper Checklist · pp. 13–14
• The checklist reports confidence intervals for experimental results and sufficient resource details in Appendix A.Reported resources include the compute information needed to reproduce experiments.Source: NeurIPS Paper Checklist · pp. 14–15
• The checklist states that the research conforms to the NeurIPS Code of Ethics and that the paper discusses potential positive and negative societal impacts.The stated rationale is the importance and viability of models that take musical inputs intuitive to composers.Source: NeurIPS Paper Checklist · p. 15
• The evaluation is automatic, with a human listening study deferred to future work; the checklist also indicates that the paper discusses its limitations.The checklist marks theoretical assumptions and proofs as not applicable because the paper contains no theoretical claims or results.Source: NeurIPS Paper Checklist · pp. 12, 17