PaperlayerView exact revision on arXiv

Source-linked AI summary

MI-MIDI 2: From Observation to Control with Sparse Autoencoders in Text-to-MIDI Generation

Jakub Poćwiardowski, Mateusz Modrzejewski

arXiv:2610.32007v1cs.SDcs.AI

TL;DR

Observational interpretations of SAE features have been an unreliable guide to steering in language models, and whether they predict control in music generators had not been tested. The paper develops a controlled protocol for text-to-MIDI models and finds that some controlled observations identify features that steer generated MIDI.

  • Problem

    Whether observational evidence about unsupervised SAE features predicts their effect on generated music had not been tested, despite such evidence being an unreliable guide to control in language models.

    Source: §1 · pp. 1–2

  • Method

    The protocol characterizes SAE features with controlled metric correlations, instrument-program patterns, and descriptions, then tests decoder-direction interventions in MIDI against baseline and matched random directions.

    Source: §1 · p. 2, §5.1 · p. 6

  • Results

    A description-selected synth-lead feature steered instrument choice in confirmation and replication; density-adjusted selections showed a small positive family-level target shift, unlike raw-correlation selections.

    Source: §6 · p. 9, §7 · p. 10

  • Takeaways & Limitations

    The authors recommend applying observational controls before selecting steering candidates and confirming individual features on new prompts.

    Source: §1 · p. 2

    All endpoints are symbolic or visual, and the study did not run a listening study.

    Source: §6 · p. 10

Abstract

from arXiv · show

Sparse autoencoders (SAEs) can expose features of a generative model without predefined labels, but in language models the evidence used to interpret these features has proven an unreliable guide to which of them steer the output. Whether interpretations of SAE features predict control in music generators has not, to our knowledge, been tested. We develop an end-to-end protocol for discovering and evaluating SAE features in text-to-MIDI generation and apply it to text2midi and MIDI-LLM. The protocol characterizes features through correlations with musical metrics, shared instrument programs, and automated descriptions, each with its own control: note-density adjustment, matched inactive excerpts, and permutation calibration. It then tests candidates by intervention, measured directly in the generated MIDI against unmodified generation and a matched random direction, ranks them in screens, and confirms selected features on new prompts. Evidence that survives these controls points to features that steer the output. Descriptions exceed permutation controls for instrumentation, and a feature whose description names a synth-lead program introduces that program in a prespecified confirmation and in a replication on disjoint prompts and reappears in independently trained dictionaries. Features selected by density-adjusted correlations show a small positive shift of their target metric as a family, whereas features selected by raw correlations show none. An atlas provides MIDI, piano-roll, and audio examples for every feature of the targeted screen.

1 Introduction

The introduction asks whether observational interpretations of unsupervised SAE features predict control in music generation, a relation not yet tested for music generators. It presents a controlled text-to-MIDI protocol and reports evidence that some controlled observations identify steerable features.

1 Introduction

  • In language models, observational evidence about feature meaning—including active examples, correlations, and generated descriptions—has proved an unreliable guide to steering effects.Prior work also reports that SAE steering can underperform simple baselines and that automated interpretability scores do not distinguish trained from randomly initialized transformers.Source: §1 · p. 1
  • Whether observational evidence about unsupervised features predicts their effect on generated music had not been studied, and text-to-MIDI scores make notes, durations, and instrument programs exactly measurable.Earlier music work used audio-based SAE steering or supervised directions for predefined symbolic attributes.Source: §1 · pp. 1–2
  • The study applies four stages to text2midi and MIDI-LLM: characterize SAE features, intervene along decoder directions, screen candidates, and confirm selected features on new prompts.Observational routes include metric correlations, shared instrument programs, and automated descriptions, each paired with a control.Source: §1 · p. 2, Figure 1 · p. 2
  • A feature from the description group that survives permutation control steers its instrument program in confirmation and replication, while density-adjusted selections show a small positive family-level shift unlike raw-correlation selections.The protocol also offers an atlas of MIDI, piano-roll, and audio examples for all 80 features of the targeted screen.Source: §1 · p. 2

2 Related Work

Related work situates the study among SAE methods, feature steering, and mechanistic research on music generators. It highlights prior limits in interpreting features and the distinction between discovering unanticipated concepts and steering predefined attributes.

Sparse autoencoders

  • SAEs address neural superposition with overcomplete dictionaries of sparse coefficients; this study uses TopK activations and separately evaluates reconstruction error and downstream-loss preservation.Its held-out recognition task asks whether a language-model description predicts feature activation on new examples.Source: §2 · pp. 2–3

Steering with dictionary features

  • Prior steering work selects SAE features by output scores or describes them by output effects, while one account argues SAEs better discover unanticipated concepts than act on known ones.SAE features can also differ across training seeds, motivating checks for whether confirmed features reappear in independent dictionaries.Source: §2 · p. 3

Music generators

  • Music-generation research includes MusicGen steering and SAE studies, symbolic steering of pitch and duration, and TopK SAE feedback control of predefined attributes.The studied models build on captioned MIDI, and earlier work analyzed them with probing, lenses, activation patching, and supervised steering.Source: §2 · p. 3

3 Models and Dictionaries

The study trains SAE dictionaries on residual states from two differently structured text-to-MIDI models, using Lakh MIDI activations and excerpt splits for characterization. It reports dictionary diagnostics and notes that adjusting for note and token counts removes most replicated metric associations.

Models

  • text2midi uses an 18-layer decoder conditioned on frozen Flan-T5, while MIDI-LLM uses a 16-layer Llama 3.2 1B decoder; dictionaries and interventions use selected residual-layer states.The analyzed layers are 2, 8, 13, and 16 for text2midi and 2, 7, 12, and 14 for MIDI-LLM.Source: §3 · p. 3

Data

  • Activations come from 1,500 Lakh MIDI files, while characterization uses file-split excerpts divided into discovery, validation, and test sets.The excerpt sets contain 2,526 examples for MIDI-LLM and 2,498 for text2midi.Source: §3 · p. 3

Sparse autoencoders

  • The TopK SAE uses 32 active features, with dictionaries of 6,144 features for text2midi and 8,192 for MIDI-LLM; each final dictionary is a single training run.States are centered and scaled by the median norm over 50,000 training tokens because unusually large norms dominated mean-based pilot scaling.Source: §3 · p. 3

Dictionary quality

  • Replacing residual states with SAE reconstructions recovers 99.22% [98.94, 99.51] of loss for text2midi L13 and 95.57% [93.95, 97.09] for MIDI-LLM L12; count adjustment removes most replicated metric associations.Features firing in 1–25% of extraction windows are retained, leaving 1,746–4,875 features per dictionary; the other six dictionaries have reconstruction diagnostics only.Source: §3 · p. 3, Table 2 · p. 4

4 Observational Evidence

This section compares three controlled ways to characterize SAE features—musical-metric correlations, automated descriptions, and shared instrument programs—as evidence about their musical patterns.

4.1 Correlations with musical metrics

  • At τ = 0.20, density adjustment leaves 862 of 10,018 replicated raw associations in the eight final dictionaries.Activation and metric are regressed on log(1 + notes) and log(1 + tokens); each dictionary retains 3.4–12.5% of its raw count.Source: §4.1 · p. 4

4.2 Automated descriptions

  • Instrumentation descriptions alone exceeded permutation calibration, with H = 2.792 [2.569, 3.014] and a difference of 0.692 [0.364, 1.033].For the other description groups, intervals for the difference included zero; 12 of 48 perfect-recognition cards passed the audit.Source: §4.2 · p. 5, Figure 2 · p. 5

4.3 Shared instrument programs

  • Twenty-seven atlas features met the shared-program criterion: 23 in text2midi and four in MIDI-LLM.Their program occurred in at least 80% of active discovery excerpts and three of four held-out active excerpts, and in none of the matched inactive excerpts.Source: §4.3 · p. 5
  • The synth-lead feature’s four active held-out excerpts all contained program 81, Lead 2 (sawtooth), while none of four distractors did.The feature entered interventions through its description despite not meeting the shared-program selection criterion.Source: §4.3 · pp. 5–6

5 Interventions During Generation

The intervention protocol compares feature edits with baseline and matched-random generation, then evaluates selected SAE features in the MIDI outputs. A prespecified synth-lead test succeeds on new prompts, while family-level shifts differ by how metric features were selected.

5.1 Protocol

  • Interventions compare feature edits with a baseline and an orthogonal matched-random direction, using P50, P75, and P90 targets and a separately controlled ablation.The targeted screen uses ten prompts per feature and prespecified feature-minus-random contrasts at P90; screens rank candidates for confirmation and estimate family-level effects.Source: §5.1 · p. 6

5.2 Confirmation of an instrument feature

  • Program 81 appears in 14/20 synth-lead feature outputs, versus 1/20 random and 2/20 baseline; on disjoint prompts, it appears in 6/10 feature outputs and neither comparison output.At P90, the confirmation has p = 0.000122 and the replication p = 0.0156; program share rises by 6.6 percentage points [2.9, 10.4] in the replication.Source: §5.2 · p. 7, Table 5 · p. 7

5.3 Family-level effects

  • At P90, density-adjusted features shift targets 0.17 baseline standard deviations [0.02, 0.33] versus random, while raw-correlation features show −0.23 [−1.23, 0.91] versus random.The density-adjusted shift relative to baseline is 0.18 [−0.04, 0.37]; the random direction changes the metric by +0.01 [−0.15, 0.19].Source: §5.3 · pp. 7–8, Table 7 · p. 8

5.4 Baseline and random references

  • For 12 of 15 metric features, the largest standardized change is on a metric other than the hypothesized target, and some random-relative gains barely differ from baseline.For example, a pitch-class feature exceeds the random direction by 1.10 classes but the baseline by only 0.10.Source: §5.4 · pp. 8–9

5.5 Blinded LLM ratings of descriptions

  • In blinded LLM ratings, the synth-lead description has the largest difference, +2.40 ± 0.55, and rates higher on all five prompts; without it, the primary subset difference is −0.036 [−0.400, 0.291].The audit-passing subset overall is rated 0.167 points higher than random [−0.300, 0.700] (p = 0.3125).Source: §5.5 · p. 9

6 Discussion

Controlled observations separate SAE features that predict steering from those that do not: density-adjusted selections show a small positive family-level shift, while raw-correlation selections show none. The discussion relates these findings to tokenization, supervised directions, recommended practices, and study limitations.

The role of the controls

  • The density-adjusted feature family shows a small positive shift in its target metric, while the raw-correlation family shows none.Because the screens also differ in prompts, dictionaries, and metrics, this contrast does not isolate the adjustment’s effect; the instrumentation-description group that exceeds permutation calibration yields a feature that controls its instrument in confirmation and replication.Source: §6 · p. 9

Instrument programs and explicit tokens

  • A program-81 direction can alter a small set of explicit instrument tokens, unlike polyphony, pitch range, and pitch entropy, which aggregate many token decisions.The authors propose that decoder directions projecting strongly onto a few output tokens may be more steerable; exploratory program features introduced absent programs more often than random directions.Source: §6 · p. 9

Unsupervised and supervised directions

  • The SAE study found a controllable feature for an attribute not chosen in advance, while its description route selected a feature missed by the program-frequency criterion.That feature’s program occurs in its strongest activations but not its typical ones, so combining routes widens the candidate set; the earlier supervised study used both intervention orientations to separate directional responses from symmetric drift.Source: §6 · p. 10

Recommendations

  • The protocol recommends count-adjusted correlations, permutation-controlled descriptions, baseline and random-direction references, small-prompt screens, new-prompt confirmation, and checks across independently trained dictionaries.Individual features should be confirmed on new prompts with a fixed endpoint.Source: §6 · p. 10

Limitations

  • The study uses single-run final dictionaries and small screens, while its family-level estimates were specified after screening and depend on the resampling scheme.Other boundaries include no listening study, possible LLM reviewer contribution to instrumentation recognition, no file-level overlap audit, and a MIDI program-number endpoint that does not establish perceived timbre.Source: §6 · p. 10

7 Conclusion

The study concludes that controlled observational evidence can identify SAE features that steer text-to-MIDI outputs. The synth-lead feature was confirmed and replicated, while density-adjusted metric selections showed a small family-level shift and raw-correlation selections showed none.

Conclusion

  • A description-identified feature introduced its instrument program in prespecified confirmation and replication, reappeared in independently trained dictionaries, and density-adjusted selections showed a small family-level shift while raw-correlation selections showed none.The atlas provides piano rolls, MIDI files, and synthesized audio for all 80 targeted-screen features.Source: §7 · p. 10

A Dictionary Selection and Fidelity

Pilot comparisons informed dictionary choices that balanced reconstruction, sparsity, and cost. The appendix also reports functional fidelity for the two evaluated final dictionaries.

Dictionary selection

  • MIDI-LLM L12 at EF 32 and k = 32 left 83.72% of the dictionary dead; k = 128 improved reconstruction and reduced dead features but increased active features per token.EF 4 and k = 32 were retained as a compromise between reconstruction, sparsity, and cost.Source: A Dictionary Selection and Fidelity · p. 13, Table 8 · p. 13
  • In text2midi, EF 8 retained more features after frequency filtering and had slightly lower FVU than EF 4 at both pilot layers.The pilot runs and earlier final-configuration runs form the 13 earlier checkpoints used in the density reanalysis and density-adjusted screen.Source: A Dictionary Selection and Fidelity · p. 13

Dictionary fidelity

  • Table 9 reports functional fidelity for the two evaluated final dictionaries, including next-token cross-entropy increase and recovery with 95% file-bootstrap intervals.Recovery is computed from file-balanced means on 200 excerpts from 146 files.Source: Table 9 · p. 13, A Dictionary Selection and Fidelity · p. 13

B Metric Characterization

Metric characterization defines twelve MusPy measurements and their missing-data handling, then reports replication counts, associations across metrics, and the strongest feature per metric and model.

Metric definitions

  • The twelve MusPy measurements leave pitch range missing for drum-only excerpts and drum metrics missing for excerpts without drums; correlations use pairwise complete data.Scale consistency measures pitch-class membership in a diatonic scale, not tonal center.Source: B Metric Characterization · p. 13, Table 10 · p. 13

Replication across splits

  • Table 11 reports metric-association replication counts across characterization splits, including discovery-validation agreement and how many also agree on test.Eligible features fire on 1–50% of discovery excerpts and pass the window-frequency filter.Source: Table 11 · p. 14

Associations per metric

  • Table 12 sums associations per metric across four final dictionaries of each model, allowing a feature to count under several metrics.It reports 7,876 eligible text2midi features and 8,423 MIDI-LLM features, with separate eligible counts for drum metrics.Source: Table 12 · p. 14

Metric feature pool

  • Table 13 identifies the strongest feature per metric and model to define a pool of 24 hypotheses across 15 features.It reports the smaller absolute discovery or validation correlation when signs match, followed by test correlation after selection.Source: Table 13 · p. 14

C Density Reanalysis of the Earlier Checkpoints

The appendix reanalyzes earlier checkpoints after adjusting activations and metrics for note density, and reports how description-recognition scores vary across groups against a permutation calibration.

C Density Reanalysis of the Earlier Checkpoints

  • The reanalysis evaluates dictionary features active in 1–50% of discovery segments, residualizing activations and metrics on note count and token count.Coefficients are fitted on discovery segments; Table 14 reports 13 earlier checkpoints.Source: C Density Reanalysis of the Earlier… · p. 15

D Description Recognition by Group

  • Recognition intervals resample cards, but treat them as independent even though cards share an LLM reviewer and sometimes source excerpts.The expected perfect-card count under random choice is the number of cards divided by 70.Source: D Description Recognition by Group · p. 15
  • Description recognition is summarized by mean correct selections out of four, with random choice at 2 and perfect-card counts compared with random expectations.The group table also defines differences relative to the 30-card permutation calibration.Source: Table 15 · p. 15

E Complete Intervention Screens

The screens report no primary per-feature rejection in three exploratory tests and document family-level sensitivity, screen coverage, and intervention diagnostics.

E Complete Intervention Screens

  • None of the three exploratory screens yielded a primary per-feature rejection at q < 0.05.The exact-test table reports the smallest possible p as 2^-n, with undefined pairs or ties potentially raising that bound.Source: E Complete Intervention Screens · p. 16, Table 16 · p. 16
  • At P90, the density-adjusted family’s feature-minus-random interval includes zero under both alternative schemes; the raw-correlation interval is not used for main inference.That raw-correlation interval retains 4,849 of 20,000 draws after excluding 75.8% of draws.Source: E Complete Intervention Screens · p. 16, Table 17 · p. 16
  • The density-adjusted screen covers 42 features, with comparisons using six to eight prompts because some metric values are undefined.The first MIDI-LLM baseline prompt produces drums only.Source: E Complete Intervention Screens · p. 16, Table 21 · p. 19
  • The targeted screen tests 24 metric hypotheses and 27 program hypotheses, while Figure 5 displays each metric feature’s effects across all twelve metrics.Table 20 compares text2midi metric effects between two references, and Table 22 reports intervention diagnostics.Source: Table 18 · p. 17, Table 19 · p. 17, Figure 5 · p. 18
  • Figure 4 pairs feature and random outputs by prompt at P90; the synth-lead effect is a replication, while the other displayed effects are exploratory.The paired comparisons include target-program note share and the hypothesized metric.Source: Figure 4 · p. 18

F Feature-Level Blinded LLM Ratings

The section reports pooled blinded LLM ratings and displays feature-level differences for all description candidates, including the six largest observed differences.

F Feature-Level Blinded LLM Ratings

  • Pooled blinded LLM ratings show no advantage for the feature direction over the random direction.Ratings are averaged over five prompts per feature and then over features; the primary subset excludes the synth-lead feature.Source: Table 23 · p. 20
  • Table 24 lists the six largest feature-level rating differences, with favored-prompt counts and token-concentration, token-syntax, and note-count flags.Differences are feature-minus-random means over five prompts.Source: Table 24 · p. 20, F Feature-Level Blinded LLM Ratings · p. 20
  • Figure 6 plots mean rating differences for all 48 description candidates across five paired prompts, with error bars showing standard deviation.Color identifies audit status, which defines the primary subset.Source: Figure 6 · p. 20
Loading 2610.32007v1…