TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models

• A generated piece reflects multiple training samples with varying influence, making sample-level contribution difficult to quantify for creator credit and compensation.Source: §1 · p. 1
• Existing attribution methods include parameter-based white-box estimates and black-box similarity retrieval using external music encoders.White-box methods use influence functions or machine unlearning; black-box methods compare outputs with training samples.Source: §1 · p. 1
• TrueMuse provides known fine-tuning source sets to address the lack of reliable ground truth for evaluating text-to-music attribution.The benchmark fine-tunes three diffusion-based models across four musical attribution sets, covering 133 attributes, 648 models, and 95,456 generated samples.Source: §1 · p. 2
• The benchmark evaluates black-box attribution across fine-tuning improvement, prompt type, joint multi-task training, and fine-tuning data size.Source: §1 · p. 2
• Attribution remains challenging, and existing methods vary substantially across evaluation settings, motivating more reliable and generalizable approaches.Source: §1 · p. 2

2.1 Music Generative Models

• Direct audio synthesis now includes diffusion, autoregressive transformer, and flow-matching approaches, following acoustically limited symbolic MIDI systems.Examples include AudioLDM2 [18], Stable Audio Open [11], and Mustango [22] for diffusion; MusicGen [7] and Lyria Realtime for autoregressive models; and JASCO [30] for flow matching.Source: §2.1 · p. 2

2.2 Data Attribution

• Attribution methods developed for image classification include influence functions [14], scalable gradient-estimation approaches, and later text-to-image diffusion methods.The text-to-image benchmark introduced by Wang et al. [32] was later adopted by Concept-TRAK [24] and A&D [20].Source: §2.2 · pp. 2–3
• TrueMuse uses selected source clips to fine-tune attribute-specific models, then generates contextual and style-prompt samples with known fine-tuning sources.Source: Figure 1 · p. 3, §2.2 · p. 3
• Text-to-music attribution includes white-box influence-function and unlearning methods [9] [6] and black-box similarity retrieval [3] [2], but lacks reliable ground truth.Earlier black-box work either retrieves clips without controlling generation or evaluates similarity on synthetic audio.Source: §2.2 · p. 3

3.1 Dataset Curation

• Known fine-tuning source sets provide controlled targets for evaluating attribution across four musical characteristics and three text-to-music models.The benchmark instantiates each attribute independently; the dataset statistics span the four sets and three models.Source: §3.1 · pp. 3–4, Table 1 · p. 4

3.1.1 Melody-Centric

• Melody-Centric tests whether a method identifies a melody despite changes in instrumentation or style, using one FMA clip per melody.The set includes 50 melodies; each is fine-tuned with the prompt “a recording of [V] melody.”Source: §3.1 · p. 4, §3.1.1 · p. 5

3.1.2 Instrument-Centric

• Instrument-Centric tests timbral attribution across melodies, accompaniments, and styles, using 3- and 6-clip configurations for 25 instruments.The clips were collected from YouTube for fine-tuning and are not redistributed.Source: §3.1 · p. 4, §3.1.2 · p. 5

3.1.3 Musician-Centric

• Musician-Centric tests recognition of an artist’s recurring style across instrumentations and genres using 3 or 6 clips from different tracks per artist.The 50 musicians were sampled from FMA; the section identifies artist attribution as a common context for copyright and royalty disputes.Source: §3.1 · p. 4, §3.1.3 · p. 5

3.1.4 Genre-Centric

• Genre-Centric tests whether attribution identifies patterns shared across artists, using clips from six musicians for each of eight genres.The set uses 3- and 6-clip configurations and varies instruments in style prompts while fixing the target genre.Source: §3.1 · p. 4, §3.1.4 · p. 5

3.2 Evaluating for Attribution

• Evaluation treats attribution as retrieval: generated audio queries a gallery of source clips, ranked by frozen-encoder cosine similarity or mapper-projected embeddings.The mapper approach is trained contrastively on generated samples and their ground-truth clips. The benchmark focuses on black-box methods because white-box methods lack released code and are often tied to particular models and training data.Source: §3.2 · pp. 5–6

4.1 General Attribution Performance

• MERT-v0-public led encoder performance at R@10 of 0.397, followed by MERT-v0 at 0.357, Music2Vec-v1 at 0.226, and DAC-16k at 0.129.The authors associate the difference with encoders’ training objectives and report a 4.0 percentage-point gap between MERT-v0 and MERT-v0-public.Source: §4.1 · pp. 6–7
• Stable Audio Open was easiest to attribute at R@10 of 0.487, versus 0.205 for Mustango and 0.140 for AudioLDM2.The authors suggest Stable Audio Open outputs may more closely resemble its fine-tuning examples.Source: §4.1 · p. 7, Figure 2 · p. 7
• Instrument-Centric was easiest to attribute at R@10 of 0.465, followed by Musician-Centric at 0.305, Melody-Centric at 0.239, and Genre-Centric at 0.101.The authors relate this ordering to the increasing abstraction of the target attributes, from instrument timbre to genre.Source: §4.1 · p. 7, Figure 2 · p. 7
• MERT-v0 rose from 0.357 to 0.400 and MERT-v0-public from 0.397 to 0.441 with mappers; gains were minimal on Stable Audio Open.The results indicate that mappers help when encoders capture attribution-relevant information but generated samples and ground-truth clips remain misaligned.Source: §4.1 · pp. 7–8, Figure 2 · p. 7

4.2 Attribution Performance Under Different Prompt Types

• Style-trained mappers outperformed contextual-trained mappers on average; MERT-v0-public reached 0.478 versus 0.400, while combining prompt types provided no consistent benefit.Gains were concentrated on Melody-Centric and Musician-Centric attribution.Source: §4.2 · p. 8, Figure 3 · p. 8
• Melody-Centric R@10 rose from 0.178 with contextual prompts to 0.365 with style prompts, while Instrument-Centric and Genre-Centric remained largely flat.The authors suggest contextual prompts can leave the target melody underconstrained.Source: §4.2 · p. 8

4.3 Single-Task versus Joint-Task Attribution

• Joint training changed overall average R@10 by only 0.003; Musician-Centric rose from 0.352 to 0.364, while Genre-Centric fell from 0.101 to 0.098.The four attribution sets were described as targeting largely distinct acoustic characteristics.Source: §4.3 · p. 8, Figure 4 · p. 9

4.4 Attribution Performance with Varying Clip Counts

• Increasing fine-tuning clips from 3 to 6 reduced average R@10 from 0.309 to 0.210 across the reported conditions.The largest set-level drop was for Musician-Centric, from 0.352 to 0.214; the authors suggest more clips broaden acoustic variation and make retrieval harder.Source: §4.4 · p. 9, Figure 5 · p. 9

5 Discussion and Limitations

• Source retrieval remains challenging and varies substantially across musical attributes and generators.The benchmark is intended to support more rigorous study of attribution in text-to-music systems.Source: §5 · p. 9
• TrueMuse initially covers a representative set of musical attributes, text-to-music generators, and prompt conditions.The authors identify broader coverage of models, musical concepts, and generation scenarios as possible future extensions.Source: §5 · p. 9
• The authors hope the benchmark supports development of attribution methods that are more robust, interpretable, and suitable for real-world credit assignment.This is presented as an intended contribution, not a demonstrated outcome.Source: §5 · p. 9

A.1 Prompt Construction with Claude

• Claude receives official demonstration prompts and separate instructions to generate contextual and style prompts for each model and attribution set.The contextual prompts vary scene and accompaniment while keeping the attribute fixed; style prompts vary genre while keeping it fixed.Source: §A.1 · p. 12

A.1.1 Official Example Prompts

• Official demonstration prompts range from short descriptions, such as Stable Audio Open’s “128 BPM tech house drum loop,” to more detailed musical descriptions.The Mustango examples specify details including chords, tempo, time signature, or key.Source: §A.1 · p. 12, §A.1.1 · p. 12

A.1 Prompt Construction with Claude

• The generated contextual and style prompts are used as-is during sample generation.The benchmark includes the full generated prompt set for remaining model and attribution-set combinations.Source: §A.1 · p. 12

A.1.2 Generated Contextual Prompts (Musician-Centric, AudioLDM2)

• Contextual prompt examples place a target musician in varied scenes, including a campfire folk tune and a late-night groove with Rhodes keys.Each example retains the [V] musician placeholder while changing the surrounding scene or accompaniment.Source: §A.1 · p. 12, §A.1.2 · p. 12
• Other contextual examples span orchestral, boom-bap, synth-pad, and horn-and-percussion settings around the target musician.The examples illustrate variation in accompaniment and musical scene while the target attribute remains fixed.Source: §A.1 · p. 12, §A.1.2 · p. 12

A.1.3 Generated Style Prompts (Musician-Centric, AudioLDM2)

• Style prompts follow “a [genre] song by [V] musician,” with 28 genre tags spanning pop, classical, hip hop, house, and ambient.The genre tags vary while the musician placeholder stays fixed.Source: §A.1 · p. 12, §A.1.3 · p. 13

A.2 Generator Details

• All three TRUEMUSE generators are text-conditioned latent diffusion models, but differ in architecture and conditioning scope.Table 2 summarizes their specifications.Source: §A.2 · p. 13, Table 2 · p. 13
• AudioLDM2 uses a U-Net conditioned on joint CLAP and GPT-2 audio-language embeddings, while Mustango adds beat and chord predictions as music-domain conditioning.Mustango’s added branch is intended to let prompts specify music-theoretic constraints.Source: §A.2 · p. 13
• Stable Audio Open uses a U-Net conditioned on T5 text embeddings and is the largest of the three models at 1.1B parameters.Together, the three generators span 712M–1.4B parameters, conditioning strategies, and training corpora.Source: §A.2 · p. 13
• Each fine-tuned model generates four waveforms per prompt, producing four samples per attribute-prompt pair.All outputs are 10-second clips; Stable Audio Open’s 44.1 kHz output is resampled to 16 kHz for consistency with the other generators.Source: §A.2 · p. 13

A.3 Attribution Performance on 6-Clip Setting

• Performance consistently degrades relative to the 3-clip setting across encoders, generators, and Instrument-, Musician-, and Genre-Centric attribution.Melody-Centric remains in the 1-clip setting rather than the 6-clip setting.Source: §A.3 · p. 13
• MERT-based encoders continue to outperform Music2Vec-v1 and DAC-16k, while Stable Audio Open remains the easiest generator to attribute.This relative ordering is unchanged in the 6-clip experiments.Source: §A.3 · p. 13
• For MERT-based encoders, learned mappers still provide the largest gains on Mustango, and style prompts outperform contextual prompts in most conditions.Both patterns persist in the 6-clip setting.Source: §A.3 · p. 13

A.4 Music Encoder Details

• MERT-v0 and MERT-v0-pub share a 12-layer, D=768 RoBERTa-style backbone pretrained via masked acoustic modelling; only MERT-v0 additionally uses private-corpus data.MERT-v0-pub is trained exclusively on publicly licensed audio, and both produce identical-dimensional embeddings.Source: §A.4 · pp. 13–14
• Music2Vec-v1 uses data2vec-based self-distillation, whereas DAC-16k is a residual vector quantization codec with 12 codebooks of size 1024.Source: §A.4 · pp. 13–14
• Figure 6 compares standard and mapper-enhanced attribution R@10 across three generators, using six clips per attribute.Source: Figure 6 · p. 14
• All four encoders operate on 16 kHz mono audio and remain frozen; Table 3 summarizes their raw output formats, aggregation procedures, and final embedding dimensions.Source: §A.4 · p. 14, Table 3 · p. 14

A.5 Attribution Performance Under Different Metrics

• Across R@1, R@5, R@100, and MAP, MERT-based encoders outperform Music2Vec-v1 and DAC-16k, Stable Audio Open scores highest, and the relative ordering remains stable.Source: §A.5 · p. 14
• R@100 yields uniformly higher scores than R@10, while R@1 and MAP are more stringent and reveal larger performance gaps between encoders.Source: §A.5 · p. 14
• Learned mappers provide the most benefit for MERT-based encoders on Mustango; the figures include R@1 at three clips and R@5 at six clips per attribute.Source: §A.5 · p. 14, Figure 9 · p. 15

A.6.1 Finetuning Details for Text-to-Music Models

• Generator fine-tuning binds each attribute to reference clips using a rare identifier; only the denoising U-Net is fine-tuned, with steps and batch sizes varying by clip count.Melody attributes use one clip for 300 steps at batch size 1; three clips use 1,000 steps at batch size 3, and six clips use 1,500 steps at batch size 4.Source: §A.6.1 · p. 15

A.6.2 Training Details for Feature Mappers

• Mapper training starts from the identity and optimizes symmetric NT-Xent loss with delta-weight regularization; Figures 10–11 report R@1 at six clips and R@5 at three clips.Source: §A.6.2 · p. 15, Figure 10 · p. 16, Figure 11 · p. 16

A.6.3 Computational Resource Usage

• 637 GPU-hours covered all experiment stages; generator fine-tuning and sample generation consumed most of the computation, while embedding extraction and mapper training added relatively little.Source: §A.6.3 · p. 16

A.7 Attribution Performance Under Different Prompt Types per Generator

• Style prompts outperform contextual prompts across all three generators, with a smaller gap on Stable Audio Open and a more pronounced gap on Mustango and AudioLDM2, especially for Melody-Centric.For Melody-Centric, contextual prompts consistently yield near-zero R@10 on Mustango and AudioLDM2, while style prompts produce substantially higher scores.Source: §A.7 · p. 16, Figure 17 · p. 18, Figure 18 · p. 19