PaperlayerView exact revision on arXiv

Source-linked AI summary

MIRA: A Musical Intent Refinement Agent for Aligning Text-to-Music Generation with User Intent

Zekai Liu, Zhilin Wang, Xuzheng He, Yu Cheng, Yang Yang

arXiv:2610.10355v1cs.SDcs.AI

TL;DR

Text-to-music evaluation can miss whether generated tracks meet explicit requirements and intent implied by references. The paper introduces expert-revised, request-specific rubrics and MIRA, which uses verifier-guided prompt search; experiments report consistent alignment improvements, including an open-source system comparable to representative commercial systems.

  • Problem

    Broad text–audio relevance and holistic quality scores do not identify which explicit or reference-implied requirements a generated track satisfies.

    Source: §1 · p. 1

  • Method

    MuRA-Bench uses expert-revised rubrics for explicit and inferred intent, while MIRA uses requirement-level audio verification and bounded prompt search for black-box generators.

    Source: §1 · pp. 1–2

  • Results

    At B = 9, MIRA increases Overall by 12.9% on ACE-Step relative to direct generation, with consistent alignment improvements across three backends and performance comparable to representative commercial systems.

    Source: §5 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    Request-specific evaluation exposes differences that global relevance scores obscure, and MIRA improves intent alignment across the evaluated open-source backends.

    Source: §1 · p. 2, §6 · p. 9

Abstract

from arXiv · show

Text-to-music systems produce increasingly convincing audio, yet evaluation reveals little about whether the result matches user intent. A global text-audio relevance score can overlook the implicit intent in underspecified prompts and mask failures in specific requirements, such as instrumentation, structure, rhythm, or mood progression. To bridge this gap, we formulate text-to-music intent alignment as satisfying a per-request rubric of independently verifiable items covering both a request's explicit requirements and its implied musical intent. Scoring items individually makes evaluation diagnostic by intent source and musical dimension, rather than a single opaque score. We instantiate this as MuRA-Bench, a benchmark of real-world platform requests curated by music experts. We further propose MIRA (Musical Intent Refinement Agent), a test-time agent that first grounds a request's intent into rubrics, then searches over prompt revisions for a black-box generator under a bounded budget, iteratively generating music, verifying it against the rubrics, and using this feedback to guide a trajectory-aware tree search. Experiments across open-source and commercial backends show that MIRA improves intent alignment, enabling an open-source generator to achieve performance comparable to representative commercial systems (e.g. Suno and Mureka). Project page: https://mirareview.github.io/.

1 INTRODUCTION

The introduction frames text-to-music intent alignment as more than audio plausibility or broad text–audio relevance: systems should satisfy explicit constraints and contextually implied intent. It presents MuRA-Bench for request-specific assessment and MIRA for verifier-guided refinement of black-box generation.

1 INTRODUCTION

  • MuRA-Bench and MIRA address request-specific intent alignment, which broad relevance scores can leave opaque when explicit requirements or reference-implied traits are missed.The benchmark scores independently auditable rubric items; MIRA uses requirement-level feedback to guide prompt revisions.Source: §1 · pp. 1–2
  • MuRA-Bench represents each request with expert-revised items covering stated constraints and intent inferred from references and context.Its 100 controlled requests come from anonymized Mureka platform intent seeds.Source: §1 · pp. 1–2
  • MIRA grounds request intent in rubrics, verifies generated audio against requirements, and searches prompt revisions under a bounded generation budget.Its trajectory-aware feedback preserves satisfied requirements while targeting unresolved ones, without modifying the backend generator.Source: §1 · p. 2
  • Experiments report consistent intent-alignment improvements and show that an open-source generator can match some representative commercial systems under MuRA-Bench evaluation.The evaluation compares stated and completed musical intent and validates item-level scores against expert judgments.Source: §1 · p. 2

2 RELATED WORK

Related work spans music-generation quality, text–audio alignment, audio-language evaluation, and inference-time refinement. The paper positions MuRA-Bench and MIRA as combining request-grounded musical criteria with audio-based feedback for budgeted prompt search.

2 RELATED WORK

  • Existing quality and relevance measures characterize system performance, but diagnosing intent alignment also requires tracing scores to individual musical requirements.FAD measures distribution-level fidelity, CLAP broad semantic correspondence, and expert ratings or preferences capture complementary perceptual aspects.Source: §2 · p. 2
  • Audio-language evaluators assess musical properties or decompose instructions into targeted questions and binary rubric items, while MuRA-Bench grounds criteria in real requests.Its expert-revised criteria distinguish explicit requirements from intent inferred from references and context, with MusicFlamingo question-answering scores.Source: §2 · pp. 2–3, Figure 1 · p. 3
  • Inference-time controls and optimization offer ways to refine generated music, but open-ended requests raise the question of how audio feedback should guide revisions.Prior work includes controllable generation and diffusion-noise optimization without retraining the generator.Source: §2 · p. 3
  • MIRA combines tool-grounded intent completion, requirement-level audio feedback, and branch-specific memory in budgeted prompt search across backends without parameter updates.This draws on agent mechanisms for tool use, iterative feedback, memory, and trajectory search while grounding observations in generated audio.Source: §2 · p. 3

3 MURA-BENCH: RUBRIC-LEVEL MUSIC INTENT ALIGNMENT

MuRA-Bench evaluates text-to-music intent through request-specific rubrics that separate prompt-supported requirements from traits inferred from references and context. Its construction uses expert revision, and its evaluator scores audible rubric-item satisfaction using music-specialized audio question answering.

3 MURA-BENCH: RUBRIC-LEVEL MUSIC INTENT ALIGNMENT

  • A request-specific rubric contains prompt-supported requirements and contextually completed musical traits, each independently judgeable from generated audio.This allows multiple valid musical realizations without requiring a reference track.Source: §3 · p. 3

3.1 BENCHMARK CONSTRUCTION

  • MuRA-Bench contains 100 controlled requests from anonymized Mureka intent seeds, including partially specified, reference-based, and hybrid prompts.Seeds lacking stable musical criteria or requiring unavailable context or sensitive information are excluded; raw user logs are not released.Source: §3.1 · p. 4
  • Three music experts revise model-drafted criteria for musical plausibility, audible atomicity, and support from the request or reference context.Named works become generalized musical traits, and seven dimensions organize criteria without imposing the same checklist on every request.Source: §3.1 · p. 4

3.2 RUBRIC-BASED EVALUATION

  • The evaluator treats each gold rubric item as a yes/no audio question and uses MusicFlamingo for the final evaluation.The paper compares this design with direct structured judgment and caption-plus-critic alternatives through expert-agreement experiments.Source: §3.2 · p. 4
  • Soft item probabilities retain evaluator confidence, avoiding an arbitrary threshold when rubric judgments are aggregated.The probabilities are derived from logits assigned to the candidate yes and no answers.Source: §3.2 · p. 4
  • Rubric-subset scores average soft item scores within each audio and then weight audios equally.Only request–audio pairs with at least one item in the selected subset are included.Source: §3.2 · p. 4
  • Overall scores all gold-rubric items, while Base GT and Completion GT separately score prompt-stated requirements and context-completed traits.Dim-Macro equally averages scores across seven dimensions to limit domination by dimensions with more items; Overall instead averages all items within each audio.Source: §3.2 · p. 4
  • Expert item-level ratings validate grader agreement, while gold rubrics and human grading labels remain unavailable to the generation and refinement system.Experts rate satisfaction on a 20-point scale, with uncertain responses excluded from agreement calculations.Source: §3.2 · p. 4

4 MIRA: MUSICAL INTENT REFINEMENT AGENT

MIRA is a verifier-guided agent for refining prompts with black-box text-to-music generators. It grounds implicit intent in rubrics and uses tree search with branch-specific memory to guide iterative improvement.

4 MIRA: MUSICAL INTENT REFINEMENT AGENT

  • MIRA externalizes latent intent as grounded, verifiable rubrics, balances exploration and exploitation through tree search, and preserves optimization history in branch-specific memory.Source: §4 · p. 5

4.1 TOOL-GROUNDED ONLINE RUBRIC

  • Tool outputs are admitted only when traceable, supported, and sufficiently confident; the resulting online rubric contains independently assessable requirements derived without access to the expert gold rubric.Source: §4.1 · p. 5

4.2 RUBRIC-GUIDED SEARCH

  • Because stochastic generations allow multiple repairs and revisions can trade off rubric items, MIRA evaluates independent prompts per layer and retains high-aggregate and stronger lower-tail candidates.Source: §4.2 · pp. 5–6
  • Search evaluates at most B nodes, including the initial generation; the tested budgets are B = 3 and B = 9.Source: §4.2 · p. 6

4.3 TREE-STRUCTURED FEEDBACK MEMORY

  • Branch-specific memory combines parent-child changes with same-layer contrasts, treats changes within 0.03 as stable, and carries item-level improvements and regressions forward.Memory remains within its branch, and different branches do not access one another’s complete prompts or private trajectories.Source: §4.3 · p. 6

4.4 CANDIDATE SELECTION

  • Final selection prioritizes aggregate satisfaction among successfully visited candidates, breaking ties by item pass rate, lower-tail satisfaction, minimum item score, then generation order.Source: §4.4 · p. 7

5 EXPERIMENTS

Experiments show consistent intent-alignment gains from MIRA across three open-source backends, with evaluation and component analyses supporting the approach. Results also compare refined generators with commercial systems and blinded expert ratings.

Experimental Setup

  • The main results cover all 100 MuRA-Bench requests, while equal-budget comparisons and component analyses use a fixed subset of 20.The three refinement backends are ACE-Step v1.5 Turbo, SongGeneration2 Large, and YuE2-3B; MIRA is evaluated at B = 3 and B = 9.Source: §5 · p. 7

Agreement with Expert Judgments

  • MF-AQA achieves the highest agreement with expert judgments across all six evaluation metrics.The validation uses 1,536 rubric-item ratings from 100 clips covering 25 requests.Source: §5 · p. 7

Main Results

  • At B = 9, MIRA raises Overall by 12.9% on ACE-Step, 19.0% on SongGeneration2, and 7.3% on YuE2-3B versus direct generation.Overall and D-Macro improve at both budgets; increasing B from 3 to 9 further improves explicit and inferred intent across all three backends.Source: §5 · p. 7, Table 2 · p. 8

Commercial Comparisons

  • YuE2-3B with MIRA records the highest Overall score among evaluated systems: 82.1 versus 81.9 for Suno v5.5.ACE-Step with MIRA reaches 76.3, compared with 75.9 for MiniMax music-2.6.Source: §5 · p. 8

Human Evaluation

  • Blinded experts rate ACE-Step with MIRA above direct ACE-Step and MiniMax music-3.0, but below Suno, on overall, explicit, and inferred intent.Figure 3 reports mean ratings on a 1–5 scale with 95% confidence intervals.Source: §5 · p. 8, Figure 3 · p. 8

Component Analysis

  • Tool grounding, adaptive search, and structured memory each contribute to component-analysis gains on a fixed subset of 20 requests.Tool grounding raises Comp. from 0.719 to 0.748 on ACE-Step and from 0.775 to 0.800 on YuE2-3B; adaptive search raises ACE-Step Comp. from 0.748 to 0.781, and memory further improves Overall and D-Macro on both backends.Source: Table 3 · p. 9, §5 · p. 9

6 CONCLUSION

The conclusion presents MuRA-Bench and MIRA as a unified framework for evaluating and improving text-to-music intent alignment. Across three open-source backends, MIRA consistently improves alignment, and an open-source generator reaches performance comparable to representative commercial systems.

6 CONCLUSION

  • The framework combines expert-revised, request-specific rubrics with tool-grounded intent completion, verifier-guided search, and tree-structured memory for black-box generation.The conclusion reports consistent alignment improvements across three open-source backends and gains over repeated sampling at equal generation budgets.Source: §6 · p. 9

AI USE STATEMENT

The authors disclose using AI tools for language refinement, coding assistance, and candidate rubric generation, followed by expert revision and author review.

AI USE STATEMENT

  • LLMs generated candidate benchmark rubrics, which music experts revised; the authors reviewed AI-assisted materials and take responsibility for the final content.ChatGPT was also used for grammar checking and text refinement, and AI coding assistants for portions of the code.Source: AI USE STATEMENT · p. 15

A.1 BENCHMARK COMPOSITION

MuRA-Bench turns anonymized Mureka requests into expert-revised rubrics whose independently assessable items cover explicit requirements and context-inferred musical intent. The appendix details expert review, human assessment, and how automatic scores are matched to expert targets.

A.1 BENCHMARK COMPOSITION

  • MuRA-Bench contains 100 controlled requests—68 instrumental and 32 vocal—with 1,566 rubric items derived from anonymized Mureka intent seeds.Requests needing unavailable context or lacking stable evaluation criteria are excluded; each request has 6–31 items (mean 15.66; median 14.5).Source: §A.1 · p. 15

A.2 EXPERT REVISION PROCEDURE

  • Three music experts independently revised disjoint benchmark subsets using shared guidelines and recorded edits, decisions, and comments.The checklist checks explicit requirements, interprets references or contexts only when needed, retains context-supported and listenable completions, assigns dimensions, and records usability.Source: §A.2 · p. 15

A.3 A COMPLETE RUBRIC EXAMPLE

  • The RPG battle-music example pairs six base items for requested elements and atmosphere with six completion items for gameplay-context traits.The example request specifies upbeat combat music for a 2D plains-biome RPG, with flutes, wind textures, and African drums.Source: §A.3 · p. 16, Table 5 · p. 16

B HUMAN ASSESSMENT AND EVALUATOR AGREEMENT

  • Three experts independently assessed assigned samples for requirement fulfillment rather than personal preference, after common instructions and full-clip listening.Item-level ratings use a 20-point scale, while system-level ratings use a separate five-point scale.Source: B HUMAN ASSESSMENT AND EVALUATOR… · p. 16, Table 6 · p. 16

B.1 RUBRIC-LEVEL ANNOTATION

  • The rubric-level agreement study uses 1,536 clip–item ratings from 100 clips and 25 requests, with one expert rating per target.Uncertain responses are excluded from numerical analysis.Source: §B.1 · p. 17

B.2 MATCHING AUTOMATIC SCORES TO HUMAN TARGETS

  • Automatic and human scores are matched on the same clip–item targets; pairwise accuracy uses 147 non-tied comparisons that share clips, not 147 independent requests.Clip-level correlations compare averages of matched rubric items across clips.Source: §B.2 · p. 17

B.3 SYSTEM-LEVEL LISTENING INTERFACE

The system-level listening study presents four candidates for each MuRA-Bench request and collects separate expert ratings of explicit, inferred, and overall intent. Results average ratings across requests, with confidence intervals describing request-level variation.

B.3 SYSTEM-LEVEL LISTENING INTERFACE

  • The listening study covers all 100 requests and 400 clips, presenting four candidates per request with anonymous labels and stably shuffled order.Candidates include direct ACE-Step, ACE-Step with MIRA (B = 9), MiniMax music-3.0, and Suno v5.5; system names and automatic scores are hidden.Source: §B.3 · p. 17
  • Experts rate explicit, inferred, and overall intent separately on a five-point scale, focusing on request fulfillment rather than music quality.Overall intent is an independent holistic judgment of whether the music achieves the intended outcome.Source: §B.3 · p. 17
  • Each system’s dimension score is the mean of 100 request ratings, with two-sided 95% Student-t intervals summarizing variation across requests, not inter-rater reliability.Each request receives equal weight.Source: §B.3 · p. 17

C.1 EVALUATOR BASELINE IMPLEMENTATIONS

The section specifies evaluator baselines, rubric use, search configuration, and comparison scope. Across three generation backends and two alternative evaluators, both MIRA budgets improve intent-alignment measures over direct generation.

C.1 EVALUATOR BASELINE IMPLEMENTATIONS

  • The evaluator baselines include raw CLAP cosine similarity, direct MusicFlamingo ratings, caption-mediated judging, and requirement-specific MusicFlamingo yes/no scoring.Caption-mediated judging reuses one clip caption across requirements, while MF-AQA evaluates each requirement directly against the audio.Source: §C.1 · p. 18

C.2 GOLD RUBRICS AND ONLINE RUBRICS

  • Expert-revised gold rubrics evaluate selected audio offline, while MIRA constructs a separate online rubric before search; gold items and human labels do not enter refinement.The online observations guide candidate refinement and selection.Source: §C.2 · p. 18

C.3 SOFT SCORES AND REPAIR THRESHOLDS

  • Online selection averages soft yes-probabilities over the fixed rubric; η = 0.60 flags items for possible repair, while ε = 0.03 classifies score changes.The repair threshold does not binarize probabilities used to rank candidates.Source: §C.3 · p. 18

C.4 SCOPE OF THE HUMAN COMPARISONS

  • Human comparisons assess intent fulfillment through requirement-level scoring, within-request ranking, and track-level intent ratings, not general quality or listener preference.The study compares requirement-level scores and candidate rankings with expert judgments.Source: §C.4 · p. 18

D.1 CONFIGURATION AND BUDGET ACCOUNTING

  • Experiments use Kimi 2.6 and MusicFlamingo with three open-source backends, direct prompting and MIRA at B = 3 or B = 9, plus ten commercial API baselines.The generation budget counts generated candidates, including the initial one, but not planner or retrieval calls, verifier questions, or elapsed time.Source: §D.1 · p. 19, Table 7 · p. 19

D.2 SEARCH CONFIGURATION AND CANDIDATE SELECTION

  • Parent selection retains the highest-mean candidate and a distinct candidate with the strongest lower-tail satisfaction; final ranking prioritizes mean satisfaction, then pass rate and other criteria.ACE-Step and YuE2 keep a fixed seed within each request’s adaptive search; SongGeneration2 uses its native stochastic interface.Source: §D.2 · p. 19

D.3 EQUAL-BUDGET AND COMPONENT CONFIGURATIONS

  • Equal-budget comparisons contrast direct generation, Best-of-9, tool-grounded Best-of-9, adaptive search without memory, and full MIRA with memory.Configurations use the same 20 requests with ACE-Step and YuE2; full MIRA and adaptive search share the budget and controller, differing in memory inputs.Source: §D.3 · pp. 19–20

D.4 CROSS-EVALUATOR ASSESSMENT

  • Both MIRA budgets improve Overall, D-Macro, Base, and Comp. over direct generation across all three backends under both alternative evaluators.At B = 9, improvements are also reported in all seven musical dimensions; comparisons are within each evaluator because their absolute score scales differ. Qwen3-Omni has lower expert agreement than MusicFlamingo across all six metrics, so its scores are complementary evidence.Source: §D.4 · p. 20

D.5 TRACE FIELDS FOR QUALITATIVE ANALYSIS

The section describes how MIRA’s traces and prompt interfaces support inspection of prompt edits, evidence, verifier feedback, and search memory. It also documents tool routing, requirement construction, and audio-verification procedures.

D.5 TRACE FIELDS FOR QUALITATIVE ANALYSIS

  • Curated case records link prompt edits to their motivation and observed rubric changes through generation, search, verifier, tool, and evidence-card fields.The trace connects each branch action to supporting evidence and observed rubric changes; the cited table and figure compare component contributions and independent sampling under the same budget.Source: §D.5 · p. 20

E PROMPT TEMPLATES AND IMPLEMENTATION INTERFACES

  • The reference implementation provides five templates for tool routing, evidence fusion, requirement decomposition, candidate refinement, and audio verification.Templates are sent as one user-role message; the verifier also receives audio, and memory fields appear only when enabled.Source: E PROMPT TEMPLATES AND IMPLEMENTATION… · p. 22

E.1 TOOLS AND EVIDENCE ADMISSION

  • The router selects reference, scene, or track-audio tools according to the request; a deterministic gate admits tool-added claims only with usable evidence and confidence of at least 0.45.The gate does not remove references already present in the user’s request.Source: §E.1 · p. 22, §E.5 · p. 23

E.2 REQUIREMENT AND CANDIDATE INTERFACES

  • Online requirements receive type and repair-priority tags, while refinement supplies a complete replacement prompt using up to eight low-scoring checks.Output filtering clips prompts to 110 words and can fall back to the original request with failure cues.Source: §E.2 · p. 22, §E.5 · p. 24

E.3 STRUCTURED MEMORY INPUT

  • Tree-structured memory supplies branch-specific comparisons, parent–child changes, resolved or weakened items, and preserve-or-repair cues to candidate refinement.Branches do not access one another’s complete prompts or private trajectories, and memory resets between user requests.Source: §E.3 · p. 22, §E.5 · p. 25

E.4 MODEL INVOCATION AND ANSWER-TOKEN SCORING

  • MF-AQA estimates soft satisfaction from the model’s next-token logits for deduplicated yes and no answer tokens.The normalized ratio yields the soft satisfaction probability used in the main text.Source: §E.4 · p. 22

E.5 CORE PROMPT TEMPLATES

  • The prompt templates direct the verifier to use audible evidence and answer yes only when a requirement is clearly satisfied.Candidate refinement preserves passed requirements and uses weak checks and targeted feedback as priority cues, while avoiding edits linked to regressions when contrast memory is present.Source: §E.5 · p. 25

F.1 DATA PROVENANCE AND PRIVACY

The benchmark uses anonymized platform requests and expert-reviewed rubrics, with safeguards for privacy, reference interpretation, and the limits of expert judgments.

F.1 DATA PROVENANCE AND PRIVACY

  • MuRA-Bench releases curated requests from anonymized Mureka intent seeds, excluding sensitive information and personal identifiers while withholding raw user logs.The platform permitted research use and public release of the curated requests.Source: §F.1 · p. 25

F.2 EXPERT PARTICIPATION

  • Experts followed shared instructions, independently completed assigned subsets, and rated requirement fulfillment with system identities and automatic scores hidden.Source: §F.2 · p. 26

F.3 MUSICAL REFERENCES AND CREATOR RIGHTS

  • Artist, song, and album references are assessed as general musical characteristics, not direct imitation targets; alignment scores do not establish originality or copyright status.Their inclusion does not imply endorsement or authorization from referenced creators.Source: §F.3 · p. 26

F.4 INTERPRETIVE AND CULTURAL BIAS

  • Expert guidelines constrain unsupported interpretations of implicit intent, but model judgments and three experts’ perspectives may reflect particular musical conventions.MuRA-Bench measures alignment with request-specific rubrics, not a universal standard of musical quality or artistic value.Source: §F.4 · p. 26

Related papers

Loading 2610.10355v1…