Source-linked AI summary
MusGU+: Toward a Musician-Centered Evaluation Framework and Discovery Tool for Generative Music AI
Laura Ibáñez-Martínez, Roser Batlle-Roca, Xavier Serra, Martín Rocamora
TL;DR
Generative music systems are widely framed as democratizing tools, but their practical suitability for musicians is not adequately captured by existing evaluations. MusGU+ introduces a three-axis framework and interactive discovery tool, finding persistent asymmetries in adaptability, usability, and musically meaningful control despite improved accessibility.
Problem
Existing evaluations provide limited support for comparing generative music systems according to musicians’ practical workflow needs.
Method
MusGU+ evaluates systems across Adaptability, Usability, and Controllability and presents the results through an interactive discovery tool.
Results
The evaluation reveals persistent asymmetries across the three dimensions, particularly for adaptability and musically meaningful control, despite improved accessibility.
Takeaways & Limitations
Assessing accessibility alone is insufficient for judging democratization or creative empowerment without considering musicians’ workflows and practical constraints.
Takeaways & Limitations
MusGU+ lacks a formal user study or systematic workflow-based evaluation with musicians, especially for validating controllability in practice.
Abstract
from arXiv · showhide
Generative music systems are increasingly presented as tools that democratize music creation, yet their practical suitability for musicians remains underexplored. Prior work includes openness-focused evaluation frameworks, such as MusGO (Music-Generative Open AI), as well as qualitative studies of musicians' experiences with generative systems. However, these approaches do not support systematic comparison or early-stage discovery of models for creative use. Motivated by such limitations, we introduce MusGU+, a musician-centered evaluation framework organized around three dimensions: Adaptability, Usability, and Controllability. Together, these capture whether a model can be feasibly trained or fine-tuned on personal data, integrated into real-world music workflows, and controlled in musically meaningful ways. We evaluate 10 representative generative music systems and present an interactive discovery tool that enables musicians to explore and filter models according to these criteria. While MusGO remains valuable for promoting responsible research practices, MusGU+ supports informed selection and practical adoption of generative systems by musicians.
1 INTRODUCTION
Generative music systems are promoted as democratizing tools, but existing evaluations provide limited insight into their suitability for musicians. MusGU+ addresses this gap with a musician-centered framework and interactive discovery tool.
- Motivation: Existing evaluations focus mainly on outputs and provide limited insight into adaptation, workflow integration, and meaningful control.
- Contribution: MusGU+ evaluates whether models can be adapted to personal data, integrated into real-world workflows, and controlled musically.
- Contribution: The framework organizes assessment around Adaptability, Usability, and Controllability, covering training pathways, interfaces, workflow integration, and control mechanisms.
- Contribution: MusGU+ is applied to 10 generative music systems through an interactive tool for exploring, comparing, and filtering models.
- Contribution: The tool supports model selection and practical adoption according to musicians’ evaluation priorities.
2 BACKGROUND AND RELATED WORK
Prior work evaluates generative music through outputs, representations, openness, descriptive surveys, and workflow studies. These perspectives leave limited support for systematically comparing systems by musicians’ practical needs.
- Existing Evaluation Approaches: Output- and representation-level evaluations address quality, behavior, interpretability, and controllability but not fully how systems are used in practice.
- Openness-Focused Evaluation: MusGO evaluates openness and reproducibility through source-code availability, training-data disclosure, licensing, and documentation.
- Openness-Focused Evaluation: Openness-focused evaluation may group systems with different practical accessibility and usability, obscuring workflow-integration differences.
- Musician-Relevant Perspectives: Descriptive surveys examine musician-relevant concerns such as inputs, real-time generation, licensing, and training options without providing a structured evaluation.
- Creative Practice: Workflow studies report challenges involving prompt control, steerability, latency, configuration complexity, and integration into iterative creative processes.
- Creative Practice: Musicians often use generative models within broader creative pipelines, where technical expertise and community support can be required.
3 MUSGU+: A MUSICIAN-CENTERED EVALUATION FRAMEWORK
MusGU+ adapts graded, composite evaluation toward practical suitability for musicians. Its three axes and 15 criteria assess adaptation, workflow use, and musically meaningful control.
- Framework Design: MusGU+ uses graded multidimensional criteria rather than treating evaluation properties as binary.
- Framework Design: The framework draws on prior literature, researcher-musician experience, cross-review, practitioner consultation, and workshop feedback.
- Adaptability: Adaptability assesses feasible adaptation to musicians’ data, including hardware, dataset size, pathways, technical barriers, and redistribution.
- Usability: Usability assesses access, interaction, reliability, real-time capability, workflow integration, licensing, and community support.
- Controllability: Controllability assesses input options and control mechanisms, including conditioning inputs, time-varying control, disentanglement, and control parameters.
- Evaluation Structure: The three axes contain 15 criteria scored as fully, partially, or not supported, with explanations defining each level.
4 MODEL EVALUATION WITH MUSGU+
MusGU+ evaluates a diverse set of maintained generative music systems through evidence-based, cross-reviewed assessments and presents results in an interactive display. Tags and filters support task-oriented comparison rather than a single definitive ranking.
- Model Selection: The evaluation covers 10 maintained audio-domain systems spanning academic, industrial, and product-oriented contexts.
- Evaluation Process: Authors score each criterion using developers’ papers, websites, documentation, repositories, interfaces, and supplementary ecosystem resources.
- Evaluation Process: Independent cross-review resolves discrepancies through joint evidence inspection and iterative refinement toward consensus.
- Interactive Display of Results: The interactive display supports exploration, reordering, and filtering across musical needs and creative priorities.
- Interactive Display of Results: Aggregate scores assign 0, 0.5, or 1 to unsupported, partially supported, or fully supported criteria, but do not define a definitive ranking.
- Interactive Display of Results: Reusable tags encode input modalities, workflow environments, real-time capability, control concepts, and musical applications for interpretable comparison.
5 DISCUSSION
The evaluation reveals strong asymmetries across Adaptability, Usability, and Controllability, with only a small group of systems consistently aligned with musicians’ practical needs. MusGU+ also functions as a discovery tool, while its framework remains limited by the absence of experiential validation and coverage beyond general-purpose audio systems.
- Adaptability: Only DDSP-VST, Neutone Morpho, and RAVE accommodate individual musicians’ adaptation constraints, including modest computation, small personal datasets, and accessible adaptation pathways.Other research-oriented models provide formal adaptation mechanisms but impose substantial technical, computational, or data-related barriers, while Suno and Udio offer no practical user-driven adaptation.
- Usability: DDSP-VST, Neutone Morpho, RAVE, and AFTER show the strongest usability alignment through interfaces, workflow integration, and real-time interaction.Commercial platforms provide accessible interfaces and community support but restrict access, output use, or ownership; Udio prohibits downloading generated outputs.
- Controllability: DDSP-VST, Neutone Morpho, RAVE, and AFTER provide the most extensive fine-grained, time-varying control over interpretable musical features and meaningful conditioning inputs.Suno and Udio rank lowest, offering mainly global conditioning and largely entangled features; Udio exposes no explicit control parameters for shaping generation.
- Cross-axis findings: Adaptability is the most consistently challenging dimension, Usability varies most across systems, and Controllability commonly remains partial rather than precise or time-localized.These asymmetries indicate substantial room for better alignment between control mechanisms and musicians’ ways of shaping sound over time.
- Discovery tool: The interactive MusGU+ table lets musicians filter systems by broad applications and axis-specific support, foregrounding practical adoption conditions rather than a single notion of suitability.It supports exploration from broad use cases to specific musical and technical requirements and is intended as a publicly accessible, evolving resource.
- Limitations: MusGU+ lacks formal musician user studies and systematic workflow evaluation, especially for assessing how controllability works in practice during fine-grained musical interaction.The framework is derived from prior literature, comparative analysis, and the authors’ experience as researchers and musicians.
- Limitations: The framework currently covers general-purpose audio-domain music generation, excluding symbolic or MIDI systems, instrument-specific models, and intermediate musical representations.Future work aims to expand coverage, refine criteria through community contributions, and assess inter-rater agreement.
- Limitations: MusGU+ does not explicitly evaluate ethical or legal dimensions, serving instead as a complementary lens on how system design shapes musicians’ engagement.The framework intersects with concerns including authorship, agency, copyright, cultural bias, and labor impacts.
6 CONCLUSION
MusGU+ provides a musician-centered framework for evaluating the practical suitability of generative music systems across adaptability, usability, and controllability. Its interactive tool supports comparison and informed selection while emphasizing musician workflows and creative contexts.
- MusGU+ evaluates generative music systems through Adaptability, Usability, and Controllability using graded criteria.The framework focuses on practical characteristics relevant to musical practice.
- The evaluation reveals persistent asymmetries in adaptability and musically meaningful control despite improvements in accessibility.These findings qualify accessibility-based claims about democratization or creative empowerment.
- An interactive tool enables musicians to explore and compare systems using musician-centered criteria and tags rather than a single ranking.This supports informed selection of systems for creative use.
- MusGU+ contributes a shared vocabulary for considering generative music systems as creative tools rather than solely technical artifacts.The framework is intended to support more reflective development practices and dialogue across technical, creative, and ethical perspectives.
A.1 Adaptability
The Adaptability dimension assesses whether musicians can feasibly adapt a model to their own data under practical constraints. It considers supported training or fine-tuning pathways.
- Adaptability assesses how feasible it is for musicians to adapt a model to their own data.
- The assessment focuses on practical constraints affecting model adaptation.
- The framework considers supported training or fine-tuning pathways when evaluating adaptability.
A.1.1 Hardware Requirements
Hardware Requirements evaluates whether a model can be trained or fine-tuned with hardware accessible to end users. The criteria distinguish practical computing setups from premium or institutional-level hardware requirements.
- CPU-only training or fine-tuning is considered feasible within a practical timeframe for end users.
- Mid-range gaming laptops and common cloud GPU instances represent practical hardware settings for end users.
- Requiring dedicated high-end GPUs, multi-GPU rigs, or TPUs makes adaptation dependent on premium or institutional-level hardware.
A.1.2 Dataset Size
Dataset Size evaluates whether a model can be effectively trained or fine-tuned on the scale of data available to individual musicians. The criteria contrast small personal datasets with large-scale data requirements.
- Small personal datasets, ranging from minutes to a few hours of audio, can support effective training or fine-tuning for some models.
- Requirements exceeding most musicians’ own libraries constrain adaptation to models needing larger curated datasets.
- Large-scale datasets may require hundreds of hours of diverse, high-quality material, making adaptation infeasible without institutional or commercial-scale data.
A.1.3 Adaptation Pathways
The framework distinguishes practical pathways for adapting generative music models from barriers involving documentation, access, workflow integration, and musical control. It evaluates whether musicians can train or fine-tune models, use them in creative workflows, and guide their outputs through meaningful controls.
- Adaptation Pathways: Complete training or fine-tuning materials, checkpoints, or dedicated interfaces provide practical adaptation pathways, whereas missing elements force users to assemble or infer them.Interfaces designed for non-programmers and documented notebooks lower the technical burden; raw code with little guidance requires significant programming knowledge.
- Adaptation Pathways: Redistribution criteria distinguish systems that explicitly permit sharing adapted models from those that restrict or prohibit redistribution.Permissions may be unconditional or limited by non-commercial, research-only, platform-bound, contractual, or technical constraints.
- Usability: Usability examines setup, access, latency, workflow integration, output licensing, and support channels relevant to musicians’ creative practice.The framework considers dedicated interfaces, unrestricted access, real-time performance, integration with music workflows, permitted output use, and community support.
- Controllability: Controllability evaluates the diversity, temporal precision, interpretability, and configurability of inputs and internal mechanisms guiding generation.Criteria range from multiple musical conditioning modalities and time-varying controls to attribute-specific pathways, configurable parameters, and access to internal representations.