Source-linked AI summary
NONOTO: A Model-agnostic Web Interface for Interactive Music Composition by Inpainting
Théis Bazin, Gaëtan Hadjeres
TL;DR
Interactive AI-based interfaces for music creators are still emerging, and local inpainting offers a way to let users reshape portions of a score. NONOTO provides a model-agnostic web interface for that interaction, with playback and DAW connections. The authors present it as an open-source, hackable tool for researchers and artists to adapt.
Problem
Interfaces designed for music creators remain an emerging area, while unambiguous quantitative evaluation of generative models is difficult and should reflect their intended applications.
Source: Introduction · p. 1Method
NONOTO is a web interface for symbolic-music inpainting that lets users regenerate score regions and lets researchers connect models through a decoupled interface and server.
Source: Introduction · p. 1, Framework · p. 2Results
NONOTO provides an interactive, open-source, hackable inpainting interface with real-time audio playback and options for MIDI and DAW synchronization.
Source: Interface · p. 2, DAW integration · p. 2, Conclusion · p. 2Takeaways & Limitations
The authors hope tools emphasizing usability, accessibility, affordance, and hackability will encourage a more modest view of AI as assisting music creation rather than replacing musicians.
Source: Conclusion · p. 3
Abstract
from arXiv · showhide
Inpainting-based generative modeling allows for stimulating human-machine interactions by letting users perform stylistically coherent local editions to an object using a statistical model. We present NONOTO, a new interface for interactive music generation based on inpainting models. It is aimed both at researchers, by offering a simple and flexible API allowing them to connect their own models with the interface, and at musicians by providing industry-standard features such as audio playback, real-time MIDI output and straightforward synchronization with DAWs using Ableton Link.
Introduction
Interactive music interfaces remain an emerging area, and NONOTO focuses on inpainting so users can reshape local score regions while retaining an active role in composition. Its web interface is designed to connect generative models to real production and performance settings.
Introduction
- NONOTO offers a general web interface for researchers to test and promote music-generation models in actual production and performance settings.The contribution list emphasizes an intuitive user interface, easy model integration, and a web-based, model-agnostic framework.Source: Introduction · p. 1
- Inpainting models recompose part of a score given its other portions, letting users participate in composition rather than receive only a complete machine-generated score.The authors argue that quick, playful exploration can support creativity and accessibility for musicians and non-experts.Source: Introduction · p. 1
Existing approaches
- FlowComposer performs local updates, but lacks real-time audio and MIDI playback, which the authors say limits it to studio use.The authors contrast this with NONOTO’s more spontaneous and reactive interaction.Source: Existing approaches · p. 1
- Magenta Studio and StyleMachine use continuation or complete generation, which the authors believe breaks musical flow and limits live-performance use compared with local inpainting.The cited Magenta Studio models include rhythm and melody generation (Roberts et al., 2018).Source: Existing approaches · p. 2
Suggested mode of usage
Users interact with a looping score by selecting regions for regeneration, with playback continuing through updates. Dataset-specific controls and DAW integration extend this interaction into collaborative music-making.
Suggested mode of usage
- Users can click or touch any region of the looping score to regenerate it, and the display updates without interrupting playback.The score can be integrated into a DAW for users to shape sounds, add effects, play drums, or create live mixes.Source: Suggested mode of usage · p. 2
- Dataset-specific controls include fermata placement for Bach chorales and chord-progression control for Jazz leadsheets or Folk songs.These metadata are sent to the generative model along with the score during regeneration.Source: Suggested mode of usage · p. 2
- The authors describe a potential jam-like, collaborative production setting and say NONOTO can be used alongside Magenta Studio or StyleMachine.In this setting, the person controlling the AI can participate as one of multiple instrument players.Source: Suggested mode of usage · p. 2
Technology
NONOTO separates its web interface from the music-inpainting server and uses browser technologies to keep the system modular. MusicXML, a small HTTP API, and real-time audio, MIDI, and Ableton Link support connect models to music-production workflows.
Framework
- A strict separation between the interface and inpainting server lets researchers connect new models by implementing how a model responds to user input.The browser-based codebase is designed to be modular and hackable, including for new interactions or corpus-specific metadata.Source: Framework · p. 2
Interface
- OpenSheetMusicDisplay renders MusicXML scores, while Tone.js adds real-time audio preview that continues uninterrupted during regeneration.This combines score rendering with playback inside the web interface.Source: Interface · p. 2
Generation back-end and communication
- The server API uses MusicXML and two commands: /generate initializes a session with a fresh score, and /timerange-change requests regeneration of a selected interval in its musical context.The passage specifies these as the two required server-side commands.Source: Generation back-end and communication · p. 2
DAW integration
- Users can send generated music to a DAW through real-time audio or MIDI and synchronize hosts over a local network with Ableton Link.The authors say these integrations, paired with arbitrary generation back-ends, support testing models in a standard production environment with minimal overwork.Source: DAW integration · p. 2
Conclusion
The authors present NONOTO as an open-source, hackable interface for inpainting-based music generation. They hope tools emphasizing usability and accessibility will support a more modest view of AI’s role in music creation.
Conclusion
- NONOTO is an interactive, open-source, hackable interface for music generation with inpainting models.The authors invite researchers and artists to develop models or new ways of interacting with it.Source: Conclusion · p. 2
- Figure 1 shows the interface used with melody and symbolic chords and with four-part chorale music.Source: Figure 1 · p. 3
- The authors hope usability, accessibility, affordance, and hackability will shift views from “robot music” replacing musicians toward AI-assisted music.The passage frames this as an aspiration, not an established outcome.Source: Conclusion · p. 3
Ackowledgements
The authors thank the anonymous reviewers for their helpful comments.
- The authors thank the anonymous reviewers for their helpful comments.Source: Ackowledgements · p. 3