Source-linked AI summary
madmom: a new Python Audio and Music Signal Processing Library
Sebastian Böck, Filip Korzeniowski, Jan Schlüter, Florian Krebs, Gerhard Widmer
TL;DR
Audio-based MIR requires processing pipelines that can handle both low-level signal extraction and high-level, machine-learning-based analysis, while existing libraries leave this integration limited. madmom provides a Python framework combining these stages with reusable processors and musically meaningful state-of-the-art features, but it focuses on applying pre-trained models rather than training them. The paper reports strong performance in aggregated MIREX evaluations and presents madmom as a reproducible framework for MIR applications.
Problem
Existing MIR libraries provide low-level or selected high-level features, but generally lack integrated machine-learning components despite their importance in current MIR applications.
Method
madmom combines low-level audio processing, high-level MIR feature analysis, callable Processors, and reusable pre-trained machine-learning models in one Python framework.
Results
madmom includes state-of-the-art systems for onset, beat, downbeat, rhythm, tempo, and piano-transcription tasks, with outstanding results reported in aggregated 2006–2015 MIREX evaluations.
Takeaways & Limitations
madmom supports rapid prototyping, conversion to standalone MIR programs, and reproducible experiments through callable and serializable processing chains.
Takeaways & Limitations
madmom applies pre-trained machine-learning models but does not provide training algorithms, deliberately prioritizing MIR application over an extensive learning toolkit.
Abstract
from arXiv · showhide
In this paper, we present madmom, an open-source audio processing and music information retrieval (MIR) library written in Python. madmom features a concise, NumPy-compatible, object oriented design with simple calling conventions and sensible default values for all parameters, which facilitates fast prototyping of MIR applications. Prototypes can be seamlessly converted into callable processing pipelines through madmom's concept of Processors, callable objects that run transparently on multiple cores. Processors can also be serialised, saved, and re-run to allow results to be easily reproduced anywhere. Apart from low-level audio processing, madmom puts emphasis on musically meaningful high-level features. Many of these incorporate machine learning techniques and madmom provides a module that implements some in MIR commonly used methods such as hidden Markov models and neural networks. Additionally, madmom comes with several state-of-the-art MIR algorithms for onset detection, beat, downbeat and meter tracking, tempo estimation, and piano transcription. These can easily be incorporated into bigger MIR systems or run as stand-alone programs.
1. INTRODUCTION
Audio-based MIR is increasingly important as audio data grows beyond manual manageability, but existing libraries generally separate low-level extraction from higher-level analysis and lack integrated machine-learning components. madmom addresses this gap with a Python framework spanning the full processing chain and state-of-the-art MIR systems.
- Audio-based MIR has become increasingly important because available audio data has expanded beyond what can be managed manually.
- Most state-of-the-art audio-based MIR algorithms extract low-level features before analyzing them to retrieve requested information.
- Existing libraries commonly focus on low-level extraction or selected high-level features, while integrated machine-learning components remain largely unavailable.
- madmom combines low-level extraction, machine-learning-based high-level analysis, and trained state-of-the-art systems within one framework.
- The library is written in Python under a BSD license, while its pre-trained models use the CC BY-NC-SA 4.0 license.
1.1 Design and Functionality
madmom uses an object-oriented, NumPy-compatible design to bundle data with metadata and reduce the effort needed to prototype MIR processing workflows.
- madmom encapsulates data and metadata in objects, often subclassing NumPy’s ndarray to retain array-handling routines.
- Simple object instantiation and sensible defaults allow a log Melspectrogram to be created from an audio-file path in one line of code.
1.1.3 Simple conversion into runnable programs
madmom’s Processor concept converts prototyped audio workflows into standalone programs with consistent interfaces, while supporting reproducible saved configurations. Its machine-learning integration deliberately focuses on applying externally trained models rather than training them.
- Processors transform complete audio-processing workflows into runnable standalone programs with a consistent calling interface.
- madmom limits machine-learning support to testing capabilities with pre-trained models because emerging training methods are difficult to maintain comprehensively.
1.1.5 State-of-the-art features
madmom emphasizes high-quality, musically meaningful state-of-the-art feature extractors for core MIR tasks and supports reproducible processing chains through saved settings.
- madmom provides state-of-the-art feature extractors for downbeats, beats, onsets, tempo, and piano transcription.
- Its pickle-based functionality saves complete processing chains, including settings, so experiments can be reloaded with their specific configurations.
1.1.7 Few dependencies
madmom is built on established scientific-computing libraries and supports parallel execution across multiple CPU cores.
- NumPy, SciPy, and Cython form madmom’s underlying library stack.They provide array handling, optimized numerical routines, and speedups for time-critical code.
- Cython accelerates time-critical library components by generating and compiling C code from Python-like syntax.
- Processors can run several programs or processing tasks in parallel across modern multi-core architectures.
1.1.9 Extensive documentation
madmom provides extensive documentation and publicly developed code, supported by automated testing and quality checks.
- The source files include thorough NumPy-format documentation.
- The online documentation includes the API reference, build and installation instructions, and interactive IPython notebooks.
- Public development uses GitHub, continuous integration, code-quality testing, and test-coverage monitoring.The project uses TravisCI, QuantifiedCode, and Coveralls for automated quality processes.
2. LIBRARY DESCRIPTION
madmom’s architecture combines concise class interfaces with function-level access and processor-based transformations between NumPy-compatible data objects.
- The API combines class-based interfaces with functions that expose detailed processing functionality.
- Data classes represent entities such as audio signals and spectrograms as NumPy ndarray subclasses with added attributes and methods.
- Processor classes store transformations between data classes, enabling chains that convert algorithm prototypes into callable processing pipelines.For example, a processor can transform a Signal into a Spectrogram.
2.1 Packages
madmom organizes audio processing, MIR features, machine-learning methods, and reusable processors into packages that support both custom pipelines and standalone programs.
- Processors: Processors define callable data-processing operations and can write outputs to files through OutputProcessor.SequentialProcessor and ParallelProcessor combine operations sequentially or across multiple CPU cores.
- madmom.audio: The audio package provides low-level signal processing, including signal loading, framing, and spectral representations.Signal and FramedSignal support format conversion, while STFT and Spectrogram support windowing, shifting, and zero-padding.
- madmom.audio: Signal and FramedSignal support arbitrary frame rates independent of sample rate while preserving temporal alignment across frame sizes.
- madmom.features: The features package emphasizes musically meaningful features for onset detection, beat and downbeat tracking, rhythm analysis, tempo estimation, and piano transcription.Most features are implemented as Processors and commonly include standalone programs and evaluation code.
- madmom.ml: madmom implements GMMs, HMMs, and neural networks, including feed-forward, recurrent, convolutional, and LSTM architectures.
- madmom.ml: The machine-learning module applies techniques without third-party dependencies but does not provide training algorithms.Pre-trained models can be converted for use with madmom and are distributed under a research-oriented Creative Commons license.
2.2 Standalone Programs
madmom provides standalone MIR programs through simple command-line wrappers that support single-file or batch processing. Their aggregated MIREX ranks demonstrate strong performance across evaluations.
- Standalone programs wrap madmom.features functionality behind a simple command-line interface.
- Programs operate in single mode for one input file or batch mode for multiple files.
- Programs are implemented as Processors and can be pickled with their parameters for later execution using identical settings.
- Table 1 reports program ranks from MIREX evaluations aggregated across 2006-2015, with rank 1 representing the best-performing submission in that combined period.
- The reported results highlight the state-of-the-art features provided by madmom.
3. CONCLUSION
The paper introduces madmom's design principles and library structure, while directing readers to its online documentation and source repository for current functionality. Planned extensions include real-time streaming, additional algorithms, and model conversion tools.
- The conclusion summarizes madmom's design principles and library structure and points readers to its online documentation and source repository.
- Future work aims to add memory-efficient streaming for online real-time audio processing instead of processing complete audio files.
- Future work also includes expanding the feature and algorithm set and adding tools to convert models trained with popular machine learning libraries.