InspireMusic: Integrating Super Resolution and Large Language Model for High-Fidelity Long-Form Music Generation

1 INTRODUCTION

• Autoregressive systems such as Jukebox and MusicGen model musical structure, but MusicGen is limited to short output lengths and prior systems still struggle with long-range dependencies.Diffusion- and flow-based alternatives can deliver audio fidelity but may sacrifice efficiency or long-term coherence, while long prompts can limit fine-grained control.Source: §1 · pp. 1–2
• Existing systems have not effectively combined high-fidelity audio, long-form coherence, and precise control over global structure and local acoustic detail.The paper describes this combination as a substantial remaining gap.Source: §1 · p. 2
• InspireMusic combines a 75Hz, one-codebook audio tokenizer with an autoregressive model and flow-matching super-resolution to target global structure and local acoustic detail.The tokenizer is described as capturing global musical structure while facilitating faster autoregressive training and inference.Source: §1 · p. 2
• The unified framework supports controllable high-fidelity generation with long-form coherence of up to 8 minutes.The authors also describe configurations using the LLM alone or together with super-resolution for improved fidelity.Source: §1 · p. 2

2 RELATED WORK

• The related-work review frames music generation as a field with several recent approaches and directions.It introduces autoregressive transformers, diffusion models, flow matching, and combined music-generation systems.Source: §2 · p. 2

2.1 AUTOREGRESSIVE TRANSFORMER MODELS

• Autoregressive models generate audio-token sequences and can capture musical structure, but often struggle to maintain coherence over extended durations.MusicLM uses hierarchical autoregressive transformers; MusicGen streamlines token-based generation by conditioning directly on text.Source: §2.1 · p. 2

2.2 DIFFUSION MODELS

• Diffusion methods such as Stable Audio 2.0 and AudioLDM can produce high-fidelity audio, but iterative inference can hamper real-time use and sometimes disrupt global musical structure.Stable Audio 2.0 uses latent diffusion, while AudioLDM uses text-conditioned latent diffusion.Source: §2.2 · p. 3

2.3 FLOW-MATCHING MODELS

• Flow matching learns a continuous mapping between distributions and is presented as a faster, less complex sampling alternative to iterative diffusion.InspireMusic applies a super-resolution variant to upscale coarse token sequences while preserving musical semantics and synthesizing detailed output.Source: §2.3 · p. 3

2.4 MUSIC GENERATION

• The paper argues that combining generative paradigms can pair autoregressive models’ long-form structure with the high-quality audio of diffusion or flow matching.It presents InspireMusic as a hybrid architecture intended to maintain coherence while producing high-fidelity, controllable music.Source: §2.4 · p. 3

3 INSPIREMUSIC

• The framework converts audio to discrete tokens, uses an autoregressive transformer to generate tokens from audio and text inputs, then applies flow matching and a vocoder to produce high-quality 48kHz audio.The overview describes the flow-matching stage as mapping generated tokens to high-resolution latent features with fine-grained acoustic detail.Source: §3 · p. 3, Figure 1 · p. 4

3.1 AUDIO TOKENIZATION

• WavTokenizer compresses 24kHz audio into 75Hz discrete tokens with one codebook, capturing coarse audio information and global musical structure while shortening autoregressive sequences.The tokenizer operates at 0.9kbps bandwidth and is described as containing rich semantic information.Source: §3.1 · pp. 3–4

3.2 AUTOREGRESSIVE TRANSFORMER

• A Qwen 2.5-backed autoregressive transformer predicts each audio token from preceding tokens to generate long sequences with musical structure and long-form coherence.The training objective can condition on text descriptions, captions, tags, timestamps, musical structures, labels, and audio tokens.Source: §3.2 · p. 4

3.3 SUPER-RESOLUTION FLOW-MATCHING

• Super-resolution flow matching maps coarse low-rate audio tokens to fine-grained high-resolution latent features extracted from 48kHz audio, which a vocoder decodes into a waveform.The model learns a continuous mapping from discrete token space to high-resolution latent space rather than using many iterative diffusion refinements.Source: §3.3 · p. 5

3.4 MODEL VARIANTS

• The model series includes lightweight and larger 30-second variants, plus InspireMusic-1.5B-Long, designed for compositions up to 8 minutes with structural coherence.The framework also defines configurations with and without flow matching for different generation tasks.Source: §3.4 · pp. 5–6, Table 1 · p. 6

3.5 MODEL TRAINING AND INFERENCE

• Training separately covers the audio tokenizers, autoregressive transformer, and flow-matching model, with paired low- and high-resolution tokens used to train the upscaling transformation.The autoregressive model uses pre-training on audio-text pairs followed by fine-tuning on curated human-captioned data; the authors state that each stage optimizes its task while maintaining overall coherence.Source: §3.5 · p. 6

4 DATASET

• Pre-training data comprises about 29 billion audio tokens, or approximately 100,000 hours, and around 740 million text tokens.Audio is resampled to 24kHz or 48kHz and segmented into 30-second clips; it is downmixed to mono unless otherwise specified.Source: §4 · p. 7
• Evaluation uses MusicCaps, Song Describer, and an in-house set of 30 music pieces with text and audio prompts.MusicCaps contains 5,521 ten-second samples with captions and expert-provided aspects; Song Describer contains approximately 1,100 captions for 706 recordings.Source: §4 · p. 7

4.1 AUDIO TOKENIZATION

• WavTokenizer produces one-dimensional discrete audio tokens at 75 Hz with a single quantizer layer; non-causal HiFi-Codec handles 48 kHz monophonic audio at 150 Hz using four residual quantizers.WavTokenizer contains approximately 430M parameters, and HiFi-Codec approximately 128M.Source: §4.1 · p. 7

4.2 TEXT CAPTIONS

• A large language model generates music text descriptions from tags including genres, instruments, styles, and soundscapes.The section points to a figure showing the dataset’s top 20 genre distribution.Source: §4.2 · p. 7

5 EXPERIMENTS

• InspireMusic is evaluated against MusicGen and Stable Audio 2.0 using subjective and objective measures, chiefly on an in-house test set unless otherwise specified.Source: §5 · p. 7

5.1 EXPERIMENTAL SETUP

• Experiments use MusicCaps, Song Describer, and in-house datasets covering genres such as electronic, classical, and jazz.Source: §5.1 · p. 7
• Training uses H800 GPUs, Adam, a base learning rate of 1 × 10−4, and a scheduler with 5,000 warm-up steps.Source: §5.1 · p. 7
• Figure 2 shows the distribution of the top 20 genres in the pre-trained dataset, whose text descriptions are generated from music tags.Source: Figure 2 · p. 8
• The LLM training proceeds through pre-training on audio tokens, caption-paired audio-token fine-tuning, and fine-tuning with human-labeled captions and audio tokens.Source: §5.1 · p. 8
• Flow matching maps 24kHz audio-token inputs to latent features extracted from corresponding 48kHz waveforms with a 150Hz HiFi-Codec.Source: §5.1 · p. 8

• The evaluation includes Qwen 2.57, MusicGen-Small, MusicGen-Medium, MusicGen-Large, and Stable Audio 2.0.Source: §5.2 · p. 8
• Models are evaluated on the in-house dataset and the public MusicCaps and Song Describer datasets.Source: §5.2 · p. 8
• Evaluation uses both objective and subjective methods.Source: §5.2 · p. 8

5.3.1 OBJECTIVE EVALUATIONS

• Objective evaluation uses FD, KL, and CLAP: FD and KL compare generated with reference audio, while CLAP measures alignment between music descriptions and generated audio.Source: §5.3 · p. 8, §5.3.1 · p. 9

5.3 EVALUATION METRICS

• Tables 2 and 3 report KL, FD, and CLAP for text-to-music and music continuation, while Tables 4 and 5 compare text-to-music results on MusicCaps and Song Describer.Source: Table 2 · p. 9, Table 3 · p. 9, Table 4 · p. 10

5.3.2 SUBJECTIVE EVALUATIONS

• Subjective evaluation uses Comparative Mean Opinion Scores from 10 raters with professional music backgrounds.Source: §5.3.2 · p. 9
• Subjective dimensions include audio-text alignment, audio quality, musicality, and overall performance.Source: §5.3.2 · pp. 9–10

• Text-to-music generates high-fidelity music with long-form coherence from text prompts.Source: §5.4 · p. 10
• Music continuation extends a composition based on an input audio prompt.Source: §5.4 · p. 10

5.5 RESULTS

• The default InspireMusic setting combines an autoregressive transformer with SRFM; without SRFM, the model generates 24kHz audio rather than 48kHz audio.The series configuration without flow matching uses the WavTokenizer decoder directly on generated tokens.Source: §5.5 · pp. 10–11
• InspireMusic-1.5B-Long outperforms the compared systems across evaluation dimensions and scores 7% higher in text-to-music CMOS than Stable Audio 2.0.The subjective listening tables report CMOS for text-to-music and music continuation.Source: Table 6 · p. 11, Table 7 · p. 11, §5.5 · pp. 11–12
• On MusicCaps, InspireMusic-1.5B performs better than MusicGen on KL and FD, but its CLAP score is lower than MusicGen and Stable Audio 2.0.The authors note differences in training data and caption style as context for CLAP comparisons.Source: §5.5 · p. 11
• Without flow matching, InspireMusic-1.5B-Long outperforms MusicGen and Stable Audio 2.0 on KL and FD on Song Describer, but trails Stable Audio 2.0 on CLAP.Its CLAP score is reported as better than the MusicGen models.Source: §5.5 · p. 11
• SRFM is reported to correct some artifacts in generated tokens, and InspireMusic-1.5B-Long with flow matching performs better than InspireMusic-1.5B with or without it.The subjective tests report CMOS on a 1.0–5.0 range with 95% confidence intervals.Source: §5.5 · p. 12
• A 5-minute InspireMusic piece retains intro, verse, chorus, and outro structure with minimal artifacts; MusicGen often repeats and loses structure after about 30 seconds.The section also reports clearer high-resolution output from SRFM than Stable Audio 2.0, supported by KL, FD, and listening tests.Source: §5.5 · p. 12

5.6 ABLATION STUDIES

• Removing SRFM causes a notable drop in audio fidelity, while the autoregressive model without SRFM reduces long-term coherence in extended compositions.Using only the WavTokenizer decoder also reduces musical detail and dynamic range.Source: §5.6 · p. 12
• At 5 minutes, InspireMusic-1.5B-Long outperforms Stable Audio 2.0 at 3 minutes on KL and FD.The length comparison includes 30-second, 1-minute, and 5-minute text-to-music generations.Source: §5.6 · p. 12, Table 11 · p. 13
• Table 10 evaluates InspireMusic-1.5B-Long with and without SRFM across CFG values on text-to-music and music continuation tasks.The reported metrics are KLpasst, FDopenl3, and CLAPscore.Source: §5.6 · p. 12, Table 10 · p. 13

6 CONCLUSION

• InspireMusic combines an autoregressive transformer with super-resolution flow matching for long-form, high-fidelity music generation.The framework uses Qwen 2.5 to predict audio tokens.Source: §6 · p. 13
• The super-resolution component maps lower-token-rate audio representations to higher-rate acoustic-codec features, then decodes them into higher-sampling-rate audio.The representations contain both semantic and acoustic information.Source: §6 · p. 13
• InspireMusic supports text-to-music and music continuation with text and audio prompts, and has comparable objective and subjective performance to MusicGen series models and Stable Audio 2.0.The authors characterize the framework as an efficient and scalable pathway for diverse, long-form, coherent, high-quality audio compositions.Source: §6 · p. 13