Source-linked AI summary
Music generation with variational recurrent autoencoder supported by history
Ivan P. Yamshchikov, Alexey Tikhonov
TL;DR
The paper addresses low-quality and repetitive outputs in neural music generation, proposing VRASH, a variational recurrent autoencoder that uses previously generated notes as history. Combined with filtering heuristics, the approach produces pseudo-live music described as acoustically pleasing, melodically diverse, and more structurally interesting, while style-specific filtering remains necessary.
Problem
Method
VRASH combines a variational autoencoder with history support, and automated heuristics filter generated tracks using a classifier trained on human assessments.
Source: §4 · p. 7Results
LM and VRASH achieve cross-entropies of 2.34 and 2.11 respectively, while VRASH yields the highest percentage of tracks with more interesting temporal and melodic structures.
Source: §4 · pp. 5–6Takeaways & Limitations
VRASH balances global and local musical structure and, with filtering, supports pseudo-realtime generation of diverse and acoustically pleasing music.
Source: §4 · p. 7, §5 · p. 9Filtering behavior is style-dependent: a filter trained on Bach stylizations must be retrained for another style to preserve the complexity needed for entertaining music.
Source: §4 · p. 9
Abstract
from arXiv · showhide
A new architecture of an artificial neural network that helps to generate longer melodic patterns is introduced alongside with methods for post-generation filtering. The proposed approach called variational autoencoder supported by history is based on a recurrent highway gated network combined with a variational autoencoder. Combination of this architecture with filtering heuristics allows generating pseudo-live acoustically pleasing and melodically diverse music.
1 Introduction
Music generation is difficult because musical sequences contain long-distance dependencies and neural systems can produce statistically plausible but human-judged poor output. The paper proposes VRASH and filtering heuristics to improve melodic diversity and acoustic quality.
- Long-distance dependencies in music limit Markov-chain approaches, while recurrent neural networks better process extended note sequences.Source: §1 · p. 1
- Variational recurrent autoencoders learn latent representations through variational inference, supporting diverse generation and control over output parameters.Source: §1 · p. 2
- VRASH is introduced as a new architecture for monotonic music generation that uses history support alongside filtering heuristics.Source: §1 · p. 2
- The combined approach is intended to generate pseudo-live music that is acoustically pleasing and melodically diverse.Source: §1 · p. 2
2 Music representation and data
The experiments use a large proprietary MIDI collection that is normalized into melodic note sequences and filtered to remove unsuitable temporal or low-entropy tracks.
- A proprietary collection of four gigabytes of MIDI files spanning different epochs and genres was split into separate tracks for preprocessing.Source: §2 · p. 2
- The representation focuses on pitch and temporal note parameters while omitting note strength, with pitch transposed to the fourth octave.Source: §2 · p. 2
- Tracks with more than eleven pause values were filtered, and pause durations were normalized using each track’s median pause.Source: §2 · pp. 2–3
- Low-entropy tracks were excluded to reduce overfitting and avoid repetitive outputs, leaving more than 15,000 normalized tracks.Source: §2 · p. 3
- Each note was encoded with pitch, octave, delay corresponding to note length, and MIDI-track metadata.Source: §2 · p. 3
3 Architecture
The paper compares a language model, a variational autoencoder, and VRASH for note-by-note music generation. VRASH extends the variational architecture with previously generated notes as historical inputs to address slowly declining mutual information in sequences.
Baseline language model
- The study trains three architectures for music generation, using a classical language model as the baseline.Source: §3 · p. 3, Figure 1 · p. 3
Variational autoencoder
- The variational autoencoder compresses melody macrostructure through a bottleneck while injecting variational Bayesian noise to support diverse, human-like structure.Source: §3 · p. 3, Figure 2 · p. 4
VRASH
- VRASH adds previously generated notes as historical inputs, allowing the model to listen to its own composed sequence while generating subsequent notes.Source: §3 · pp. 3–4, Figure 3 · p. 5
- VRASH is presented as the first application of this generative approach to music rather than text.Source: §3 · p. 4
History support
- The history mechanism is designed to compensate for slowly declining mutual information between distant tokens in natural discrete sequences.Source: §3 · p. 4
4 Experiments and discussion
Experiments compare VRASH with language-model and VAE baselines using cross-entropy, filtering accuracy, and mutual-information profiles. VRASH offers marginally better cross-entropy than LM, more interesting subjective structures, and style-dependent long-distance dependencies, while macrostructure remains imperfect.
4 Experiments and discussion
- 2.11 versus 2.34 cross-entropy: VRASH and LM perform comparably near saturation, with VRASH marginally better.The comparison uses an untrained random network as a reference baseline.Source: §4 · p. 5, Figure 4 · p. 6
- VRASH produces the highest percentage of tracks judged to have interesting temporal and melodic structures, despite only marginally improving formal performance over LM.The paper presents this as a subjective assessment and cites qualitative feedback from listeners and professional musicians.Source: §4 · pp. 5–6
- VAE and especially VRASH reduce repeated-note problems more effectively than the Language Model, although all architectures require large, high-quality training data.The models generally generate sufficiently diverse and interesting music under that data condition.Source: §4 · p. 6
- VAE and VRASH aim to capture macrostructure but do not always reproduce the distinct structural dynamics of human-written tracks.The authors nevertheless describe VRASH as a promising direction.Source: §4 · p. 6
- Up to 87% of good tracks are accurately highlighted by the filtering mechanism across test sets and methods.The classifier was trained from human ratings of more than 1000 tracks, treating ratings of 4 or 5 as acceptable.Source: Table 1 · p. 7, §4 · p. 7
- Jazz-stylized VRASH tracks most closely match real-track mutual-information profiles, whereas filtered jazz tracks decline differently and require style-specific retraining.Bach-stylized tracks retain higher long-distance mutual information and may sound harmonious but mechanistic; jazz tracks are described as more diverse and human-like.Source: §4 · pp. 8–9, Figure 5 · p. 8
5 Conclusion
The paper compares several architectures for monotonic music generation and presents VRASH as a model balancing global and local musical structure. It also introduces style control and filtering for inconsistent outputs.
5 Conclusion
- VRASH balances global track structure with locally diverse and interesting musical patterns.The VAE focuses on macrostructure, while the history-supported extension advances local diversity.Source: §5 · p. 9
- The paper compares Language Model, Variational Autoencoder, and VRASH architectures for monotonic music generation.Source: §5 · p. 9
- VRASH allows output style control through the latent representation of the input vector.The model can generate tracks corresponding to given parameters.Source: §5 · p. 9
- A simple filtering method addresses non-consistent generative output, alongside analysis comparing generated tracks with empirical data.Source: §5 · p. 9
- The proposed structure is relatively easy to implement and train.Source: §5 · p. 9