Source-linked AI summary
Copying Versus Randomization in Lempel-Ziv Music Synthesis
Nadav Mishan, Ram Zamir
TL;DR
Standard LZ’s preference for long, contiguous copies can yield verbatim training excerpts in generated music. The paper controls average sequence length using tree-data limits and sequence extension; it reports control over copying and randomization with low complexity, while noting qualitative issues with rhythm and overfitting.
Problem
Standard LZ favors copying long, contiguous sequences, so generated music can contain verbatim training excerpts rather than original compositions.
Source: §I · p. 1Method
The paper balances copying and randomization by controlling average sequence length through per-tree training-data limits and sequence extension.
Source: §I · p. 1, B. Building the LZ tree · pp. 1–2Results
The authors report that the algorithm controls the balance between copying and randomization with low complexity, and that decreasing per-tree data or increasing “steps” decreases or increases Ls, respectively.
Source: §IV · p. 2, A. Limiting Training Data · p. 2, B. Choosing the Number of Steps · p. 2Takeaways & Limitations
The authors identify using different universal-compression algorithms and improving songs through preprocessing and a different generation stage as future directions.
Source: §IV · p. 2Qualitative evaluation notes rigid timing because rhythm was omitted, and larger Ls can yield coherence that stems from verbatim copying of training data.
Source: C. Qualitative Evaluation · p. 2
Abstract
from arXiv · showhide
We utilize Lempel-Ziv universal compression for music note generation. We control the algorithm's tendency to over-copy or under-copy training data by manipulating the average sequence length saved in the dictionary.
I. INTRODUCTION
Lempel-Ziv compression can produce verbatim excerpts because it favors copying long, contiguous sequences. The paper proposes two ways to adjust average sequence length and balance copying against randomization.
I. INTRODUCTION
- Lempel-Ziv compression is presented as an efficient alternative for generative AI, with prior work exploring it for music generation.The introduction contrasts universal compression’s real-time efficiency with the computational demands of diffusion, transformer-based, and GAN methods.Source: §I · p. 1
- Standard LZ favors long, contiguous copies, so generated music can consist of verbatim training excerpts rather than original compositions.The introduction frames this as a challenge because new compositions are typically inspired by existing works rather than reproduced from them.Source: §I · p. 1
- The proposed controls target the trade-off between exact copying and randomization by manipulating average sequence length.The paper describes limiting each tree’s training data to decrease average sequence length and extending sequences to increase it.Source: §I · p. 1
II. METHODOLOGY
The methodology section introduces a training-generation process and points readers to its block diagram. A Python implementation is available in a GitHub repository.
II. METHODOLOGY
- The methodology section presents an overview of the algorithm’s learning and generation process.The overview is identified as Fig. 1, a block diagram of the training-generation process.Source: §II · p. 1, Figure 1 · p. 1
- Fig. 1 is identified as a block diagram of the training-generation process.The supplied passage identifies the figure’s subject but does not describe its individual components.Source: Figure 1 · p. 1
- A Python code implementation is available in the cited GitHub repository.Source: §II · p. 1
A. Training Data and Preprocessing
The study uses annotated piano performances from MAESTRO and preprocesses MIDI pitches as relative intervals so the algorithm processes interval structure rather than specific pitch frequencies.
A. Training Data and Preprocessing
- The study uses about 200 hours of annotated piano performances from Google-magenta’s MAESTRO database.Only MIDI pitch data is used; velocity and Note-On/Off events are ignored.Source: A. Training Data and Preprocessing · p. 1
- A delta transform converts absolute note values into differences between consecutive notes.The resulting sequence represents relative intervals rather than absolute pitches.Source: A. Training Data and Preprocessing · p. 1
- The transform is applied to accentuate repeating patterns across compositions that vary in octave and key.The LZ algorithm therefore processes interval structure rather than specific pitch frequencies.Source: A. Training Data and Preprocessing · p. 1
B. Building the LZ tree
The paper builds an LZ78 dictionary as a tree and adjusts sequence length through branch extension and limits on the data used to train each tree.
B. Building the LZ tree
- LZ78 stores seen sequences in a dictionary, represented here as a tree.The algorithm begins at an empty tree’s root and checks whether the next symbol exists among the current node’s children.Source: B. Building the LZ tree · p. 1
- When a symbol exists, the algorithm increments the child node’s frequency and moves the pointer to that node.Source: B. Building the LZ tree · p. 1
- When a symbol is new, the algorithm adds a child node for it and returns the pointer to the root.Source: B. Building the LZ tree · p. 1
- The “steps” parameter delays the return to the root after sequence extension, increasing average sequence length.This departs from regular LZ practice, which returns to the root after extending a sequence by one symbol.Source: B. Building the LZ tree · p. 1
- For a stationary ergodic process, the number of LZ phrases is described as a function of sample length and entropy rate.The passage defines c(n) as the number of phrases in a distinct parsing of a sample of length n, and H(x) as the process entropy rate.Source: B. Building the LZ tree · p. 2
- The tree-ensemble approach gives each tree a subset of the data to decrease the heuristic average sequence length, Ls.The heuristic is introduced after noting that average sequence length grows with n.Source: B. Building the LZ tree · p. 2
C. Generation
Generation repeatedly selects a tree and samples symbols according to their observed frequencies until a leaf is reached, then switches trees until the target number of symbols is generated.
C. Generation
- A tree is chosen uniformly, and symbols are sampled probabilistically according to how often each has appeared.Sampling continues until a leaf is reached, after which another tree is selected; the cycle repeats until the requested number of symbols is generated.Source: C. Generation · p. 2
- Each tree supplies a probabilistic symbol sequence that ends when generation reaches a leaf node.Source: C. Generation · p. 2
- Fig. 3 is labeled as an example of the generation process.Source: Figure 3 · p. 2
III. EXPERIMENTS
The experiments examine two methods for controlling average sequence length.
III. EXPERIMENTS
- The experiments try two different methods to control Ls.Source: §III · p. 2
A. Limiting Training Data
The section examines limiting training data per tree as a way to control average sequence length; empirically, reducing each tree’s data decreases that length.
A. Limiting Training Data
- Limiting the data assigned to each tree is used to control average sequence length.Source: A. Limiting Training Data · p. 2
- As each tree receives less training data, average sequence length decreases.Fig. 4 plots average sequence length against the number of songs the tree was trained on.Source: A. Limiting Training Data · p. 2, Figure 4 · p. 2
B. Choosing the Number of Steps
The section tests the steps parameter as a way to increase average sequence length; the reported result is that increasing steps does so.
B. Choosing the Number of Steps
- Increasing steps increases average sequence length.Fig. 5 plots average sequence length as a function of steps.Source: B. Choosing the Number of Steps · p. 2, Figure 5 · p. 2
- The steps parameter sets how many steps occur before the algorithm returns to the root, thereby increasing average sequence length.Source: B. Choosing the Number of Steps · p. 2
C. Qualitative Evaluation
Listening to generated songs reveals rigid rhythm because training omitted rhythm, while greater sequence length is associated with musical coherence that may come from copying training material. The authors propose plagiarism detection to assess uniqueness.
C. Qualitative Evaluation
- Examples of songs generated with different parameters are available in the GitHub repository.Source: C. Qualitative Evaluation · p. 2
- Larger Ls values produce samples with established chord progressions and consistent tone, but this coherence often stems from verbatim copying.The authors propose using plagiarism detection to test song uniqueness.Source: C. Qualitative Evaluation · p. 2
- Omitting rhythm during training yields a strict beat and a perceptually robotic quality.The authors identify encoding expressive timing features as a future need.Source: C. Qualitative Evaluation · p. 2
IV. CONCLUSIONS
The algorithm controls the balance between copying and randomization with low complexity. Future work will explore different universal compression algorithms and ways to improve song structure.
IV. CONCLUSIONS
- The algorithm controls the copying–randomization balance with low complexity.Source: §IV · p. 2
- Future work will explore different universal compression algorithms.Source: §IV · p. 2
- Future work will seek to improve song structure through preprocessing and a different generation stage.Source: §IV · p. 2