PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Potential of Simple Framewise Approaches to Piano Transcription

Rainer Kelz, Matthias Dorfer, Filip Korzeniowski, Sebastian Böck, Andreas Arzt, Gerhard Widmer

arXiv:1612.05153v1cs.SDcs.LG

TL;DR

Piano transcription with neural networks is complicated by choices of input representation, architecture, optimization, and regularization. The paper systematically studies these choices and develops simple framewise acoustic models, finding that they outperform the published state of the art on MAPS and can serve as a new baseline. The approach remains constrained by the computational cost of exhaustive hyperparameter search and by its limited-budget selection assumption.

  • Problem

    Neural-network piano transcription requires selecting among many input representations and hyperparameters, creating a difficult model-design problem.

    Source: §1 · p. 1

  • Method

    The paper systematically evaluates input representations, uses classifier performance to assess suitability, and develops increasingly powerful framewise acoustic architectures with tuned training and regularization.

    Source: §1 · p. 1, §2.2 · p. 2, §2.4 · p. 2

  • Results

    Simple framewise networks outperform the current published state of the art on the MAPS dataset without complex post-processing.

    Source: §6 · p. 6

  • Takeaways & Limitations

    Straightforward framewise transcription networks are proposed as a new state-of-the-art baseline for MAPS.

    Source: §6 · p. 6

    Exact exhaustive selection of input representations and model classes is left for future work because it would require roughly 800–900 days of computation, and the selection strategy assumes overfitting can be reliably mitigated.

    Source: §2.2 · p. 2, §5 · p. 5

Abstract

from arXiv · show

In an attempt at exploring the limitations of simple approaches to the task of piano transcription (as usually defined in MIR), we conduct an in-depth analysis of neural network-based framewise transcription. We systematically compare different popular input representations for transcription systems to determine the ones most suitable for use with neural networks. Exploiting recent advances in training techniques and new regularizers, and taking into account hyper-parameter tuning, we show that it is possible, by simple bottom-up frame-wise processing, to obtain a piano transcriber that outperforms the current published state of the art on the publicly available MAPS dataset -- without any complex post-processing steps. Thus, we propose this simple approach as a new baseline for this dataset, for future transcription research to build on and improve.

1. INTRODUCTION

Neural networks offer strong performance for audio and music tasks, but transcription requires choices about both hyperparameters and input representation. This work uses a mixed strategy to identify suitable representations and architectures for framewise piano transcription while focusing on acoustic models.

1. INTRODUCTION

  • Neural networks can outperform hand-crafted audio pipelines, but their use requires extensive hyperparameter and input-representation choices.Relevant choices include architecture, optimization, regularization, spectrogram variants, constant-Q transforms, and raw audio.Source: §1 · p. 1
  • The study systematically selects an input representation, then uses expert-guided architecture development for framewise piano transcription.This combines systematic representation search with manual hyperparameter selection rather than relying exclusively on either manual or automatic optimization.Source: §1 · p. 1
  • The analysis focuses on acoustic models and omits complementary musical language models, although prior work reports that language-model systems perform better by a small margin [26].The comparison concerns neural architectures for monaural framewise piano-note transcription.Source: §1 · p. 1

2. INPUT, METHODS AND PARAMETERS

The paper compares spectrogram and constant-Q input representations, defines suitability through performance with simple classifiers, and then examines higher-capacity models under a limited computational budget. Its representation search varies spectrogram parameters while using model performance as the selection criterion.

2.1 Input Representation

  • The study compares four input representations—S, LS, LM, and CQT—while varying sampling rate, bands per octave, circular shifts, zero padding, and filter normalization.Spectrogram magnitudes are rescaled to [0, 1], and Table 1 specifies which parameters vary for each input type.Source: §2.1 · pp. 1–2, Table 1 · p. 2

2.2 Model Class and Suitability

  • A model class fixes the number and widths of layers and their nonlinearities, whereas an individual model is determined by its weights.The paper uses this distinction to organize comparisons among classifier architectures.Source: §2.2 · p. 2
  • Input-representation suitability is defined by the performance of a simple classifier given exactly that representation.The procedure first uses low-variance logistic regression and then a one-hidden-layer shallow net to test whether conclusions transfer to a higher-capacity model class.Source: §2.2 · p. 2
  • The representation-first strategy assumes overfitting can be reliably mitigated and is presented as appropriate under a limited computational budget.The paper frames this as the best available procedure under that assumption and budget constraint.Source: §2.2 · p. 2

2.3 Nonlinearities and Initialization

The section motivates ReLU nonlinearities through their gradient and convergence behavior, then emphasizes initialization choices for stable training, especially with logistic outputs. The networks therefore use ReLUs throughout with a uniform initialization scale of ±2.

Nonlinearities

  • ReLUs provide better-behaved gradients and faster convergence because they do not saturate.The passage contrasts ReLUs with saturating nonlinearities and cites [15].Source: §2.3 · p. 2

Initialization

  • Initialization scale depends on the nonlinearity and the number of weights contributing to each activation.Proper initialization is especially important when training networks with more than one hidden layer [31].Source: §2.3 · p. 2
  • The transcription networks use ReLUs throughout because their logistic output layer is prone to saturation effects.This design choice links the hidden-layer nonlinearity to the behavior of the output layer.Source: §2.3 · p. 2
  • The ReLU networks are initialized with a uniform distribution having a scale of ±2.The stated scale follows the recommendation in.Source: §2.3 · p. 2

2.4 Weight Decay

Weight decay regularizes the network through L1 and L2 penalties, while dropout randomly removes hidden activations to reduce co-adaptation and overfitting. These mechanisms impose different sparsity and robustness effects on the model.

2.4 Weight Decay

  • Weight decay adds L1 or L2 penalties to the training cost, with λ controlling regularization strength; combining both yields Elastic Net.L2 regularization can play a role similar to early stopping, whereas L1 tends to produce sparser weights [1].Source: §2.4 · p. 2

2.5 Dropout

  • Dropout randomly zeroes hidden-layer activations during training, preventing units from co-adapting and improving robustness, generalization, and resistance to overfitting.It can also be interpreted as averaging exponentially many models.Source: §2.5 · p. 2

2.6 Batch Normalization

Batch normalization stabilizes layer activations during training, while the examined layer types provide complementary transformations, feature extraction, pooling, and invariance mechanisms.

2.6 Batch Normalization

  • Batch normalization enforces approximately zero-mean, unit-variance activations per mini-batch, limiting distribution drift during training.It also reduces saturation, eases adaptation in subsequent layers, accelerates training, and provides additional regularization.Source: §2.6 · pp. 2–3

2.7 Layer Types

  • The architectures use dense, convolutional, or mixed layer arrangements, with pooling, normalization, and dropout interleaved as applicable.These choices determine how inputs are transformed and regularized throughout the network.Source: §2.7 · p. 3
  • Dense layers apply an affine transformation followed by a nonlinearity, whereas convolutional layers produce feature maps from shared kernels.Convolutional layers use multiple kernels, each followed by the same nonlinearity.Source: §2.7 · p. 3
  • Max pooling selects the largest activation in local time-frequency neighborhoods, providing limited translational invariance and robustness to small tuning changes.Global average pooling instead computes the mean value of feature maps in all-convolutional networks.Source: §2.7 · p. 3

2.8 Architectures

The study compares dense, convolutional, and mixed architectures for framewise piano transcription, using 88 sigmoid outputs trained with element-wise binary crossentropy and thresholded predictions.

2.8 Architectures

  • Architectures vary between all-dense, all-convolutional, and mixed networks, with layer widths, depths, filters, and regularization placements requiring selection.Convolutional designs additionally vary kernel counts and temporal and frequency extents.Source: §2.8 · p. 3

2.9 Networks for Framewise Polyphonic Piano Transcription

  • Each network outputs 88 logistic-function activations, corresponding to playable notes on most modern pianos.The logistic outputs lie in the interval [0, 1].Source: §2.9 · p. 3
  • Training minimizes frame- and element-wise binary crossentropy between the network output vector and the ground-truth vector at each time.The overall training loss is the mean over the training set.Source: §2.9 · p. 3
  • For performance measurement, network outputs are thresholded at 0.5 to produce binary note predictions.This thresholding is applied after the continuous logistic outputs are generated.Source: §2.9 · p. 3

2.10 Optimization

Optimization uses gradient-based updates, with mini-batch stochastic gradient descent replacing full-training-set gradients and extensions adapting smoothing or learning rates.

2.10 Optimization

  • The basic weight update takes a small step of length α in the direction of steepest descent to reduce the loss.The optimization framework also permits a learning-rate schedule over time.Source: §2.10 · p. 3
  • Mini-batch stochastic gradient descent approximates the costly full-training-set gradient using an i.i.d. sample of size M much smaller than T.Momentum, Nesterov momentum, and Adam respectively smooth estimates, correct missteps, or adapt the learning rate dynamically.Source: §2.10 · p. 3

3. DATASET AND MEASURES

Experiments use the MAPS dataset and framewise validation measures, with two fold configurations that differ in split sizes and realism of the test recordings.

3. DATASET AND MEASURES

  • MAPS provides MIDI-aligned classical pieces rendered with high-quality piano samples and real upright Disklavier recordings, yielding clean annotations with little label noise.Framewise performance measures are computed on the validation set.Source: §3 · p. 3
  • For performance comparisons, the validation set uses framewise measures, including a supplied expression involving TP[t] and FN[t].Source: §3 · p. 3
  • Configuration I uses a 173-43-54 train-validation-test division, with validation tracks randomly removed from training and the published test sets retained.The validation set contains 43 tracks, differing from the 26-track validation set used in [26].Source: §3 · pp. 3–4
  • Configuration II uses a 180-30-60 division and restricts testing to real-piano recordings, creating a more realistic transcription setting.Its exact splits were selected using the same methodology because the original configuration-II splits were unpublished.Source: §3 · p. 4

4. ANALYSIS OF RELATIVE HYPER-PARAMETER IMPORTANCE

The study separates input-representation effects from broader hyper-parameter effects, finding that spectrogram type dominates representation variability while learning rate dominates across the larger shallow-net search.

4.1 Types of Representation

  • Spectrogram Type is the most important representation hyper-parameter for both model classes, followed by pairwise interactions.Table 2 reports the three most influential parameters and the percentage of variability attributed to them.Source: §4.1 · p. 4, Table 2 · p. 4
  • LM, using logarithmically spaced bins and logarithmically scaled magnitude, performs best for the shallow net despite mismatching logistic regression.The error bars represent standard deviation caused by the other varied parameters.Source: §4.1 · p. 4, Figure 1 · p. 4
  • The constant-Q transform underperforms relative to the tested alternatives for both model classes, an unexpected result requiring further investigation.Source: §4.1 · p. 4, Figure 1 · p. 4

4.2 Greater context

  • Learning Rate accounts for 47.11% of performance variability, compared with 5.28% for Spectrogram Type across 3000 random parameterizations.The broader analysis predicts marginalized performance from a random-forest model fitted to measured runs.Source: §4.2 · p. 4
  • The broader hyper-parameter study varies optimizer, learning rate, momentum, scheduling, batch normalization, dropout, and L1/L2 penalties.The study samples these settings alongside representation choices and models their predicted marginalized performance.Source: §4.2 · pp. 4–5, Table 3 · p. 5

5. STATE OF THE ART MODELS

The study evaluates increasingly powerful framewise piano-transcription models, using selected representations, expert tuning, and simple architectures. Convolutional models are compared with dense networks and prior hybrid systems on MAPS under standard and realistic splits.

Model classes

  • Convolutional models were evaluated alongside dense and all-convolutional alternatives, with the architectures detailed in Table 4.The model classes were DNN, ConvNet, and AllConv; the paper identifies AllConv as a first proposal for framewise piano transcription.Source: §5 · p. 5, Table 4 · p. 5

Model selection

  • Training and architecture choices were selected by a human expert after automatic hyper-parameter search for these larger models was deferred.The authors estimate roughly 800–900 days of compute to determine the best input representation exactly, given the model and fold combinations.Source: §5 · p. 5

Training

  • Batch normalization, dropout, and simple optimization produced low validation error quickly, while learning-rate initialization and scheduling remained important choices.The reported practical strategy searched learning rates logarithmically and then reduced them after validation error flattened.Source: §5 · p. 5

Evaluation

  • The final evaluation compares simple, largely unsmoothed systems with hybrid systems on both standard and realistic train-test splits.All systems use the same standard splits and additional realistic splits modeled after configurations I and II in [26].Source: §5 · pp. 5–6, Table 5 · p. 6, Table 6 · p. 6

Results

  • Table 5 reports four-fold MAPS test performance for configuration I, while Table 6 reports performance for configuration II.The supplied passages identify the evaluation scope and averaging scheme but do not provide the table cell values.Source: Table 5 · p. 6, Table 6 · p. 6

6. CONCLUSION

The authors conclude that input representation and hyper-parameter tuning, especially learning-rate choices, are important for framewise piano transcription. They argue that convolutional networks outperform dense siblings, while all-convolutional networks approach mixed networks with fewer parameters.

Conclusion

  • The study argues that input representation and hyper-parameter tuning, especially learning rate and schedule, are important for transcription performance.These conclusions are presented as consequences of the reported experiments.Source: §6 · p. 6
  • Convolutional networks are argued to have an advantage over deep dense networks because they use a context window.The conclusion attributes this advantage specifically to the convolutional models’ temporal context.Source: §6 · p. 6
  • All-convolutional networks are reported to perform nearly as well as mixed networks while using far fewer parameters.The paper proposes the straightforward framewise networks as a new state-of-the-art baseline for MAPS.Source: §6 · p. 6

Related papers

Loading 1612.05153v1…