PaperlayerView exact revision on arXiv

Source-linked AI summary

auDeep: Unsupervised Learning of Representations from Audio with Deep Recurrent Neural Networks

Michael Freitag, Shahin Amiriparian, Sergey Pugachevskiy, Nicholas Cummins, Björn Schuller

arXiv:1712.04382v2cs.SDeess.AS

TL;DR

Handcrafted audio features are common, while deep networks face challenges with sequential inputs of variable length. auDeep uses recurrent sequence-to-sequence autoencoders to learn unsupervised audio representations, and its features match or outperform several approaches on evaluated classification datasets, with SoundNet stronger on ESC-10 and ESC-50 under a different pretraining setup.

  • Problem

    Audio processing commonly uses handcrafted features, and sequential audio challenges deep networks that typically require fixed-dimensional inputs.

    Source: §1 · p. 1

  • Method

    auDeep learns audio representations with recurrent sequence-to-sequence autoencoders trained on spectrogram sequences, using the trained model's fully connected layer activations as features.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    auDeep matches or outperforms several named approaches on ESC-10, ESC-50, and GTZAN, while SoundNet performs better on ESC-10 and ESC-50.

    Source: §4 · p. 4

  • Takeaways & Limitations

    The authors conclude that auDeep is an open-source toolkit with competitive performance on various audio classification tasks.

    Source: §5 · p. 4

    The SoundNet comparison is not straightforward because SoundNet was pretrained on an external corpus of over 2 million videos, whereas auDeep used only ESC-10 and ESC-50 data.

    Source: §4 · p. 4

Abstract

from arXiv · show

auDeep is a Python toolkit for deep unsupervised representation learning from acoustic data. It is based on a recurrent sequence to sequence autoencoder approach which can learn representations of time series data by taking into account their temporal dynamics. We provide an extensive command line interface in addition to a Python API for users and developers, both of which are comprehensively documented and publicly available at https://github.com/auDeep/auDeep. Experimental results indicate that auDeep features are competitive with state-of-the art audio classification.

1. Introduction

Deep representation learning offers an alternative to handcrafted audio features, while sequential audio poses fixed-dimensional input challenges for deep networks. auDeep is presented as a toolkit for unsupervised representation learning from acoustic data, using recurrent sequence-to-sequence autoencoders and offering both a Python API and command-line interface.

1. Introduction

  • Deep representation learning has drawn attention as an alternative to handcrafted features, with reported advantages on tasks including speech recognition and music transcription (Bengio et al., 2013; Amiriparian et al., 2016).Audio sequences pose a challenge because deep neural networks typically require fixed-dimensional inputs.Source: §1 · p. 1
  • auDeep is presented as a TensorFlow-based Python toolkit for deep unsupervised representation learning from acoustic data.The supplied passage ends before completing the description of how this is achieved.Source: §1 · p. 1
  • The toolkit uses recurrent sequence-to-sequence autoencoders and provides both a Python API and an extensive command-line interface.The introduction points to Section 3.1 for a description of the procedure.Source: §1 · p. 2

2. Recurrent Sequence to Sequence Autoencoders

The method learns fixed-length sequence representations with a recurrent encoder-decoder autoencoder, trained to reconstruct its input. For acoustic representation learning, it trains on spectrogram sequences and uses the fully connected layer's activations as features.

2. Recurrent Sequence to Sequence Autoencoders

  • The encoder summarizes an input sequence in its final hidden state; a fully connected layer initializes the decoder, which reconstructs the sequence.Training minimizes root mean squared error between the input and reconstruction, with the previous expected decoder output fed back at each step to accelerate convergence.Source: §2 · p. 2
  • After training, activations of the fully connected layer serve as the input sequence's representation.The encoder and decoder are multilayered RNNs.Source: §2 · p. 2
  • For acoustic data, the autoencoders use LSTM cells or gated recurrent units trained on spectrograms treated as sequences of frequency vectors.The approach is fully unsupervised and accounts for sequence temporal dynamics.Source: §2 · p. 2

3. System Overview

auDeep combines a sequence-to-sequence autoencoder core with audio-focused feature-learning functionality. Its workflow extracts features from spectrograms for optional classifier evaluation or export, while the design supports reusable models and flexible data and compute settings.

3. System Overview

  • The system pairs a domain-independent sequence-to-sequence autoencoder implementation with additional audio representation-learning functionality.The core implementation is described as high-performing.Source: §3 · p. 2

3.1 Practical Usage

  • The practical workflow extracts spectrograms from raw audio, trains an autoencoder, and extracts a feature vector for each instance; features and metadata can then be exported.When labels are available, a classifier can be trained and evaluated on the features.Source: §3.1 · pp. 2–3

3.2 Design

  • The design provides reusable TensorFlow autoencoder checkpoints, unified handling of diverse datasets, and conversion between NetCDF and CSV/ARFF formats.The Python API and command-line interface offer fine-grained control; the system runs on CPU and uses GPU acceleration when available.Source: §3.2 · p. 3

4. Experiments

Across four audio classification tasks, auDeep's features match or outperform several named baseline and state-of-the-art approaches, though SoundNet performs better on ESC-10 and ESC-50 under a different pretraining setup. The experiments evaluate fused autoencoder representations with an MLP across acoustic scene, environmental sound, and music genre classification.

4. Experiments

  • The evaluation covers acoustic scene classification on TUT AS 2017, environmental sound classification on ESC-10 and ESC-50, and music genre classification on GTZAN.The TUT AS 2017 evaluation uses its development partition.Source: §4 · p. 4
  • The experiments fuse representations from multiple autoencoder configurations and evaluate them with an MLP using baseline-matched cross-validation on TUT AS 2017, ESC-10, and ESC-50.For GTZAN, the authors randomly generated five stratified folds because no predefined cross-validation setup was available; they distribute experimental code and parameter choices for reproducibility.Source: §4 · p. 4
  • auDeep matches or outperforms a CNN and a representation-learning approach on ESC-10 and ESC-50, and a sparse-coding approach on GTZAN; SoundNet performs better on ESC-10 and ESC-50.The SoundNet comparison is not straightforward: it was pretrained on an external corpus of over 2 million videos, whereas auDeep used only ESC-10 and ESC-50 data.Source: §4 · p. 4

5. Conclusions

auDeep is an easy-to-use, open-source toolkit for unsupervised audio representation learning, with competitive performance on various audio classification tasks. Its long-term goal is to become a general-purpose deep audio toolkit through additional algorithms and regression support.

5. Conclusions

  • auDeep is an easy-to-use, open-source toolkit for deep unsupervised audio representation learning, with competitive performance on various audio classification tasks.Source: §5 · p. 4
  • The long-term goal is to grow auDeep into a general-purpose deep audio toolkit by integrating other deep representation learning algorithms.Examples include conditional variational sequence to sequence autoencoders and deep convolutional generative adversarial networks.Source: §5 · p. 4
  • The planned feature-learning extension includes regression tasks on continuously labelled data.Source: §5 · p. 4
Loading 1712.04382v2…