PaperlayerView exact revision on arXiv

Source-linked AI summary

Weakly Supervised Tabla Stroke Transcription via an Adaptive Dynamic Rhythm Language Model (ADRM)

Rahul Bapusaheb Kodag, Vipul Arora

arXiv:2608.30314v1eess.AS

TL;DR

TST lacks scalable onset-level annotations because existing supervised systems require precise expert alignment. The paper combines CTC acoustic modelling with ADRM lattice rescoring and releases real and synthetic tabla datasets; ADRM consistently reduces SER versus acoustic-only decoding, although performance varies with data and recording conditions.

  • Problem

    Existing TST approaches rely on costly onset-level annotations, motivating weakly supervised learning from ordered stroke sequences.

  • Method

    The framework combines a CTC acoustic model with ADRM, which uses global rhythmic priors and local adaptive transitions to rescore candidate stroke lattices.

  • Results

    ADRM reduces SER relative to acoustic-only decoding by 18.9–35.4% across four datasets and outperforms the fully supervised S&C baseline on TSD and HMR.

  • Takeaways & Limitations

    Symbolic rhythmic regularities and local adaptive transition information improve weakly supervised tabla transcription across diverse datasets.

  • Takeaways & Limitations

    Performance varies with limited original training data, interference from vocals and accompaniment, and substantial tempo variation within the same t¯ala.

Abstract

from arXiv · show

Tabla Stroke Transcription (TST) is central to the analysis of rhythmic structure in Hindustani music, yet it remains challenging due to complex and dynamic rhythmic organization and the scarcity of strongly annotated data. Existing approaches largely rely on fully supervised learning with onset-level annotations, which are costly and impractical at scale. This work addresses TST in a weakly supervised setting, using only symbolic stroke sequences without temporal alignment of onsets. We propose a framework that combines a Connectionist Temporal Classification (CTC)-based acoustic model with a sequence-level rhythmic language model for rescoring, similar to that used in automatic speech recognition. The acoustic model produces a decoding lattice, which is refined using an Adaptive Dynamic Rhythm Language Model (ADRM) that combines $t\bar{a}la$-conditioned symbolic rhythmic regularities with local stroke dynamics. Moreover, we release a new performance-recorded tabla dataset, named \emph{Tabla Improvisation Dataset}, along with a complementary synthetic dataset for sequence-level weakly supervised TST. Experiments demonstrate consistent and substantial reductions in stroke error rates with ADRM compared to those with acoustic-only decoding, confirming the benefit of incorporating symbolic rhythmic regularities during lattice rescoring for accurate transcription.

I. INTRODUCTION

Tabla Stroke Transcription is important for analyzing Hindustani rhythmic structures, but existing onset-supervised approaches require costly expert annotations. This work frames TST as weakly supervised sequence modelling and introduces datasets and ADRM-based rhythmic rescoring.

  • Tabla Stroke Transcription supports analysis of cyclic t¯ala structures and pedagogical and archival applications.
  • Fully supervised TST systems require precise onset-level annotations that are costly for tabla music and demand expert alignment.
  • Weak supervision uses ordered stroke sequences without temporal alignment, making scalable data collection feasible within t¯ala’s metrical framework.
  • The work formulates TST as weakly supervised sequence modelling and proposes ADRM to incorporate rhythmic information during CTC lattice decoding.
  • The released datasets include 133 minutes of real-world recordings across six t¯alas and a 121-minute synthetic dataset covering the same structures.
  • Prior tabla research addressed onset-based segmentation, stroke classification, and unsupervised representations, while related repetition-based methods require separate formulations for Hindustani tabla.

B. Weakly Supervised Sequence Learning in ASR

The framework adapts alignment-free sequence learning and language-model rescoring from ASR to weakly supervised tabla transcription. CTC generates acoustically plausible stroke lattices, which ADRM refines using global rhythmic regularities and local transitions.

  • ASR demonstrates that CTC can train sequence models without frame-level alignment, while language-model rescoring improves acoustically generated hypotheses.
  • Existing music language-model research mainly addresses strongly supervised Western-instrument transcription, leaving non-Western applications limited.
  • The acoustic model converts tabla audio into discrete stroke sequences and uses CTC loss to marginalize over valid frame-level alignments.
  • Beam search produces a decoding lattice of acoustically plausible stroke sequences for subsequent rhythmic rescoring.
  • ADRM assigns rhythm-aware scores to lattice paths using a t¯ala-conditioned global prior and a path-dependent local adaptive model.

1) Global Rhythmic Prior Model:

The global rhythmic prior models t¯ala-conditioned symbolic stroke regularities while allowing performance variation. It uses either analytical n-gram statistics or a neural next-stroke model, with latent t¯ala inference for multi-t¯ala recordings.

  • The global prior captures statistical stroke dependencies conditioned on t¯ala rather than enforcing a fixed stroke pattern, count, phase, or canonical thek¯a.
  • The Bayesian global prior uses t¯ala-conditioned n-gram transition statistics with a recent stroke window to reduce sparsity from long histories.
  • For multi-t¯ala recordings, the decoder treats t¯ala as latent and marginalizes over possible t¯alas instead of selecting one in advance.
  • During decoding, the posterior over t¯alas is inferred from observed stroke-history windows and weights the corresponding t¯ala-specific priors.
  • A neural next-stroke language model provides an alternative global prior by learning symbolic sequence regularities implicitly rather than explicitly conditioning on t¯ala.

2) Local Adaptive Rhythm Model with Temporal Forgetting:

The local adaptive rhythm model captures short-term, performance-specific stroke dynamics that a global canonical model cannot represent. It updates a Dirichlet-parameterized transition model with exponentially weighted recent observations and combines it with the global prior.

  • The local adaptive model addresses local repetition and improvisation by updating transition statistics from recently observed stroke sequences.
  • It uses a Dirichlet-parameterized first-order Markov model with exponentially weighted updates for interpretable transition probabilities.
  • Temporal Adaptation with Exponential Forgetting: After each observed transition, exponential forgetting keeps the model responsive to local rhythmic variation while preventing unbounded count growth.
  • Temporal Adaptation with Exponential Forgetting: Although its parameters encode past history, the predictive distribution remains first-order Markov and depends only on the immediately preceding stroke.
  • Initialization of Dirichlet Parameters: The model initializes its Dirichlet parameters from global training-set transition counts and uses uniform initialization for the sequence-start symbol.
  • Global and local rhythmic models are linearly interpolated to obtain the next-stroke probability and sequence-level rescoring score.

C. Lattice-Based Rhythmic Rescoring

The framework rescored CTC lattice paths with rhythmic information while preserving path-specific stroke history. Its adaptive weighting gives rhythmic evidence more influence when acoustic predictions are ambiguous.

  • Lattice-based rhythmic rescoring: CTC lattice paths can merge at one node despite differing stroke histories, making history-dependent rhythmic scoring ambiguous.Different paths may require different rhythmic scores for later extensions.
  • Lattice-based rhythmic rescoring: State expansion preserves path-specific rhythmic context by treating histories reaching the same acoustic node as distinct decoding states.Each expanded state augments a lattice node with Dirichlet parameters encoding observed stroke transitions.
  • Adaptive rhythm weighting: Each outgoing arc combines its acoustic log-score with a rhythmic probability weighted by the adaptive coefficient β_k.After rescoring, the state’s Dirichlet parameters are updated and the expanded successor is inserted for further decoding.
  • Adaptive rhythm weighting: Acoustic confidence is estimated from entropy over softmax-normalized competing arc scores, with higher confidence indicating dominance by a single candidate.The confidence measure distinguishes clear predictions from ambiguity among stroke hypotheses.
  • Adaptive rhythm weighting: β_k = β(1 − C_k), so rhythmic information contributes more under ambiguous acoustic evidence and less under reliable evidence.This makes the rhythmic contribution depend on the local reliability of the acoustic model.

3) Search Strategy and Beam Pruning:

Search explores expanded decoding states in best-first order and uses beam pruning to retain only states close to the current best score. The procedure preserves rhythmic context while controlling computational growth.

  • Search strategy: Expanded states are explored by best-first traversal, ordered by their accumulated log-scores in a priority queue.The queue is initialized with the start state and successor states are inserted as paths expand.
  • Beam pruning: Beam pruning retains only states whose scores are within Δ_beam of the maximum queue score.The rule removes hypotheses that fall outside the specified score range from the best current state.
  • Search strategy: The optimal stroke sequence is recovered by backtracking from the best final expanded state after search.The expanded states preserve the path-specific rhythmic context needed for reconstruction.

IV. CURATED TABLA DATASET

The paper curates complementary real and synthetic tabla datasets to address the scarcity of annotated material for sequence-level weakly supervised transcription. The datasets support rhythmic modelling across multiple t̄ālas.

  • Dataset motivation: Existing Indian percussion datasets are limited in size, scope, and coverage of extended tabla performances, motivating two complementary resources.The proposed resources are the performance-recorded Tabla Improvisation Dataset and the Synthetic Tabla Improvisation Dataset.
  • Sequence-level TST: The dataset design targets sequence-level weakly supervised TST using symbolic stroke sequences and t̄āla-structured recordings.The algorithm inputs a CTC lattice, rhythmic priors, transition counts, and hyperparameters, then returns an optimal stroke sequence.
  • Rhythmic rescoring: The rescoring algorithm computes adaptive and global rhythmic probabilities, combines them, and applies the resulting scores to outgoing lattice arcs.The procedure uses the CTC lattice and updates expanded states while processing candidate strokes.
  • Decoding output: The framework returns the highest-scoring final expanded state and reconstructs its stroke sequence by backtracking.Beam pruning is applied during expansion to retain computationally tractable hypotheses.

A. Tabla Improvisation Dataset (TID)

The Tabla Improvisation Dataset combines performance-recorded tabla audio with symbolic annotations and is paired with a synthetic corpus for controlled evaluation. Together, the resources cover multiple rhythmic structures and performance variation.

  • Tabla Improvisation Dataset: The performance-recorded dataset contains approximately 133 minutes across 23 recordings and six commonly used t̄ālas.Its recordings include canonical theka patterns, performer-specific variations, and improvisatory passages.
  • Tabla Improvisation Dataset: The recordings use a 30-stroke vocabulary spanning basic articulations and compound dayan–bayan strokes.Audio was recorded in stereo at 44.1 kHz, with performances spanning 110 to 240 BPM.
  • Synthetic dataset: The complementary synthetic dataset is generated from expert-defined symbolic sequences and isolated stroke recordings while preserving underlying t̄āla structure.Its sequences include canonical theka patterns, valid improvisatory variations, and fillers.
  • Evaluation setting: The two datasets provide acoustic realism and controlled rhythmic diversity for systematic evaluation across multiple t̄ālas.The experimental setup compares acoustic-only decoding with ADRM-based rhythmic rescoring and includes supervised baselines and ablations.
  • Evaluation setting: Evaluation uses the curated datasets and public benchmarks while discarding onset information for the weakly supervised setting.Training, validation, and test partitions are created for each dataset, with augmentation applied to training data except for HMR.

B. Acoustic Model and Training Configuration

The system trains a CTC-based acoustic model to generate decoding lattices, then applies rhythmic rescoring and evaluates against common symbolic-sequence baselines.

  • Acoustic model: The C-TDNN-F acoustic model uses log-Mel spectrograms and 12 layers with bottleneck projections, temporal convolutions, residual connections, normalization, and dropout.Features use a 46.4 ms window, 10 ms shift, and 128 Mel filter banks.
  • Training: Training uses the CTC objective with Adam at a 0.001 learning rate for 100 epochs and gradient clipping.
  • Rhythmic decoding: The framework searches rhythmic memory and acoustic–rhythmic weighting hyperparameters independently for each dataset.The search varies ρ, β, and λ across predefined grids.
  • Decoding: Beam search generates full CTC lattices, which are decoded acoustically or rescored by expanding states with path-specific rhythmic context.Acoustic-only decoding uses Viterbi scoring over the acoustic lattice.
  • Rhythmic model: The NN-LM learns 64-dimensional stroke embeddings and processes them with a two-layer bidirectional GRU for next-stroke prediction.Each GRU direction has 256 hidden units.
  • Evaluation: All baselines are evaluated using the same symbolic representation and Stroke Error Rate, including supervised, segmentation-based, and unsupervised systems.
  • Temporal analysis: Temporal alignment is assessed by aligning ADRM outputs to CTC posteriors and comparing recovered onsets with ground truth across four tolerance windows.The windows are 50 ms, 100 ms, 200 ms, and 500 ms.

H. Ablation Studies

The experiments show that rhythmic rescoring consistently improves transcription, while ablations examine data scale, global–local interpolation, and adaptive weighting under dataset-specific constraints.

  • Main results: 18.9–35.4% relative SER reductions are obtained with Acoustic Model + ADRM across four datasets versus acoustic-only decoding.The selected settings are ρ = 0.03, β = 0.6, and λ = 0.4.
  • Prior comparison: The Bayesian prior performs best on TSD, TID, and TID-S, whereas the neural prior is slightly better on HMR.
  • Baseline comparison: The framework outperforms the fully supervised S&C baseline on TSD and HMR, while unsupervised stroke clustering has the highest SER on all four datasets.The paper attributes the advantage over S&C to sequence-based CTC predictions using preceding strokes and temporal context.
  • Ablations: Combining global and local rhythmic components performs better than either alone, with best performance at λ = 0.4.The Bayesian prior is more effective with limited symbolic data, while the neural prior becomes more competitive as training data increase.
  • Ablations: Confidence-modulated rhythmic weighting provides small but consistent improvements over fixed-weight decoding across datasets.
  • Temporal alignment: Temporal alignment improves with wider tolerance windows, but accurate onset localization remains challenging without explicit temporal modeling.

A. Analysis of Global and Adaptive Rhythmic Modelling

The analysis shows that global and adaptive rhythmic models provide complementary information: the global prior stabilizes predictions, while the adaptive model responds to recently observed local patterns.

  • Model behavior: The probability trajectories expose distinct roles for the global rhythmic prior and the local adaptive model during a representative five-cycle sequence.
  • Model behavior: The global prior decreases across increasingly varied phrases, whereas adaptive probabilities remain comparatively narrower and respond to local repetition.
  • Adaptive response: Repeated Tita Kata Gadi Gana phrases produce higher adaptive probabilities on the second occurrence because recent transitions influence subsequent predictions.
  • Global prior: The global prior remains largely unchanged across repeated phrases because it derives from t̄ala-conditioned statistics rather than recent stroke history.
  • Complementary roles: The global prior supplies stability, while the adaptive model supplies responsiveness to recurring local patterns.
  • Combined model: The combined model uses both components, and the lowest SER occurs at λ = 0.4.The figure trends agree with the interpolation ablation results.

B. Error Analysis

ADRM improves transcription chiefly by using rhythmic context to suppress inconsistent insertions and reorder acoustically plausible alternatives. Its benefits are strongest for insertion errors, while acoustic confusions and absent hypotheses remain difficult to correct.

  • Error patterns: Substitution errors dominate across datasets, with recurring confusions including Dhin→Dha in TID and Dhin→Dhi in TSD.HMR shows similar patterns plus confusions associated with singing voice and harmonic accompaniment.
  • Error patterns: Frequent strokes are more likely to be predicted, whereas less frequent stroke classes are more likely to be deleted.Inserted strokes generally belong to dominant labels in the corresponding datasets.
  • ADRM effects: Insertion errors decrease from 7.7% to 2.8% for TID, 5.7% to 3.0% for TSD, and 9.4% to 4.0% for HMR after ADRM rescoring.The adaptive model updates transition probabilities from decoded history, lowering probabilities for inconsistent continuations.
  • ADRM effects: ADRM yields smaller substitution improvements because acoustically similar strokes can retain comparable scores among competing lattice hypotheses.Deletion errors improve least because missing strokes often do not survive in the decoding lattice.
  • Limitations: Strong acoustic confusions and missing stroke hypotheses remain challenging because rescoring cannot reliably correct them.ADRM more effectively exploits rhythmic context for insertion and substitution errors than for absent hypotheses.
Loading 2608.30314v1…