Source-linked AI summary
CASM: Context-Aware Semi-Markov Post-Processor for Beat Tracking
Zhanhong He, Hanyu Meng, Yaolong Ju
TL;DR
Beat tracking needs a post-processor that balances local activation evidence with rhythmic regularity, while existing approaches can rely on preset constraints or miss ambiguity among competing periods. CASM adapts interval regularization to local periodicity and ambiguity; across the tested backbones and datasets, it improves reported accuracy and continuity measures and is less sensitive than DBN to calibration data.
Problem
Beat post-processing must balance local activations against temporal regularity, while predefined DBN constraints can require retuning and PLPDP confidence does not capture near-tied half- or double-tempo alternatives.
Source: §1 · p. 1Method
CASM is an offline semi-Markov decoder that selects activation-supported beats and adapts interval regularization to local period strength and ambiguity.
Source: §1 · p. 1, §2.1 · p. 2, §2.2 · p. 2Results
CASM improves every reported metric over Direct across three backbones on SMC and GTZAN, and its calibration results are less variable than DBN’s across fold combinations.
Source: §4.1 · p. 3, §4.4 · p. 4, §5 · p. 4Takeaways & Limitations
A single frozen configuration lets CASM improve accuracy and continuity across tested backbones without backbone retraining, and it remains above Direct at every calibration scale.
Source: §1 · p. 1, §4.4 · p. 4, §5 · p. 4CASM is offline and uses future activation context, so its performance is not directly comparable with causal online systems.
Source: §1 · p. 1
Abstract
from arXiv · showhide
In music beat tracking, model-predicted activations must be decoded into a discrete, musically coherent event sequence. Direct peak picking closely follows local evidence but can retain spurious peaks or miss weak beats. Dynamic Bayesian networks (DBNs), a widely used structured post-processor, improve sequence consistency under predefined global tempo, meter, and transition constraints, but their behavior can depend strongly on these settings. We introduce CASM, a context-aware semi-Markov decoder that instead conditions its temporal constraint on local activation evidence. CASM also accounts for ambiguity among competing periodic interpretations, including half- and double-tempo alternatives. Deterministic safeguards prevent implausible outputs and preserve beat-downbeat consistency. Applied to fixed activations from three neural beat trackers (BeatThis, MSCNN, and TCN), CASM improves temporal continuity while preserving event-level F1 across the GTZAN and SMC datasets, without backbone retraining or dataset-specific retuning. Further analysis shows that CASM is less sensitive than the DBN baseline to the composition of the calibration data.
1. INTRODUCTION
Beat post-processing must balance local activation evidence against rhythmic regularity: peak picking is local, while strong temporal constraints can suppress valid expressive-timing peaks. CASM adapts that balance to local periodicity and its ambiguity, unlike methods that rely on predefined constraints or a single favored period.
1. INTRODUCTION
- Direct peak picking follows local evidence without considering sequence periodicity, while post-processing combines that evidence with temporal regularity.Weak constraints can leave missed or spurious beats uncorrected; overly strong constraints can suppress valid peaks during expressive timing or rapid local tempo variation.Source: §1 · p. 1
- DBNs use preset tempo ranges, meter states, and transition probabilities, and may require per-track or corpus-specific retuning when those assumptions do not match a recording.Earlier methods also include global-tempo dynamic programming and CRFs that discourage abrupt IBI changes.Source: §1 · p. 1
- PLPDP conditions constraints on a locally estimated period and confidence, but its peak-height confidence does not capture a near-tied half- or double-tempo alternative.Strongly enforcing a period that is nearly tied with a metrical alternative risks committing to the wrong metrical level.Source: §1 · p. 1
- CASM uses local periodicity and competing-period ambiguity to strengthen regularity when cues are clear and let activations dominate when they are ambiguous.It is an offline, model-agnostic semi-Markov post-processor with deterministic safeguards and a beat-synchronous meter decoder.Source: §1 · p. 1, Figure 1 · p. 2
- With one fixed configuration, CASM improves every reported metric across three backbones on SMC and GTZAN without retraining or per-track or per-dataset BPM and meter retuning.The paper also presents CASM as an open-source Python package with an activation-to-event interface.Source: §1 · p. 1
2. METHODOLOGY
CASM selects a beat path over activation-supported peaks, scoring beat evidence against interval regularity informed by local period and its ambiguity. Its safeguards bound the resulting beat and downbeat sequences and keep decoding deterministic.
2.1. CASM overview
- CASM is an offline, model-agnostic decoder that adapts temporal regularization to local periodicity, relaxing it when competing periods make the evidence ambiguous.It operates on activations from a frozen tracker and uses a semi-Markov path to select beats among activation peaks.Source: §2.1 · p. 2
2.2. Semi-Markov beat path
- The semi-Markov path rewards beat evidence and penalizes rhythmically inconsistent intervals, with transitions directly between selected beats rather than tempo hidden states.The deterministic hand-specified decoder's state is the last selected beat; each transition scores the elapsed interval as a whole.Source: §2.2 · p. 2
2.3. Local period and period margin
- For each candidate, CASM estimates local period from normalized autocorrelation in a centered 8-second window and computes a margin that reflects competing-period ambiguity.A small margin marks nearby or half- or double-time alternatives; both period and margin are median-filtered over five candidates.Source: §2.3 · p. 2
2.4. Margin-conditioned duration cost
- The interval cost uses both endpoints’ geometric-mean period and margin; lower margins reduce penalty weight and widen timing tolerance.The Gaussian log-interval penalty is zero at the preferred interval and treats doubled and halved intervals alike; decreasing the margin relaxes the cost toward zero.Source: §2.4 · p. 2
2.5. Decoding, safeguards, and downbeats
- A Viterbi-style decoder backtracks the best beat path, but returns the Direct path if CASM’s beat count is below 0.85 or above 1.8 times Direct’s.A second dynamic program decodes downbeats on the selected beat grid and falls back to snapped Direct downbeat peaks when its F-measure is below 0.6 at ±70 ms, ensuring every output downbeat coincides with a beat.Source: §2.5 · p. 2
3. EXPERIMENTS
The experiments compare post-processors on identical frozen-backbone activations, using SMC and held-out GTZAN evaluations. The setup also freezes CASM’s calibration across tracks, backbones, and datasets and tests sensitivity to calibration-set composition.
3.1. Evaluation protocol
- Evaluation uses eight-fold cross-validation, out-of-fold SMC partitions, and held-out GTZAN, reporting F1 at ±70 ms plus CMLt and AMLt.The backbone checkpoints are trained on the other seven folds; SMC has beat but not downbeat annotations, while GTZAN is excluded from the cross-validation pool.Source: §3.1 · p. 3
3.2. Post-processors and tempo support
- All post-processors receive the same fixed activations, with offline methods compared against Direct, DBN, PLPDP, and CRF; DBN is tested with default and expanded tempo support.The standard DBN uses a 55–215-BPM range, while its 30–300-BPM variant changes only that range; CASM is also tested with its range restricted to 55–215 BPM.Source: §3.2 · p. 3, §3.1 · p. 3
3.3. CASM calibration
- CASM’s configuration is calibrated by grid search on SMC folds 1–7, then frozen for every track, backbone, and dataset; sensitivity tests evaluate 64 fold-subset calibration sets.Each configuration is evaluated on held-out SMC fold 0 and GTZAN, while DBN’s grid also varies tempo range, transition weight, observation weight, and threshold.Source: §3.3 · p. 3
4. RESULTS
Across fixed backbone activations, CASM improves beat-tracking metrics over direct peak picking, while component ablations identify the adaptive margin and count safeguard as important. CASM also shows less calibration-data sensitivity than DBN.
4.1. Comparison of post-processing methods
- CASM improves every reported metric over direct peak picking across all three backbones on fixed activations.The 30–300 BPM DBN gains continuity but degrades F1, and its results vary with tempo support; PLPDP and CRF offer isolated rather than consistent gains.Source: §4.1 · p. 3, Table 1 · p. 3
4.2. Ablation of CASM components
- The adaptive margin is the most critical component, and removing the count safeguard improves continuity but drops F1 below direct peak picking.Using the margin for only penalty weight or tolerance lowers F1 on both datasets and removes GTZAN gains; a single-endpoint context or restricted tempo range reduces gains.Source: §4.2 · p. 4, Table 2 · p. 4
4.3. Comparison with state-of-the-art systems
- CASM needs no retraining and uses the same parameters across backbones, while BeatThis-MDM improves every GTZAN metric but has lower SMC F1 and CMLt than BeatThis direct.The comparison also reports that systems with large pretrained frontends retain the best scores.Source: §4.3 · p. 4, Table 3 · p. 4
4.4. Sensitivity to calibration data
- CASM remains tightly clustered across calibration-fold combinations and its median stays above Direct at every scale, including single-fold calibration.DBN varies more, stays below Direct, and responds divergently across SMC and GTZAN as calibration folds increase.Source: §4.4 · p. 4, Figure 2 · p. 4
5. CONCLUSION AND FUTURE WORK
CASM is an offline, model-agnostic post-processor that balances local beat evidence with broader rhythmic patterns. The authors report gains in beat accuracy and continuity, and identify multimodal use and integration into training loops as future directions.
5. CONCLUSION AND FUTURE WORK
- CASM balances local beat evidence with broader rhythmic patterns and uniformly enhances F1, CMLt, and AMLt across tested backbones.The conclusion also reports that it uses one frozen calibration setup and is less sensitive to calibration data than DBN.Source: §5 · p. 4
- The authors identify applying CASM to video, motion, and other multimodal beat-tracking systems as a future direction.They note that CASM analyzes neural activations directly.Source: §5 · p. 4
- The authors plan to integrate CASM into neural-network training loops, following examples from dPLP and piano transcription.The conclusion cites PLPDP's evolution into dPLP and SemiCRF's use in piano transcription as precedents.Source: §5 · p. 4