Source-linked AI summary
Pitch Smoothing Using Relative Interval Networks
Chin-Yun Yu, Chi-Jen Peng, Li Su, György Fazekas
TL;DR
Per-frame pitch estimators do not exploit audio’s temporal structure, and Viterbi’s adjacent-frame chain can lock into octave errors across corrupted frames. RIN combines per-frame estimates with multi-hop VQT pitch differences in a global optimization; across benchmarks it improved YIN by up to 21.8 points in RPA, while robustness benefits depended on the acoustic degradation.
Problem
Standalone per-frame pitch estimators cannot exploit audio’s temporal structure, while Viterbi’s adjacent-frame chain can lock into incorrect octaves across corrupted frames.
Source: §1 · p. 1Method
RIN combines per-frame pitch estimates with multi-hop VQT pitch differences and confidence weights in a global optimization solved as a minimum cost circulation problem.
Source: §1 · p. 1, §2.3 · p. 2, §2.4 · p. 3Results
Across speech, singing, and instrumental benchmarks, RIN improved YIN by up to 21.8 points in RPA but did not consistently benefit stronger estimators.
Source: §4 · p. 4Takeaways & Limitations
RIN performed comparably to Viterbi in clean conditions and showed superior robustness under low-SNR white noise while remaining computationally efficient.
Source: §1 · p. 1VQT pitch-difference estimation assumes harmonic amplitudes remain unchanged up to a scale factor across m frames; when this does not hold, the corresponding confidence weight is adjusted.
Source: §2.2 · p. 2
Abstract
from arXiv · showhide
Pitch tracking systems typically couple a per-frame fundamental frequency ($F_0$) estimator with a temporal smoothing stage to obtain continuous trajectories. Conventional Viterbi smoothers enforce first-order continuity but lack long-term temporal awareness and could lock into octave errors across corrupted frames. We propose Relative Interval Networks (RIN), a trajectory smoothing framework that reconciles per-frame pitch estimates with data-driven multi-hop pitch differences. We extract robust relative pitch intervals across arbitrary frame offsets using Variable-Q Transform cross-correlation. We formulate pitch smoothing as an $L_1$-norm optimization problem and prove its equivalence to a minimum cost circulation problem, solved efficiently via linear programming. Evaluations across speech, singing, and instrumental datasets show that RIN substantially improves weak estimators, matches or outperforms Viterbi decoding at a comparable computational cost, and provides superior robustness under certain acoustic degradation.
1. INTRODUCTION
Pitch tracking needs temporal smoothing because per-frame estimators cannot exploit audio’s temporal structure, while Viterbi’s adjacent-frame chain can lock into octave errors across corrupted frames. Relative Interval Networks (RIN) combine per-frame estimates with multi-hop pitch differences in a global optimization.
1. INTRODUCTION
- Pitch tracking extracts temporally coherent F0 trajectories, a task that couples front-end pitch estimation with back-end smoothing.Standalone per-frame estimators cannot exploit audio’s temporal structure.Source: §1 · p. 1
- Viterbi’s adjacent-frame chain can lock a contour into an incorrect octave across consecutive corrupted frames.The HMM also uses static, signal-agnostic transition penalties, and its trellis requires a fine pitch grid.Source: §1 · p. 1
- Fig. 1 depicts a Viterbi decoding failure on an MIR-1K excerpt.The accompanying text identifies the failure as octave locking across corrupted frames.Source: Figure 1 · p. 1, §1 · p. 1
- RIN combines per-frame pitch estimates with robust multi-hop pitch differences to optimize trajectory smoothing globally.The framework uses Variable-Q Transform measurements and is equivalent to a minimum cost circulation problem.Source: §1 · p. 1
2. METHODOLOGY
RIN smooths pitch by reconciling per-frame estimates with data-driven pitch differences across multiple offsets. It estimates and confidence-weights those differences from VQT correlations, then solves a global optimization and uses the resulting cues to enhance voicing detection.
2.1. Problem formulation
- RIN regularizes pitch estimates toward data-driven differences across arbitrary frame offsets, using an L1 objective that is robust to octave outliers and hard note transitions.With only immediate-neighbor offsets and zero differences, the regularizer reduces to a first-order smoothness penalty.Source: §2.1 · pp. 1–2
2.2. Computing pitch differences using VQT
- VQT shift invariance supports estimating pitch differences from cross-correlations between frames separated by each offset.The method assumes harmonic amplitudes remain unchanged up to a scale factor across the frames; when that assumption fails, the corresponding estimates receive less weight.Source: §2.2 · p. 2
2.3. Confidence weighting
- Confidence weights combine peak-to-mean prominence with peak correlation, so broadly high correlations do not by themselves imply high confidence.An arcsin mapping counteracts saturation of skewed high correlation values.Source: §2.3 · p. 2
2.4. Network flow formulation
- The weighted L1 pitch objective is equivalent to a minimum cost circulation problem, whose flow-conservation dual yields the pitch contour as node potentials.The LP formulation uses bounded edge variables, and the authors report that it roughly halves solve time using HiGHS.Source: §2.4 · pp. 2–3
2.5. Enhancing voicing detection
- RIN enhances existing voicing detection with correlation-derived confidence weights, combining evidence across valid offsets using a geometric mean.The mixing weights are set to β_m = 1/m^2.Source: §2.5 · p. 3
3. EXPERIMENTS AND RESULTS
Across three annotated datasets, RIN is evaluated against unsmoothed estimates and Viterbi using multiple pitch metrics. Its benchmark performance depends on estimator and acoustic condition, while offset ablations show most gains arrive with the first offsets.
3.1. Experimental setup and metrics
- The evaluation uses MIR-1K, PTDB-TUG, and URMP, comparing unsmoothed, Viterbi, and two RIN offset sets with RPA, RCA, OA, MAE, and runtime.The two RIN configurations use JA = {1} and JB = {1, 5}.Source: §3.1 · p. 3
3.2. Benchmark performance on clean audio
- Across MIR-1K and URMP, RIN generally matches or outperforms Viterbi in RPA and often lowers MAE, while PTDB-TUG shows no smoother MAE gain for the strongest unsmoothed estimators.RIN has runtime comparable to Viterbi for most estimators. The table covers clean audio on PTDB-TUG, MIR-1K, and URMP, with RPA, RCA, and OA in percent and MAE in cents.Source: §3.2 · p. 3, Table 1 · p. 4
3.3. Robustness and ablation studies
- At −10 dB white noise, CREPE RPA is 76.36 for RIN, 68.19 for Viterbi, and 67.18 without smoothing; under babble, results vary with estimator and SNR.For SWIPE under white noise at −10 dB, RIN reaches 73.75 RPA versus 71.48 for Viterbi; under babble, RIN falls below both alternatives for SWIPE at negative SNRs.Source: §3.3 · p. 4, Figure 2 · p. 4
- 72.75 RPA for SWIPE and 75.19 for CREPE follow adding the first offset, with gains largely plateauing beyond two offsets.The greedy paths are in-sample references; at |J| = 2, JB is 0.53 and 0.61 points below the greedy sets for SWIPE and CREPE, respectively, while peak offset sets differ by estimator.Source: §3.3 · p. 4, Figure 3 · p. 4
4. CONCLUSION
RIN combines per-frame pitch estimates with VQT-based multi-hop pitch differences in a network-flow smoother. It improves weak estimators in some benchmarks, but benefits stronger estimators inconsistently and responds differently to white and babble noise.
4. CONCLUSION
- RIN combines per-frame pitch estimates with multi-hop pitch differences estimated using VQT and confidence weights.The paper describes RIN as a network-flow pitch smoother.Source: §4 · p. 4
- RIN improved YIN by up to 21.8 points in RPA across the speech, singing, and instrumental benchmarks, but did not consistently benefit stronger estimators.Source: §4 · p. 4
- RIN’s advantage over Viterbi increased as SNR decreased under white noise, while it remained comparable under babble except for SWIPE at the lowest SNRs.Effective offsets depended on the estimator; future work includes other F0 domains and more efficient network-flow solvers.Source: §4 · p. 4
6. COMPLIANCE WITH ETHICAL STANDARDS
The study used human-subject data from open-access datasets and states that ethical approval was not required.
6. COMPLIANCE WITH ETHICAL STANDARDS
- The retrospective study used open-access human-subject data from MIR-1K, PTDB-TUG, and URMP; ethical approval was not required.Source: §6 · p. 5