PaperlayerView exact revision on arXiv

Source-linked AI summary

Few-Shot Calibration for Sim-to-Real Single-Channel Speaker Distance Estimation

Michael Neri, Archontis Politis, Tuomas Virtanen

arXiv:2609.29203v1eess.AScs.SDeess.SP

TL;DR

Real distance labels are scarce, so speaker distance estimators are commonly trained on synthetic audio, but their transfer to real corpora is poor. The paper evaluates zero-shot transfer and studies few-shot post-hoc calibration of a frozen estimator; across the three evaluated corpora, a corpus-mean constant predictor beats every learned model without calibration.

  • Problem

    Real audio recordings labelled with talker-to-microphone distance are scarce, motivating training speaker distance estimators on synthetic data.

    Source: §1 · p. 1

  • Method

    The paper studies affine calibration and a shrinkage mapping for a frozen synthetic-trained estimator, using a small target-corpus calibration set without retraining.

    Source: §2.1 · p. 1, §2.3 · p. 2

  • Results

    A constant predictor at the corpus mean beats every learned model on all three real corpora, while calibrated error depends on correlation with target distances rather than absolute error.

    Source: §2.3 · p. 2, §5 · p. 4

  • Takeaways & Limitations

    Checkpoint selection should favor correlation with target distances, because affine calibration removes constant bias and wrong output scale.

    Source: §2.3 · p. 2

    Calibration assumes access to N labelled utterances drawn from the target corpus training split, with N on the order of a few tens.

    Source: §2.1 · p. 1

Abstract

from arXiv · show

Speaker distance estimators are trained almost exclusively on simulated room acoustics, because real recordings annotated with the true talker-to-microphone distance are scarce. We show that models trained this way transfer poorly. On three real corpora we evaluate, simply predicting the average distance of the corpus is more accurate than any learned model. Then, we ask how few labelled real utterances are needed to make a frozen, synthetic-trained estimator useful, and study post-hoc calibration maps that rescale its output without gradients or retraining. An analysis of the achievable error shows that what the calibration is not limited by the absolute accuracy of the estimator, but how well it orders utterances by distance, since a constant bias or a wrong output scale is removed exactly by the calibration itself. Balancing this against the cost of estimating each coefficient from few samples yields a criterion that accounts for which map wins on which corpus and at which annotation budget, together with a shrinkage variant that requires no hard decision. Our findings suggest selecting synthetic checkpoints by linear correlation with true distances rather than by absolute error. Code, datasets, and analysis are available at https://github.com/michaelneri/audio-distance-estimation.

1. INTRODUCTION

Single-channel speaker distance estimation relies on acoustic cues, but real recordings with labelled distances are scarce, motivating synthetic training despite limited evidence on sim-to-real transfer. This work evaluates that transfer, develops few-shot calibration, and modifies a CRNN head to improve correlation under domain shift.

  • Distance cues include direct-to-reverberant ratio, early reflections, and spectral envelope, and degraded cues impair human and machine distance perception.Source: §1 · p. 1
  • Earlier approaches used binaural hand-crafted features and classifiers, while later DNN work largely focused on binary or coarse-bin prediction and generalized poorly across acoustic environments.Source: §1 · p. 1
  • Real audio with labelled distances is scarce, so the research community has converged on training with synthetic data.Source: §1 · p. 1
  • Prior work reported near-chance zero-shot results on three real corpora, while sim-to-real transfer without time calibration remained unstudied in the relevant single-channel setting.Source: §1 · p. 1
  • The work analyzes zero-shot mismatch, fits a two-parameter affine map to adapt a frozen synthetic-trained estimator using labelled utterances, and modifies the CRNN head to improve correlation under domain shift.The feature-prediction head leaves synthetic MAE largely unchanged, but substantially improves correlation with true distance labels under domain shift.Source: §1 · p. 1

2. FEW-SHOT CALIBRATION

A frozen synthetic-trained estimator is calibrated on a small set of labeled utterances by fitting maps that correct systematic offset or scale errors without retraining. Which map is useful depends on target-corpus correlation and the sample cost of estimating its parameters; a shrinkage map avoids a hard choice between affine and constant calibration.

2.2. Foundations of distance calibration

  • A two-parameter affine map, fitted on N labeled target-corpus examples, corrects offset and scale errors without gradients, domain adaptation, or fine-tuning.The estimator remains frozen, and calibration maps its raw output to a calibrated distance.Source: §2.1 · p. 1, §2.2 · p. 2
  • The constant predictor is the reference: it estimates the calibration-set mean distance, while offset-only and scale-only maps each constrain one affine coefficient.These one-parameter maps trade modelling bias against estimation variance.Source: §2.2 · p. 2

2.3. Estimating the affine mapping

  • Affine calibration's population error depends on prediction–distance correlation, not raw absolute error, because correlation is unchanged by bias or scale transformations.The passage defines target-label and raw-prediction statistics and identifies correlation as the remaining determinant of calibrated error.Source: §2.3 · p. 2
  • A fitted slope is worthwhile only when correlation is high enough for the calibration-set size; otherwise, its sampling noise can dominate.The supplied criterion gives thresholds of ρ > 0.38 at N = 5, 0.29 at N = 10, and 0.21 at N = 20.Source: §2.3 · p. 2
  • The proposed shrinkage map scales the fitted slope by evidence for correlation, tending to the constant predictor as estimated correlation approaches zero and to the affine map as N grows.It avoids a hard threshold for choosing whether to fit a slope.Source: §2.3 · p. 2

3. MATERIALS

The materials specify a convolutional-recurrent distance estimator and its training variants, a synthetic training dataset, and three real evaluation corpora.

3.1. Distance estimator

  • Both estimator variants supervise clip- and per-frame predictions; the full-stack adds T60, Tmix, and volume prediction heads, and all variants use five-fold cross-validation.The estimator uses STFT log-magnitude and sine/cosine phase features; training uses clean or WHAM!-noise-mixed audio.Source: §3.1 · p. 2

3.2. Synthetic dataset

  • The synthetic corpus contains 2,500 ten-second, 16 kHz recordings, with five-fold splits and simulated distances from 1.00 to 11.00 m.Anechoic EARS speech was convolved with simulated pyroomacoustics room impulse responses.Source: §3.2 · p. 3

3.3. Real datasets

  • The real-data evaluation uses VoiceHome2, STARSS23, and QMUL-TIMIT, spanning domestic recordings, moving-speaker scenes, and measured room impulse responses.VoiceHome2 and STARSS23 use a single channel from microphone arrays; QMUL-TIMIT includes clean and 0 dB WHAM!-noise conditions.Source: §3.3 · p. 3

4. RESULTS

The synthetic-trained estimator’s real-data performance varies substantially by corpus, and no calibration map wins everywhere. Few-shot performance depends on distance correlation and the calibration budget, while shrinkage avoids a hard map choice.

4.1. Results on synthetic data

  • With noise training, 0 dB MAE falls from 1.83 m to 1.47 m; the full stack further reduces it to 1.41 m, while clean MAE falls from 1.30 m to 1.23 m.Only the full stack significantly improves both regimes; individual component contributions are not monotonic.Source: §4.1 · p. 3, Table 1 · p. 3

4.2. Zero-shot real data analysis

  • VoiceHome2 correlations never exceed +0.08, while STARSS23 models reach significant ρ=0.09–0.18 despite its narrower distance range.On STARSS23, full-stack has ρ=+0.18 and the lowest MAE, 1.51 m.Source: §4.2 · pp. 3–4, Table 2 · p. 3
  • On QMUL-TIMIT clean, full-stack reaches ρ=0.63 versus baseline ρ=0.17, despite MAE of 8.32 m versus 7.30 m.At 0 dB, full-stack reaches ρ=+0.29 while baselines retain ρ ≤+0.05.Source: Table 2 · p. 3, §4.2 · p. 4

4.3. Few-shot calibration

  • On QMUL-TIMIT, offset-only wins at every 0 dB budget and affine wins clean from N=10; VoiceHome2 maps lose to the constant, while STARSS23 gains only 0.01 m from N=20.On QMUL-TIMIT clean, affine reaches 2.54 m versus 3.08 m for the constant predictor; at N=50 offset-only beats affine, 2.67 m versus 2.70 m.Source: §4.3 · p. 4, Table 3 · p. 4
  • The shrunk estimator stays within 1% of the better affine or constant map for N ≥10 and degrades toward the constant when slope information is weak.It inherits significant QMUL-TIMIT gains in both conditions and costs 0.07 m on VoiceHome2 at N=5, matching the constant thereafter.Source: §4.3 · p. 4, Table 3 · p. 4

5. CONCLUSION

The conclusion is that post-hoc calibration can make a frozen synthetic-trained estimator useful when its predictions preserve distance ordering. Selecting checkpoints by linear correlation, rather than absolute error, reflects that calibration can repair scale but not uncorrelated predictions.

5. CONCLUSION

  • On all three real datasets, a constant corpus-mean predictor beats every learned model in zero-shot evaluation.Source: §5 · p. 4
  • Calibrated error depends on how well estimator outputs correlate with real distances, not on their absolute error.Calibration removes constant bias or an incorrect output scale, but cannot repair uncorrelated predictions.Source: §5 · p. 4
  • The finite-sample criterion ρ^2 > 1/(N +2) motivates choosing between maps by annotation budget, while shrinkage interpolates without a hard threshold.Source: §5 · p. 4
Loading 2609.29203v1…