Toward accurate speaker inventories for online speaker diarization

1. INTRODUCTION

• A fixed inventory capacity can merge speakers, while distance-only registration can split them after noise or turn boundaries.Once emitted, labels cannot be revised, so registrations made on too little evidence persist.Source: §1 · p. 1
• DER reports neither count-error direction, and the lowest-DER VoxSRC-23 system has the worst inventory in the existing-system comparison.JER weights reference speakers equally but also remains silent on speaker count and its direction.Source: §1 · p. 1
• A candidate speaker pool registers speakers on accumulated evidence, reducing macro speaker-count error from 225% to 21% of the reference and macro DER by 0.77 %p.The pool keeps evidence separate per candidate.Source: §1 · p. 1
• Commit-gated label assignment holds undecided outputs until candidates are confirmed or rejected, reducing macro DER a further 0.87 %p at 0.109 s mean added latency over all outputs.The mean is 1.27 s over the 8.6% of outputs held.Source: §1 · p. 1

2. BACKGROUND

• The tracker maintains committed speaker centroids and routes each embedding to its nearest centroid under cosine distance when it falls within the adaptive threshold; otherwise it enters a shared buffer.The threshold grows linearly with the number of registered speakers.Source: §2 · p. 1
• A buffered embedding receives a label immediately, and the buffer's medoid is registered as a speaker once its size reaches preset K.If the embedding went to the buffer, its label is the next unused identifier.Source: §2 · p. 2
• One shared buffer can mix alternating unseen speakers into one registered identity, while a single distant embedding can seed a speaker who does not exist.The tracker bases each decision on one observation and does not undo emitted labels.Source: §2 · p. 2

3.1. Evidence-gated speaker registration

• The candidate pool keeps embeddings for different unseen speakers apart, replacing one shared buffer and applying the registration size K to each candidate separately.Rejected embeddings join the nearest candidate when their median member-wise distance passes the admission threshold; otherwise they open a new candidate.Source: §3.1 · p. 2, §3 · p. 2
• A candidate registers after accumulating K mutually compatible embeddings, whose normalized mean becomes its centroid; a singleton candidate can be discarded after expiry Te.Candidates below K receive no label, so one noisy observation leaves no trace in the inventory.Source: §3.1 · p. 2

3.2. Commit-gated label assignment

• Outputs attached to unresolved candidates wait in an ordered queue and receive the candidate identity if it registers, or the fallback label when the delay budget expires.A timed-out output is not revisited, so a later registration leaves that output merged under its fallback label.Source: §3.2 · p. 2

4.1. Datasets

• The nine-dataset evaluation covers 1,907 recordings spanning DIHARD acoustic conditions, in-the-wild Vox recordings with many speakers, and overlapping English and Mandarin meetings.The three tracker variants are compared on all nine datasets; existing-system comparisons use VoxSRC-23 and DIHARD III evaluation sets.Source: §4.1 · p. 2

4.2. Evaluation protocol

• The protocol reports DER and its FA, MS, and speaker-confusion components, speaker-count error, and algorithmic added latency.Macro averages are rescored under a common 0.15-s collar; the per-dataset protocols use 0.25 s on Vox datasets and none elsewhere.Source: §4.2 · pp. 2–3

4.3. Implementation details

• The system uses FireRed-VAD and a released ReDimNet2 checkpoint, with tracker hyperparameters fixed on DIHARD III and VoxConverse development sets.The tracker uses K = 5, τpool = 0.45, Lprov = 2.0 s, and a 30-s singleton expiry; no evaluation set enters these choices.Source: §4.3 · p. 3

4.4. Systems compared

• The comparison includes three shared-front-end variants with different decision policies, released systems with their stated settings, and a quoted S2SND-M result.Streaming Sortformer uses four output slots, LS-EEND fixes S=8 attractors, and DIART has no speaker capacity; Fig. 1 groups VoxSRC-23 results by reference speaker count.Source: §4.4 · p. 3, Figure 1 · p. 3

5.1. Comparison with existing systems

• On VoxSRC-23, + Pool + Delay has the lowest DER and |e|; on DIHARD III it trails Streaming Sortformer by 4.6 %p of DER and LS-EEND on |e|.On DIHARD III, both fixed-capacity systems count speakers more accurately than the proposed system.Source: §5.1 · p. 3

5.2. Both axes across nine datasets

• Across nine datasets, macro count error improves by an order of magnitude and DER improves on eight of nine.Table 2 reports per-dataset results across both axes and segmentation conditions.Source: Table 2 · p. 4, §5.2 · p. 4

5.3. Sensitivity

• Across hyper-parameter sweeps, macro DER moves within 3.3 %p while |e| ranges from 19.5 to 63.6, still far below the baseline rule’s 225.0.The operating point sits 0.18 and 0.13 %p from each sweep’s DER optimum, trading for 5.2 and 8.6 %p of |e|, respectively.Source: Table 3 · p. 4, §5.3 · p. 4

5.2. Both axes across nine datasets

• On AliMeeting, DER is flat, 34.60 to 34.76, while |e| falls from 440% to 13%; sustained overlap yields mixtures the evidence gate cannot reject.This is the one dataset where the pool does not help.Source: §5.2 · p. 4
• With reference boundaries, DER improves on all nine datasets, while count errors barely move: macro 225.0 to 230.8 and 20.7 to 22.4.The count-error increases come from the three DIHARD sets.Source: §5.2 · p. 4

5.4. The delay budget and what it costs

• At Lprov = 2.0 s, the delay sweep saturates after (K−1) embedding steps; the queue holds 8.6% of outputs, adding 0.109 s on average overall.2.4% of outputs time out and leave under the fallback.Source: Table 4 · p. 4, §5.4 · p. 4

• Across nine datasets, macro speaker-count error falls from 225% to 21%, while macro DER drops by 1.64 %p, at mean added latency 0.109 s over all outputs.Source: §6 · p. 4
• The candidate pool settles speaker counts, while commit-gated label assignment delays outputs until candidates are decided.Source: §6 · p. 4
• One label per step misses overlapped speech, which accounts for four fifths of meeting datasets’ residual after segmentation.Source: §6 · p. 4