VoxLock: Target-Speaker Extraction for Voice Agents Without an Enrollment Step

• Noise suppressors do not remove nearby speech, and Krisp, ai-coustics, and Sanas sell closed, per-minute or per-device voice-isolation filters.Source: §1 · p. 1
• TSE typically requires an enrollment recording and is usually evaluated with signal metrics on simulated mixtures, unlike agents’ need for recognizer output on real audio.Source: §1 · p. 1
• VoxLock takes its voice sample from the call itself, so the agent passes in one audio stream.Source: §1 · p. 1
• VoxLock fine-tunes a public causal TSE baseline [4] on one NVIDIA L4.Source: §1 · p. 1
• Its evaluation compares recognizer output against seven models from three commercial vendors and five open models on Krisp’s benchmark, 134 AMI windows, and 400 AMI windows frozen before scoring.Source: §1 · p. 1
• VoxLock releases runtime version 1.2.1, which runs a live call on one CPU thread.Source: §1 · p. 1

2. Related work

• SpeakerBeam, VoiceFilter, VoiceFilter-Lite, and the personalized DNS track condition separation on a pre-recorded enrollment utterance or clip.VoiceFilter-Lite supports streaming and on-device recognition.Source: §2 · p. 1
• Lin et al. derive a speaker cue from the mixture, while REAL-T and the REAL-TSE challenge [4] released causal BSRNN baselines.Band-split RNNs were extended to personalized enhancement and are the TSE backbone in WeSep; VoxLock starts from one of the causal baselines.Source: §2 · p. 1
• TIGER and SAM Audio [13] output multiple sources without identifying the caller, whereas DeepFilterNet suppresses noise while retaining all speech.The paper reports no published comparison of open TSE with commercial voice-isolation products on real recordings by downstream recognition.Source: §2 · p. 1

3.1. Model

• VoxLock starts from REAL-TSE’s causal tfmap_context BSRNN [4], using 16 kHz audio, 512-sample STFT, 128-sample hop, six band-split blocks, unidirectional modeling, and 27M parameters; algorithmic latency is 24 ms.Each call-spectrum frame attends over sample-spectrum frames; separator band features attend over frame-level ECAPA-TDNN sample features, whose result multiplies into each band.Source: §3.1 · p. 1

3.2. Voice sample from the call

• The model uses exactly 3 s of caller speech, selected from the loudest 3 s in the first 15 s offline or after 3 s of detected speech live.Both selection rules assume the caller is the loudest, earliest talker, as with a headset or caller-held phone; live audio passes through unchanged until the sample is collected.Source: §3.2 · p. 1

3.3. Training

• Every training stage minimizes negative SI-SDR with AdamW, updates all weights including ECAPA-TDNN, and starts from the previous stage’s model.The training description gives a 10^-4 learning rate in S1 and S2, decaying to 10^-5 over S2’s final 1,000 steps, and 3 × 10^-5 in S3 and S4.Source: §3.3 · pp. 1–2
• S1 trains for 4,000 steps on 6 s simulated mixtures of a near-field LibriSpeech caller and one or two quieter background talkers, with telephone-channel effects in 25% of examples.The caller sample is either the mixture’s first 3 s or another caller utterance, clean or noisy.Source: §3.3 · p. 2
• S2 adds AMI headset audio from 40 meetings for 4,000 steps, using wearer-only targets and crosstalk-only mixtures with pasted caller clips; evaluation and REAL-T meetings are excluded.The first 1,000 steps include wearer-alone windows and amplified crosstalk; the remaining 3,000 use crosstalk alone with the target silent outside a 3–4 s caller clip.Source: §3.3 · p. 2
• S3 uses 30 s examples, mostly with continuous callers, to address the 20 dB fade within 40 s observed after training on 6 s windows.Source: §3.3 · p. 2
• S4 varies caller distance independently of background-talker distance and adds real AMI room tone to counter the model’s tendency to keep the closer talker.S3 and S4 retain the real-data batch at half weight.Source: §3.3 · p. 2

Krisp benchmark

• On Krisp, evaluation uses 265 real recordings with a main talker and a second voice; its table reports caller-word loss and background-word leakage over 85.3 minutes of background-only speech.Krisp’s published VI 2.5 outputs are scored as released.Source: §4 · p. 2, Table 1 · p. 2

AMI benchmark

• On AMI, the initial evaluation uses 134 windows from seven people, while the post-final-model blind set contains 400 hash-frozen windows from 16 people across 11 standard-test meetings.The initial meetings were examined during development; the blind windows were frozen before any model was run. Each begins at a 3–15 s wearer turn and extends 4 s past it while others talk.Source: §4 · p. 2, Table 2 · p. 2

Metrics

• WER measures errors on caller words, missed measures the share of caller words lost, and leak counts recognized words during background-only speech per minute.Leak excludes audio within 0.25 s of caller speech; reported 95% bootstrap intervals use 5,000 draws over people and are uncorrected for multiple comparisons.Source: §4 · p. 2

Comparators

• Comparators include Krisp VI 2.5, ai-coustics Voice Focus, Sanas Voice Isolation, Hush, DeepFilterNet3, TIGER, SAM Audio small, and REAL-TSE.TIGER uses 10 s chunks and keeps the output closest by ECAPA embedding to the first 3 s of speech; SAM Audio uses a text prompt and the first 3 s as an anchor. Commercial versions were those available in October 2026, and the commercial filters run causally.Source: §4 · p. 2

5. Results

• On Krisp, VoxLock reaches 4.95% WER and is significantly lower than Krisp Lite, both ai-coustics models, Sanas, and every open model.Its WER is not significantly different from Krisp Default; 9 background words reached its transcript in 85.3 minutes of background-only speech, compared with 130 for Krisp Default and 5,674 unprocessed.Source: §5 · pp. 2–3
• In the phone-call scenario, VoxLock raises WER from 3.82% to 5.36%, but lowers it from 29.31% and 25.13% to 4.83% and 4.90% in work and call-center scenarios.On 200 caller-only windows, WER is 7.49% against 7.18% unprocessed, a nonsignificant 0.31-point increase.Source: §5 · p. 2
• On AMI, VoxLock has the lowest WER and fewest missed words, while ai-coustics Voice Focus 2.2 S lets fewer background words through, a significant difference.With seven people, the intervals are wide.Source: §5 · pp. 2–3
• On blind AMI windows, VoxLock cuts WER from 42.95% to 10.63% and leaked words from 98 to 1.2 per minute.The 400 windows were frozen before scoring; the live path passes each window through unprocessed until it has heard 3 s of speech.Source: Table 3 · p. 3, §5 · p. 3
• S1 reduces Krisp WER from 9.99% to 5.61% and blind AMI WER from 14.75% to 11.07%; later stages further reduce Krisp leak from 2.25 to 0.11 words per minute and WER from 5.61% to 4.95%.On blind AMI, WER moves by at most 0.6 points after S1, and S1, S2, and S3 are not significantly different from VoxLock. This was one training run with stages designed after inspecting previous results, not a controlled ablation; the first 1,000 steps of S2 were no better than 1,000 more simulated steps, and leak was significantly higher than S1 on Krisp.Source: Table 4 · p. 3, §5 · p. 3
• Using the first 3 s of speech found with Silero VAD instead of the loudest 3 s makes no significant difference on blind AMI, while raising Krisp WER by 0.13 points [+0.01, +0.28].Other systems are absent from the blind AMI comparison because the ai-coustics key was unavailable, Krisp publishes outputs only for its own benchmark, and Sanas and the open baselines were not run there.Source: §5 · p. 3
• With another meeting participant’s 3 s turn as the sample, VoxLock loses 17% of the caller’s words, versus 8% with the right sample.The REAL-TSE checkpoint loses 61% with the wrong sample; VoxLock follows the sample but also learned to keep the dominant near-field talker.Source: §5 · p. 3

6. Deployment

• VoxLock 1.2.1 uses an enroll ONNX graph once per call and a chunk graph for each call chunk, with float32 streaming output matching a whole-file pass up to float rounding.The enroll graph returns a normalized spectrum and attention keys and values for each band; the chunk graph uses those tensors and recurrent state to do only call-dependent work per chunk.Source: §6 · p. 3
• 5.11% WER on Krisp is achieved by the 8-bit model, which uses 77% of one CPU thread in 40 ms chunks with 24 ms algorithmic latency.In 80 ms chunks it uses 61% of one thread; in 20 ms chunks it uses 110%, while two threads run those chunks at 0.86 times real time. The float32 model processes recordings 40 times faster than real time on an NVIDIA L4.Source: §6 · p. 3
• 5.15% WER on Krisp is scored by live(), compared with 4.95% for the same model on the whole file.At the switch, live() runs the network over up to 30 s of the call so its recurrent state starts from the beginning; this takes 2–5 s of compute on one CPU thread and stalls output at the switch.Source: §6 · p. 3

7. Limitations

• The model is English-trained, operates at 16 kHz, and supports a single caller.Source: §7 · p. 3
• Automatic sampling assumes the caller is the loudest early talker; if a background talker dominates, VoxLock selects the wrong person.Source: §7 · p. 3
• Both comparison sets were seen during development, and every AMI window starts with the caller, matching the sampling assumption.Source: §7 · p. 3
• The Krisp benchmark comes from a vendor whose systems are compared, and its CC BY-NC 4.0 license rules out commercial use.Source: §7 · p. 3
• All transcripts come from one recognizer family, and all results come from one training run.Source: §7 · p. 3

8. Conclusion

• VoxLock removes background voices for a recognizer about as well as Krisp’s best filters and with lower WER than every other tested system.It fine-tunes a public TSE baseline on one GPU using a voice sample from the call.Source: §8 · p. 3
• VoxLock holds up on a blind set of meetings.Source: §8 · p. 3
• VoxLock runs live on one CPU thread, and its model, ONNX graphs, Python package, and per-file evaluation scores are released.Source: §8 · p. 3