PaperlayerView exact revision on arXiv

Source-linked AI summary

Normalise or condition? Noise-floor front-ends for on-board keyword spotting under UAV rotor ego-noise

Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

arXiv:2609.31699v1cs.SDcs.CVeess.AS

TL;DR

On-board keyword spotting for UAV commands must contend with rotor ego-noise, and prior KWS results do not evaluate the microphone noise encountered in flight. The paper benchmarks cross-drone command spotting and compares front-ends that track the preceding noise floor. Conditioning preserves the baseline false-alarm rate while improving detections under a fixed false-alarm budget, whereas normalising front-ends can lose ground to the baseline.

  • Problem

    Rotor ego-noise can overwhelm nearby speech, while prior KWS work does not evaluate spotting under the ego-noise of the microphone that actually flies.

    Source: §1 · p. 1

  • Method

    The study trains on one quadrotor and tests on another, comparing established front-ends with noise-floor subtraction and conditioning based on the preceding two seconds.

    Source: §1 · p. 1, §2.2 · p. 2

  • Results

    Conditioning yields +8–9 points at −10 dB under 1 FA/h, with or without PCEN, while normalising front-ends fall below plain log-mel at a fixed false-alarm budget.

    Source: §3.2 · p. 4, §4 · p. 5

  • Takeaways & Limitations

    Streaming false-alarm performance changes the front-end comparison: conditioning keeps the baseline rate and turns its per-clip gain into detections.

    Source: §4 · p. 5

    When throttle changes just before a command, floor-based front-ends lose 9–16 points at 0/−5 dB until the tracker catches up.

    Source: §3.1 · p. 4

Abstract

from arXiv · show

A microphone on the airframe of a small multi-rotor UAV is dominated by rotor ego-noise, so spoken flight commands arrive at negative signal-to-noise ratio (SNR). We study small-footprint keyword spotting (KWS) for a ten-word command vocabulary under real ego-noise, training on one quadrotor and testing on another. Besides per-clip accuracy we measure the streaming false-alarm rate on 4.4 h of continuous rotor noise. We compare classical noise-robust front-ends (CMN, PCEN, spectral subtraction), test-time adaptation, and two front-ends that track the per-band ego-noise floor over the two seconds preceding the decision window and either subtract it (normalisation) or feed it to the network as a second input channel (conditioning). Per clip, all front-ends look alike: +2-3 points on average, up to +14 at -15 dB. On continuous rotor noise they differ sharply. Normalising front-ends fire about ten times more often than plain log-mel at the same threshold and end up below it at a budget of one false alarm per hour (-9 points at 0 dB). Conditioning keeps the baseline's false-alarm rate and turns its gain into detections (+8 points at -10 dB over three seeds); on top of PCEN it gives the best per-clip accuracy and false-alarm rate, and a level-anchored variant is also invariant to the microphone gain. Real drone+interferer recordings expose the remaining failure mode, environmental sounds and bystander speech, which training negatives halve. A single script reproduces all on-device numbers on the NVIDIA Jetson Orin NX flight computer, where the complete pipeline costs 3 ms per 100 ms hop on the CPU.

1. INTRODUCTION

Small UAV operators can use spoken commands when their hands and eyes are occupied, but rotor noise makes on-board recognition difficult. The paper benchmarks command spotting under real drone ego-noise and compares noise-floor front-ends for this setting.

1. INTRODUCTION

  • Rotor ego-noise is broadband and harmonic, 20–60 dB louder than nearby speech, and recorded on-board human sounds can have SNRs from −57 to −2.5 dB.Prior drone-audition approaches use microphone arrays or motor telemetry and target intelligible speech rather than spotting a small word set.Source: §1 · p. 1
  • Existing small-network KWS results on Google Speech Commands do not establish performance under the ego-noise of the microphone that actually flies.The paper identifies this as an evaluation gap for on-board UAV command spotting.Source: §1 · p. 1
  • The benchmark trains on one quadrotor and tests on another, comparing noise-floor normalisation and conditioning with existing front-ends and training or adaptation approaches.It also evaluates streaming false alarms and reports a Jetson Orin NX implementation with measured cost and an on-device replay that reproduces the PC evaluation.Source: §1 · pp. 1–2

2. ON-BOARD KEYWORD SPOTTING UNDER EGO-NOISE

The study evaluates ten command words plus unknown and silence using speech mixed with ego-noise from two quadrotors. It tracks a per-band noise floor and tests ways to subtract it or provide it as input context, alongside compact networks and streaming decision logic.

2.1. Task and data

  • The test uses ten commands plus unknown and silence, with training and validation noise from drone2 and test noise exclusively from unseen drone1.Ego-noise mixtures span −15 to 20 dB at test; silence examples are noise-only segments used to assess rotor-noise false alarms.Source: §2.1 · p. 2

2.2. Front-end: tracking and using the ego-noise floor

  • The noise-floor profile is the per-band 20th percentile of the 200 frames preceding each decision window; NFN subtracts it, while NPC and NPC-a provide floor context as a second channel.NPC-a anchors both channels to the overall floor level, so microphone gain shifts cancel; the approach is related to log-domain spectral subtraction and noise-aware training.Source: §2.2 · p. 2

2.3. Networks and training

  • The evaluated back-ends are BC-ResNet-3/-8 and KWT-1, trained from scratch for 50 epochs with augmentation and validation-based checkpoint selection.The models have 57 k, 349 k, and 0.56 M parameters, respectively.Source: §2.3 · p. 2

2.4. Streaming decision logic on the UAV

  • Every 100 ms, the streaming detector classifies a second of log-mel frames with the preceding two-second floor profile; smoothed posteriors trigger commands subject to consecutive-hop and inhibition rules.The pipeline sends commands to the flight controller over MAVLink; the figures describe streaming hit rate versus false alarms and stress tests for input gain and throttle-context mismatch.Source: §2.4 · pp. 2–3, Figure 1 · p. 3, Figure 2 · p. 3

3. EXPERIMENTS

Across unseen-drone tests, ego-noise training matters more than the choice of robust front-end for per-clip accuracy, but streaming evaluation exposes a false-alarm cost for normalisation. Conditioning on the tracked noise floor preserves the baseline’s false-alarm rate, while interferers and throttle changes reveal remaining operational limits.

3.1. Per-clip accuracy: training noise and front-ends

  • With ego-noise training on the other drone, front-ends add 1.9–2.8 accuracy points on average, but CMN and NFN raise FAR while NPC-a keeps it lower than baseline.Clean-trained BC-ResNet-8 reaches 98.0% on clean words but only 16% at −15 dB and has FAR 100%; ego-noise training reaches 81.2% with FAR 3.6%.Source: §3.1 · p. 3, Table 1 · p. 4
  • Among floor trackers, the context mean gives NPC-a 84.2% accuracy and FAR 0.6%; training without speech in the context raises FAR to 9.8%.A 0.5 s context gives 82.7% accuracy; a throttle step can leave the two-second tracker stale.Source: §3.1 · pp. 3–4
  • A throttle change immediately before a command costs floor-based front-ends 9–16 points at 0/−5 dB until the tracker catches up; a level-jump reset is proposed.Log-mel and CMN are unaffected, while PCEN loses 4–6 points.Source: §3.1 · p. 4
  • With ±20 dB test microphone-gain mismatch, log-mel loses up to 6.6 points at 0 dB SNR, whereas CMN, PCEN, NFN, and NPC-a are invariant.For BC-ResNet-3, the log-mel drop reaches 27 points (86.3 → 59.2%), versus 85.5/87.3% with NFN/NPC-a.Source: §3.1 · p. 4

3.2. Streaming operation: hit rate at a fixed false-alarm budget

  • At a budget of 1 FA/h, NFN and CMN fall below log-mel at 0 dB (75.3% and 74.9% vs. 84.2%), while NPC-a reaches 59.7% at −10 dB, +8.0 over log-mel.At threshold 0.7, NFN and CMN fire 7.7 and 8.2 times per hour, versus 0.8 for log-mel; NPC-a fires 1.0 times per hour. Adding the anchored floor channel to PCEN yields 84.5 ± 0.2% per-clip accuracy with FAR 1.5 ± 0.1%.Source: §3.2 · p. 4

3.3. Different back-ends

  • For the 57 k BC-ResNet-3, NPC-a gives 79.2% mean accuracy with 2.0% FAR; in streaming mode it scores 55.0/85.4% at −10/0 dB, matching the six-times-larger BC-ResNet-8 baseline.Source: §3.3 · pp. 4–5

3.4. Beyond rotor noise: real recordings and negatives

  • Real interferers trigger 25–50 false alarms per hour on environmental sounds and over 100 on nearby speech; adding training negatives roughly halves these rates without changing rotor-noise FAR.Non-command words still trigger 4–6 times per 100 utterances, supporting a wake-word approach rather than relying on a better front-end.Source: Table 2 · p. 4, §3.4 · p. 5

3.5. Deployment on the Jetson Orin NX

  • On the Jetson Orin NX, the complete BC-ResNet-3 pipeline costs 3.2 ms per 100 ms hop on two CPU cores, and on-device replay reproduces the PC evaluation.BC-ResNet-8 costs 8.6 ms per hop on CPU; TensorRT FP16 brings networks to 1.2–1.5 ms.Source: §3.5 · p. 5, Table 3 · p. 5

4. CONCLUSION

Conditioning on the tracked noise floor preserves the baseline false-alarm rate and improves detections, while normalising front-ends fare worse at a fixed false-alarm rate. The remaining challenge is interference from bystander speech and environmental sounds.

4. CONCLUSION

  • +8–9 points at −10 dB under 1 FA/h: conditioning keeps the baseline false-alarm rate and turns the gain into detections, with or without PCEN.Normalising front-ends fire ten times more often than log-mel and fall below it at a fixed false-alarm rate.Source: §4 · p. 5
  • A 57–350 k parameter network can spot commands beside rotors when trained on recorded ego-noise and paired with a front-end chosen for the false-alarm budget.A clean-trained model reaches 98% accuracy but fires on every second of hovering.Source: §4 · p. 5
  • Bystander speech and environmental sounds trigger dozens of false alarms per hour in real recordings; training negatives halve that rate, and a wake word should absorb the rest..Source: §4 · p. 5
  • The complete pipeline costs 3 ms per 100 ms hop on the Jetson Orin NX.Source: §4 · p. 5
Loading 2609.31699v1…