PaperlayerView exact revision on arXiv

Source-linked AI summary

PolyChirp: Multi-Species Birdsong Classification Using TinyML on Low-Power Acoustic Sensors

Nathan Duboisset, Zhaolan Huang, Felix Bießmann, Roudy Dagher, Antoine Lavandier, Emmanuel Baccelli

arXiv:2608.23101v1cs.LGcs.AI

TL;DR

Existing low-power bird monitoring systems were limited mainly to binary detection of one species, despite deployments often targeting several species and operating under strict memory and battery constraints. PolyChirp combines site-specific dataset automation with multi-class TinyML models and microcontroller/NPU deployment. It classifies up to 10 species with accuracy above 0.98 while remaining within the stated resource envelope.

  • Problem

    Prior low-power microcontroller systems were limited to single-species detection, whereas field deployments may need to distinguish several species while conserving memory and battery.

  • Method

    PolyChirp combines geographical dataset automation, one-sigmoid-per-species TinyML models, and deployment on microcontrollers with optional NPU acceleration.

  • Results

    Accuracy above 0.98 is achieved for up to 10 species, alongside improved single-species performance and a tiny fraction of BirdNet's memory budget.

  • Takeaways & Limitations

    PolyChirp enables high-accuracy simultaneous classification of up to 10 species on common microcontroller hardware.

  • Takeaways & Limitations

    The dataset is narrowed by the BirdNET confidence threshold and under-representation of some species, while NPU support excludes learnable audio front ends and inputs exceeding tensor limits.

Abstract

from arXiv · show

Recent progress in the field of TinyML has demonstrated that low-power hardware based on microcontrollers can achieve bird species monitoring in real time based on acoustic sensor data for an entire breeding period on a single battery charge. However, the state of the art on low-power microcontrollers was so far limited to binary classification of a single species. In contrast, real fauna monitoring deployments often target multiple species simultaneously. To address this challenge we develop PolyChirp, an approach combining biological domain expertise, automated dataset curation, neural architecture optimization and novel hardware to achieve multiclass bird species detection in the wild. PolyChirp is based on newly designed tiny multiclass models that leverage recent microcontrollers and hardware acceleration with a neural processing unit (NPU). We evaluate the predictive performance of these models, and we measure their computational performance -- memory footprint, latency, energy consumption -- on common microcontroller hardware. Our results demonstrate that PolyChirp not only outperforms state-of-the-art on single species binary classification, but also achieves robust classification of up to 10 species simultaneously, while still fitting with the resource envelope of a sensor that must remain operational in the field for a full season on a single battery charge.

I. INTRODUCTION

Field acoustic monitoring must conserve memory and battery while distinguishing multiple species, but prior microcontroller systems were largely limited to single-species detection. PolyChirp addresses this gap with automated, multi-species TinyML models and evaluates their deployment costs.

  • Continuous field recording stores mostly irrelevant sounds, consuming the memory and battery budgets of acoustic recording units.
  • Prior microcontroller systems could detect a single target species but could not separate or identify several species in one deployment.
  • PolyChirp introduces a multi-species on-device classifier that supports retraining for different geographical locations and species subsets.
  • A geographical site-driven dataset builder converts a location, radius, and target species count into a deployment-specific multi-class dataset.
  • One sigmoid output per species lets a single network perform both bird-sound detection and species classification.
  • Accuracy above 0.98 is achieved for up to 10 species, while using a tiny fraction of BirdNet's memory budget and outperforming TinyChirp in the single-species case.

II. DATASETS

PolyChirp builds deployment-specific datasets from local species occurrence and BirdNET detections, then uses spectrogram representations tailored to the discriminative frequency range. Its multi-output formulation treats non-target audio as an all-zero target vector.

  • Dataset construction is driven by deployment location, radius, and target species count rather than a fixed global species list.
  • For Paris with N = 10, target classes contain 4870 to 10 000 clips, while the non_target class contains 18 800 clips from other birds, AudioSet, and no-bird windows.
  • BirdNET detections at confidence ≥0.92 produce 3 s target clips, while recordings without detections contribute random 3 s no-bird windows.
  • The light_mel representation uses 99×40 bins over 2500–8000 Hz because species separation is concentrated above 2 kHz.
  • Each species has a sigmoid output trained with binary cross-entropy, and non-target clips are represented by an all-zeros target vector rather than an extra output channel.

A. Mel Frontend

The model family adapts TinyChirp-compatible architectures and several alternative front ends to multiclass sigmoid prediction under microcontroller constraints. PolyChirp includes lightweight mel, depthwise-separable, richer spectrogram, and learned waveform-front-end variants.

  • The light_mel input is a 99×40 spectrogram, containing about 3.7× fewer values and costing 4 to 5× less to compute than tinychirp_mel on-device.
  • The baseline TinyChirp architectures are reimplemented with per-class sigmoid outputs and a channel multiplier wired through convolutional and dense widths.
  • The baseline models include CNN-Mel, CNN-Time, and Transformer-Time with 25 600, 748, and 2306 parameters respectively at m=1.
  • DS-CNN and WrenNet use tinychirp_mel, while SincNet and LEAF learn their front ends directly from waveforms.
  • Mel-PolyChirp uses light_mel with three convolutional blocks, global average pooling, and a 32m-wide dense head optimized for speed.
  • Table I describes baseline architecture shapes using the channel multiplier m and target species count N.

D. Training with Augmentation

Training uses Adam and binary cross-entropy with class-aware weighting, while waveform augmentations are applied before feature extraction. The augmentation stream combines temporal, amplitude, noise, masking, and distortion perturbations.

  • Models are trained with Adam and binary cross-entropy on a 70/15/15 split sampled at natural class proportions.
  • Sample weighting assigns half the total weight to non-target audio and distributes the other half among target classes by inverse frequency.
  • Augmentation runs only during training on raw waveforms before mel computation, ensuring every front end receives the same perturbations.
  • The augmentation set includes circular shifts, gain changes, polarity inversion, additive noise, time masking, and clipping distortion.

IV. TINYML DEPLOYMENT PIPELINE

The deployment pipeline combines a site-driven dataset builder with selectable on-device classification or detection models. Classification identifies target species, while detection answers only whether a bird was heard.

  • Classification Pipeline: The builder converts a deployment site into a labelled multi-class dataset, which is then used to train and flash a quantized model.A campaign specifies a coordinate, radius, and target count N; the builder creates N target classes and a pooled non-target bucket.
  • Classification Pipeline: The classification pipeline emits N sigmoid scores per 3 s window for species-aware detection and classification.The same forward pass supports bird-sound detection when any output exceeds threshold and species classification through active-output selection.
  • Detection Pipeline: The detection pipeline collapses bird audio into one positive class and non-bird material into the negative class.It balances bird clips across species, producing a cheaper one-output detector with a single threshold and no per-species calibration.

V. IMPLEMENTATION OVERVIEW

PolyChirp is implemented across portable Cortex-M CPU paths and an optional Axon NPU path, with int8 quantization and optimized on-device mel preprocessing. The NPU accelerates compatible convolutional spectrogram models, while waveform models remain on the CPU.

  • Hardware paths: The implementation targets the Nordic nRF54LM20 and Raspberry Pi Pico 2, using portable Cortex-M execution on both and Axon NPU acceleration only on the Nordic board.The same trained network can run through CPU paths on either board or an accelerated NPU path on the nRF54LM20.
  • Hardware paths: The Axon compiler accepts compatible int8 convolutional spectrogram models but excludes waveform models because of input-size and operator constraints.Mel-spectrogram models fit the tensor and kernel limits, whereas SincNet and LEAF exceed or lack supported operations.
  • Optimization: Post-training quantization maps weights and activations to int8, keeping model memory within the RAM budget and enabling NPU execution.The Axon NPU executes int8 only, while Cortex-M inference also uses int8 arithmetic.
  • Front-end preprocessing: The mel front-end is optimized by precomputing the Hann window and storing the sparse triangular filterbank for reuse across frames.Sparse filtering reduces application to roughly two multiply-accumulates per FFT bin instead of a dense matrix product.

VI. EXPERIMENTAL PERFORMANCE EVALUATION

The evaluation examines predictive accuracy, front-end choices, resource budgets, hardware acceleration, and pipeline shape. Measurements include two low-power boards and compare preprocessing costs alongside model performance.

  • Accuracy: The evaluation tests whether one network detects and identifies up to N = 10 species and preserves accuracy after int8 quantization.This defines the accuracy axis of the experimental evaluation.
  • Front-end: The front-end comparison asks whether raw-waveform processing reduces cost relative to mel spectrograms at an accuracy trade-off.The experiment explicitly compares skipping mel computation with using a mel-spectrogram front-end.
  • Budget and hardware: The resource evaluation measures flash, RAM, latency, and energy against a full-season sensor envelope, including the effect of the on-chip NPU.Measurements are performed on the Nordic nRF54LM20 and Raspberry Pi Pico 2.
  • Front-end preprocessing: 7 to 11× faster preprocessing than TinyChirp’s literal per-frame version comes from precomputing framing components, while light_mel is another 4 to 5× cheaper than tinychirp_mel.The Axon NPU does not accelerate the 512-point FFT over the Cortex-M and cannot run the 1024-point FFT, so preprocessing remains CPU-based.

B. Predictive Performance

PolyChirp maintains strong multiclass performance as the number of target species increases, while model architecture and execution backend determine robustness, latency, memory fit, and energy use. INT8 models support high-recall detection, and NPU acceleration substantially reduces inference cost on the nRF54.

  • Multiclass classification: Macro F2 declines gradually as target species increase, with wrennet falling from 0.98 to 0.97 and mel_polychirp_x2 remaining above 0.94.The curves average performance over resampled species draws from k=1 to k=10 at the F2-tuned threshold.
  • Multiclass classification: Time-domain baselines degrade most severely, with CNN-Time and Transformer-Time reaching macro F2 values of 0.43 and 0.52.AUC and accuracy remain nearly flat, while precision falls first at the recall-favoring F2 operating point.
  • Detection: Every model clears F2 ≥ 0.91 at recall ≥0.97 on bird-presence detection, including 1–3 KB time-domain baselines at F2 = 0.91–0.92.LEAF is excluded from the INT8 comparison because its quantized network collapses.
  • Quantization: INT8 test loss differs from float by under 0.4 for every model except LEAF, whose loss rises from 0.19 to 1.33.The LEAF collapse is attributed to learned PCEN gains not surviving a single activation scale.
  • Inference cost: On the nRF54, NPU inference takes 2 ms for mel_cnn and 39 ms for mel_polychirp_x3, versus hundreds of milliseconds to several seconds for the transpiler.Non-streaming transpilation can run out of memory at the widest width, whereas streaming still fits.
  • Energy and deployment: The NPU draws about 9.8 mW but uses far less energy per inference because it finishes one to two orders of magnitude faster than runtimes drawing 4.9 mW.On the Pico 2, recorded operations take 1.3 to 2.2× less time but consume 6.5 to 12× more energy than on the nRF54.

VII. DISCUSSION & PERSPECTIVES

The discussion examines hardware acceleration, transpiler trade-offs, dataset limitations, thresholding, and memory-aware inference for PolyChirp deployments.

  • Hardware acceleration: NPU deployment is challenging because it requires separate C firmware and a closed-source vendor add-on, tying deployments to specific hardware.This path also departs from the paper’s Rust and Ariel OS stack.
  • Hardware acceleration: NPU acceleration reduces inference latency by 2 orders of magnitude, from about a second to a few milliseconds, while proportionally lowering inference energy consumption.The Axon NPU executes the convolutional portion of mel models faster than Cortex-M cores.
  • Hardware acceleration: Waveform models cannot use the NPU because their 48 000-sample input exceeds the 1024 tensor-dimension limit and their Gabor front-end is unsupported.The authors identify lifting these hardware limitations as future work.
  • Transpilers: IREE uses less RAM, achieves lower latency, and supports more operators than microflow, but incurs a substantially larger Flash runtime footprint and weaker memory-safety guarantees.The measured Flash runtime footprint is more than 2× larger with IREE.
  • Dataset limitations: Dataset curation uses an arbitrary BirdNET confidence threshold of 0.92, which keeps labels clean but narrows training data toward loud, clear calls.Rare species are also trained on less data because of under-representation.
  • Deployment trade-offs: Thresholds must be retuned per quantized backend because int8 post-training quantization shifts scores, while field distributions can differ from test data.The pipeline favors recall because false negatives permanently lose real detections, whereas false positives can be filtered later.

VIII. RELATED WORK

Related work spans bird-recognition systems on servers, edge devices, and microcontrollers, alongside TinyML runtimes, model-search systems, and neural accelerators.

  • Birdsong recognition on edge and server hardware: BirdNET recognizes several thousand species from continuous soundscape audio, while BirdNET-Pi and Bird@Edge support real-time or local-edge bird-sound recognition.The related systems use substantially larger edge or server-oriented resources than the microcontroller focus of PolyChirp.
  • Birdsong recognition on microcontrollers: Prior microcontroller work combines compact audio models with low-power hardware for birdsong detection, vocalization analysis, biologging, or single-species screening.Examples include TinyBird-ML, WrenNet, and TinyChirp.
  • TinyML pipelines: TinyML systems use interpreters such as TensorFlow Lite Micro and CMSIS-NN or ahead-of-time transpilers such as µTVM and IREE.MCUNet combines a runtime with neural architecture search to fit models and runtimes to memory budgets.
  • Microcontroller hardware and neural accelerators: Microcontroller-scale NPUs can reduce inference time and energy substantially, but typically support int8 models and restricted CNN operator sets.The related accelerator landscape includes Ethos-U55, MAX78000, Neural-ART, and Nordic Axon.
  • Microcontroller hardware and neural accelerators: MLPerf Tiny benchmarks plain MCU cores, while recent cross-vendor µNPU benchmarks report that measured performance often departs from datasheets.This motivates evaluating accelerator behavior directly on hardware.

IX. CONCLUSION

The paper presents PolyChirp as a low-power embedded framework for training, deploying, and evaluating TinyML bird models across microcontroller hardware with or without NPU acceleration.

  • Conclusion: PolyChirp enables different TinyML models to be trained and baked into firmware for various microcontroller-based hardware.The framework supports both NPU-accelerated and non-accelerated execution.
  • Conclusion: The paper publishes its Ariel OS model-benchmarking pipeline and reproducible PolyChirp benchmark code.The listed repositories support reproduction of the reported benchmarking workflow.
Loading 2608.23101v1…