PaperlayerView exact revision on arXiv

Source-linked AI summary

Machine Learning for Microcontroller-Class Hardware: A Review

Swapnil Sayan Saha, Sandeep Singh Sandha, Mani Srivastava

arXiv:2205.14550v5cs.LG

TL;DR

Conventional machine learning’s memory and compute footprint constrains direct deployment on microcontrollers, where models must fit device budgets. This review characterizes a closed-loop development workflow and finds that several application classes adopt specific instances of it.

  • Problem

    Conventional machine learning has memory and compute demands that hinder direct deployment on resource-constrained microcontrollers, motivating a workflow suited to their device budgets.

    Source: §IX · p. 14, §XIII · p. 23

  • Method

    The review characterizes a closed-loop microcontroller model-development workflow and examines its components and application-dependent adaptations through qualitative and numerical analysis.

    Source: §I · p. 2

  • Results

    Several application classes adopt specific instances of the workflow, with the review presenting quantitative comparisons and tradeoffs across adaptations.

    Source: §I · p. 2

  • Takeaways & Limitations

    The conclusion calls for next-generation TinyML frameworks to address the challenges discussed in the review.

    Source: §XIII · p. 23

    The review characterizes current TinyML efforts as first-generation and says its discussed challenges have received limited exploration.

    Source: §XIII · p. 23

Abstract

from arXiv · show

The advancements in machine learning opened a new opportunity to bring intelligence to the low-end Internet-of-Things nodes such as microcontrollers. Conventional machine learning deployment has high memory and compute footprint hindering their direct deployment on ultra resource-constrained microcontrollers. This paper highlights the unique requirements of enabling onboard machine learning for microcontroller class devices. Researchers use a specialized model development workflow for resource-limited applications to ensure the compute and latency budget is within the device limits while still maintaining the desired performance. We characterize a closed-loop widely applicable workflow of machine learning model development for microcontroller class devices and show that several classes of applications adopt a specific instance of it. We present both qualitative and numerical insights into different stages of model development by showcasing several use cases. Finally, we identify the open research challenges and unsolved questions demanding careful considerations moving forward.

I. INTRODUCTION … B. Independence From Network Infrastructure via Remote Deployment

Microcontrollers’ severe resource limits make direct deployment of conventional ML models unsuitable, motivating a systematic onboard development workflow. The paper presents a closed-loop workflow with qualitative and quantitative insights, application-specific comparisons, and open research challenges.

I. INTRODUCTION

  • Microcontrollers’ severe resource limits make direct porting of models designed for high-end edge devices unsuitable and call for systematic deployment workflows and tools.A typical microcontroller has 128-kB RAM and 1 MB of flash, compared with 4 GB of RAM and 64 GB of storage for a mobile phone.Source: §I · p. 1
  • Prior surveys review microcontroller model-development cycles qualitatively, while this article adds quantitative comparisons across workflow choices and application-specific case studies.The authors describe their contribution as providing qualitative and numerical insights into workflow blocks, application-dependent evaluations, and tradeoffs,,,.Source: §I · pp. 1–2
  • The paper presents a closed-loop microcontroller ML workflow, with quantitative evaluation and application-specific comparisons of workflow adaptations.It also analyzes tradeoffs in existing development processes and identifies open research challenges.Source: §I · p. 2

II. TINYML WORKFLOW

  • TinyML refers to compression, ML blocks, AutoML frameworks, and hardware and software for ultralow-power (≤1 mW), always-on onboard analytics on resource-constrained platforms,,.Typical microcontroller platforms have SRAM in the order of 100–102 kB and flash in the order of 103 kB.Source: §II · p. 2

A. Applicability

  • TinyML combines the lightweightness of first-principle approaches with the accuracy of large neural networks, which can provide rich sensor-data inferences without domain expertise.It can turn IoT nodes from simple data harvesters or first-principles processors into learning-enabled inference generators.Source: A. Applicability · p. 2

B. Independence From Network Infrastructure via Remote Deployment

  • TinyML enables offline onboard inference without data offloading or cloud-based inference when time-critical applications or unreliable network bandwidth or power make remote models unsuitable.The passages identify autonomous driving, robot and industrial control, wildlife monitoring, and energy-harvesting intermittent systems as examples,,,,,,,.Source: B. Independence From Network… · p. 2

C. Security and Privacy … IV. FEATURE PROJECTION

TinyML keeps inference on resource-constrained microcontrollers to reduce privacy exposure and deployment costs, using a closed-loop workflow that covers data preparation, optional feature projection, model development, and deployment. The sections also describe data-engineering challenges and tools that help prepare and validate training data.

C. Security and Privacy

  • Onboard inference keeps private data within the IoT node, reducing the probability of privacy leaks and attack surfaces compared with streaming it to third-party clouds; privacy-preserving alternatives include FL, secure aggregation, and homomorphic encryption.Those alternatives impose expensive network and compute requirements.Source: C. Security and Privacy · p. 2

D. Low Deployment Cost

  • A Cortex M4-class microcontroller costs around 5–10 USD and can run on a coin-cell battery for months or years, enabling ultralow-power, low-cost AI inference.The paper contrasts this with GPUs that are energy-hungry and expensive to maintain continually for inference using small models,.Source: D. Low Deployment Cost · p. 2
  • The closed-loop TinyML workflow prepares sensor data, optionally projects features, selects lightweight models, and uses NAS or hyperparameter tuning to meet accuracy, SRAM, flash, and latency targets.Hardware metrics may come from proxies or real measurements; candidate-model steps are repeated until desired performance is achieved.Source: D. Low Deployment Cost · p. 3, Figure 1 · p. 3
  • Deployment ports the best-performing model to a TinyML suite for inference and operator optimization, compression, and code generation before flashing it to the microcontroller.Online learning, on-device training, or federated learning can periodically fine-tune the model for shifts in data distribution.Source: D. Low Deployment Cost · p. 3, Figure 1 · p. 3
  • MLPerf Tiny benchmarks four microcontroller-class tasks—multiclass and binary image recognition, keyword spotting, and outlier detection—with quality targets and SRAM, flash, MAC, and latency metrics.The suite uses standard datasets for each task.Source: D. Low Deployment Cost · p. 3

III. DATA ENGINEERING

  • TinyML data engineering must prepare data and learning techniques that generalize to unseen scenarios, using augmentation and cleaning such as transforms, oversampling, class balancing, and noise addition.Data engineering covers acquisition, analytics, and storage; streaming systems and analytics engines support data gathering, provenance, refinement, and sustainment.Source: §III · pp. 3–4
  • Specialized data frameworks use tests and anomaly detectors to identify problematic samples, while AI-, DSP-, and heuristic-assisted labeling helps reduce bias, labeling errors, and manual effort.MSWC and Plumerai Data support data-quality checks; Edge Impulse, MSWC, SensiML DCL, and Plumerai Data offer automated labeling tools.Source: §III · pp. 3–4

IV. FEATURE PROJECTION

  • Optional feature projection reduces input dimensionality, making models operating on intrinsic data dimensions computationally tractable and mitigating the curse of dimensionality.The passage identifies linear, nonlinear, and domain-specific feature extraction as projection approaches.Source: D. Low Deployment Cost · p. 3, §IV · p. 4

A. Linear Methods … B. Structured Sparsity

The paper contrasts dimensionality-reduction and feature-engineering options for microcontrollers, then surveys compression methods and their tradeoffs in model size, speed, accuracy, and hardware support.

A. Linear Methods

  • Linear methods include PCA, NMF, SVD, and ICA,,; they offer specialized representations but struggle with nonlinearities, global relationships, outliers, skew, and one-hot variables.PCA preserves variance, NMF suits sparse interpretable representations, and ICA identifies independent features from non-Gaussian inputs.Source: A. Linear Methods · p. 4

B. Nonlinear Methods

  • Nonlinear methods such as LLE, kernel PCA, t-SNE, and UMAP model manifold structure, but are generally unsuitable for real-time microcontroller execution.UMAP represents global structure more accurately than t-SNE and runs faster through graph approximations; nonlinear methods are often used to visualize handcrafted features.Source: B. Nonlinear Methods · pp. 4–5

C. Feature Engineering

  • Domain-specific feature engineering is often more suitable for microcontrollers because it is lightweight and can use dedicated signal processors, but designing statistically significant features requires human expertise.Feature selection can further reduce redundant features using statistical, correlation, information-theoretic, tree-ensemble, or metaheuristic methods.Source: C. Feature Engineering · p. 5

V. PRUNING, QUANTIZATION, AND ENCODING

  • Joint pruning, quantization, and encoding reduced benchmark DNN sizes by 35–49× without accuracy loss; posttraining quantization cuts size 4× and speeds inference 2–3×.Joint compression reported AlexNet 35× (6.9 MB), LeNet-5 39× (44 kB), LeNet-300-100 40× (27 kB), and VGG-16 49× (11.3 MB). Quantization-aware training can reduce model size up to 8× for the same or lower accuracy drop; mixed-precision quantization provides 7× memory reduction, and binarized quantization provides 8.5–19× speedup and 8× memory reduction. The target hardware must support the selected bitwidth.Source: §V · p. 5, A. Common Model Compression Techniques · p. 5

A. Common Model Compression Techniques

  • Among surveyed frameworks, 88% provide quantization primitives and 50% support pruning; weight pruning provides 4× speedup and 5–10× memory reduction, usually with unstructured sparsity.Intermittent-computing pruning policies sweep memory, energy, and accuracy budgets to build a Pareto frontier; some frameworks support structured channel and filter pruning.Source: A. Common Model Compression Techniques · p. 5

B. Structured Sparsity

  • At the same accuracy, Bayesian compression reduces parameter count by 80×; SIMD-aware pruning yields 3.54× speedup and 88% model-size reduction versus 1.90× and 80% for traditional pruning.SIMD-aware pruning groups weights by the microcontroller’s SIMD width and removes trivial groups using their root mean square.Source: B. Structured Sparsity · p. 6
  • Differentiable channel pruning uses salience masks and resource/pruning feedback to learn layer sizes, providing up to 1.7× speedup while compressing unpruned models by 80×.It learns layer sizes through bilevel continuous gradient descent without additional training overhead.Source: B. Structured Sparsity · p. 6
  • Doped Kronecker-product matrices achieve 2.5–5.5× speedup and 1.3–2.4× higher compression than traditional techniques, with 8% higher accuracy than weight-pruning and low-rank methods.Doping adds an extremely sparse matrix using comatrix regularization to reduce comatrix adaptation during training.Source: B. Structured Sparsity · p. 6

VI. LIGHTWEIGHT MACHINE LEARNING BLOCKS … D. Low-Rank, Stabilized, and Quantized Recurrent Models

The section surveys lightweight machine-learning blocks designed to reduce memory and latency on microcontrollers while retaining model performance. It covers sparse projections, spatial and multiscale convolutions, and recurrent models that are stabilized and compressed.

VI. LIGHTWEIGHT MACHINE LEARNING BLOCKS

  • The lightweight blocks target lower memory and latency while retaining model performance, spanning projections, convolutional modules, and recurrent models.Fig. 2 illustrates examples including sparse projections, fire modules, group convolution, low-rank recurrent matrices, and temporal convolutional networks.Source: §VI · p. 6, Figure 2 · p. 7

A. Sparse Projection

  • Sparse projection maps high-dimensional inputs onto low-dimensional prototypes to reduce parameters and computation; Bonsai and ProtoNN operate on these prototypes.The projection can be learned using stochastic gradient descent and iterative hard thresholding to mitigate accuracy loss.Source: A. Sparse Projection · pp. 6–7

B. Lightweight Spatial Convolution

  • SqueezeNet reduced AlexNet’s size by 510× (<0.5 MB), while MobileNetV1 depthwise separable convolution is 9× cheaper and saves 7–9× memory over 3 × 3 kernels.SqueezeNet uses pointwise filters, bottleneck and excitation layers, and late downsampling; MobileNetV2 introduced inverted residuals and linear bottlenecks.Source: B. Lightweight Spatial Convolution · p. 7
  • Group convolution with channel shuffle improves cross-group channel relations, while channelwise feature concatenation reuses activations and gradients and reduces network parameters.MobileNetV2 places residual connections between bottleneck layers rather than excitation layers.Source: B. Lightweight Spatial Convolution · p. 7

C. Lightweight Multiscale Spatial Convolution

  • EfficientDet combines bidirectional multiresolution feature aggregation with compound scaling and is 4–9× smaller, uses 13-42× fewer FLOPs, and outperforms the listed models in latency and mean average precision.Its feature-network topology is optimized through NAS; the reported comparisons include YOLOv3, RetinaNet, AmoebaNet, Resnet, and DeepLabV3.Source: C. Lightweight Multiscale Spatial… · p. 7
  • Scaled-YOLOv4 saves up to 50% computational budget over vanilla CNN backbones, and batch-normalization removal or fusion with input downscaling speeds multiresolution inference by 3.6–8.8×.The speedup comparison is over vanilla YOLO or MobileNetsV1.Source: C. Lightweight Multiscale Spatial… · p. 7

D. Low-Rank, Stabilized, and Quantized Recurrent Models

  • FastRNN stabilizes gradients with a weighted residual connection, while FastGRNN adds a gate and low-rank, sparse, quantized matrices, making it 35× smaller than gated or unitary RNNs.The FastRNN scalars control the extent of hidden-state updates based on inputs, without significant compute overhead.Source: D. Low-Rank, Stabilized, and Quantized… · p. 8
  • Kronecker recurrent units compress RNNs by 16–50× without significant accuracy loss, while LMUs have 10000× more capacity and are 100× smaller than LSTM.Doping, comatrix adaptation, and comatrix regularization further compress Kronecker recurrent units by 1.3–2.4×.Source: D. Low-Rank, Stabilized, and Quantized… · p. 8

E. Temporal Convolutional Networks … B. Search Strategy

The paper surveys compact temporal and attention-based models, then describes hardware-aware neural architecture search (NAS) through its search spaces and strategies. Across these sections, it highlights design trade-offs and execution-level optimizations for microcontroller constraints.

E. Temporal Convolutional Networks

  • TCNs model temporal sequences without recurrent connections, using dilated kernels and gated residual connections; time-series TCNs can be 100× smaller than CNN-LSTMs,.They convolve only current and past elements, expand receptive fields with fewer parameters or layers than vanilla RNNs, and support parallel, out-of-order training.Source: E. Temporal Convolutional Networks · p. 8

F. Attention Mechanisms, Transformers, and Autoencoders

  • Edge attention designs report 4.17× fewer parameters than MobileNetsV1, up to 507× fewer than previous work [113], and 72× fewer than RefineNet and EdgeSegNet [115].LSRA transformers can be 18× smaller than vanilla transformers with pruning and quantization; transformer distillation enables up to 7.5× smaller and 9.4× faster inference over bidirectional encoder representations. Attention condensers use encoder–decoder modules with self-attention [113], while MobileViT combines convolution and global processing; optimized layouts and quantization have also enabled transformer deployment on microcontrollers. Depthwise and pointwise convolution yield autoencoders as small as 2.7 kB for anomaly detection.Source: F. Attention Mechanisms, Transformers,… · p. 8

VII. NEURAL ARCHITECTURE SEARCH

  • Hardware-aware NAS searches for a network balancing accuracy, latency, and energy under target constraints through search-space formulation, search strategy, and cost function,,.Candidate models are sampled, trained, and evaluated; approaches include reinforcement learning, one-shot gradient-driven NAS, evolutionary algorithms with weight sharing, Bayesian optimization, and training-free NAS.Source: §VII · p. 8, B. Search Strategy · p. 9

A. Search Space Formulation

  • NAS search spaces may be layerwise, cellwise, or hierarchical, trading architectural flexibility against search cost and hardware specialization.Layerwise spaces fix macroarchitecture and optimize remaining serialized layers, but are computationally expensive and not gradient friendly. Cellwise spaces stack repeated directed acyclic graph motifs, are more time-efficient and transferable across datasets, but fix the global architecture and limit hardware specialization. Hierarchical factorized spaces recursively combine blocks and cells while enabling hardware specialization without increasing search cost.Source: A. Search Space Formulation · pp. 8–9
  • iNAS reduced intermittent inference latency by 60% versus architectural-level NAS, with a 7% increase in search overhead.For extreme memory and energy budgets, execution-level search optimizes tile dimensions, loop order, and tile outputs preserved per power cycle; micro-TVM also automates operator and inference scheduling.Source: A. Search Space Formulation · p. 9

B. Search Strategy

  • Reinforcement learning searches candidate cells under device-aware rewards but converges slowly, while differentiable NAS lowers search cost through continuous relaxation and path-level pruning.Q-learning suits small discrete spaces, whereas PPO and DDPG suit complex continuous spaces; PPO is more stable but needs more samples than DDPG,,. Gradient-driven methods use bilevel optimization and path binarization, but large supernetworks demand GPU memory and training time and often require repeating NAS for new hardware,,,.Source: B. Search Strategy · pp. 9–10
  • Evolutionary weight-sharing NAS trains a once-for-all supernetwork for multiple hardware platforms, but supernetwork training, candidate fine-tuning, and slow convergence remain costly.Evolution selects, crosses, and mutates subnetworks, while progressive shrinking and knowledge distillation jointly fine-tune them,,,,.Source: B. Search Strategy · p. 10
  • Bayesian NAS supports black-box optimization across datasets and heterogeneous architectures but degrades beyond a dozen parameters, while training-free proxies and graph neural networks have generalization limits.Bayesian performance depends on its surrogate model, and most optimizers tune fixed architectures; THIN-Bayes supports conditional search across architectures,,. Training-free approaches estimate accuracy using architectural heuristics or graph neural networks, but gradient-based proxies still require GPU calculations, and prediction networks add training cost,,.Source: B. Search Strategy · p. 10

C. Cost Function · D. Cost Function Formulation · E. Hardware Profiling

NAS uses a cost function to guide network search toward configurations that balance model performance with hardware resource use. The cost can be optimized as a single or multiobjective problem, while hardware profiling estimates target-device costs with varying speed and accuracy.

C. Cost Function

  • The cost function feeds the search strategy network accuracy, SRAM and flash use, latency, and energy to seek Pareto-optimal network configurations.Source: C. Cost Function · p. 10

D. Cost Function Formulation

E. Hardware Profiling

  • Hardware-aware NAS uses hardware-specific cost functions or search heuristics, profiling sampled models on the target device in a hardware-in-the-loop approach.Source: E. Hardware Profiling · p. 10
  • Hardware-in-the-loop profiling is slowest but most accurate at NAS runtime, whereas analytical estimation is fastest but least accurate,.Source: E. Hardware Profiling · p. 11
  • Analytical profiling can use FLOPs as a latency proxy, and standard RAM models for working memory; latency predictors improve with kernel detection and adaptive sampling.Source: E. Hardware Profiling · p. 11
  • For intermittent systems, latency includes progress preservation and recovery, battery recharge, and tiled inference; SRAM use sums input, weight, and output maps based on tile dimensions, loop order, and preservation batch size.Source: E. Hardware Profiling · p. 11

VIII. TINYML SOFTWARE SUITES · A. Operator and Inference Optimizations · B. Notable TinyML Software Suites

TinyML software suites prepare trained models for microcontroller deployment by generating embedded code, optimizing operators and inference, and sometimes managing runtime execution and resources. Their designs range from lightweight model converters to specialized inference engines.

VIII. TINYML SOFTWARE SUITES

  • Suites span open-source and closed-source frameworks, code generators, and inference engines; SONIC and TAILS target intermittent computing systems.SONIC introduced 25%–75% latency overhead versus 10× for baseline intermittent frameworks and reduced inference energy by 6.9× over competing baselines; TAILS reduced inference energy by 12.2× over competing baselines.Source: §VIII · p. 11, B. Notable TinyML Software Suites · pp. 12, 14

A. Operator and Inference Optimizations

  • Fused and in-place operators reduced memory usage by 1.6× and improved speedup by 1.2–2×, while loop transformations target execution overhead, locality, and data reuse.Transformations include reordering, reversal, fusion, distribution, unrolling, and tiling; fusion combines operators without storing intermediate feature representations.Source: A. Operator and Inference Optimizations · p. 11
  • Input reuse reduced SRAM usage by 2.57× and provided 2× speedup in TF-Net; PULP-NN register-level reuse achieved 20% speedup over CMSIS-NN and 1.9× improvement over native GAP8-NN.TF-Net reuses unpacked inputs across weight filters; PULP-NN also uses a reusable im2col buffer for partial spatial reuse.Source: A. Operator and Inference Optimizations · p. 12

B. Notable TinyML Software Suites

  • TFLM provided 2.2× speedup and 1.08× memory and flash savings over CMSIS-NN for image recognition.Its operator resolver links only needed operations, while a preallocated arena, two-stack allocation, and runtime interpreter support memory planning and execution.Source: B. Notable TinyML Software Suites · p. 13
  • uTensor generates C++ from TensorFlow models for Mbed-enabled boards, targeting models smaller than 2 kB and providing optimized, legacy, and quantized operators.Its core and library provide runtime structures, interfaces, allocators, contexts, operations, and tensors; it uses an arena and can run CMSIS-NN under the hood.Source: B. Notable TinyML Software Suites · p. 13
  • Micro-TVM generates optimized C code for bare-metal IoT devices, and AutoTVM code was only 1.2× slower than handcrafted CMSIS-NN-based code for image recognition.Its compiler stack applies operator, loop, procedural, and graph-level memory-planning optimizations; AutoTVM searches schedules and predicts their performance.Source: B. Notable TinyML Software Suites · p. 13
  • EdgeML offers lightweight algorithms and tools for Class 0 devices, while CMSIS-NN supplies fixed-point optimized neural-network functions for Cortex-M models.EdgeML includes RNNPool, which supports 4–8× downsampling without accuracy loss while reducing memory and compute; CMSIS-NN achieved 4.6× speedup and 4.9× energy savings over nonoptimized convolutional models.Source: B. Notable TinyML Software Suites · p. 13
  • X-Cube-AI provided 1.3× memory reduction and 2.2× speedup over TFLM for gesture recognition and keyword spotting; EON used 25%–55% less SRAM and 35% less flash than TFLM.EmbML reduced memory usage by 31% and latency by 92% over Sklearn Porter.Source: B. Notable TinyML Software Suites · pp. 13–14

IX. ONLINE LEARNING

On-device models need periodic updates to handle real-world feature-distribution shifts, but bandwidth and privacy can limit server-based training, while training’s memory and energy costs make conventional GPU-based techniques unsuitable for microcontrollers. This motivates on-device training and federated-learning frameworks for microcontrollers.

IX. ONLINE LEARNING

  • Periodic updates are needed after deployment to account for feature-distribution shifts in the wild.Source: §IX · p. 14
  • Limited communication bandwidth and privacy concerns can prevent sending new server-trained models to microcontrollers.Source: §IX · p. 14
  • Training’s memory and energy footprint exceeds inference’s, making traditional GPU-based training unsuitable for microcontrollers.Source: §IX · p. 14
  • On-device training and federated-learning frameworks have emerged for microcontrollers; Table X summarizes features of notable TinyML FL frameworks.Source: §IX · p. 14, Table X · p. 15

A. On-Device Training · B. Federated Learning

On-device training requires detecting data shifts, adapting models within device constraints, and selecting useful training samples. Federated learning distributes this process across devices and must address model aggregation, communication, heterogeneity, scalability, and privacy.

A. On-Device Training

  • On-device training frameworks must detect input shifts, adapt models within device and sample constraints, and select training samples to maximize learning effect.Shift detection methods include feature-distribution divergence, streaming-input means and variances, or prediction confidence scores.Source: A. On-Device Training · p. 15
  • On-device adaptation uses last-layer tuning, specialized operators, or continual and incremental learning, with tradeoffs in forgetting, network compatibility, memory, or compute.TinyTL uses lite residual modules while updating bias, and ML-MCU trains influential one-versus-one classifiers; quantized continual learning stores past activation maps as replay data.Source: A. On-Device Training · p. 15
  • Sample selection can use gradient norms, minority-class oversampling, weighted replay, or sample-importance weighting, but current frameworks are incompatible with popular TinyML suites and limited to simple architectures.The suites cannot unfreeze frozen model graphs on board, motivating additional memory constraints in neural architecture search.Source: A. On-Device Training · p. 15

B. Federated Learning

  • Federated-learning strategies commonly use FedAvg, with variants addressing client disconnection, cross-device accuracy, pruning, personalized updates, or distillation-based aggregation.FedAvg aggregates local model weights at the server rather than gradients to improve communication efficiency.Source: B. Federated Learning · p. 15
  • 5%–11% accuracy improvement over vanilla FedAvg and 10–5000× communication-cost reduction were reported for transfer-learning and knowledge-distillation variants; RL aggregation achieved 1.4%–2.7% higher accuracy over FedAvg.The first comparison is for human action recognition; the RL result is for image recognition.Source: B. Federated Learning · pp. 15–16
  • Flower and DIoT provide bidirectional gRPC and WebSocket communication that supports low-latency, concurrent, asynchronous, and language-agnostic server-client exchange.Most frameworks also rely on pruning or class-probability distillation to improve communication efficiency.Source: B. Federated Learning · p. 16
  • Federated-learning frameworks track client resources and task completion, distribute workloads using methods such as importance sampling or adaptive pruning, and must scale to many clients.Flower has been shown to scale to 15M clients with 1000 concurrent clients.Source: B. Federated Learning · p. 16
  • Vanilla FedAvg can be vulnerable to membership-inference and model-inversion attacks, while Flower uses secure aggregation designed to tolerate client dropouts without trusted hardware.Flower, PruneFL, and TinyFedTL were tested on microcontrollers and support Python, Java, and C++.Source: B. Federated Learning · p. 16

X. KEY APPLICATIONS … D. Activity and Gesture Tracking

The key applications use different variants of the TinyML workflow, with application-specific numerical insights illustrating their resource-conscious model choices and performance.

X. KEY APPLICATIONS

  • The TinyML workflow varies by application, and this section presents numerical insights from application-specific workflow variants.Source: §X · p. 16

A. Image Recognition and Visual Wake Words

  • MCUNetv2 and AttendNets achieve 72%–73% top 1% accuracy on ImageNet for microcontrollers; other approaches target CIFAR-10, segmentation, and specialized vision tasks.MCUNetv2 combines once-for-all NAS with patch-by-patch inference and receptive field redistribution; AttendNets use a visual attention condenser.Source: A. Image Recognition and Visual Wake… · p. 17
  • RaScaNet offers the best balance of accuracy and resource usage for visual wake-word detection, combining convolutional and recurrent blocks with spatial and channel attention.Source: A. Image Recognition and Visual Wake… · p. 17

B. Audio Keyword Spotting and Speech Recognition

  • Lightweight depthwise-separable convolutions, attention condensers, and recurrent units enable audio models around 100 kB; FastGRNN, ShallowRNN, and ULP RNN can run with 2-kB SRAM on AVR RISC microcontrollers.Models typically use log Mel-spectrograms; Google Speech Commands training data comprises 35 words and 105829 utterances from 2618 speakers [10].Source: B. Audio Keyword Spotting and Speech… · p. 17

C. Anomaly Detection

  • Microcontroller anomaly detectors commonly use fully connected autoencoders or depthwise CNNs; DROCC handles raw audio, sensor data, and images without feature extraction.DROCC assumes normal data lies on a low-dimensional linear manifold and augments training with generated outliers; other audio detectors generally use mel-spectrograms.Source: C. Anomaly Detection · p. 17

D. Activity and Gesture Tracking

  • Activity and gesture models commonly use lightweight architectures or shallow, polynomial-complexity conventional models; most reach 90% or more accuracy on simple macrogestures or macroactivities at roughly 100 or 101 kB.Models are mostly hand-tuned, with a few automated using NAS.Source: D. Activity and Gesture Tracking · p. 17

E. Odometry and Navigation … A. Feature Projection Versus No Feature Projection

The section surveys resource-conscious microcontroller applications in odometry, mHealth, and facial biometrics, then examines how feature projection affects model efficiency and performance. It frames these examples as parts of an end-to-end workflow whose choices interact.

E. Odometry and Navigation

  • Odometry examples span TinyOdom, VNNs, and DroNets, which bring neural position estimation, autonomous control, and navigation to microcontroller-class hardware.TinyOdom models are 31–134× smaller than existing neural inertial odometry models. VNNs achieve 97% accuracy; PULP GAP8 reduces latency 13× (0.2–1.2 mS) and energy 3.2× (3.9–18.9 µJ per inference) over Cortex-M architectures. DroNets achieve 6–18 FPS within 64 mW and cover 113 m of unseen indoor trajectory at 1.5 m/s.Source: E. Odometry and Navigation · p. 18

F. mHealth

  • TinyML mHealth applications mostly use anomaly detection to indicate health conditions, enabling ultralightweight models in the order of 100–101 kB.Examples include microphone-based eating and cough monitoring, ECG sleep monitoring and arrhythmia detection, EEG seizure recognition, and earable-sensor fall detection.Source: F. mHealth · p. 18

G. Facial Biometrics

  • Microcontroller face detectors commonly combine lightweight spatial convolution with NAS, quantization, and inference-engine optimizations.Methods include squeeze-and-excitation modules, depthwise and pointwise convolution, nonlinear pooling, rapid downsampling, patch-based scheduling, receptive-field redistribution, and dual-memory management.Source: G. Facial Biometrics · pp. 18–19

A. Feature Projection Versus No Feature Projection

  • For human activity recognition, Bonsai’s five statistical features fail to distinguish similar activities, causing a 17% accuracy drop despite size similar to FastGRNN.The passage gives sitting and sleeping as examples of statistically similar activities.Source: Table XVIII · p. 19
  • DROCC outperforms one-class SVM in anomaly detection by +20% accuracy while reducing model size 1600×; ultralightweight ML blocks can combine efficiency with accuracy.Poorly designed features can add compute overhead, so ultralightweight blocks can be more efficient and accurate than models tied to a feature-extraction pipeline.Source: Table XVIII · p. 19, A. Feature Projection Versus No… · p. 19

XI. DISCUSSION AND CASE STUDIES

  • The discussion breaks down the end-to-end workflow and analyzes how individual techniques affect execution and choices at other workflow stages.Its quantitative analyses use selected examples from Sections IV to X.Source: §XI · p. 19

A. Feature Projection Versus No Feature Projection

  • Feature extraction requires balancing domain knowledge, feasible architectures, feature acceleration, and support from model optimization and architecture-search tools.The passage cautions that lightweight models may lack parameters to generalize to new field data distributions, while many deployment and learning tools lack support for them.Source: A. Feature Projection Versus No… · p. 19

B. Compression Versus No Compression · C. Lightweight Models Versus Vanilla Models · D. Using NAS Versus Handcrafted Models

For microcontroller-class deployment, model choice and optimization must balance accuracy, memory, latency, and hardware/software compatibility. The sections compare compression with lightweight models and NAS with handcrafted design, showing when each approach helps and what constraints qualify its use.

B. Compression Versus No Compression

  • LSTM-Prune and LSTM-KP provide an average 12× model size reduction with only 2.3% accuracy loss versus FastGRNN and Bonsai, supporting sparsification of vanilla models.Across uncompressed and vanilla models, pruning yielded 13.5× average model-size reduction versus 2.1× for FastGRNN.Source: B. Compression Versus No Compression · pp. 19–20
  • Pruning yields 13.6× average flash compression versus 3.9× for quantization, while their combination yields 16×; pruning reduces accuracy by 4.9% versus 0.4% for quantization.Quantization reduces multiplication bitwidth and is more likely than pruning to save SRAM and inference latency; unstructured pruning can add execution overhead.Source: B. Compression Versus No Compression · p. 20
  • Quantization benefits depend on hardware and software support: some microcontrollers cannot exploit low precision, and TFLM does not support arbitrary weight and activation bitwidths.The selected quantization scheme must be compatible with both the microcontroller and TinyML software suite.Source: B. Compression Versus No Compression · p. 20

C. Lightweight Models Versus Vanilla Models

  • When vanilla or pretrained models are too large or non-neural, lightweight models can fit microcontroller limits while maintaining or exceeding vanilla-model accuracy.Lightweight models are also adopted after out-of-memory errors; some can improve accuracy and latency, as FastGRNN does versus RNN.Source: C. Lightweight Models Versus Vanilla… · p. 20, Table XX · p. 20
  • Lightweight models can trade one deployment metric for another or generalization: SqueezeNet has lower size but higher latency and energy than AlexNet, while TCN test accuracy is 5% lower than the listed vanilla models.Under the same test-set distribution shift, lightweight blocks had an 11.8% accuracy drop versus vanilla models; TinyML workflow support also limits model choices.Source: C. Lightweight Models Versus Vanilla… · p. 20, Table XX · p. 20

D. Using NAS Versus Handcrafted Models

  • NAS consistently finds the most accurate model within specified device constraints when accuracy, flash, SRAM, latency, and energy objectives compete.Architectures may trade accuracy against resource use or violate operator constraints, motivating constrained search.Source: D. Using NAS Versus Handcrafted Models · p. 21, Table XXI · p. 21
  • Across target hardware, TinyOdom produces four models with 1.6–30× model-size reduction and 1.2× resolution drop, while MicroNets DS-CNNs outperform handcrafted DS-CNN by 3.3–4.5%.NAS can adapt architectures across microcontrollers with distinct memory and compute budgets, reducing manual search effort.Source: D. Using NAS Versus Handcrafted Models · p. 21, Table XXI · p. 21
  • NAS can address deployment needs that prior expert designs do not: TinyOdom enabled inertial-odometry models on microcontrollers, and µNAS targeted AVR RISC devices.SpArSe pursued DNN deployment and a broader AI-IoT application spectrum; NAS can also reveal achievable performance and suitable architectures in unexplored settings.Source: D. Using NAS Versus Handcrafted Models · p. 21

E. Using Runtime Optimizations Versus No Optimizations · F. Using Online Learning Versus Static Models · XII. CHALLENGES AND OPPORTUNITIES

TinyML deployment requires choosing runtime and software optimizations suited to device constraints, while online learning offers adaptation gains but exposes workflow gaps; new challenges remain.

E. Using Runtime Optimizations Versus No Optimizations

  • With partial im2col, the CIFAR-10 CNN uses 133 kB rather than 332 kB of working memory, and optimized operators reduce latency 4.6× and energy 4.9×.Without the memory reduction, the CNN would overflow Cortex-M7 SRAM [164].Source: E. Using Runtime Optimizations Versus… · p. 21
  • MCUNetv2 achieves record ImageNet and Pascal VOC accuracy on microcontrollers by applying patch-by-patch inference and receptive field redistribution to a model that otherwise overflows SRAM.Source: E. Using Runtime Optimizations Versus… · p. 21
  • TFLM’s interpreter-based runtime supports dynamic memory management, multitenancy, and model-binary updates, but increases flash usage 1.3× and latency 2.6× versus STM32Cube.AI.TFLM’s support for these features is weighed against the STM32Cube.AI comparison in the use case.Source: E. Using Runtime Optimizations Versus… · pp. 21–22
  • Suite selection depends on device and resource requirements, model and training support, intermittent computing, online learning, schedule exploration, dynamic memory, multitenancy, portability, sparsity, and quantization.STM32Cube.AI is limited to STM32 Cortex-M devices but supports nonneural models; TFLM supports broader platforms but only neural networks, while SONIC and TAILS support intermittent computing and micro-TVM offers schedule exploration.Source: E. Using Runtime Optimizations Versus… · pp. 21–22

F. Using Online Learning Versus Static Models

  • On-device adaptation to dataset shifts improves model performance by 34%, while TinyOL adds 10% latency overhead.The comparison concerns online learning versus static models.Source: F. Using Online Learning Versus Static… · p. 22, Table XXIII · p. 22
  • Online learning constrains model and software choices, with frameworks often assuming CNNs or binary classifiers and relying on custom code generators.NAS must account for training memory and inference overhead; limited on-device-learning studies also leave FL effects of NAS, compression, and lightweight ML blocks unclear.Source: F. Using Online Learning Versus Static… · p. 22

XII. CHALLENGES AND OPPORTUNITIES

  • After first-generation work squeezed models into microcontroller memory, compute, and power limits, TinyML frameworks streamlined deployment and addressed many surveyed issues, but new challenges require research.The workflow is designed for a single noncollaborative model, whereas federated learning distributes a global model enhanced through local updates.Source: §XII · p. 22, F. Using Online Learning Versus Static… · p. 22

A. Application Specific Safety and Heuristic Requirements … XIII. CONCLUSION

The review presents a widely applicable workflow for developing and deploying machine learning on microcontroller-class devices, while identifying challenges that next-generation TinyML frameworks must address. These challenges span safety, data uncertainty, adaptation, compatibility, security, and joint hardware-software design.

A. Application Specific Safety and Heuristic Requirements

  • TinyML workflows need logical reasoning within microcontroller resource bounds because neural networks cannot guarantee compliance with domain rules and physical laws.Proposed approaches include trainable neurosymbolic reasoning,, signal temporal logic, and physics-aware embeddings,,,.Source: A. Application Specific Safety and… · p. 22

B. Data Quality and Uncertainty Awareness

  • Missing data, timestamp misalignment, and window jitter can impair complex-event-processing models, so TinyML needs uncertainty-aware training or precise time synchronization,,.These uncertainties can arise from timing delays, clock imperfections, sensor malfunction, memory overflow, or power constraints,.Source: B. Data Quality and Uncertainty… · p. 22

C. On-Device Fine-Tuning

  • Microcontrollers need a balance between on-device fine-tuning and over-the-air updates, because existing adaptation approaches either exceed device limits or have nondeterministic offloading,,.Robustness approaches such as representation learning and domain-adversarial training produce models that do not fit on microcontrollers; whether unsupervised embeddings can be ported remains open.Source: C. On-Device Fine-Tuning · pp. 22–23

D. Backward Compatibility

  • TinyML models need both high accuracy and fidelity to upstream models, since equal accuracy can conceal samplewise inconsistencies and downstream failures.Positive congruent training and backward compatible learning remain to be integrated and optimized for the TinyML workflow.Source: D. Backward Compatibility · p. 23

E. New Security and Privacy Threats

  • TinyML workflows should optimize for adversarial robustness and include attack-surface analysis, since compressed models face attacks and IoT nodes expose sensing and device-level vulnerabilities,,,,.Threats include false-data injection, side-channel leakage, malware, model extraction, access-control attacks, man-in-the-middle attacks, flooding, and routing attacks.Source: E. New Security and Privacy Threats · p. 23

F. Hardware/Software Coexploration

  • TinyML software and hardware need joint optimization, since neural-network workloads have grown while microcontroller platforms shrink and proposed architecture innovations remain largely disjoint from TinyML software,,,,.The proposed direction is attention-directed hardware design that optimizes the platform and model together.Source: F. Hardware/Software Coexploration · p. 23

XIII. CONCLUSION

  • The review introduces a widely applicable workflow for developing and deploying ML on microcontroller-class devices and identifies opportunities for a next generation of TinyML frameworks.Showcased applications illustrate tradeoffs across instances of workflow adoption.Source: §XIII · p. 23

Related papers

Loading 2205.14550v5…