PaperlayerView exact revision on arXiv

Source-linked AI summary

A Machine Learning-oriented Survey on Tiny Machine Learning

Luigi Capogrosso, Federico Cunico, Dong Seon Cheng, Franco Fummi, Marco Cristani

arXiv:2309.11932v2cs.LG

TL;DR

TinyML must operate under severe memory, power, and latency constraints. This survey reviews the literature and implementation workflows, finding model optimization well explored while self- and weakly supervised learning still need effort.

  • Problem

    TinyML is difficult under severe architectural, energy, and latency constraints, including kilobyte-scale memory and milliwatt-scale power.

    Source: §I · p. 1

  • Method

    The survey reviews TinyML literature from January 2018 to September 2023, formalizes three implementation workflows, and emphasizes the ML perspective.

    Source: §VIII · p. 15

  • Results

    Model optimization is the most explored area, while self- and weakly supervised learning still require effort and RAFs or VeLO are absent from TinyML works.

    Source: §VII · p. 14

  • Takeaways & Limitations

    The discussion identifies self- and weakly supervised learning, RAFs and VeLO, and co-design as directions for future TinyML research.

    Source: §VII · pp. 14–15

    A recognized TinyML benchmark is lacking amid power, memory, hardware, and software heterogeneity, which may hamper scientific comparison of solutions.

    Source: §VII · p. 15

Abstract

from arXiv · show

The emergence of Tiny Machine Learning (TinyML) has positively revolutionized the field of Artificial Intelligence by promoting the joint design of resource-constrained IoT hardware devices and their learning-based software architectures. TinyML carries an essential role within the fourth and fifth industrial revolutions in helping societies, economies, and individuals employ effective AI-infused computing technologies (e.g., smart cities, automotive, and medical robotics). Given its multidisciplinary nature, the field of TinyML has been approached from many different angles: this comprehensive survey wishes to provide an up-to-date overview focused on all the learning algorithms within TinyML-based solutions. The survey is based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodological flow, allowing for a systematic and complete literature survey. In particular, firstly we will examine the three different workflows for implementing a TinyML-based system, i.e., ML-oriented, HW-oriented, and co-design. Secondly, we propose a taxonomy that covers the learning panorama under the TinyML lens, examining in detail the different families of model optimization and design, as well as the state-of-the-art learning techniques. Thirdly, this survey will present the distinct features of hardware devices and software tools that represent the current state-of-the-art for TinyML intelligent edge applications. Finally, we discuss the challenges and future directions.

I. INTRODUCTION

TinyML brings learning to resource-constrained embedded devices, where memory, power, and latency limits matter especially in safety-critical applications. This survey reviews implementation workflows and learning methods from an ML perspective, covering literature through September 2023.

I. INTRODUCTION

  • Embedded IoT/IIoT systems span smart applications, but onboard ML faces severe memory, power, and latency limits, especially in safety-critical settings.Available memory averages a few kilobytes and accessible power is on the order of milliwatts.Source: §I · p. 1
  • TinyML targets edge ML with minimal processor and memory requirements and power of a few milliwatts or less, while hardware advances prioritize lower consumption.The survey links this hardware focus to constraining the complexity of implemented learning algorithms.Source: §I · p. 2, Figure 1 · p. 2
  • The survey distinguishes ML-oriented and HW-oriented workflows, which separate model design from hardware, from co-design, where ML experts and hardware engineers work together from the start.In co-design, hardware resources can shape algorithms while algorithms influence hardware choices.Source: §I · p. 2

A. MOTIVATION AND CONTRIBUTIONS

  • The survey catalogs TinyML literature through September 2023 and emphasizes the ML perspective, including topics such as meta-learning, RAFs, and VeLO [26].Its introduction notes exponential growth in TinyML research and distinguishes this survey's focus from other surveys.Source: A. MOTIVATION AND CONTRIBUTIONS · p. 2, Figure 2 · p. 3

B. ARTICLE ORGANIZATION

  • The article reviews existing surveys and selection criteria, then covers workflows, learning algorithms, hardware and software tools, future directions, and conclusions.The learning algorithms and techniques are presented as the core of the survey.Source: B. ARTICLE ORGANIZATION · p. 3

II. EXISTING SURVEYS

This section reviews recent TinyML surveys from hardware and application perspectives, describing the technologies and domains they cover.

A. HARDWARE PERSPECTIVE

  • Hardware surveys span ASICs, RISC-V cores and software stacks, MCU frameworks and evaluations,,, and FPGA accelerator design.The MCU evaluation in considers power efficiency, energy per inference, inference efficiency, and inference time.Source: A. HARDWARE PERSPECTIVE · p. 3

B. APPLICATION PERSPECTIVE

  • Application surveys cover TinyML in IoT, including service models and network integration,, and environmental challenges such as pollution monitoring.The environmental survey also considers TinyML devices' limitations in that context.Source: B. APPLICATION PERSPECTIVE · p. 4
  • Other application surveys address TinyML techniques for predictive maintenance and anomaly detection.The predictive-maintenance survey discusses libraries, hardware, datasets, models, and model optimization.Source: B. APPLICATION PERSPECTIVE · p. 4
  • Healthcare coverage includes patient selection and monitoring, data management, and drug-validation trials, especially with wearables; another survey builds a taxonomy across application domains.The taxonomy includes healthcare, smart farming, environment, and anomaly detection.Source: B. APPLICATION PERSPECTIVE · p. 4

C. FIELD VIEWPOINT

The survey distinguishes its ML-focused coverage from prior TinyML reviews and documents a PRISMA-based selection process for the literature it reviews.

C. FIELD VIEWPOINT

  • Prior work covered benchmarking use cases and metrics, deployment techniques, edge training and inference, software-engineering challenges, and reformable TinyML.The benchmarking review [27] selected four use cases and reference datasets and baselines; deployment work compared techniques for reducing inference computation and memory footprints.Source: C. FIELD VIEWPOINT · p. 4
  • Unlike and, which discuss TinyML background, tools, and enablers, this survey emphasizes advanced learning aspects and HW-SW co-design.Source: C. FIELD VIEWPOINT · p. 4
  • Most reviewed works were not based on a well-defined, widely known SLR methodology; the passage identifies and as exceptions.Source: C. FIELD VIEWPOINT · p. 4

III. SELECTION CRITERIA

  • The review followed PRISMA-based SLR guidance, included English peer-reviewed publications, and searched five databases using TinyML and related keywords.The databases were Web of Science, Scopus, IEEE Xplore, ScienceDirect, and Google Scholar.Source: §III · p. 5
  • Among 4,233 retrieved records, 3,413 duplicates, 271 automation-ineligible papers, and 549 inaccessible papers were excluded; title-and-abstract screening excluded 3,341, selected 189 articles, and left 172 eligible.Source: §III · p. 5
  • None of the 172 reviewed papers was a survey of ML-oriented TinyML techniques, motivating this work's claim to be the first systematic review on that topic.Source: §III · p. 5

IV. TINYML WORKFLOWS

TinyML systems can be developed through ML-oriented, HW-oriented, or co-design workflows, which differ in their starting point and how model and hardware development are coordinated.

IV. TINYML WORKFLOWS

  • In the ML-oriented workflow, model design, training, and evaluation receive most attention while the hardware choice remains fixed or limited.The model is optimized, evaluated against requirements, adapted for the target device, and evaluated in production.Source: §IV · p. 5
  • In the HW-oriented workflow, developers design or enhance embedded hardware for current and future ML algorithms, including accelerator modules for performance and consumption.The workflow includes hardware design, benchmark assessment of optimized hardware, and physical-device evaluation.Source: §IV · pp. 5–6
  • Co-design intertwines model optimization and hardware design from the outset, with ML and hardware specialists collaborating to co-optimize the system.Examples include bespoke FPGA architectures for ML algorithms [41] and customized analog compute-in-memory accelerators using hardware-informed training.Source: §IV · p. 6, Figure 4 · p. 6
  • The co-design workflow defines device requirements across ML and hardware groups, analyzes alternatives, selects components cooperatively, and evaluates target-specific optimizations.Source: §IV · p. 6

V. LEARNING PANORAMA UNDER THE TINYML LENS

The survey organizes TinyML learning methods into model optimization, model design, and learning algorithms, with optimization techniques and pruning among its covered topics.

V. LEARNING PANORAMA UNDER THE TINYML LENS

  • The proposed TinyML learning taxonomy groups its coverage into model optimization, model design, and learning algorithms.Source: §V · p. 6, Figure 5 · p. 7

A. MODEL OPTIMIZATION

  • Model optimization techniques aim to reduce memory footprint, energy consumption, and inference latency for deployment on resource-limited devices.The survey identifies pruning, quantization, knowledge distillation, and hyper-parameter optimization as covered techniques.Source: A. MODEL OPTIMIZATION · p. 7

1) Pruning

  • Pruning can compress models while maintaining or sometimes increasing accuracy, although differing goals and evaluations leave benchmarks and metrics unstandardized.It may be applied during training, where it regularizes the model and mitigates overfitting, or after training to remove redundant connections and parameters for efficiency.Source: 1) Pruning · p. 7

2) Quantization

The section surveys quantization, knowledge distillation, and hyper-parameter optimization as techniques for making models more suitable for resource-constrained devices. It describes their approaches and the trade-offs they address.

2) Quantization

  • Quantization reduces tensor bit widths to compact model representations and enable efficient vectorized operations, with inference-cost reductions while maintaining accuracy.QAT fine-tunes a quantized pretrained model; PTQ quantizes weights and activations after floating-point training, with 8-bit variants including weight-only and full-integer quantization.Source: 2) Quantization · pp. 7–8

3) Knowledge Distillation

  • Distillation transfers knowledge from a large teacher to a smaller student, balancing model size against acceptable accuracy for resource-constrained deployment.The survey distinguishes response-based knowledge (last-layer predictions), feature-based knowledge (last and intermediate layers), and relation-based knowledge (relationships among layers or samples).Source: 3) Knowledge Distillation · p. 8
  • Distillation schemes include offline training, simultaneous online teacher-student updates, and self-distillation, where teacher and student share an architecture.Online distillation is presented as suitable when the teacher is not significantly larger or higher performing.Source: 3) Knowledge Distillation · p. 8

4) Hyper-Parameter Optimization (HPO)

  • HPO automates searches over settings such as learning rate, batch, and network size, reducing tuning effort and time while improving model performance.The surveyed search methods include Grid Search, Random Search, and Bayesian Optimization.Source: 4) Hyper-Parameter Optimization (HPO) · p. 8

B. MODEL DESIGN

The model-design section surveys architecture exploration and modifications intended to balance accuracy and efficiency for TinyML applications. It covers NAS, rational activation functions, and convolutional-layer designs.

1) Neural Architecture Search (NAS)

  • NAS automates architecture exploration using reinforcement learning, evolutionary, or gradient-based search, and can include data granularity choices for resource-constrained systems [102].The survey reports that NAS reduces design effort and time while improving task performance –,, and discusses MCU-targeted work,, and constraints.Source: 1) Neural Architecture Search (NAS) · pp. 8–9

2) Rational Activation Functions (RAFs)

  • RAFs are trainable with backpropagation and integrate into networks like common activations; they offer flexible shapes and potential approximation advantages [105],,.The survey presents their accuracy–computational-cost balance as a potential avenue for compact TinyML models, while noting limited prior research.Source: 2) Rational Activation Functions (RAFs) · p. 9

3) Convolutional Layers

  • Depth-separable convolution replaces a standard convolution with point-wise 1 × 1 and spatial operations without an intermediate nonlinear activation, preserving regular-convolution output shape while using significantly fewer parameters.MobileNet and MobileNetV2 use depth-wise separable layers [107],; Tiny-Sepformer uses a 1D depthwise-separable convolution in its convolution-attention block [110].Source: 3) Convolutional Layers · p. 9
  • The survey also describes Xception as interpreting Inception modules as an intermediate step between regular and depthwise-separable convolution.Source: 3) Convolutional Layers · p. 9

4) Attention Mechanism

The section introduces attention and surveys learning paradigms and their TinyML applications. It covers supervised and unsupervised learning, including examples in anomaly detection.

4) Attention Mechanism

  • Attention dynamically assigns importance to input parts relevant to context, supporting salient-information extraction and context-aware predictions across tasks such as vision, speech, and language.The survey cites AttendNets as a self-attention architecture for on-device visual perception, including image recognition [114].Source: 4) Attention Mechanism · p. 9

C. LEARNING ALGORITHMS

  • The learning-algorithm survey covers supervised, unsupervised, self-supervised, and deep reinforcement learning, plus weak supervision, meta-learning, and continual learning for resource use.Table 3 summarizes the popularity of learning algorithms over the years based on referenced research contributions.Source: C. LEARNING ALGORITHMS · pp. 9–10, Table 3 · p. 10

1) Supervised Learning

  • Supervised learning trains on labeled input-output pairs but depends on annotated data; TinyML work addresses annotation gaps through sample generation and uncertainty forecasting,.Some approaches first train larger supervised models and distill their knowledge into smaller models, or use parallel computation.Source: 1) Supervised Learning · p. 10

2) Unsupervised Learning

  • Unsupervised learning finds patterns in unlabeled data and is used for anomaly detection in industrial settings and other scenarios,,.Examples include an Arduino Nano autoencoder for washing-machine anomalies, road-anomaly detection using TEDA, and urban-vehicle CO2 monitoring.Source: 2) Unsupervised Learning · p. 10

3) Self-Supervised Learning (SSL)

The section describes self-supervised, deep reinforcement, and weakly supervised learning in TinyML. It presents their training approaches and examples involving anomaly detection, constrained-device deployment, and reduced labeling needs.

3) Self-Supervised Learning (SSL)

  • SSL trains models on surrogate prediction or reconstruction tasks without external annotations, learning data structure useful when labeled anomaly examples are scarce or costly.The survey describes an IoT anomaly-detection method using a multivariate LSTM autoencoder, which treats deviations from a learned distribution as anomalous data.Source: 3) Self-Supervised Learning (SSL) · pp. 10–11

4) Deep Reinforcement Learning

  • Deep reinforcement learning models can be transferred to constrained devices; surveyed MCU examples include autonomous driving with joint computation-energy and caching-cost consideration.That policy-gradient approach is reported to reduce final-model energy cost; another example trains a tiny CNN for an electrically actuated tilting maze.Source: 4) Deep Reinforcement Learning · p. 11

5) Weakly Supervised Learning

  • Weakly supervised learning uses partial or noisy labels, reducing labeling needs; one anomaly-detection approach uses expert labels to remove unnecessary Isolation Forest trees.An industrial anomaly-detection ensemble is described as scalable, with predictable effects on memory footprint and inference delay.Source: 5) Weakly Supervised Learning · p. 11

6) Meta Learning

Meta-learning adapts the learning algorithm across episodes and is organized around meta-representation, meta-optimizer, and meta-objective. Continual learning aims to learn from new data while retaining acquired knowledge, addressing catastrophic forgetting.

6) Meta Learning

  • Meta-learning is organized around meta-representation, meta-optimizer, and meta-objective, the three axes shaping neural-network meta-learning for TinyML.Unlike fixed algorithms applied independently to tasks, meta-learning improves the learning algorithm using experience across episodes.Source: 6) Meta Learning · p. 11
  • Meta-representation learns shared patterns across tasks as a knowledge base for adapting to new tasks with limited data.MetaLDC [143] meta-trains low-dimensional classifiers offline and uses closed-form loss gradients to adapt the binary class layer.Source: 6) Meta Learning · p. 11
  • Meta-optimizers learn to update model parameters for fast adaptation; TinyReptile [47] collaboratively learns a neural-network initialization across tiny devices.Source: 6) Meta Learning · p. 11
  • Meta-objectives guide adaptation and generalization; related work includes ANP and applying LTH to meta-learning for IoT-device adaptation.Source: 6) Meta Learning · pp. 11–12

7) Continual Learning

  • Continual learning seeks to incorporate new data while preserving prior knowledge, because conventional models can suffer catastrophic forgetting.Source: 7) Continual Learning · p. 12
  • Continual-learning methods are classified as regularization-based, replay-based, or dynamic architectures, which respectively penalize loss, replay stored data, or adjust model capacity.Source: 7) Continual Learning · p. 12
  • For TinyML, regularization approaches add loss penalties to prevent overfitting and forgetting; demonstrates an IoT MCU setting with gesture-recognition and image-classification tasks.Source: 7) Continual Learning · p. 12
  • Replay-based approaches include latent-space replay with 8-bit execution using less than 64MB, local MCU training in Train++ [148], and replay-generated incremental models in TyBox.Source: 7) Continual Learning · p. 12

VI. TINYML DEVICES AND TOOLS

TinyML relies on hardware platforms including CPUs, GPUs, FPGAs, and TPUs. The survey compares their advantages and disadvantages and describes CPU, GPU, and FPGA characteristics relevant to ML workloads.

VI. TINYML DEVICES AND TOOLS

A. CENTRAL PROCESSING UNIT (CPU)

  • Microcontrollers, particularly Arm Cortex-M devices, are widely used for low-power ML deployment, but can include unnecessary operations and fail to exploit deep-learning parallelism.Their minimal power needs suit settings where battery replacement is difficult.Source: A. CENTRAL PROCESSING UNIT (CPU) · p. 12

B. GRAPHICS PROCESSING UNIT (GPU)

  • GPUs use thousands of ALUs for parallel operations, suiting ML matrix and vector computations; their high-bandwidth memory access supports handling large datasets and rapid data transfer.Source: B. GRAPHICS PROCESSING UNIT (GPU) · p. 13

C. FIELD-PROGRAMMABLE GATE ARRAY (FPGA)

  • FPGAs configure logic blocks and interconnects for different ML applications, while specialized DSP and memory blocks accelerate operations such as convolutions and dot products.The survey describes FPGAs as targeting performance, efficiency, and scalability for ML computations.Source: C. FIELD-PROGRAMMABLE GATE ARRAY (FPGA) · p. 13

D. TENSOR PROCESSING UNIT (TPU)

The TPU section describes hardware organized for tensor operations, while the software section surveys tools for deploying and optimizing TinyML models. The discussion maps research activity and identifies unresolved constraints in models, data, and benchmarking.

D. TENSOR PROCESSING UNIT (TPU)

  • A TPU’s systolic array executes tensor operations in parallel, supported by high-bandwidth memory and an interconnect; Edge TPUs are available in several Coral form factors.The array consists of processing elements in a two-dimensional grid, each connected to neighboring elements.Source: D. TENSOR PROCESSING UNIT (TPU) · p. 13

E. SOFTWARE TOOLS

  • TinyML software tools range from OS-free microcontroller inference in TensorFlow Lite Micro to cloud-based training and edge export in Edge Impulse, STM model conversion in X-CUBE-AI, and MinUn’s resource optimizations.MinUn addresses lower-bit number representations, bandwidth assignment, and memory fragmentation.Source: E. SOFTWARE TOOLS · pp. 13–14

VII. DISCUSSION

  • Model optimization is the most extensively explored area, with work on pruning, quantization, and knowledge distillation, while HPO research remains scarce.The survey attributes HPO’s scarcity to its complexity, limited awareness of its potential, and resource requirements.Source: §VII · p. 14
  • NAS dominates model-design research, but the survey found no TinyML work using RAFs or VeLO and identifies them as possible future directions.Attention mechanisms and depth-separable convolutions are also prominent in model design, [117].Source: §VII · p. 14
  • Unsupervised and continual learning receive substantial attention, while self-supervised and weakly supervised learning still need further work; anomaly detection dominates applications, which mostly use deep learning.The survey notes TEDA-based approaches as a non-deep-learning exception.Source: §VII · p. 14
  • There is no one-size-fits-all choice among CPUs, GPUs, FPGAs, and TPUs, and the survey expects co-design to be a primary direction for future developments.The authors describe early co-design work as promising.Source: §VII · p. 15
  • TinyML lacks a recognized benchmark amid low-power, memory, hardware, and software heterogeneity, and the survey highlights the open question of a broadly representative dataset.Existing TinyML datasets include examples for on-device online training, but the field is mainly concerned with sensor processing.Source: §VII · p. 15
  • TinyML remains constrained by small SRAM and flash, the need for relevant real-world data to assess failures beyond accuracy, and the lack of a widely accepted MCU model.The survey also notes that conventional infrastructure has widely accepted models such as MobileNet, unlike the MCU ecosystem.Source: §VII · p. 15

VIII. CONCLUSION

This survey reviews TinyML research from January 2018 to September 2023 and formalizes three implementation workflows. It emphasizes the ML perspective alongside hardware and software, and identifies future research directions and unresolved problems.

VIII. CONCLUSION

  • The survey presents a systematic review of TinyML literature from January 2018 to September 2023.It summarizes the review process in Figure 3.Source: §VIII · p. 15
  • It formalizes three workflows for implementing TinyML systems and emphasizes the ML perspective, including recent ML technologies practitioners may explore.The review also examines hardware-device advantages and disadvantages and current TinyML frameworks.Source: §VIII · p. 15
  • The survey identifies promising areas for future research and lists unresolved problems that need to be addressed to advance the field.Source: §VIII · p. 15
Loading 2309.11932v2…