Source-linked AI summary
Intelligence at the Extreme Edge: A Survey on Reformable TinyML
Visal Rajapakse, Ishan Karunanayake, Nadeem Ahmed
TL;DR
TinyML commonly runs inference on frugal MCUs, while prior surveys have not comprehensively addressed reformability and deployment methods. This paper surveys reformable TinyML, proposes a taxonomy, and discusses method suitability, applications, challenges, and future directions.
Problem
Prior TinyML surveys largely focus on static systems or specialized domains and do not comprehensively discuss reformability or deployment methodologies.
Source: §1 · p. 3Method
The paper surveys reformable TinyML, proposes a taxonomy of approaches, and reviews TinyML workflow, deployment schemes, and available benchmarking tools.
Source: §1 · p. 3, §7 · pp. 25–26Results
The survey classifies reformable approaches and discusses their suitability, limited benchmarking methods, selected industrial applications, and future directions.
Source: §7 · pp. 25–26Takeaways & Limitations
The paper identifies potential applications for reformable TinyML and outlines challenges and future research directions within the surveyed scope.
Source: §7 · p. 26Reformable TinyML remains immature: available toolsets target static TinyML, and its reliability in real-world applications has yet to be demonstrated.
Source: §6.1.4 · p. 22
Abstract
from arXiv · showhide
Tiny Machine Learning (TinyML) is an upsurging research field that proposes to democratize the use of Machine Learning and Deep Learning on highly energy-efficient frugal Microcontroller Units. Considering the general assumption that TinyML can only run inference, growing interest in the domain has led to work that makes them reformable, i.e., solutions that permit models to improve once deployed. This work presents a survey on reformable TinyML solutions with the proposal of a novel taxonomy. Here, the suitability of each hierarchical layer for reformability is discussed. Furthermore, we explore the workflow of TinyML and analyze the identified deployment schemes, available tools and the scarcely available benchmarking tools. Finally, we discuss how reformable TinyML can impact a few selected industrial areas and discuss the challenges and future directions.
1 INTRODUCTION
Cloud-based processing can add delays to delay-critical IoT applications, motivating computation closer to the data source. TinyML brings ML to frugal MCUs, but reformable solutions address the inference-only assumption by allowing deployed models to be updated.
1 INTRODUCTION
- Cloud distance can delay delay-critical IoT applications, while processing at the extreme edge can reduce latency.The survey presents TinyML as a way to address computational load associated with pure-edge systems that can burden cloud resources.Source: §1 · pp. 1–2
- TinyML uses inexpensive, low-power MCUs for always-on, battery-driven ML applications at the extreme edge.The paper notes that most MCUs operate in the milliwatt range and describes an industry consensus that similar power levels can support coin-cell operation for over a year.Source: §1 · p. 2
- Many TinyML implementations assume inference only, while real-world data can change and motivate models that improve after deployment.The authors argue that such improvement should require minimal human intervention.Source: §1 · p. 3
- Reformable TinyML covers solutions that update embedded models to improve their performance in the environments where they are deployed.Unlike static TinyML’s train-then-test approach, reformable solutions can update through network mechanisms or adaptivity.Source: §1 · p. 3
- The survey proposes a taxonomy of reforming methodologies and discusses their suitability, deployment methods, limited benchmarking frameworks, industrial potential, and future directions.It addresses a gap in prior surveys, which the authors say do not discuss TinyML deployment methodologies.Source: §1 · p. 3
2 OVERVIEW OF TINYML
TinyML brings machine learning to highly constrained MCUs through a workflow of training, optimization, porting, and deployment. Its deployment approaches trade off performance, portability, and reproducibility, while MCU characteristics support edge use but impose practical constraints.
2.1 Definition
- TinyML refers to ML applications using miniaturized models for energy-efficient inference on constrained IoT devices without a resource-rich operating system.The paper identifies Arduino UNO, ESP8266, and ATmega328 as examples of MCU boards.Source: §2.1 · p. 4
2.2 Workflow
- The TinyML workflow has three phases: training, optimization, and deployment, with porting and deployment specific to TinyML.Training and optimization encompass data collection, algorithm selection, model training, and optimization; model optimization can use pruning, distillation, quantization, or encoding, with low-precision quantization potentially degrading performance.Source: §2.2 · pp. 4–5
2.3 Deployment Methods
- TinyML deployment uses manual programming, code generators, or ML interpreters because raw models from popular frameworks exceed MCU memory requirements.Manual programming can optimize performance but costs time and reproducibility; code generation favors convenience but can impede portability; interpreters offer portability with memory and performance overhead.Source: §2.3 · p. 6
2.4 Characteristics of Embedded Systems
- MCUs combine low energy use and low cost with highly heterogeneous resource capacities that make benchmarking difficult.Their small energy budgets can support miniature batteries or energy harvesting, while device resources vary widely across manufacturers and use cases.Source: §2.4.1 · p. 6, §2.4.2 · p. 7, §2.4.3 · p. 7
2.5 Characteristics of TinyML infused MCUs
- Local TinyML processing can improve responsiveness and reduce transmission of raw data, while wireless communication and reformable TinyML resources remain relevant constraints.The survey also notes that beginner resources largely address static TinyML, while reformable implementations have comparatively few resources for replication.Source: §2.5.1 · p. 8, §2.5.2 · p. 8
3 INTRODUCING REFORMABLE TINYML
The survey frames reformable TinyML as a response to static inference and changing data, then classifies solutions by how models are updated. Its taxonomy distinguishes on-device learning approaches from network-reliant approaches and discusses their suitability and constraints.
3 INTRODUCING REFORMABLE TINYML
- The survey introduces local and over-the-air updates, concept drift, article-selection criteria, and a taxonomy for classifying reformable TinyML solutions.The authors selected relevant literature through database searches, iterative reference review, and criteria focused on IoT, highly energy-efficient devices, and reformable learning or update approaches.Source: §3 · p. 8, §3.1 · p. 9
3.1 Overview
- Reformable TinyML can update models locally or over the air, addressing the static-inference assumption in resource-constrained devices.The survey connects reformability to changing environments and model drift, described as performance degradation as data changes.Source: §3.1 · pp. 8–9
3.2 Taxonomy of Reformable TinyML
- The taxonomy divides reformable TinyML into online-learning and network-reliant approaches, then into eager/lazy learning and distributed/over-the-air approaches, respectively.The authors present the taxonomy to organize a fragmented research area and examine distinguishing factors at each level.Source: §3.2 · p. 9
- Online learning updates a model as data arrives, whereas offline learning updates parameters after processing a complete batch.The survey notes that on-device offline learning has been demonstrated, but its overall feasibility remains open because of resource overhead.Source: §3.2 · p. 10
- Lazy learning stores samples for later processing, while eager learning builds a generalized model that requires more training resources but predicts faster.Lazy learning’s knowledge base can scale poorly on MCUs; eager learning avoids that storage requirement but increases training cost.Source: §3.2 · pp. 10–11
- Network-reliant approaches use external entities and include distributed learning and over-the-air updates; distributed learning can use centralized federated or decentralized gossip learning.The survey identifies higher communication overhead and potential energy disparities as concerns for gossip learning, while OTA updates can address post-deployment bugs or security vulnerabilities.Source: §3.2 · p. 11
- Repeated writes to MCU flash or EEPROM can eventually deteriorate storage because these memories have limited write endurance.The survey says storage drawbacks should be considered when designing systems that use these reforming approaches.Source: §3.2 · p. 11
4 STATE OF THE ART IN REFORMABLE TINYML
This section surveys reformable TinyML work across on-device, online, and network-reliant learning, then reviews software implementations and benchmarking suites. It describes varied mechanisms and evaluation results, alongside gaps and constraints in current approaches.
4.2 Online Learning
- Online TinyML spans lazy, eager, and continual learning; reported approaches include streaming-trained layers, class-balancing plugins, and quantized latent replay.The lazy kNN approach was 5–10% less accurate than a pretrained SVM on Raspberry Pi 3B+, while Train++ and QLR-CL report energy, accuracy, and retraining trade-offs.Source: §4.2.1 · p. 13, §4.2.2 · pp. 13–14
4.1 On-device Offline Learning
- Train++ improved accuracy by 5.15–7.3% and reduced training time by ≈10–226 seconds; against Edge2Train, it used ≈34000–65000× less training energy and ≈34–66× less single-inference energy.Edge2Train’s highest reported accuracy was 96.67% versus 90.0% on Iris and 92.85% versus 91.66% on MNIST for CPUs and MCUs, respectively; MCU-ML results indicate on-device training is feasible on low-resource devices.Source: §4.1 · p. 12, §4.2.2 · p. 13
4.3 Network Reliant Approaches
- Network-reliant approaches include federated and geographically collaborative training, plus cloud-to-device model delivery and wireless flashing.Reported federated results include validation accuracy declining with more devices and loss increasing every ten epochs for non-IID data; OTA-TinyML stores retrieved models on MCUs, while FOTA flashes them wirelessly.Source: §4.3.1 · p. 15, §4.3.2 · pp. 15–16
4.4 Analysis of the Software implementations
- The software analysis finds C++ predominant among surveyed implementations, with accuracy, memory usage, and inference time recurring as evaluation metrics.The review describes C++ as energy efficient and close to device hardware; it also links C++ development with greater portability across commodity MCUs.Source: §4.4 · pp. 16, 18, Table 4 · p. 17
4.5 Benchmarking Frameworks for TinyML
- TinyML benchmarking suites address the difficulty of comparing heterogeneous MCUs, but the review describes only two suites.TinyMLPerf covers four benchmarks and multiple metrics with open and closed divisions; another suite evaluates seven MCUs on ten datasets using 30 fully connected networks.Source: §4.5 · p. 18, Table 5 · p. 18
5 POTENTIAL OF REFORMABLE TINYML IN INDUSTRIAL APPLICATIONS
Reformable TinyML can untether low-resource devices from parent nodes, opening application possibilities across industrial IoT, healthcare, smart environments, agriculture, and AR/VR. The section considers how local adaptation and learning could address each domain’s specific needs.
5.1 Industrial IoT
- Always-on TinyML can flag machinery anomalies before escalation, but batch maintenance models need recorded failures and costly MCU data extraction; online learning and federated sharing are proposed alternatives.Previously unseen failures can be difficult to reproduce for training, and extracting data can be cumbersome when MCUs are embedded in machinery.Source: §5.1 · p. 19
5.2 Healthcare
- Wearables could personalize health models through incremental learning and use FL for sensitive measurements, while local TinyML may reduce networking dependence and charging needs.The section also notes that telemetering data outside an MCU’s local network can delay inference, potentially affecting monitored patients.Source: §5.2 · pp. 19–20
5.3 Smart Environments/Cities
- Rural wildlife cameras can incrementally adapt to geographic shifts in illumination, background, and noise; smart buildings can leverage existing networks for OTA model updates.The section presents online learning as prospective where rural communities lack OTA updates, and existing smart-space networks as a basis for updates in cities.Source: §5.3.1 · p. 20, §5.3.2 · p. 21
5.4 Agriculture
- Seasonal crop changes challenge fixed diagnosis models; incremental or online adaptation and federated learning are proposed, with LoRa needed to compensate for weak radio access.The section links network-reliant approaches to the broad coverage needs of agricultural areas.Source: §5.4 · p. 21
5.5 Augmented/Virtual Reality
- Local TinyML reduces off-device gesture-processing latency, while online learning can accommodate person-specific gestures and personalize classifiers.The section notes that gestures vary between individuals, making static offline-trained models prone to false predictions.Source: §5.5 · p. 21
6 OPEN CHALLENGES, FUTURE DIRECTIONS AND NEXT GENERATION TINYML
Reformable TinyML faces constraints in resources, benchmarking, networking, reliability, and security; the paper outlines research directions and potential integrations with other technologies.
6.1 Challenges
- Device heterogeneity complicates benchmarking and energy comparisons, while reformable tools remain immature and their real-world reliability is unproven.Higher resource overheads also limit generalization to lower-resource devices and may require drastic implementation changes that adversely affect performance.Source: §6.1.2 · p. 22, §6.1.4 · pp. 22–23, §6.1.9 · pp. 23–24
- MCU resource scarcity constrains online learning: replay requires storing old-task examples, while training on new classes can cause catastrophic forgetting.Replay is described as fine-tuning with data from old and new tasks, but the required storage is improbable on limited MCU memory.Source: §6.1.3 · p. 22, §6.1.1 · p. 22
- Battery-powered, network-reliant TinyML must manage costly communications and adapt across varied network contexts, including implanted and machine-embedded settings.The paper suggests reducing radio use, choosing among radio access technologies, and limiting transmissions or scheduling them when delay permits.Source: §6.1.5 · p. 23
- Volatile MCU SRAM can erase learning progress on reset or power-down, motivating investigation of ways to save updates.The paper points to Flashing Over-the-Air concepts as a possible mitigation.Source: §6.1.6 · p. 23
6.2 Future Directions
- Future work includes offloading intensive learning to resource-rich devices and reducing redundant model communications through concise application-layer mechanisms.Fog processing is presented as a way to balance radio energy and latency against available processing resources.Source: §6.2.1 · p. 24, §6.2.2 · p. 24
- Online learning needs to address incomplete or noisy data, which can hinder post-deployment algorithm performance without advanced preprocessing.Source: §6.2.3 · p. 24
6.3 Next Generation TinyML
- Blockchain is proposed for secure model or software propagation, while decentralization could support distributed model refinement but increase resource use across devices.Source: §6.3.1 · p. 24, §6.3.3 · p. 25
- Dynamic online-learning models complicate explainability because explanations can become obsolete; serverless computing and newer ML methods are also identified as possible TinyML integrations.For static models transmitted through OTA updates, the paper says explainability can be imperative; serverless resource management may serve demand-driven tasks.Source: §6.3.2 · p. 25, §6.3.4 · p. 25, §6.3.5 · p. 25
7 CONCLUSION
The conclusion frames reformable TinyML as a response to environmental change that many embedded solutions cannot accommodate. The paper surveys its workflows, deployment methods, taxonomy, applications, and future directions.
7 CONCLUSION
- Many TinyML solutions cannot improve embedded models to address environmental dynamicity and volatility.Source: §7 · p. 25
- The paper surveys reformable TinyML as updatable solutions and describes the generic workflow and deployment methods for producing them.Source: §7 · p. 25
- Its proposed taxonomy classifies reformable TinyML by the technology used for improvement and accompanies discussion of suitability, benchmarking, industrial applications, and future directions.Source: §7 · pp. 25–26
8 GLOSSARY
The glossary expands abbreviations used throughout the paper, including terms for learning methods, hardware, networking, and AI.
8 GLOSSARY
- The glossary expands ML, DL, NN, CNN, SGD, GD, SVM, kNN, KD, and FL.Source: §8 · p. 26
- It defines hardware and systems abbreviations including MCU, SRAM, OS, RB, TFLM, TinyML, and CC.Source: §8 · p. 26
- It also expands IoT, RAT, LLN, WSN, OTA, AR, VR, and XAI.Source: §8 · p. 26