PaperlayerView exact revision on arXiv

Source-linked AI summary

TinyOL: TinyML with Online-Learning on Microcontrollers

Haoyu Ren, Darko Anicic, Thomas Runkler

arXiv:2103.08295v3cs.LGcs.DCeess.SY

TL;DR

TinyOL addresses the limitation that TinyML models on MCUs are typically static, inference-only deployments that cannot readily adapt to streaming field data and changing conditions. It adds incremental online learning to existing networks and evaluates the approach with autoencoder-based fine-tuning and classification. The experiments report effectiveness and feasibility on an Arduino board, while offline training remains better when sufficient data and computation are available.

  • Problem

    Conventional TinyML trains models offline on powerful machines and deploys static inference-only models, leaving MCU systems difficult to adapt to new data and changing conditions.

  • Method

    TinyOL attaches a trainable online-learning layer to an existing MCU neural network, updating it from streaming samples and modifying its structure when new classes appear.

  • Results

    Experiments on an Arduino board with an autoencoder in anomaly detection and classification scenarios demonstrate TinyOL’s effectiveness and feasibility for on-device post-training.

  • Takeaways & Limitations

    TinyOL can adapt pretrained MCU models to new working conditions and improve performance over time under constrained-resource settings.

  • Takeaways & Limitations

    Offline training outperforms online learning when enough data and computational power are available.

Abstract

from arXiv · show

Tiny machine learning (TinyML) is a fast-growing research area committed to democratizing deep learning for all-pervasive microcontrollers (MCUs). Challenged by the constraints on power, memory, and computation, TinyML has achieved significant advancement in the last few years. However, the current TinyML solutions are based on batch/offline settings and support only the neural network's inference on MCUs. The neural network is first trained using a large amount of pre-collected data on a powerful machine and then flashed to MCUs. This results in a static model, hard to adapt to new data, and impossible to adjust for different scenarios, which impedes the flexibility of the Internet of Things (IoT). To address these problems, we propose a novel system called TinyOL (TinyML with Online-Learning), which enables incremental on-device training on streaming data. TinyOL is based on the concept of online learning and is suitable for constrained IoT devices. We experiment TinyOL under supervised and unsupervised setups using an autoencoder neural network. Finally, we report the performance of the proposed solution and show its effectiveness and feasibility.

I. INTRODUCTION

TinyML brings neural-network inference to resource-constrained MCUs, but conventional deployments leave models static and unable to adapt efficiently to changing devices and environments. TinyOL addresses this gap with incremental online learning on streaming data, demonstrated through MCU experiments.

  • MCUs support widespread IoT sensing and communication but operate under severe power, memory, and computation constraints.MCUs are used as inexpensive tiny computers, with worldwide shipments estimated at 30 billion per year.
  • Existing TinyML frameworks generally train models on powerful machines and flash them to MCUs for inference only.This makes the deployed model a static object that must be retrained and re-uploaded to learn from new data.
  • Remote deployment, machine-specific behavior, limited field-data storage, and evolving input distributions make repeated centralized model updates costly or delayed.Concept drift can reduce performance when the environment changes.
  • TinyOL adds an online-learning layer that updates weights from streaming samples, can modify its structure for new classes, and avoids storing historical training data.The system is implemented in C++ and can attach to an arbitrary existing network on an MCU.
  • Experiments on an Arduino Nano 33 BLE Sense use an autoencoder in supervised and unsupervised scenarios to evaluate on-device post-training and remodeling.The reported results show adaptation to new field data and incremental improvement of the pretrained model.

II. RELATED WORK

TinyML research has advanced efficient edge inference, hardware, and applications, while online learning and edge training remain less developed. The identified gap is training neural networks directly on bare-metal, resource-constrained MCUs.

  • TinyML research spans efficient deep-learning algorithms, ultra-low-power hardware, and deployment-oriented applications.Examples include model compression, architecture design, battery-powered hardware, and MCU inference libraries.
  • Widely used TinyML libraries assume neural networks are trained on powerful machines and flashed to MCUs without on-device training.The cited frameworks support inference optimization but not training on the MCU.
  • Online machine learning receives less attention than traditional batch or offline learning because training data are often assumed to be available in advance.Prior work addresses incremental learning, catastrophic forgetting, or streaming-data libraries.
  • Prior edge-training systems use devices with substantially more resources than constrained MCUs, limiting their applicability to tiny edge devices.The paper characterizes training neural networks on bare-metal MCUs as an unresolved gap.

III. APPROACH

TinyOL extends an existing MCU neural network with an online-learning component that processes each incoming sample and adapts to new classes. This design targets field-specific updates without centralized retraining.

  • A general neural-network model is insufficient because users and machines exhibit different patterns requiring field-data updates.Training models separately for every user is described as time-consuming, while MCU training remains resource-intensive.
  • TinyOL can attach to an existing MCU neural network as a new layer or replace a specific layer.The system processes streaming samples sequentially and updates its weights online.
  • When a new class appears, TinyOL can modify the output-layer structure to accommodate the additional information.The design addresses data-storage and computation constraints on MCUs.

A. The TinyOL System

TinyOL adds a trainable layer after a frozen MCU neural network, updating it incrementally as streaming data arrive. The system is evaluated with autoencoder-based anomaly detection and runtime conversion to anomaly classification.

  • A. The TinyOL System: MCU inference naturally processes streaming samples sequentially, enabling TinyOL to add post-training after each inference.The existing network is extended rather than retrained as a whole.
  • A. The TinyOL System: The additional TinyOL layer runs in RAM and can be customized, initialized, and updated while the pretrained network remains a frozen C array in Flash.The arrangement resembles transfer learning, with fine-tuning concentrated in later layers.
  • A. The TinyOL System: Each streaming sample passes through the existing network, is optionally standardized, produces a prediction, and updates metrics and weights when a label is available.Algorithm 1 summarizes this inference-and-update workflow.
  • A. The TinyOL System: Real-time training on field data adapts models to changing statistical conditions and is presented as protection against concept drift.Without post-training, performance can drop when deployment data differ from training data.
  • B. Concept Design: The evaluation uses an anomaly-detection autoencoder, whose reconstruction error distinguishes normal from abnormal observations.The first use case fine-tunes the existing network; the second modifies the layer structure to create supervised anomaly classification.

1) Use case – fine-tune:

TinyOL addresses concept drift by replacing an autoencoder’s final layer with an online-trainable layer. The layer updates from individual samples, enabling runtime fine-tuning without batch accumulation.

  • Motivation: TinyOL replaces the existing autoencoder’s final layer to post-train it online for field data affected by concept drift.The paper identifies differing field signatures as a source of disrupted performance.
  • Online fine-tuning: The fine-tuning layer uses sigmoid activation and cross-entropy loss for its online update procedure.The update is derived through the chain rule for each connection weight.
  • Online fine-tuning: Each weight is updated after processing one sample rather than accumulating losses across a batch.The learning-rate-scaled update uses the reconstruction target, prediction, activation derivative, and input activation.

2) Use case – multi-anomaly classification:

TinyOL converts a pretrained autoencoder into a runtime multi-anomaly classifier. It uses encoder representations and reconstruction error as features, while expanding the classifier when new anomaly classes appear.

  • Motivation: TinyOL classifies anomaly patterns on the fly, addressing the absence of categorical anomaly information before deployment.The use case targets monitoring and interpretation of different machine irregularities.
  • Features: The classifier receives the encoder output and reconstruction error after scaling them with running mean and variance.These features are extracted from an under-complete autoencoder.
  • Classifier: TinyOL initializes a multi-class classification layer with a weight vector for each anomaly class.The described forward-propagation uses input features X and class weights W.
  • Online updates: With a provided ground-truth label, the system updates classifier weights incrementally for each streaming sample.The update uses the current anomaly classes and learning rate.
  • Online updates: When a new class label appears, TinyOL automatically initializes additional connections for that class.This changes the layer structure to accommodate newly observed anomaly categories.

C. Experimental Design and th Data

The experiments use an Arduino Nano 33 BLE board and a USB fan to collect three-axis vibration data from three fan conditions. The data pipeline reduces the sensor input with PCA before autoencoder processing.

  • Hardware: The test hardware is an Arduino Nano 33 BLE board with a 64 MHz Cortex-M4 CPU, 256KB SRAM, and a three-axis accelerometer.A USB fan simulates an industrial rotating machine.
  • Data: The fan produces three vibration categories: normal operation, stuck, and tilted.Each pattern is recorded from the accelerometer over 40 timestamps at a 119Hz sampling rate.
  • Deployment: TinyOL is integrated with the pretrained autoencoder to run inference, post-training, and fan-mode classification at runtime.The board setup therefore evaluates the system directly on a constrained MCU.
  • Data: Five minutes of data are recorded for each category; normal data trains the autoencoder, while abnormal data supports evaluation.PCA reduces the three-axis data to one dimension, and the autoencoder input size is (40, 1).

IV. EVALUATION

The evaluation examines autoencoder behavior on the board and the effect of concept drift, using vibration patterns and reconstruction outputs. Reconstruction-error distributions separate normal from abnormal samples.

  • Evaluation procedure: The evaluation first measures laptop-trained autoencoder performance, then tests the model on the board under concept drift.It subsequently evaluates TinyOL’s effectiveness and online classification against batch/offline training.
  • Vibration data: Figure 4 compares 3-axis vibration patterns for normal, stuck, and tilted fan states.The patterns correspond to the three operating categories used in the experiment.
  • Autoencoder reconstruction: Figure 5 presents autoencoder inputs alongside their corresponding reconstructed outputs.The plotted examples include anomalous and normal samples after PCA.
  • Anomaly detection: The mean squared error of anomalous samples is much higher than that of normal data.The distributions are computed from 3000 normal and 3000 abnormal samples.
  • Anomaly detection: An anomaly can be classified when its reconstruction error exceeds a fixed threshold.The paper identifies this thresholding procedure as applicable in an industry setting.

B. Fine-Tune

Fine-tuning TinyOL on streaming field data aligns reconstruction-error distributions with training data and supports adaptation to changing working conditions with modest runtime overhead.

  • Fine-Tune: TinyOL was tested by flashing the autoencoder to an Arduino Nano 33 BLE Sense and enabling post-training on the board.The system learned new field patterns incrementally using an attached additional layer.
  • Fine-Tune: Field measurements produced a different reconstruction-error distribution from training data, making a fixed anomaly threshold generate too many false alarms.Minor board-position deviations changed the MSE distribution, so training and inference data could not be assumed to share the same statistical properties.
  • Fine-Tune: After 2000 online fine-tuning iterations, field reconstruction errors had a similar distribution to training-phase data.Each streaming sample was used to fine-tune TinyOL weights via Bluetooth.
  • Fine-Tune: On-device incremental training consumed slightly more time per iteration than inference.Post-training can be disabled after satisfactory accuracy is achieved, limiting its ongoing computational cost.

C. Multi-Anomaly Classification

TinyOL incrementally classifies normal, stuck, and tilted fan states using autoencoder-derived features. Online performance improves with more streaming data and remains competitive with batch learning under limited shuffling, while batch learning can outperform it with sufficient data and epochs.

  • Multi-Anomaly Classification: TinyOL uses the autoencoder’s encoder output and reconstruction error to classify normal, stuck, and tilted fan states incrementally.The classifier learns class patterns from streaming vibration data at runtime.
  • Multi-Anomaly Classification: A new class label automatically expands TinyOL’s layer structure; otherwise, each run updates the existing weights incrementally.Classes are presented sequentially rather than frequently shuffled to reflect the practical fan-switching setup.
  • Multi-Anomaly Classification: Online and offline learning are compared using F1-score and macro F1-score on a pre-collected test dataset.Macro F1-score is the unweighted mean of class-specific F1-scores.
  • Multi-Anomaly Classification: Online performance improves with more streaming data, while the normal-class F1-score remains higher than those of the two abnormal classes.The authors attribute weaker abnormal-class feature extraction to freezing the autoencoder encoder during post-training.
  • Multi-Anomaly Classification: Offline batch learning outperforms online learning with enough data and more than 50 epochs, but online learning remains overall competitive with limited shuffling.Batch training uses more computational resources because it processes the whole dataset at each epoch.

V. CONCLUSIONS AND FUTURE WORK

TinyOL enables on-device incremental online learning, adapting existing neural networks to new working conditions while supporting flexible layer changes. The paper reports effectiveness on an Arduino board and identifies several directions for extending the system.

  • Conclusions: TinyOL adapts arbitrary existing neural networks to new working conditions through incremental online learning and can modify layer structure on the fly.The system is designed for on-device adaptation as data arrive.
  • Conclusions: TinyOL’s performance was evaluated by comparing online learning with offline learning in autoencoder-based anomaly detection and anomaly classification.The evaluation used vibration data from a USB fan on an Arduino board.
  • Conclusions: Offline training outperforms online training when enough data and computational power are available, but TinyOL satisfies the expectations for constrained tiny devices.This comparison frames TinyOL’s feasibility under resource constraints rather than as a universal replacement for offline training.
  • Future Work: Future work includes efficient algorithms for full-size on-device training and support for more neural-network optimizers and operations.Examples include recurrent neural networks.
  • Future Work: Further directions include integrating online dimension reduction, reduced-precision training on 8-bit MCUs, mass management of intelligent IoT devices, and open-sourcing the repository.These directions broaden TinyOL’s algorithmic, hardware, deployment, and accessibility scope.
Loading 2103.08295v3…