Source-linked AI summary
On the Instance Hardness as a Decision Criterion in TinyML Systems
Tobiasz Puslecki, Krzysztof Walkowiak
TL;DR
TinyML needs inference methods that respect limited device resources while maintaining classification quality and controlling energy use. This paper applies TDP-based instance hardness with DES-Clustering, routing samples by a tunable threshold; preliminary results indicate substantial energy reductions with limited classification-quality decreases. The work remains a proof of concept, with threshold control based on context proposed for future work.
Problem
TinyML inference must balance limited memory and computing resources, classification quality, and energy consumption as operating conditions change.
Method
The paper uses normalized pruned-tree depth as instance hardness to route easy samples to the tree and difficult samples to DES-Clustering through a tunable threshold.
Results
Appropriate threshold selection significantly reduces energy consumption with only a limited decrease in classification quality.
Takeaways & Limitations
Threshold control can adjust classification accuracy and thereby influence computational complexity and inference energy consumption in TinyML applications.
Takeaways & Limitations
The approach relies on latency as a leaf-depth proxy, although prediction cost is not perfectly linear because path lengths and tree structures vary.
Abstract
from arXiv · showhide
TinyML includes the implementation of machine learning on devices with limited memory and computing resources. With the development of technology, AI systems continue to scale in terms of size and computational requirements. This forces researchers to adapt methods to be environmentally sustainable by designing techniques for reducing computational costs and energy consumption in inferring AI models, even in small devices. In this work, we present preliminary findings on a novel application of the tree depth prune instance hardness method to the TinyML system. The results indicate that threshold control can change energy consumption with limited classification quality changes. This method allows us to adjust classification accuracy, thereby influencing computational complexity and energy consumption for inference. We present a work in progress with initial results as a proof of concept.
1 Introduction
TinyML must balance constrained-device inference with accuracy and energy use as operating conditions change. The paper proposes TDP-based instance hardness with DES-Clustering to route easy and difficult samples differently.
- TinyML systems operate under limited memory and computing resources while needing lightweight, accurate, and energy-efficient inference.
- Changing data distributions, battery status, stream intensity, and external energy availability complicate TinyML operation.
- Instance hardness measures classification difficulty, enabling easier samples to use simpler models and harder samples to use more complex ones.
- Dynamic Ensemble Selection chooses competent classifiers for each test instance, using ensembles whose diversity supports generalization.
- Prior work used instance hardness to send easy samples to simple classifiers and difficult samples to complex dynamic ensembles, targeting lower complexity without significant accuracy loss.
- The paper explores TDP as an instance-hardness metric combined with DES-Clustering for a memory- and time-efficient TinyML solution.
2 Method
The method uses normalized pruned-tree leaf depth to estimate instance hardness and route samples between a pruned tree and DES-Clustering. A threshold controls this routing and therefore the accuracy–complexity–energy trade-off.
- TDP uses normalized leaf depth from a pruned decision tree as instance hardness, routing easy cases to the tree and difficult cases to DES-Clustering.
- DES-Clustering selects classifiers using accuracy and diversity within K-means-defined competence regions.
- Latency can proxy leaf depth to save memory, assuming harder instances traverse deeper paths and require longer inference.
- Cost-complexity pruning removes branches with small impurity gains, leaving difficult near-boundary samples at greater node depths.
- Higher thresholds send more instances to the simple TDP tree, whereas lower thresholds send more to DES, with intermediate values trading accuracy against energy and latency.
3 Experimental evaluation
The evaluation varies instance-hardness thresholds across datasets and dynamic-selection methods using repeated stratified cross-validation. Accuracy generally decreases with threshold changes, while the energy proxy consistently decreases.
- Experiments use Digits, Vehicle, and Wine with 5-times repeated stratified 2-fold cross-validation.
- The classifier pool is a Random Forest with 25 estimators and maximum depth 10, while K-means uses k=5 to define competence regions.
- The study varies IH thresholds and compares DES-Clustering, KNORA-U, and KNORA-E, using accuracy and an energy proxy based on samples sent to the heavier DES classifier.
- Accuracy generally trends downward as thresholds change, except for KNORA-U on Wine and KNORA-E on Digits.
- Energy-proxy values consistently decrease and never increase across the evaluated methods and thresholds.
4 Conclusions
The paper presents TDP plus DES-Clustering as a TinyML approach for adjusting classification quality and inference cost. Preliminary results indicate that threshold selection can reduce energy consumption with limited classification-quality loss.
- The proposed TDP and DES-Clustering combination is intended as a memory- and time-efficient solution tailored to TinyML systems.
- Appropriate threshold selection can significantly reduce energy consumption with only a limited decrease in classification quality.
- Threshold adjustment changes classification accuracy and thereby influences inference computational complexity and energy consumption.
- The work is preliminary, with future plans to control thresholds using context such as battery level or photovoltaic energy availability.