Source-linked AI summary
SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers
Igor Fedorov, Ryan P. Adams, Matthew Mattina, Paul N. Whatmough
TL;DR
MCUs enable widespread, low-power IoT deployment, but severe RAM and flash limits make CNN inference difficult and have encouraged simpler alternatives. SpArSe jointly searches CNN architectures and pruning strategies under these constraints. It finds CNNs that fit devices with as little as 2 KB RAM and are smaller and more accurate than previous SOTA non-CNN models across standard datasets.
Problem
Severe MCU memory constraints prevent conventional CNNs from fitting, while existing MCU-compatible alternatives trade away prediction accuracy.
Method
SpArSe jointly performs neural architecture search and network pruning while optimizing accuracy, model size, and working memory.
Results
SpArSe finds CNNs that are smaller and more accurate than previous SOTA non-CNN models across datasets while satisfying MCU constraints, including devices with 2 KB RAM.
Takeaways & Limitations
CNNs can be designed for deployment on highly memory-constrained MCUs rather than being excluded from these platforms.
Takeaways & Limitations
The memory models used for SpArSe are reductive, though they can be updated as MCU deployment toolchains mature.
Abstract
from arXiv · showhide
The vast majority of processors in the world are actually microcontroller units (MCUs), which find widespread use performing simple control tasks in applications ranging from automobiles to medical devices and office equipment. The Internet of Things (IoT) promises to inject machine learning into many of these every-day objects via tiny, cheap MCUs. However, these resource-impoverished hardware platforms severely limit the complexity of machine learning models that can be deployed. For example, although convolutional neural networks (CNNs) achieve state-of-the-art results on many visual recognition tasks, CNN inference on MCUs is challenging due to severe finite memory limitations. To circumvent the memory challenge associated with CNNs, various alternatives have been proposed that do fit within the memory budget of an MCU, albeit at the cost of prediction accuracy. This paper challenges the idea that CNNs are not suitable for deployment on MCUs. We demonstrate that it is possible to automatically design CNNs which generalize well, while also being small enough to fit onto memory-limited MCUs. Our Sparse Architecture Search method combines neural architecture search with pruning in a single, unified approach, which learns superior models on four popular IoT datasets. The CNNs we find are more accurate and up to $4.35\times$ smaller than previous approaches, while meeting the strict MCU working memory constraint.
1 Introduction
MCUs are inexpensive, power-efficient platforms whose severe memory limits make CNN deployment difficult. SpArSe addresses this challenge by jointly searching architectures and pruning strategies to find accurate CNNs that satisfy MCU memory constraints.
- Motivation: MCUs are small, inexpensive, and power-efficient processors widely used in everyday devices and IoT systems.The paper describes MCUs as approximately 1 cm^2, costing about $1, and consuming about 1 mW.
- Challenge: 420 KB of parameters and 177 KB of intermediate feature maps make LeNet inference impossible on an Arduino Uno with 32 KB flash and 2 KB RAM.The example illustrates why conventional CNNs exceed MCU memory budgets even after 8-bit weight quantization.
- Challenge: CNN deployment on MCUs is primarily blocked by hard RAM and flash-memory constraints.Intermediate feature maps must fit RAM, while model parameters must fit ROM or flash memory.
- Approach: SpArSe combines neural architecture search with pruning to balance generalization performance against model-size and working-memory constraints.The search includes pruning strategies alongside architectural and training hyperparameters, enabling rapid evaluation of many subnetworks.
- Results: SpArSe finds high-accuracy CNNs that fit within 2 KB RAM while outperforming previous MCU-specific and pruned popular architectures across four datasets.The paper also reports models with high accuracy and small model size, while optimizing memory objectives can reduce working memory by more than 11.2x.
2 Related work
Prior efficient-CNN research largely targets mobile phones, whose resources remain far greater than those of MCUs. Existing MCU-focused alternatives and restricted NAS or pruning approaches leave room for jointly optimizing CNN structure under MCU constraints.
- Mobile platforms: Efficient CNN techniques such as depthwise separable layers, deployment-centric pruning, quantization, and NAS have primarily targeted mobile phones.These methods were developed to enable inference on mobile platforms rather than the far more constrained MCU setting.
- MCU gap: Mobile phones still have many orders of magnitude more memory and compute capacity than MCUs.The paper contrasts mobile platforms with MCUs, which represent a much more constrained deployment target.
- MCU-specific models: MCU-oriented prior work includes Bonsai, a pruned shallow decision tree, and ProtoNN, a compressed k-nearest-neighbors method.These alternatives address MCU deployment without using CNN architectures.
- NAS and pruning: Related NAS and structured-pruning methods restrict architectures by optimizing choices such as kernel size and feature-map count within a predefined backbone.The paper positions SpArSe as a broader search over architectures and pruning strategies.
3 SpArSe framework: CNN design as multi-objective optimization
SpArSe formulates CNN design for constrained MCUs as a multi-objective search balancing validation accuracy, model size, and working memory. It explores diverse architectures and pruning configurations using Pareto optimization, Bayesian search, network morphism, and staged refinement.
- Design objectives and representation: SpArSe represents each CNN design by connectivity, weights, operations, and training hyperparameters in a unified design space.The connectivity is a directed acyclic graph, while operations include choices such as convolution and pooling.
- Design objectives and representation: The optimization minimizes validation error, model size, and maximum layer working memory because no single design simultaneously optimizes all three objectives.Model size counts bits for parameters, while working memory accounts for the largest intermediate requirement across layers.
- Design objectives and representation: Pareto optimality identifies CNN configurations that cannot improve one objective without worsening another, allowing designs to be selected under different hardware constraints.This is useful when MCU deployments impose arbitrary limits on subsets of the objectives.
- Search space: The search space varies network depth, width, connectivity, convolution types, kernel sizes, channels, residual connections, multiscale outputs, training parameters, and pruning settings.It includes regular, depthwise separable, and downsampled convolutions, plus optional normalization, activation, pooling, and residual operations.
- Memory and pruning: Pruning reduces model size and working memory by zeroing parameters or removing structured groups while preserving classification accuracy as much as possible.SpArSe considers both unstructured and channel pruning.
- Search procedure: SpArSe combines multi-objective Bayesian and random sampling with network morphism, inherited weights, and coarse-to-fine stages to search efficiently.Morphisms allow pruning proposals to use 2-8× fewer epochs than training comparable proposals from scratch.
4 Results
SpArSe is evaluated across four datasets and against pruning, Bonsai, and other competing methods under parameter and memory objectives. It finds smaller, more accurate CNN configurations and shows that jointly optimizing architecture and pruning is especially valuable for tiny models.
- Evaluation setup: SpArSe evaluates MNIST, CIFAR10, CUReT, and Chars4k, including binary variants, across classification tasks with 10–62 classes.The reported dataset splits and class counts establish the breadth of the evaluation.
- Parameter minimization: SpArSe finds smaller and more accurate models than competing methods across the parameter-minimization experiments.Dominating configurations match or exceed competing test accuracy while minimizing parameter count.
- Memory-constrained deployment: SpArSe outperforms Bonsai across accuracy, model size, and working memory on most reported datasets, including the full ten-class MNIST task.Under the working-memory model, Bonsai exceeds SpArSe only on CIFAR10 in the reported comparison.
- Architecture and pruning: Including pruning in architecture optimization produces roughly an 80x parameter reduction compared with SpArSe without pruning on MNIST.Both experiments sample the same number of configurations, while pruning enables exploration of high-performing sparse subgraphs.
5 Conclusion
The paper concludes that MCUs are a viable target for machine learning despite their severe resource constraints. SpArSe designs CNNs that satisfy MCU memory limits while improving on prior non-CNN approaches across standard datasets.
- Conclusion: SpArSe demonstrates that CNNs can be designed for MCUs with as little as 2KB RAM.The conclusion directly challenges the view that CNNs are unsuitable for MCU deployment.
- Conclusion: SpArSe optimizes CNNs for multiple MCU hardware constraints rather than parameter count alone.The approach combines architecture design and pruning to target deployment requirements.
- Pruning formulation: The pruning formulation supports both unstructured and structured pruning through different choices of weight groups.Unstructured pruning uses individual weights as groups, whereas structured pruning groups related weights such as feature-map parameters.
Appendix B Search space details
The search space exposes architectural and training-relevant choices for convolutional networks, including branches, merge operations, layer types, kernels, and feature-map counts. Discrete variables are specified with bounded ranges and conditional activation rules.
- Search-space notation: Table 5 defines search-space variables using bounded discrete ranges written as lower bound, increment, and upper bound.The table caption specifies the range notation for discrete variables.
- Convolutional blocks: The search space includes convolutional layer types, kernel sizes, and numbers of output feature maps for each block and layer.Supported convolutional types include Conv2D, DownsampledConv2D, and SeparableConv2D.
- Convolutional blocks: Downsampled convolution can optionally prepend a 1×1 convolution that reduces input feature maps before the main operation.This choice is active only for DownsampledConv2D layers.
- Branch structure: Optional left and right branches can route intermediate or input features through fully connected layers before merging with the main path.Each branch is controlled by a Boolean variable and connects to a final fully connected layer after the merge operation.
- Branch structure: Branch-specific weight-fraction variables allocate portions of total fully connected weights to active branches.These variables are conditional on the corresponding branch being enabled and range from 0.01 to 1.
Appendix C Morphism detals
SpArSe generates nearby candidate architectures by applying morphing operations to reference configurations. It then reuses compatible posterior information from the reference during pruning and training.
- Configuration morphisms: A configuration is a morph of a reference when one or more operations from the morphism table transform that reference.Morphism operations define how candidate architectures are generated from existing configurations.
- Configuration morphisms: Each proposed configuration applies randomly selected morphisms to a randomly chosen earlier reference configuration.This procedure keeps each proposal relatively close to its reference architecture.
- Parameter inheritance: Posterior parameters are inherited for identical or operation-compatible nodes before the remaining parameters are learned.For compatible layers with different widths, corresponding parameter subsets are copied whenever possible.
Appendix D Visualization of discovered CNNs
Figure 6 visualizes the architectures that dominated competing methods in Table 2.
- Figure 6 shows the architectures that dominated the competing methods in Table 2.
Appendix E Extended results on interaction of pruning and architecture
This appendix examines how pruning interacts with architecture across the Chars4k, CUReT, MNIST, and CIFAR10-binary experiments.
- Figure 4 shows pruning–architecture interactions for the Chars4k and CUReT experiments.
- Figure 3 presents the corresponding MNIST and CIFAR10-binary experiments.
Appendix F Evolution of winning CNNs
Figure 5 traces the evolution of the CNN architectures that ultimately dominated competing methods on MNIST in Table 2.
- Figure 5 shows the evolution of CNN architectures that ended up dominating competing methods on MNIST.
- The visualized architectures are the ones reported as dominating competing methods in Table 2.
Appendix G Visualization of winning CNNs
The appendix visualizes winning CNNs and related optimization behavior, including parameter-minimizing MNIST architectures, Pareto configurations, and comparisons among competing methods.
- Figure 6 visualizes winning CNNs for MNIST classification with the minimum number of parameters.
- Table 6 lists allowable morphs, while random sampling is always uniform.
- Figure 4 plots |V | against ∥¯ω∥0 for the best-performing configurations.
- Figure 5 shows MNIST configurations over the optimization process, with lighter samples indicating later sampling.
- Against KNN and ProtoNN, the dominating configuration is the same; against RBF-SVM and Bonsai, it is also the same.