PaperlayerView exact revision on arXiv

Source-linked AI summary

PrototypeNAS: Rapid Design of Deep Neural Networks for Microcontroller Units

Mark Deutel, Simon Geis, Axel Plinge

arXiv:2603.15106v2cs.AIcs.CVcs.LG

TL;DR

Running DNNs on diverse resource-constrained embedded systems at low latency is a major challenge. PrototypeNAS decouples architecture and compression search from training, using an MCU-constrained zero-shot multi-objective search and Hypervolume subset selection. PrototypeNAS finds competitive-accuracy DNNs for 12 datasets across three tasks and outperforms TinyNAS and NATS-Bench by 5% in accuracy on average on CIFAR10.

  • Problem

    Running DNNs on diverse resource-constrained embedded systems at low latency is a major challenge.

    Source: §1 · p. 2

  • Method

    PrototypeNAS decouples architecture and compression search from training, using an MCU-constrained zero-shot multi-objective search and Hypervolume subset selection.

    Source: §1 · p. 2, §3 · p. 3, §3.2 · p. 6

  • Results

    PrototypeNAS finds competitive-accuracy DNNs for 12 datasets across three tasks and outperforms TinyNAS and NATS-Bench by 5% in accuracy on average on CIFAR10.

    Source: §1 · p. 2, §5 · p. 16

  • Takeaways & Limitations

    The search considers hundreds of architectures while reducing training to 3–5 DNN candidates.

    Source: §1 · p. 2

    The search space assumes predefined baseline architectures that can be decomposed into repeatable superblocks and use pruning and quantization during training.

    Source: §3.1 · p. 5

Abstract

from arXiv · show

Enabling efficient deep neural network (DNN) inference on edge devices with different hardware constraints is a challenging task that typically requires DNN architectures to be specialized for each device separately. To avoid the huge manual effort, one can use neural architecture search (NAS). However, many existing NAS methods are resource-intensive and time-consuming because they require the training of many different DNNs from scratch. Furthermore, they do not take the resource constraints of the target system into account. To address these shortcomings, we propose PrototypeNAS, a zero-shot NAS method to accelerate and automate the selection, compression, and specialization of DNNs to different target microcontroller units (MCUs). We propose a novel three-step search method that decouples DNN design and specialization from DNN training for a given target platform. First, we present a novel search space that not only cuts out smaller DNNs from a single large architecture, but instead combines the structural optimization of multiple architecture types, as well as optimization of their pruning and quantization configurations. Second, we explore the use of an ensemble of zero-shot proxies during optimization instead of a single one. Third, we propose the use of Hypervolume subset selection to distill DNN architectures from the Pareto front of the multi-objective optimization that represent the most meaningful tradeoffs between accuracy and FLOPs. We evaluate the effectiveness of PrototypeNAS on 12 different datasets in three different tasks: image classification, time series classification, and object detection. Our results demonstrate that PrototypeNAS is able to identify DNN models within minutes that are small enough to be deployed on off-the-shelf MCUs and still achieve accuracies comparable to the performance of large DNN models.

1 Introduction

PrototypeNAS targets the challenge of running increasingly large DNNs on diverse, resource-constrained MCUs at low latency. It combines zero-shot multi-objective search with architectural and compression choices, then selects a small set of candidates for training.

1 Introduction

  • Growing DNN size and computational cost make low-latency inference on diverse, resource-constrained embedded systems a major challenge.The paper frames its problem as quickly and efficiently designing and training DNNs for MCU inference.Source: §1 · pp. 1–2
  • PrototypeNAS combines an ensemble of zero-shot proxies, a search space spanning architecture and compression choices, and Hypervolume subset selection of Pareto-optimal models.The selected 3–5 models represent tradeoffs between accuracy and resource consumption.Source: §1 · p. 2
  • PrototypeNAS architectures outperform TinyNAS and NATS-Bench by 5% in accuracy on CIFAR10 on average.The evaluation covers 12 datasets across image classification, time series classification, and object detection, using an ARM Cortex-M MCU.Source: §1 · p. 2
  • PrototypeNAS considers hundreds of architectures while reducing training to only 3–5 DNN candidates.The method is presented as a resource- and time-efficient alternative to training hundreds of architectures to find one candidate.Source: §1 · p. 2

2 Related Work

Prior work uses NAS to automate architecture design and hardware-aware multi-objective optimization to account for memory and inference speed. The paper identifies imprecise zero-shot proxies and reliance on a single supernet as limitations, alongside established DNN compression methods.

2 Related Work

  • NAS automates DNN architecture design, while hardware-aware NAS treats memory and inference speed alongside accuracy as multi-objective goals.The cited work also includes MCU-focused NAS.Source: §2 · p. 2
  • Zero-shot NAS uses proxy metrics to avoid training many candidates, with proxies based on features such as intermediate-map correlations or input-space linear regions [20] [22].Some hardware-aware approaches derive smaller networks from a large pretrained supernet.Source: §2 · pp. 2–3
  • Zero-shot proxies can be imprecise, bias architectures, and correlate poorly with actual training accuracy; a single supernet also limits exploration to variants around a pre-existing optimum.The paper cites these as challenges in prior zero-shot NAS approaches.Source: §2 · p. 3

3 Method

PrototypeNAS separates architecture and compression search from target-dataset training. Its constrained search explores architecture and compression choices, selects a compact subset of Pareto solutions, then trains and evaluates those candidates.

3.1 Architectural Prototype Exploration

  • The proposed search space combines baseline-architecture selection with structural and size optimization, including pruning and quantization configurations.It assumes predefined baselines decomposable into an initial pattern, repeatable superblocks, and a classifier; Table 1 describes 14 tunable hyperparameters.Source: §3 · p. 3, Table 1 · p. 4, §3.1 · p. 5
  • The constrained multi-objective search minimizes FLOPs and maximizes four zero-shot proxies, subject to target-MCU memory and FLOPs limits.The proxies are MeCo, ZiCo, NASWOT, and SNIP; hardware constraints define the optimization limits.Source: §3 · p. 4, §3.1 · p. 4

3.2 Hypervolume Subset Selection

  • Hypervolume subset selection chooses k solutions from the full Pareto set to maximize the subset's hypervolume.The evolutionary procedure represents solutions as bits, uses mutation and crossover, and repairs genes so exactly k solutions remain.Source: §3.2 · pp. 6–7

3.3 Dataset Evaluator

  • Dataset evaluators train, prune, and quantize selected models, using PTQ for image tasks and object detection and QAT for time-series classification.The time-series experiments used 15 additional QAT epochs after PTQ had produced significant accuracy drops on those datasets.Source: §3.3 · p. 7

4 Evaluation

PrototypeNAS is evaluated across image classification, time-series classification, and person detection, including comparisons with other NAS methods and a random-search baseline. Results cover accuracy and resource tradeoffs, proxy behavior, and search time and emissions.

4 Evaluation

  • Across 12 datasets and three tasks, the evaluation used 500 search trials and selected five architectures for training, with iMXRT1062 memory and FLOPs constraints for the classification experiments.The tasks were image classification, time-series classification, and person detection; detection also included an unconstrained Raspberry Pi 5 experiment.Source: §4 · p. 7, §4.3 · p. 13

4.1 Results for Image and Time Series Classification

  • HSS selected five architectures from a Pareto front jointly considering FLOPs and four proxies, rather than following any single proxy’s or FLOPs’ ranking.The image and time-series searches used different sets of baseline architectures.Source: §4.1 · pp. 8, 10
  • For image and time-series classification, selected models achieved accuracies competitive with or better than related embedded DNNs; RAM needs were similar across candidates, while latency and energy scaled linearly with FLOPs.Tables report test accuracy, RAM, ROM, latency, and energy; image models used PTQ and time-series models used QAT.Source: Table 2 · p. 9, §4.1 · pp. 10–11, Table 3 · p. 10

4.2 Proxy Ensemble Analysis

  • None of the four proxies had consistently high positive Kendall’s τ with accuracy across datasets, but each dataset had at least one proxy outperforming FLOPs; proxy pairs generally showed weak relationships.The proxy-to-accuracy relationships sometimes became negative, while the pairwise proxy relationships were generally weak in both task optimizations.Source: §4.2 · p. 12, Figure 4 · p. 12

4.3 Object Detection

  • For person detection on COCO, the five selected architectures provided meaningful mAP 50–FLOPs tradeoffs at both 128 × 128 and 320 × 320 input resolutions.The experiments used a YOLOv5 detector with an MBedNet backbone; the 128 × 128 setup targeted an iMXRT1062, while the unconstrained 320 × 320 setup targeted a Raspberry Pi 5.Source: §4.3 · pp. 12–13, Figure 5 · p. 13

4.4 Comparison with MCUNet and NATS-Bench

  • On CIFAR10, PrototypeNAS consistently outperforms MCUNet by up to 5% in accuracy at roughly the same computational cost and is competitive with NATS-Bench in both search spaces.For the NATS-Bench comparison, PrototypeNAS models were retrained at 69 × 69 pixels; 5 models were trained for 100 epochs versus 48,393 NATS-Bench models trained for up to 200 epochs.Source: Figure 6 · p. 14, §4.4 · pp. 14–15

4.5 Impact on Performance and Emissions

  • Training five PrototypeNAS models instead of 17 feasible random models reduced execution time by 62%, with a similar average emissions decrease across four regions.Average emissions differed noticeably among the USA, Germany, Japan, and Egypt.Source: Figure 7 · p. 15, §4.5 · p. 16
  • Randomly sampled feasible CIFAR10 models were often only 2–3% behind the full PrototypeNAS models in accuracy, though they were dominated by those models.The comparison evaluated trained and quantized models.Source: §4.5 · p. 16

5 Conclusion

PrototypeNAS is a three-step zero-shot NAS method for rapidly designing DNNs for MCUs. It finds deployable models across 12 datasets and three tasks, with a CIFAR10 comparison favoring its models over those found by TinyNAS and NATS-Bench.

5 Conclusion

  • PrototypeNAS is a novel three-step zero-shot NAS method for rapidly designing DNNs for MCUs.It performs multi-objective optimization with an ensemble of zero-shot proxies over many baseline architectures.Source: §5 · p. 16
  • On CIFAR10, PrototypeNAS models outperform architectures found by TinyNAS and NATS-Bench.The method also finds DNNs that run on standard MCUs.Source: §5 · p. 16
  • Within minutes, PrototypeNAS finds DNNs with competitive accuracies across 12 datasets spanning image classification, time series classification, and object detection.The conclusion states that the found DNNs run on standard MCUs.Source: §5 · p. 16

Related papers

Loading 2603.15106v2…