PaperlayerView exact revision on arXiv

Source-linked AI summary

NAS-Driven Hardware Accelerator Exploration for Edge AI and Quantization Effects on the Pareto Space

Eleftherios Mylonas, Angelos Kouprizas, Michael Birbas, Alexios Birbas

arXiv:2608.13293v1cs.AI

TL;DR

Post-search quantization is used for deployment, but its effects on NAS-discovered Pareto structure remain uncharacterised. This paper pairs a three-stage architecture search, quantization, and CGRA mapping pipeline with a full-search-space INT4 study. Its FP32 zero-shot surrogate outperforms a dedicated INT4-trained surrogate in normalised hypervolume across two search strategies.

  • Problem

    Post-training quantization’s effects on NAS-discovered Pareto structure remain uncharacterised at full-search-space scale, despite its use in deployment-focused work.

    Source: C. State-of-the-art & Related work · p. 3

  • Method

    A three-stage pipeline integrates architecture search, Pareto-filtered quantization, and evolutionary CGRA mapping, alongside a full-search-space study of INT4 PTQ.

    Source: §I · p. 2

  • Results

    The FP32 zero-shot surrogate outperforms the dedicated INT4-trained surrogate in normalised hypervolume across two search strategies.

    Source: §I · p. 2

  • Takeaways & Limitations

    The cleaner FP32 training signal appears to transfer effectively across precision regimes despite quantization-induced Pareto reorganization.

    Source: §V · p. 6

    The empirical study covers INT4 PTQ on NAS-Bench-201, with more complex mixed-precision schemes left for future work.

    Source: §V · p. 6

Abstract

from arXiv · show

Edge AI deployment demands neural architectures that are simultaneously accurate, computationally efficient, and hardware-deployable - a challenge addressed by hardware-aware Neural Architecture Search (NAS). While recent works incorporate quantization directly into the NAS loop, these approaches expand search complexity and tightly couple architecture and quantization design. The simpler post-search quantization strategy has received little analytical attention: the effects of Post-Training Quantization (PTQ) on the NAS-discovered Pareto structure remain uncharacterised, and no framework combines quantized architecture mapping onto reconfigurable accelerators with automated hardware exploration. This paper addresses both gaps. First, a three-stage pipeline is proposed: a hardware-agnostic Pareto rank surrogate frontend on NAS-Bench-201, a quantization bridge with Pareto-aware filtering and feedback control, and an evolutionary Domain Space Exploration (DSE) backend on CGRA4ML for optimal hardware mapping. Second, an empirical study characterises how INT4 PTQ perturbs the NAS-Bench-201 Pareto space through formal stability metrics on ground-truth data for all 15,625 architectures, and demonstrates that an FP32 zero-shot surrogate outperforms a dedicated INT4-trained surrogate in Pareto space coverage across two standard search strategies.

I. INTRODUCTION

The paper proposes a three-stage hardware-aware NAS workflow linking full-precision architecture selection, quantization, and evolutionary CGRA hardware exploration. Its INT4 study characterises Pareto stability and finds that an FP32 zero-shot surrogate outperforms a dedicated INT4-trained surrogate in normalised hypervolume.

I. INTRODUCTION

  • Edge AI’s privacy, latency, and offline-autonomy benefits accompany computing, memory, and power constraints, motivating accurate, hardware-deployable model design through NAS [1].Automated architecture search addresses the combinatorial design space without manual trial-and-error across heterogeneous hardware platforms.Source: §I · p. 1
  • The introduction identifies gaps in NAS benchmark coverage, quantization effects on multi-objective estimators, and DSE for flexible reconfigurable accelerators.Most benchmarks prioritise FP32 accuracy; HW-NAS-Bench [2] adds performance data for fixed, almost exclusively FP32 hardware targets without examining quantization effects. Exhaustive search is intractable, while estimation quality depends on principled strategies and informative architecture encodings.Source: §I · p. 1
  • The three-stage workflow combines full-precision candidate selection, quantization and failure filtering, and evolutionary optimisation of a CGRA accelerator for each surviving architecture.The hardware-agnostic frontend is trained on NAS-Bench-201 [3] and CIFAR-10.Source: §I · pp. 1–2
  • The FP32 zero-shot surrogate outperforms a dedicated INT4-trained surrogate in normalised hypervolume across two search strategies.Using Brevitas, the INT4 PTQ study computes formal Pareto stability metrics across all 15,625 NAS-Bench-201 architectures and provides three-way transferability analysis of FP32 Pareto rank predictions under quantization-induced distribution shift.Source: §I · p. 2
  • The CGRA4ML [5] DSE backend maps quantized Pareto survivors using a free analytical performance oracle and a three-term normalised scalar fitness function.The environment seeks the optimal hardware mapping for each architecture surviving the quantized Pareto filter.Source: §I · p. 2

II. THEORETICAL BACKGROUND · A. NAS

NAS automates neural architecture design through a search space, a search strategy, and performance estimation. Precomputed benchmarks support reproducible comparisons without training every candidate from scratch.

A. NAS

  • NAS automates neural architecture design, seeking an architecture and associated weights that maximize validation accuracy across the search space.Source: A. NAS · p. 2
  • The search space defines candidate architectures, including cell-based designs constrained by fixed operations and connectivity patterns.Source: A. NAS · p. 2
  • The search strategy samples candidate architectures from the search space using approaches such as evolutionary algorithms or reinforcement learning.Source: A. NAS · p. 2
  • Performance estimation evaluates or predicts each candidate’s quality using one or more metrics, most commonly accuracy.Source: A. NAS · p. 2
  • Architecture encoding transforms a candidate’s structural details into a vector and can improve performance prediction accuracy.Source: A. NAS · p. 2
  • NAS-Bench-201 [3] and NAS-Bench-360 enable reproducible comparisons through precomputed architecture metrics, avoiding the cost of training each candidate from scratch.These benchmarks act as lookup tables for evaluating candidates and comparing NAS methods and search strategies.Source: A. NAS · p. 2

B. Hardware-aware NAS

Hardware-aware NAS formulates architecture search as multi-objective optimization over hardware performance metrics. Searches for non-dominated architectures on the Pareto front use Pareto rank surrogates.

B. Hardware-aware NAS

  • Hardware-aware NAS extends architecture optimization to multiple hardware performance metrics.Each objective corresponds to a performance metric.Source: B. Hardware-aware NAS · p. 2
  • NAS benchmarks fix weights for each candidate architecture, reducing optimization to architecture-dependent metrics.The resulting problem can be constrained by the user's computation budget and desired performance metrics.Source: B. Hardware-aware NAS · p. 2
  • Prominent hardware-aware NAS works [7], seek a Pareto front: a set of solutions that no other solution dominates.This non-dominated set captures the multi-objective search target.Source: B. Hardware-aware NAS · p. 2
  • Pareto rank surrogates guide searches for Pareto-front architectures, and this paper builds on the surrogate and architecture-encoding work in.These performance estimators are also called Pareto rank predictors.Source: B. Hardware-aware NAS · p. 2

C. State-of-the-art & Related work

Existing hardware-aware NAS benchmarks and joint quantization searches leave post-search PTQ’s effect on FP32 Pareto rankings uncharacterised. This paper addresses that gap through an INT4 quantization bridge and a systematic Pareto-stability and surrogate-transferability study.

C. State-of-the-art & Related work

  • HW-PR-NAS trains directly on Pareto ranks with a listwise ranking loss, addressing independent-surrogate dominance errors and underpinning this paper’s Stage I surrogate.Its ranking objective preserves architectures’ multi-objective ordering rather than predicting each objective independently.Source: C. State-of-the-art & Related work · p. 3
  • HW-NAS-Bench extends NAS-Bench-201 with latency and energy measurements across six hardware platforms, but provides only FP32 metrics without studying quantization’s effect on Pareto rankings.These measurements enable hardware-aware architecture comparison without physical hardware access during search.Source: C. State-of-the-art & Related work · p. 3
  • HAQ, APQ, SimQ-NAS [11], and QuantNAS incorporate quantization into search, substantially expanding the search space rather than characterising post-search PTQ.HAQ uses reinforcement learning for per-layer mixed precision; APQ jointly searches architecture and quantization policy. SimQ-NAS [11] searches both across CNNs and transformers, while QuantNAS searches a fully quantized supernet to address post-search quantization’s suboptimality.Source: C. State-of-the-art & Related work · p. 3
  • NAAS [13] uses two-level co-search over neural architectures and accelerator configurations, demonstrating better accuracy-efficiency trade-offs than sequential approaches.This represents a parallel line of work on architecture–accelerator co-search.Source: C. State-of-the-art & Related work · p. 3
  • The paper addresses uncharacterised PTQ-induced Pareto changes through an INT4 quantization bridge and a systematic NAS-Bench-201 stability and surrogate-transferability study.The bridge combines INT4 PTQ, budget-aware filtering, and feedback. The study introduces formal stability analysis at full search-space scale and a three-way surrogate transferability analysis.Source: C. State-of-the-art & Related work · p. 3

III. SYSTEM OVERVIEW

The proposed three-stage system follows a NAS-then-quantized approach to simplify deployment and decouple model optimization from hardware accelerator optimization.

III. SYSTEM OVERVIEW

  • Three stages constitute the proposed NAS-then-quantized system.Architecture search precedes quantization.Source: §III · p. 3
  • Decoupled model and accelerator optimization enables separate frontend solutions for NAS and backend solutions for hardware design and exploration.Source: §III · p. 3
  • The NAS-then-quantized approach is chosen partly because it is relatively straightforward to deploy.Source: §III · p. 3

A. Stage I: Hardware-Agnostic Frontend · B. Stage II: Quantization Bridge

The hardware-agnostic frontend searches NAS-Bench-201 using an accuracy–FLOPs Pareto-rank surrogate. The quantization bridge applies INT4 quantization, re-ranks and filters candidates, and triggers a new search if none survive.

A. Stage I: Hardware-Agnostic Frontend

  • Ten architectures are returned as Pareto-front candidates after Random Search (RS) or Multi-Objective Evolutionary Algorithm (MOEA) runs guided by a target FLOPs value.The target FLOPs value guides each search toward the intended region of Pareto space.Source: A. Stage I: Hardware-Agnostic Frontend · p. 3
  • The Pareto-rank surrogate uses two hardware-agnostic objectives: accuracy derived from NAS-Bench-201 and FLOPs extracted directly from the models.The frontend performs the system’s main NAS procedure.Source: A. Stage I: Hardware-Agnostic Frontend · p. 3
  • The frontend redesigns HW-PR-NAS, restricting architecture-encoder training to NAS-Bench-201 and adding a Pool output node to its LSTM.The modification aims to improve Kendall’s Tau correlation between predicted Pareto-rank scores and actual architecture rankings.Source: A. Stage I: Hardware-Agnostic Frontend · p. 3

B. Stage II: Quantization Bridge

  • The Quantization Bridge re-ranks quantized candidates and filters dominated or undesired architectures arising from the quantization-induced Pareto-space shift.It connects the hardware-agnostic and hardware-aware parts of the system.Source: B. Stage II: Quantization Bridge · p. 3
  • The bridge applies INT4 quantization to candidate architectures using AMD-Xilinx’s open-source Brevitas tool.The rationale for choosing INT4 is deferred to Section IV.Source: B. Stage II: Quantization Bridge · p. 3
  • A feedback loop triggers a new Stage I search when no candidate architectures survive the Quantization Bridge.The restart is conditional on the bridge leaving no survivors.Source: B. Stage II: Quantization Bridge · p. 3

C. Stage III: Hardware-Aware Backend

The hardware-aware backend maps surviving quantized models to customized CGRA4ML accelerators. An evolutionary design-space search selects hardware configurations using analytical performance estimates under resource constraints.

C. Stage III: Hardware-Aware Backend

  • A QONNX-to-QKeras translation layer converts surviving Brevitas models into CGRA4ML-compatible models while copying their quantized weights.CGRA4ML compiles QKeras models into deployment-ready CGRA designs, supporting INT1/2/4/8 activations and weights and INT8/16/32 biases.Source: C. Stage III: Hardware-Aware Backend · pp. 3–4
  • Approximately 1,200 valid configurations remain after constraining PE array dimensions, weight SRAM depth, and AXI width by R × C ≤ MAX PEs.Rows R ∈ {8, 10, 12, 16}; columns C range from 12 to 96 in dominant-kernel-width steps. Weight SRAM depth ranges from 256 to 1024 in steps of 64, and AXI width is 64 or 128.Source: C. Stage III: Hardware-Aware Backend · p. 4
  • The evolutionary search uses CGRA4ML’s analytical oracle to jointly minimise latency, PE idle ratio, and array area without RTL generation or synthesis.The oracle returns inference clock cycles, PE utilisation, and idle PE ratio. Fitness terms are normalised to approximately [0, 1], with α=0.2, β=0.1, and λ governing the hard-constraint penalty.Source: C. Stage III: Hardware-Aware Backend · p. 4
  • Tournament selection, constraint-preserving mutation, and elite carryover drive evolution until fitness improvement falls below 10^-4 over eight generations.The winning CGRA configuration is exported for testing through FPGA or ASIC flows.Source: C. Stage III: Hardware-Aware Backend · p. 4

IV. EXPERIMENTAL RESULTS · A. Quantization Scheme Selection

The quantization evaluation selected INT4 PTQ Id 6 for this work, although most promising configurations used 8-bit activations. Two-epoch finetuning restored INT4 accuracy from 9.49% to 83.53%.

IV. EXPERIMENTAL RESULTS

  • The experimental results evaluate the previously proposed three-stage hardware-aware NAS system.The evaluation follows the presentation of the system.Source: §IV · p. 4

A. Quantization Scheme Selection

  • A sample architecture selected using size and complexity served as the baseline for identifying complexity-robust PTQ methods.These selection criteria guided the quantization-strategy evaluation.Source: A. Quantization Scheme Selection · p. 4
  • 325 PTQ configurations were tested without finetuning, yielding six promising configurations summarized in Table I.This screening identified complexity-robust PTQ configurations.Source: Table I · p. 4, A. Quantization Scheme Selection · p. 4
  • 83.53% accuracy, restored from 9.49% after two finetuning epochs, supported selecting INT4 PTQ Id 6 for this work.INT4 PTQ offered half the complexity of INT8 PTQ; the authors interpreted the accuracy recovery as evidence that original model information was preserved.Source: A. Quantization Scheme Selection · p. 4
  • Most selected schemes used 8-bit activation quantization, and statistical analysis of INT8 runs further supported the reported superiority of INT8 PTQ.Figure 3 presents the statistical analysis of INT8 PTQ.Source: A. Quantization Scheme Selection · p. 4, Figure 3 · p. 4
  • Six Brevitas PTQ features represented by Table I’s columns were identified as the most prominent, with equally important contributions to quantization results.These six features are distinct from the six selected configurations.Source: A. Quantization Scheme Selection · p. 4, Table I · p. 4

B. Quantization Error Against Model Complexity · C. Quantization’s Impact on Pareto Front & Surrogate Performance

INT4 quantization disproportionately harms low-complexity models and reorganizes Pareto-front membership, yet preserves enough ranking structure for effective FP32-surrogate transfer. The FP32 zero-shot surrogate achieves higher search hypervolume than the INT4-trained surrogate under both RS and MOEA.

B. Quantization Error Against Model Complexity

  • Approximately 5% median accuracy loss at 7–15 MFLOPs contrasts with near-zero medians and substantially reduced variance at higher complexity under INT4 quantization.Models at 7–47 MFLOPs exhibit greater sensitivity, wide interquartile ranges, and numerous outliers exceeding 50% degradation.Source: B. Quantization Error Against Model… · p. 4
  • Nearly 45% of operations at 82–220 MFLOPs are nor conv 3x3, whose noise filtering helps explain greater INT4 robustness.At lower complexity, nearly uniform primitive mixtures create sparse, heterogeneous cells: none removes edges, while skip connect propagates noise unfiltered. Convolutions attenuate quantization noise, especially nor conv 3x3; other operation types each fall below 25% at higher complexity.Source: B. Quantization Error Against Model… · p. 5

C. Quantization’s Impact on Pareto Front & Surrogate Performance

  • Approximated FP32 and INT4 Pareto fronts closely track ground truth, and re-evaluated FP32-optimal architectures align closely with the ground-truth INT4 front.This visual alignment suggests that zero-shot FP32-surrogate transfer may suffice for effective INT4-aware search.Source: Figure 6 · p. 5, C. Quantization’s Impact on Pareto… · p. 5
  • 0.8352 FP32 KT rank correlation drops to 0.7219 with zero-shot INT4 transfer and recovers to 0.8219 after fine-tuning on quantized fully-trained data.The retained ranking consistency supports zero-shot transfer as a practical alternative to full surrogate retraining.Source: Table II · p. 5, C. Quantization’s Impact on Pareto… · p. 5
  • 12.26% and 6.77% higher normalized global hypervolume favor the FP32 zero-shot surrogate over the INT4-trained surrogate for RS and MOEA, respectively.Hypervolume ratios were 1.12 for RS and 1.07 for MOEA over 50 independent runs within a 0.625–10 MFLOPs budget. Accuracy differences across surrogates and search strategies remained within one standard deviation.Source: C. Quantization’s Impact on Pareto… · pp. 5–6, Table III · p. 6
  • 0% front survival and a 21.73% dominance flip rate reveal complete frontier reorganisation under INT4 despite sufficiently preserved global ranking correlation.Ground-truth KT was 0.6655, indicating moderate ranking reordering. Pareto rank sensitivity of 0.2404 indicates that quantization fragility is largely independent of FP32 Pareto optimality.Source: Table III · p. 6, C. Quantization’s Impact on Pareto… · p. 6
  • Cleaner FP32 training signals transfer across correlated Pareto structures, while the INT4 surrogate appears to favor higher-FLOPs architectures and underrepresent hypervolume-rewarded low-FLOPs regions.The paper attributes the FP32 surrogate’s search advantage to this combination of training-signal reliability and computational-range coverage.Source: C. Quantization’s Impact on Pareto… · p. 6

D. Hardware DSE Results

Stage III hardware DSE finds the same optimal CGRA4ML configuration for three Pareto-front architectures with shared structural features. Architecture 8592 has a higher clock count because a 3×3 convolution increases its per-cell workload across all recurrences.

D. Hardware DSE Results

  • 15 cell recurrences repeat nor conv 1x1 and skip connect operations, providing the shared structural profile reflected in the architectures’ common optimal configuration.Source: D. Hardware DSE Results · p. 6
  • Architecture 8592 has a higher clock count because its single nor conv 3x3 operation at the first cell node increases per-cell computation.The workload increase is proportional to the 3×3 versus 1×1 kernel-area ratio across all recurrences.Source: D. Hardware DSE Results · p. 6
  • All three Pareto-front architectures converge to the same optimal CGRA4ML configuration in Stage III hardware DSE.Table IV presents the DSE results for these architectures.Source: Table IV · p. 6, D. Hardware DSE Results · p. 6

V. CONCLUSION

The paper presents a three-stage hardware-aware NAS pipeline for edge AI deployment on CGRA-based accelerators and an empirical study of INT4 PTQ on NAS-Bench-201. Despite full Pareto structure reorganization, the FP32 zero-shot surrogate outperforms the dedicated INT4-trained surrogate in Pareto space coverage.

V. CONCLUSION

  • The FP32 zero-shot surrogate outperforms the dedicated INT4-trained surrogate in Pareto space coverage despite INT4 PTQ fully reorganizing the NAS-Bench-201 Pareto structure.The authors suggest that the cleaner FP32 training signal transfers effectively across precision regimes.Source: §V · p. 6
  • The proposed three-stage hardware-aware NAS pipeline targets edge AI deployment on CGRA-based accelerators.The pipeline is presented alongside an empirical study of INT4 PTQ effects on the NAS-Bench-201 Pareto space.Source: §V · p. 6
  • Possible future work includes optimizing the hardware-agnostic frontend for more robust architecture search and extending the empirical study to more complex mixed-precision schemes.Source: §V · p. 6

Related papers

Loading 2608.13293v1…