PaperlayerView exact revision on arXiv

Source-linked AI summary

NAS-Driven Hardware Accelerator Exploration for Edge AI and Quantization Effects on the Pareto Space

Eleftherios Mylonas, Angelos Kouprizas, Michael Birbas, Alexios Birbas

arXiv:2608.13293v1cs.AI

TL;DR

Post-search quantization’s effect on NAS-discovered Pareto structures remains insufficiently characterized, especially for hardware deployment. This paper combines Pareto-aware INT4 quantization with hardware mapping exploration and finds that an FP32 zero-shot surrogate outperforms a dedicated INT4-trained surrogate in Pareto-space coverage.

  • Problem

    The effect of post-training quantization on relative architectural rankings and Pareto structure remains uncharacterized at full NAS search-space scale.

  • Method

    A three-stage workflow performs hardware-agnostic NAS, Pareto-aware INT4 quantization with feedback filtering, and evolutionary CGRA hardware design-space exploration.

  • Results

    INT4 PTQ reorganizes the Pareto structure, while the FP32 zero-shot surrogate outperforms the dedicated INT4-trained surrogate in Pareto-space coverage across two search strategies.

  • Takeaways & Limitations

    The findings indicate that FP32 surrogate rankings can transfer effectively across precision regimes despite substantial INT4-induced Pareto-space reorganization.

  • Takeaways & Limitations

    The empirical study is limited to INT4 PTQ, with extension to more complex mixed-precision schemes left for future work.

Abstract

from arXiv · show

Edge AI deployment demands neural architectures that are simultaneously accurate, computationally efficient, and hardware-deployable - a challenge addressed by hardware-aware Neural Architecture Search (NAS). While recent works incorporate quantization directly into the NAS loop, these approaches expand search complexity and tightly couple architecture and quantization design. The simpler post-search quantization strategy has received little analytical attention: the effects of Post-Training Quantization (PTQ) on the NAS-discovered Pareto structure remain uncharacterised, and no framework combines quantized architecture mapping onto reconfigurable accelerators with automated hardware exploration. This paper addresses both gaps. First, a three-stage pipeline is proposed: a hardware-agnostic Pareto rank surrogate frontend on NAS-Bench-201, a quantization bridge with Pareto-aware filtering and feedback control, and an evolutionary Domain Space Exploration (DSE) backend on CGRA4ML for optimal hardware mapping. Second, an empirical study characterises how INT4 PTQ perturbs the NAS-Bench-201 Pareto space through formal stability metrics on ground-truth data for all 15,625 architectures, and demonstrates that an FP32 zero-shot surrogate outperforms a dedicated INT4-trained surrogate in Pareto space coverage across two standard search strategies.

I. INTRODUCTION

Edge AI requires accurate, efficient, and deployable models under computing, memory, and power constraints, motivating hardware-aware Neural Architecture Search (NAS). This work proposes a three-stage workflow combining NAS-Bench-201 architecture selection, INT4 post-training quantization analysis, and evolutionary CGRA hardware mapping.

  • Motivation: The shift toward edge AI increases demands for computing, memory, and power efficiency while requiring privacy, lower inference latency, and autonomous operation.These constraints arise as deployment moves away from centralized cloud AI toward resource-constrained edge devices.
  • Motivation and problem: The study addresses limited NAS benchmark coverage by considering hardware performance beyond the predominantly FP32 accuracy focus.The introduction identifies limited task coverage and the neglect of hardware performance metrics as open problems for edge AI deployment.
  • Proposed workflow: The proposed three-stage hardware-aware NAS procedure selects full-precision candidates, filters architectures after quantization, and searches evolutionary CGRA mappings for survivors.The frontend is trained on NAS-Bench-201 [3] and CIFAR-10, while the backend optimizes a CGRA architecture per surviving architecture.
  • Quantization study: The work evaluates INT4 Post Training Quantization on all 15,625 NAS-Bench-201 architectures using formal Pareto stability metrics and quantization-aware surrogate transferability analysis.The analysis uses ground-truth quantized data and examines FP32 Pareto-rank predictions under quantization-induced distribution shift.
  • Hardware exploration: A CGRA4ML [5] hardware DSE environment finds optimal mappings for quantized architectures surviving Pareto filtering using an analytical performance oracle and normalized scalar fitness.The fitness function combines three normalized terms for hardware exploration.

II. THEORETICAL BACKGROUND · A. NAS

Neural Architecture Search (NAS) automates neural network design by combining a search space, a search strategy, and performance estimation. NAS benchmarks such as NAS-Bench-201 [3] and NAS-Bench-360 enable reproducible evaluation through pre-computed architecture metrics.

  • A. NAS: NAS automates neural network architecture design within a defined candidate search space.Search spaces may use cell-based architectures with fixed operations and connectivity patterns.
  • A. NAS: A typical NAS procedure combines a search space, a strategy for sampling architectures, and a performance estimation strategy.Search strategies include evolutionary algorithms and reinforcement learning.
  • A. NAS: NAS seeks an architecture and corresponding weights that maximize validation accuracy across the search space.The optimization considers both candidate architectures and their weight parameters.
  • A. NAS: NAS benchmarks provide collections of pre-trained architectures from defined search spaces with associated metrics.They support reproducible comparison while avoiding the prohibitive cost of training every candidate from scratch.
  • A. NAS: NAS-Bench-201 [3] and NAS-Bench-360 are widely used benchmarks for architecture-search evaluation.These benchmarks organize architectures and their metrics for comparison within specific search spaces.
  • A. NAS: Benchmark lookup tables let NAS methods evaluate candidate architectures by querying pre-computed results.This enables performance comparisons without retraining each architecture.

B. Hardware-aware NAS

Hardware-aware NAS formulates architecture search as multi-objective optimization over hardware performance metrics, potentially under computation constraints. It seeks non-dominated Pareto-front architectures, using Pareto rank surrogates to estimate candidate performance [7],.

  • B. Hardware-aware NAS: Hardware-aware NAS treats architecture search as multi-objective optimization over n hardware performance metrics.With fixed weights per candidate on a NAS benchmark, the formulation reduces to evaluating each architecture across those metrics.
  • B. Hardware-aware NAS: The multi-objective formulation can incorporate a user-assigned computation budget and desired performance metrics as constraints.
  • B. Hardware-aware NAS: The Pareto front is the set of solutions that cannot be further optimized because no other solution dominates them [7],.An architecture dominates another only when the required objective conditions are satisfied.
  • B. Hardware-aware NAS: Pareto rank predictors, or Pareto rank surrogates, estimate performance for NAS strategies seeking Pareto-front architectures.The paper bases its work on prior research covering Pareto rank surrogates and suitable architecture encoding.

C. State-of-the-art & Related work

Prior work advances hardware-aware NAS through dedicated benchmarks and joint quantization-search methods, but leaves the effects of post-search quantization on Pareto rankings and structure insufficiently characterised.

  • Hardware-aware NAS benchmarks: HW-NAS-Bench extends NAS-Bench-201 with latency and energy measurements across six hardware platforms, enabling hardware-aware architecture comparison without physical hardware access during search.Its metrics are collected in FP32, so quantization and its effect on Pareto rankings are not studied.
  • Quantization-aware NAS: HAQ, APQ, and SimQ-NAS [11] incorporate quantization directly into NAS by searching layer bit-widths, architecture, and quantization policies.These approaches address quantization during search rather than through post-search quantization.
  • Research gap: Existing work does not characterise how PTQ perturbs the Pareto structure of FP32 NAS-discovered architectures or provide formal stability metrics for this perturbation.Joint quantization-NAS methods avoid post-search quantization, hardware-aware benchmarks ignore it, and deployment-focused works apply PTQ without studying relative architectural rankings.

III. SYSTEM OVERVIEW · A. Stage I: Hardware-Agnostic Frontend · B. Stage II: Quantization Bridge

The proposed NAS system uses a three-stage NAS-then-quantized pipeline that separates NAS from hardware-accelerator optimization. Its frontend predicts Pareto rankings hardware-agnostically, while the quantization bridge quantizes, re-ranks, and filters candidate architectures.

  • III. SYSTEM OVERVIEW: The complete system comprises three stages and follows a NAS-then-quantized approach for deployment-oriented design.This choice decouples model optimization from hardware-accelerator optimization and enables separate NAS and hardware-exploration solutions.
  • III. SYSTEM OVERVIEW: The pipeline separates NAS and hardware design exploration, allowing dedicated frontend and backend solutions.The separation is intended to simplify deployment while decoupling the two optimization problems.
  • A. Stage I: Hardware-Agnostic Frontend: Stage I, the Hardware-Agnostic Frontend, performs the main neural architecture search procedure.It is based on a redesign of HW-PR-NAS for the NAS-Bench-201 search space.
  • A. Stage I: Hardware-Agnostic Frontend: The frontend trains its architecture encoding module only for NAS-Bench-201 and enhances the LSTM predictor with a Pool output layer.These changes target improved Kendall’s Tau correlation between predicted Pareto scores and actual architecture rankings.
  • B. Stage II: Quantization Bridge: Stage II, the Quantization Bridge, mediates between hardware-agnostic NAS and hardware-aware system components.It uses the open-source AMD-Xilinx Brevitas tool for quantizing candidate architectures.
  • B. Stage II: Quantization Bridge: The bridge re-ranks quantized candidates and filters architectures dominated or rendered undesirable by the Pareto-space shift after quantization.Its responsibilities therefore include quantization, Pareto re-ranking, and final candidate filtering.

C. Stage III: Hardware-Aware Backend

Stage III maps surviving quantized models onto customized CGRA accelerator architectures using CGRA4ML. A QONNX-to-QKeras translation layer enables compatibility, while evolutionary search selects configurations using analytical performance estimates.

  • Hardware-Aware Backend: The backend maps survivor models to customized, deployment-ready CGRA designs through CGRA4ML, with a QONNX-to-QKeras translation layer supporting Brevitas quantized models.CGRA4ML comprises Python and SystemVerilog scripts for compiling QKeras models into flexible CGRA designs.
  • Hardware-Aware Backend: An evolutionary algorithm minimizes normalized latency, PE idle ratio, and array area using CGRA4ML’s analytical oracle, which predicts clock cycles and PE utilization without RTL generation or synthesis.The oracle also returns idle PE ratio for each candidate configuration.
  • Hardware-Aware Backend: Search uses tournament selection, constraint-preserving mutation, and elite carryover, stopping after fitness improvement remains below 10^-4 for 8 generations before exporting the winning configuration.The fitness weights are α=0.2 and β=0.1, with λ enforcing the hard constraint penalty; the exported design can enter FPGA or ASIC flows.

IV. EXPERIMENTAL RESULTS · A. Quantization Scheme Selection

The evaluation selects complexity-robust post-training quantization configurations using architecture size and complexity, finding that most promising schemes use 8-bit activation quantization. Statistical analysis further supports INT8 PTQ as the preferred strategy.

  • IV. EXPERIMENTAL RESULTS: The study evaluates the proposed three-stage hardware-aware NAS system after presenting its design.This evaluation provides the context for the quantization-strategy selection experiments.
  • A. Quantization Scheme Selection: Six promising PTQ configurations were selected from 325 no-finetuning tests using architecture size and complexity as selection criteria.The configurations are summarized in Table I and use six Brevitas PTQ features identified as having equally important contributions.
  • A. Quantization Scheme Selection: Most selected schemes use 8-bit activation quantization, indicating the superiority of INT8 PTQ among the tested strategies.The preference for INT8 is further supported by statistical analysis of different INT8 PTQ runs.

B. Quantization Error Against Model Complexity · C. Quantization’s Impact on Pareto Front & Surrogate Performance

INT4 quantization is most damaging for low-complexity NAS-Bench-201 models, while convolutional structure can attenuate quantization noise. Although INT4 reorganizes the efficient frontier, FP32 Pareto structure and surrogate rankings remain sufficiently correlated for zero-shot transfer to outperform an INT4-specific surrogate in search hypervolume.

  • B. Quantization Error Against Model Complexity: 7–15 MFLOPs models show median INT4 accuracy drops of approximately 5%, with numerous outliers exceeding 50% degradation; sensitivity progressively decreases as complexity increases.The reported low-complexity range is 7–47 MFLOPs, with the largest median degradation concentrated at 7–15 MFLOPs.
  • B. Quantization Error Against Model Complexity: NAS-Bench-201’s nearly uniform primitive-operation distribution creates heterogeneous cells in which none and skip connect amplify quantization errors, whereas convolutions attenuate high-frequency noise.The lowpass effect is most pronounced for nor conv 3x3 operations.
  • C. Quantization’s Impact on Pareto Front & Surrogate Performance: The migrated FP32 Pareto front aligns closely with the ground-truth INT4 front, indicating that FP32 Pareto structure is substantially preserved for zero-shot INT4-aware search.The analysis compares true FP32 and INT4 fronts with approximated fronts and the migrated FP32 front re-evaluated under INT4.
  • C. Quantization’s Impact on Pareto Front & Surrogate Performance: The surrogate’s KT correlation decreases from 0.8352 on FP32 to 0.7219 after zero-shot transfer to INT4, indicating meaningful predictive structure without retraining.The passage states that fine-tuning on quantized fully-trained data was also evaluated, but the supplied text does not provide its result.
  • C. Quantization’s Impact on Pareto Front & Surrogate Performance: 12.26% and 6.77% higher normalized global hypervolume were achieved by the FP32 zero-shot surrogate than the INT4-trained surrogate across both RS and MOEA searches.The comparison used a 0.625–10 MFLOPs budget, 50 independent runs, RS with 300 samples, and MOEA with 250 queries and population 150.
  • C. Quantization’s Impact on Pareto Front & Surrogate Performance: 0% front survival, a 21.73% dominance-flip rate, and ground-truth KT of 0.6655 show that INT4 completely reorganizes the FP32 efficient frontier while preserving moderate ranking correlation.The dominance-flip rate indicates that roughly one in five FP32 dominance relationships changes under INT4.

D. Hardware DSE Results

Stage III DSE maps all three Pareto-front architectures to the same optimal CGRA4ML configuration, reflecting their shared repeated operator structure. Architecture 8592 has a higher clock count because of one initial 3×3 convolution.

  • D. Hardware DSE Results: All three Pareto-front architectures converge to the same optimal CGRA4ML configuration in Stage III DSE.The result is reported for three Pareto-front architectures in Table IV.
  • D. Hardware DSE Results: The shared configuration reflects repeated normal 1×1 convolutions and skip connections across 15 cell recurrences.These common structural features explain why the three architectures select the same hardware configuration.
  • D. Hardware DSE Results: Architecture 8592 has a higher clock count because its first cell node contains a single normal 3×3 convolution, whose workload scales with kernel area across recurrences.The 3×3-versus-1×1 kernel area ratio drives the proportional workload increase.

V. CONCLUSION

The paper presents a three-stage hardware-aware NAS pipeline for edge AI deployment on CGRA-based accelerators and empirically studies INT4 PTQ effects on the NAS-Bench-201 Pareto space. INT4 PTQ fully reorganizes the Pareto structure, yet the FP32 zero-shot surrogate achieves greater Pareto space coverage than the dedicated INT4-trained surrogate.

  • V. CONCLUSION: The proposed contribution is a three-stage hardware-aware NAS pipeline for edge AI deployment on CGRA-based accelerators.
  • V. CONCLUSION: INT4 PTQ causes a full reorganization of the NAS-Bench-201 Pareto structure.
  • V. CONCLUSION: The FP32 zero-shot surrogate outperforms the dedicated INT4-trained surrogate in Pareto space coverage.The finding suggests that the cleaner FP32 training signal transfers effectively across precisions.
Loading 2608.13293v1…