PaperlayerView exact revision on arXiv

Source-linked AI summary

Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks

Lei Yang, Zheyu Yan, Meng Li, Hyoukjun Kwon, Liangzhen Lai, Tushar Krishna, Vikas Chandra, Weiwen Jiang, Yiyu Shi

arXiv:2002.04116v1cs.LGeess.SPstat.ML

TL;DR

Co-exploring multiple DNN architectures and ASIC designs is challenging because edge tasks share design specs and may need different dataflows. NASAIC combines dataflow templates with a controller that searches architectures and hardware allocations together. In the reported workloads, it met specs where NAS→ASIC did not, with lower hardware costs and limited average accuracy loss.

  • Problem

    Multiple edge-task DNNs must be optimized under unified design specs, while their distinct architectures may not fit a single dataflow.

    Source: §II · p. 2

  • Method

    NASAIC uses dataflow templates and a multi-task RNN controller to co-explore multiple neural architectures and ASIC resource allocations.

    Source: §IV · p. 4, §VI · p. 6

  • Results

    NASAIC met all specs where NAS→ASIC did not; for W1, latency, energy, and area fell by 17.77%, 2.49×, and 2.32%, respectively, with 0.76% average accuracy loss.

    Source: C. Results on Multiple Tasks for… · p. 6

  • Takeaways & Limitations

    The experiments underscore the necessity of co-exploring neural architectures and ASIC designs.

    Source: C. Results on Multiple Tasks for… · p. 6

Abstract

from arXiv · show

Neural Architecture Search (NAS) has demonstrated its power on various AI accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs). However, it remains an open problem, how to integrate NAS with Application-Specific Integrated Circuits (ASICs), despite them being the most powerful AI accelerating platforms. The major bottleneck comes from the large design freedom associated with ASIC designs. Moreover, with the consideration that multiple DNNs will run in parallel for different workloads with diverse layer operations and sizes, integrating heterogeneous ASIC sub-accelerators for distinct DNNs in one design can significantly boost performance, and at the same time further complicate the design space. To address these challenges, in this paper we build ASIC template set based on existing successful designs, described by their unique dataflows, so that the design space is significantly reduced. Based on the templates, we further propose a framework, namely NASAIC, which can simultaneously identify multiple DNN architectures and the associated heterogeneous ASIC accelerator design, such that the design specifications (specs) can be satisfied, while the accuracy can be maximized. Experimental results show that compared with successive NAS and ASIC design optimizations which lead to design spec violations, NASAIC can guarantee the results to meet the design specs with 17.77%, 2.49x, and 2.32x reductions on latency, energy, and area and with 0.76% accuracy loss. To the best of the authors' knowledge, this is the first work on neural architecture and ASIC accelerator design co-exploration.

I. INTRODUCTION

Integrating NAS with ASIC design is difficult because ASICs offer a large design space, and multiple tasks can require heterogeneous accelerators. NASAIC uses dataflow-based templates to co-explore neural architectures and hardware while meeting design specifications.

I. INTRODUCTION

  • Successive NAS and ASIC optimization produced solutions violating hardware specs, while NAS aware of one ASIC design yielded lower accuracy than the 10,000-run Monte Carlo optimum.Choosing the solution closest to the specs was also suboptimal.Source: §I · p. 1, Figure 1 · p. 1
  • Multiple edge tasks may run concurrently, and their distinct DNN architectures can favor different dataflows, motivating heterogeneous sub-accelerators.Integrating these sub-accelerators further complicates the design space.Source: §I · p. 1
  • NASAIC narrows ASIC design choices to existing dataflow templates and resource allocation, then jointly predicts multiple DNN hyperparameters and hardware allocation parameters.Its framework evaluates mappings and schedules and prunes architectures unable to meet specs before training.Source: §I · pp. 1–2
  • NASAIC met all design specs; for W1, it reduced latency by 17.77%, energy by 2.49×, and area by 2.32× versus NAS→ASIC, with 0.76% average accuracy loss.The brute-force accelerator designs for NAS-identified architectures did not satisfy all specs.Source: §I · p. 2

II. BACKGROUND AND CHALLENGES

The section identifies two challenges: co-exploring neural architectures with the flexible ASIC design space, and optimizing multiple DNNs under shared specifications. Distinct task architectures may also require different dataflows, adding complexity to heterogeneous accelerator design.

Challenge 1

  • ASIC flexibility creates a large design space, but existing accelerator designs offer dataflow styles that can serve as fixed-structure templates and reduce it.The paper names Shidiannao, NVDLA, and Eyeriss as examples of existing designs.Source: §II · p. 2

Challenge 2

  • Edge workloads require simultaneous optimization of multiple DNN architectures under unified latency, energy, and area specs, rather than sequential hardware-aware NAS.The tasks may involve multiple DNNs that execute on the same accelerator.Source: §II · p. 2
  • One dataflow may not suit all task architectures: NVDLA favors ResNets, whereas Shidiannao favors U-Nets.The section describes their different preferences for activation-channel size and resolution.Source: §II · p. 2
  • Connecting heterogeneous sub-accelerators through a network-on-chip can support distinct dataflows, but further complicates the design space.The cited example uses a network interface controller to connect the sub-accelerators.Source: §II · p. 2

III. PROBLEM DEFINITION

The problem is framed as co-exploring task-specific DNN architectures, heterogeneous ASIC resources, and layer mappings and schedules under shared design constraints. The design uses dataflow templates to define accelerator choices and evaluates the resulting synthesis against accuracy and hardware specs.

III. PROBLEM DEFINITION

  • The co-exploration has application, accelerator, and synthesis layers: choose architectures, create template-based ASIC options, then allocate, map, and schedule them.The synthesis layer bridges application choices and accelerator resources.Source: §III · p. 2, Figure 3 · p. 3
  • Each task has a DNN defined by its backbone, layers, hyperparameters, and accuracy; NAS selects hyperparameters for the task's architecture.The examples use ResNet9 for classification and U-Net for segmentation, with task-specific search parameters.Source: §III · p. 3
  • Each sub-accelerator is specified by dataflow, PE count, and NoC bandwidth, with the dataflow selected from predefined templates.Memory size is determined from the template and mapped layers, so it is excluded from the search space.Source: §III · p. 3
  • Resource allocation assigns templates and hardware resources to sub-accelerators, subject to the available PE and bandwidth limits.Using multiple dataflow templates yields a heterogeneous accelerator; one template can instead support homogeneous designs.Source: §III · p. 3
  • The synthesis problem maps network layers to sub-accelerators and orders their execution to maximize weighted accuracy while satisfying latency, energy, area, and resource constraints.The accuracy objective can maximize average accuracy or minimum accuracy across networks.Source: §III · p. 3

IV. PROPOSED CO-EXPLORATION FRAMEWORK: NASAIC

NASAIC jointly explores architectures for multiple DNNs and hardware-resource allocations, then evaluates accuracy and hardware costs to find high-accuracy solutions meeting design specs. Its controller uses reinforcement learning, while the optimizer selector and evaluator support the search.

IV. PROPOSED CO-EXPLORATION FRAMEWORK: NASAIC

  • NASAIC samples neural architectures and hardware allocations, evaluates accuracy and hardware cost, and uses the resulting reward to update its controller.The framework applies reinforcement learning, though the paper notes that other optimization approaches could also use its reward function.Source: §IV · p. 3
  • The multi-task RNN controller predicts architectures for multiple DNNs and accelerator design parameters in a single co-exploration process.Its segments assign outputs to DNN hyperparameters or sub-accelerator resource parameters.Source: §IV · p. 4, Figure 5 · p. 4
  • The optimizer selector switches between architecture-only, hardware-only, and joint exploration, including repeated hardware exploration for previously identified architectures.The joint mode cycles between generating new architecture-hardware samples and exploring hardware designs for earlier architectures.Source: §IV · p. 4
  • The evaluator’s training-and-validation path obtains each DNN’s accuracy by training it from scratch and measuring it on a held-out validation dataset, then uses task weights to compute weighted accuracy for reward calculation.Source: §IV · p. 4
  • For hardware evaluation, NASAIC uses MAESTRO to estimate area and per-layer latency and energy, then maps and schedules layers across sub-accelerators.The mapping problem minimizes energy subject to a latency constraint; the paper uses a heuristic to accelerate the search rather than time-consuming ILP.Source: §IV · p. 4
  • The reward penalizes the degree to which latency, energy, or area exceeds its spec and applies no penalty when all specs are met.Upper bounds for the metrics are obtained by exploring hardware designs using a neural architecture identified by NAS.Source: §IV · p. 4

V. EXPERIMENTAL EVALUATION

The evaluation tests NASAIC across application workloads and hardware configurations. The authors report that it identifies accurate neural architectures and ASIC accelerator designs that meet the specified constraints across multiple AI tasks.

V. EXPERIMENTAL EVALUATION

  • Across different workloads and hardware configurations, NASAIC identifies accurate neural architectures and ASIC accelerator designs that are guaranteed to meet design specs.The evaluation describes the resulting accuracy as high across multiple AI tasks.Source: §V · p. 5
  • The evaluation covers different application workloads and hardware configurations.The authors use these settings to assess NASAIC's efficacy.Source: §V · p. 5
  • The authors report high accuracy for multiple AI tasks alongside hardware designs that meet the given specs.This is the evaluation section's stated outcome for NASAIC.Source: §V · p. 5

A. Evaluation Environment

The evaluation uses three AR-glasses workloads involving classification and segmentation, with ResNet9 and U-Net backbones and two sub-accelerators. Workload-specific latency, energy, and area limits and fixed exploration settings define the test environment.

A. Evaluation Environment

  • The study models AR-glasses workloads involving classification and segmentation, using CIFAR-10, STL-10, and Nuclei across three workload combinations.The workloads include classification with segmentation, two classification datasets, or two tasks on CIFAR-10.Source: A. Evaluation Environment · p. 5
  • Classification uses ResNet9 and segmentation uses U-Net, with architecture search spaces tailored to each dataset and task.For STL-10, the network is deeper because input images have higher resolution; the setup also specifies searched layer and filter-channel parameters.Source: A. Evaluation Environment · p. 5
  • The hardware case study integrates two sub-accelerators using Shidiannao, NVDLA, or row-stationary dataflows, with up to 4096 PEs and 64 GB/s NoC bandwidth.The framework supports arbitrary numbers of sub-accelerators, but this evaluation uses two.Source: A. Evaluation Environment · p. 5
  • Workload-specific latency, energy, and area specs are paired with 500 exploration episodes and 10 accelerator designs per episode.The reported specs are ⟨8e5, 2e9, 4e9⟩ for W1, ⟨1e6, 3.5e9, 4e9⟩ for W2, and ⟨4e5, 1e9, 4e9⟩ for W3; each workload exploration takes around 3.5 GPU hours.Source: A. Evaluation Environment · p. 5

B. Design Space Exploration

NASAIC’s explored solutions meet the design specs while achieving higher accuracy than the smallest-network lower bounds. The results also show that resource boundaries can constrain accuracy, while the solution closest to the specs need not be best.

B. Design Space Exploration

  • NASAIC met all design specs and exceeded the smallest-network accuracy lower bounds on CIFAR-10, STL-10, and Nuclei.Reported accuracies were 92.85%, 92.62%, 93.23%, and 91.11% versus 78.93% on CIFAR-10; 75.72% versus 71.57% on STL-10; and Nuclei IOU 0.8374 versus 0.6462.Source: B. Design Space Exploration · p. 5
  • Table I compares successive NAS→ASIC, ASIC→HW-NAS, and NASAIC.The supplied caption identifies the compared approaches but gives no table values.Source: Table I · p. 6
  • For W1 and W3, the best solutions approached a spec boundary; W1 energy was 97.12% of its spec, and W3 latency was 93.4%.The authors interpret these cases as resource-bounded and say the boundary can help designers identify hardware bottlenecks.Source: B. Design Space Exploration · p. 6
  • For W2, a solution closer to the specs had 2.86% lower CIFAR-10 and 2.91% lower STL-10 accuracy than NASAIC’s best solution.The authors use this result to caution that choosing the solution closest to the specs can fail.Source: B. Design Space Exploration · p. 6

C. Results on Multiple Tasks for Multiple Datasets

Across two workloads, NASAIC met all design specs, whereas brute-force ASIC exploration for NAS-identified architectures found no legal solution. Against successive NAS→ASIC, it reduced latency, energy, and area; against ASIC→HW-NAS, it achieved higher accuracy on specified datasets.

C. Results on Multiple Tasks for Multiple Datasets

  • For W1, NASAIC met all specs and reduced latency 17.77%, energy 2.49×, and area 2.32× versus NAS→ASIC; for W2, the reductions were 30.39%, 29.58%, and 30.85%.NASAIC’s average accuracy losses were 0.76% and 1.17% for the two workloads, respectively.Source: C. Results on Multiple Tasks for… · p. 6
  • Brute-force ASIC exploration found no legal solution for NAS-identified architectures; versus ASIC→HW-NAS, NASAIC had 0.87% higher CIFAR-10 accuracy on W1 and 3.65% higher STL-10 accuracy on W2.Accuracy was similar for Nuclei on W1 and CIFAR-10 on W2, even though ASIC→HW-NAS was closer to the specs.Source: C. Results on Multiple Tasks for… · p. 6
  • The authors conclude that these results underscore the importance of co-exploring neural architectures and ASIC designs.Source: C. Results on Multiple Tasks for… · p. 6

D. From Single and Homogeneous to Heterogeneous ASIC Accelerator

On CIFAR-10 (W3), NASAIC’s heterogeneous-accelerator exploration produced two distinct networks, while unconstrained NAS produced a higher-accuracy architecture that did not meet specs. The comparison covers different accelerator configurations.

D. From Single and Homogeneous to Heterogeneous ASIC Accelerator

  • The comparison includes NAS, Single Acc., Homo. Acc., and Hetero. Acc. configurations on CIFAR-10 (W3).Single Acc. runs the network sequentially twice; Homo. Acc. runs the same network simultaneously on two sub-accelerators, with corresponding constraint adjustments.Source: Table II · p. 6, D. From Single and Homogeneous to… · p. 6
  • NASAIC’s heterogeneous accelerators produced two networks with 93.23% and 91.11% accuracy; NAS reached 94.17% but could not satisfy the specs, even using all hardware resources.The lower-accuracy heterogeneous network is described as comparable to Single Acc. and potentially useful for ensemble learning.Source: D. From Single and Homogeneous to… · p. 6, Table II · p. 6
  • Single Acc. met specs at 91.45% accuracy, while Homo. Acc. reached 92.00% after boosting performance through parallelism.Single Acc. could not further improve accuracy because it was bounded by latency.Source: D. From Single and Homogeneous to… · p. 6

VI. CONCLUSION

NASAIC co-explores neural architectures and ASIC accelerator designs for multiple AI tasks on edge devices. Its template set and multi-task RNN controller support this joint search under unified design specs.

VI. CONCLUSION

  • NASAIC co-explores neural architectures and ASIC accelerator designs for multiple AI tasks on edge devices.Source: §VI · p. 6
  • Its accelerator template set describes designs in terms of dataflow style, linking NAS with ASIC design.Source: §VI · p. 6
  • A multi-task-oriented RNN controller simultaneously determines multiple neural architectures under a unified design spec.Source: §VI · p. 6

Related papers

Loading 2002.04116v1…