PaperlayerView exact revision on arXiv

Source-linked AI summary

CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

Qijing Huang, Minwoo Kang, Grace Dinh, Thomas Norell, Aravind Kalaiah, James Demmel, John Wawrzynek, Yakun Sophia Shao

arXiv:2105.01898v1cs.LGcs.AR

TL;DR

DNN accelerators expose vast scheduling spaces, creating demand for a fast way to find efficient schedules. CoSA formulates scheduling as a constrained-optimization problem and reports up to 2.5× speedup, 22% better energy-efficiency, and 90× shorter time-to-solution.

  • Problem

    DNN accelerator workloads can have millions or billions of valid schedules, motivating a generalized framework to quickly produce efficient scheduling options.

    Source: §I · p. 1

  • Method

    CoSA formulates DNN accelerator scheduling as a constrained-optimization problem that jointly determines loop tiling, loop permutation, and spatial mapping.

    Source: §I · p. 2, §VI · p. 11

  • Results

    Up to 2.5× speedup and 22% better energy-efficiency, with 90× shorter time-to-solution, compared with state-of-the-art schedules.

    Source: §VI · p. 11

  • Takeaways & Limitations

    CoSA offers a direct alternative to DNN scheduling approaches that incur the high cost of iterative scheduling.

    Source: §VI · p. 11

    CoSA assumes dense CNN workloads, does not exploit sparsity, and best targets deterministic systems with explicitly managed scratchpads.

    Source: E. Limitation of CoSA · p. 7

Abstract

from arXiv · show

Recent advances in Deep Neural Networks (DNNs) have led to active development of specialized DNN accelerators, many of which feature a large number of processing elements laid out spatially, together with a multi-level memory hierarchy and flexible interconnect. While DNN accelerators can take advantage of data reuse and achieve high peak throughput, they also expose a large number of runtime parameters to the programmers who need to explicitly manage how computation is scheduled both spatially and temporally. In fact, different scheduling choices can lead to wide variations in performance and efficiency, motivating the need for a fast and efficient search strategy to navigate the vast scheduling space. To address this challenge, we present CoSA, a constrained-optimization-based approach for scheduling DNN accelerators. As opposed to existing approaches that either rely on designers' heuristics or iterative methods to navigate the search space, CoSA expresses scheduling decisions as a constrained-optimization problem that can be deterministically solved using mathematical optimization techniques. Specifically, CoSA leverages the regularities in DNN operators and hardware to formulate the DNN scheduling space into a mixed-integer programming (MIP) problem with algorithmic and architectural constraints, which can be solved to automatically generate a highly efficient schedule in one shot. We demonstrate that CoSA-generated schedules significantly outperform state-of-the-art approaches by a geometric mean of up to 2.5x across a wide range of DNN networks while improving the time-to-solution by 90x.

I. INTRODUCTION · II. BACKGROUND AND MOTIVATION · A. DNN Scheduling Space

DNN scheduling has a vast, consequential search space, while existing search strategies can be costly to scale. CoSA formulates scheduling as constrained optimization that can generate schedules in one pass.

I. INTRODUCTION

  • Millions or billions of schedules may be valid for a layer, and their performance and energy efficiency can vary widely, motivating fast, generalized scheduling across hardware configurations.Programmers must decide when and where computation and data movement are mapped spatially and temporally.Source: §I · p. 1

II. BACKGROUND AND MOTIVATION

  • Exhaustive search can become intractable, while feedback-driven tuning, beam search, and machine-learning methods require extensive training data and simulation, limiting extension to new hardware,,,,,,.High performance also requires considering tiling, pipelining data movement with computation, and data reuse.Source: §I · p. 1

I. INTRODUCTION

  • CoSA formulates scheduling as a constrained-optimization problem solved in one pass, using prime-factor allocation to choose tiling, loop order, and spatial or temporal execution.Its constraints expose algorithmic layer dimensions and hardware memory and network hierarchies; the work presents it as the first one-shot constrained-optimization approach to major DNN scheduling decisions.Source: §I · pp. 1–2

A. DNN Scheduling Space

  • 7.2× separates the best and worst valid schedules in a ResNet-50 convolution example; billions of tilings arise, half of sampled schedules violate buffer limits, and latency clusters appear.The example uses a 3×3 convolution with 256 input and output channels, 14×14 output dimensions, and five memory levels.Source: Figure 1 · p. 2, A. DNN Scheduling Space · p. 2

I. INTRODUCTION

  • 2.5× better performance than state-of-the-art approaches and 90× less scheduling time are reported for CoSA-generated schedules across DNN network layers.CoSA generates schedules without expensive iterative search.Source: §I · p. 2
  • CoSA takes a communication-oriented approach, exposing data-transfer costs across on-chip memories through clearly defined objective functions.These objective functions are composable.Source: §I · p. 2

A. DNN Scheduling Space

  • Diverse DNN accelerator architectures create a need for scheduling that is fast, performant, and explainable.The paper focuses on operator-level scheduling for specific hardware.Source: A. DNN Scheduling Space · p. 2
  • Operator-level scheduling optimizes each DNN layer through loop tiling, loop permutation, and spatial mapping, which must be considered together because each can significantly affect performance.Spatial mapping binds loop dimensions to parallel processing elements rather than sequential execution.Source: A. DNN Scheduling Space · p. 2

B. State-of-the-art Schedulers … 3) Constrained-optimization Approaches:

CoSA uses constrained optimization to schedule DNN accelerators in one pass, jointly choosing tile sizes, loop permutations, and spatial mappings. This approach targets scheduling spaces with billions of valid schedules and limitations in existing search and polyhedral methods.

1) Brute-force Approaches:

B. State-of-the-art Schedulers

2) Feedback-based Approaches:

  • Feedback-driven schedulers use machine learning or statistical methods to improve cost models, but their large training-data requirements limit use on hardware under development,,,,,.The passages describe these methods as mainly applicable to post-silicon hardware where large-scale measurement is possible.Source: 2) Feedback-based Approaches · p. 3

3) Constrained-optimization Approaches:

  • Constrained optimization has been applied to large-scale problems in architecture and systems, including instruction scheduling, high-level synthesis, memory partitioning, algorithm selection, and program synthesis,,,,,,,,,,,,,.Source: 3) Constrained-optimization Approaches · p. 3
  • Polyhedral methods apply constrained optimization to auto-vectorization and loop tiling, but prior work targets general-purpose CPUs and GPUs rather than software-managed spatial accelerators,,,,,.Source: 3) Constrained-optimization Approaches · p. 3
  • Existing polyhedral approaches do not directly optimize tile sizes, preventing their co-optimization with loop transformations such as permutation and resulting in sub-optimal schedules,,.They take tile size as input and transform the program based on that given size.Source: 3) Constrained-optimization Approaches · p. 3
  • CoSA encodes tiling, loop permutation, and spatial mapping in one domain-specific formulation, optimizing utilization and communication cost in one pass for complex accelerators.The formulation accounts for multi-level memory hierarchies.Source: 3) Constrained-optimization Approaches · p. 3

III. THE COSA FRAMEWORK … 3) Target Scheduling Decisions:

CoSA formulates DNN-layer scheduling for spatial accelerators as a constrained MIP that generates schedules in one pass. The framework covers nested-loop workloads and PE-array architectures, with scheduling decisions spanning loop tiling, permutation, and spatial mapping.

III. THE COSA FRAMEWORK

  • CoSA deterministically generates high-performance schedules in one pass without exhaustive search or iterative sampling, and applies across network layers and hardware architectures.It formulates scheduling as a MIP with algorithmic and architectural constraints.Source: §III · p. 3

A. CoSA Overview

  • CoSA takes DNN-layer and accelerator specifications as constraints and uses schedule variables, constraints, and objectives to generate a valid, high-performance schedule.Objectives can include maximizing buffer utilization or improving parallelism.Source: A. CoSA Overview · pp. 3–4

3) Target Scheduling Decisions:

  • 1.7× speedup is possible for the illustrated layer by placing P outermost in the global-buffer loop order, motivating consideration of loop permutation.The schedules share loop tiling and spatial mapping; PCK and PKC place P outermost.Source: Figure 3 · p. 4, 3) Target Scheduling Decisions · p. 4

1) Target Workload:

  • CoSA targets nested-loop DNN operators with bounds R, S, P, Q, C, K, and N; this representation also expresses matrix multiplication.R and S are kernel dimensions, P and Q output dimensions, C input channels, K output channels, and N batch size.Source: 1) Target Workload · p. 4

2) Target Architecture:

  • CoSA targets spatial architectures with processing-element arrays connected by an on-chip network and multiple memory levels, a commonly adopted accelerator template,,,,,,,,,,.Source: 2) Target Architecture · p. 4

3) Target Scheduling Decisions:

  • CoSA schedules specify loop tiling, loop permutation, and spatial mapping to describe how a DNN layer executes across memory levels and parallel spatial resources.Tiling assigns loops and bounds to memory levels, permutation sets loop order within a level, and spatial mapping assigns loops to parallel resources.Source: 3) Target Scheduling Decisions · p. 4
  • 4.3× separates the best and worst spatial mappings for the illustrated layer, with the best mapping distributing P, C, and K across spatial loops.The best schedule uses s:P2C4K2 and t:P2K2; the gap reflects differences in communication traffic.Source: Figure 4 · p. 4, 3) Target Scheduling Decisions · p. 4

B. CoSA Variables and Constants · 1) Variable Representation:

CoSA represents DNN schedules as prime-factor allocations encoded in a binary matrix, with constants describing dimension-to-tensor and tensor-to-memory relationships. This representation captures the full scheduling space through memory-level, spatial-or-temporal, and loop-permutation decisions.

1) Variable Representation:

  • Table III illustrates X with K factors mapped spatially and N = 3 mapped temporally at rank 1 in the global buffer.The table caption describes the N tile as allocated at the innermost loop level in the GlobalBuf.Source: Table III · p. 5, 1) Variable Representation · p. 5
  • Each loop bound is factorized into prime factors, with large prime bounds padded before factorization; each factor receives exactly one scheduling configuration.A configuration combines the mapped memory level, permutation order, and spatial mapping.Source: 1) Variable Representation · p. 5
  • CoSA’s prime-factor encoding represents all possible schedules and lets optimization search the full scheduling space.The paper states that this full-space guarantee follows from the prime-factor decomposition.Source: 1) Variable Representation · p. 5
  • The binary matrix X indexes layer dimensions, loop-bound prime factors, spatial or temporal mapping, and memory and permutation levels.These four axes are indexed by j, n, k, and i, respectively.Source: 1) Variable Representation · p. 5
  • Tiling assigns prime factors to memory levels, while spatial or temporal columns indicate whether each factor executes spatially or temporally.The example splits K into size-2 tiles allocated to the input buffer and global buffer, with both factors spatially mapped.Source: 1) Variable Representation · p. 5
  • Permutation ranks O0 through OZ order factors within a memory level, with the lowest rank innermost and the highest rank outermost.Only one prime factor can occupy each rank; enough rank slots are reserved for all factors at all memory levels, though some may remain empty.Source: 1) Variable Representation · p. 5

B. CoSA Variables and Constants

  • CoSA’s constants include binary matrices A and B: A encodes layer-dimension associations with data tensors, and B encodes which tensors each memory level can store.The variables and constants are summarized in Table II; Table IV presents A and B.Source: B. CoSA Variables and Constants · p. 4, Table IV · p. 5

2) Constant Parameters: · C. CoSA Constraints · 1) Buffer Capacity Constraint:

CoSA encodes DNN tensor-dimension and memory-placement relations with constant binary matrices, then uses those relations to express buffer-capacity constraints in its scheduling model.

2) Constant Parameters:

  • Constant binary matrix A maps layer dimensions R, S, P, Q, C, K, and N to input, weight, and output tensors for transaction-size and communication calculations.These calculations include multicast and reduction traffic.Source: 2) Constant Parameters · p. 5
  • Constant binary matrix B specifies which memory levels can store which tensors, representing architectures with dedicated buffers and shared global buffers.The example has dedicated input and weight buffers, plus a shared global buffer for input and output activations.Source: 2) Constant Parameters · p. 5

C. CoSA Constraints

  • CoSA expresses constraints derived from the target accelerator architecture using its scheduling variables and constant parameters.Source: C. CoSA Constraints · p. 5

1) Buffer Capacity Constraint:

  • Buffer utilization must not exceed each buffer’s capacity and is computed from tensor tiling factors, including both spatial and temporal factors.Tiling sizes use relevant prime factors indicated by X, with tensor dimensions linked through matrix A and buffer placement represented by B.Source: 1) Buffer Capacity Constraint · p. 5
  • Taking logarithms transforms the buffer-utilization constraint’s products of decision variables X into a linear expression suitable for standard constraint solvers.The paper also encodes the associated if-else statement.Source: 1) Buffer Capacity Constraint · p. 6
  • CoSA adds the logarithm of each tensor’s datatype size to buffer utilization to account for different precisions.The datatype-size term is precision_v.Source: 1) Buffer Capacity Constraint · p. 6

2) Spatial Resource Constraint: … 2) Compute-Driven Objective:

CoSA constrains spatial mappings to fit available hardware resources and offers objectives for optimizing buffer utilization, compute cycles, or other performance aspects, individually or in combination. Its utilization objective targets reuse and lower communication, while its compute objective encourages mapping more iterations spatially to exploit parallelism.

2) Spatial Resource Constraint:

  • Spatial mappings must assign each problem factor to either spatial or temporal execution and keep spatial factors within the available resource limit.The limits apply to processing elements at the chip level and multiply-and-accumulate units at the processing-element level; SI denotes available spatial resources at level I.Source: 2) Spatial Resource Constraint · p. 6

D. Objective Functions

  • CoSA objectives can optimize utilization, compute, or communication individually, or combine objectives to optimize multiple performance aspects.Source: D. Objective Functions · p. 6

1) Utilization-Driven Objective:

  • CoSA maximizes buffer utilization across tensors to increase data reuse and minimize overall communication.Prior work shows that optimizing tiling block size for buffer utilization can achieve communication lower bounds in a system with one-level cache.Source: 1) Utilization-Driven Objective · p. 6
  • Maximizing the logarithmic sum of utilization across buffer levels and tensors is equivalent to maximizing their geometric mean.Source: 1) Utilization-Driven Objective · p. 6
  • Users can weight buffer levels or tensors differently to prioritize utilization of a specific memory level or data tensor.Source: 1) Utilization-Driven Objective · p. 6

2) Compute-Driven Objective:

  • CoSA’s compute-driven objective minimizes estimated compute cycles by multiplying the temporal factors for each processing element.Source: 2) Compute-Driven Objective · p. 6
  • The compute objective encourages the solver to exploit system parallelism by mapping more iterations to spatial resources rather than temporal execution.Source: 2) Compute-Driven Objective · p. 6

3) Traffic-Driven Objective:

CoSA models communication cost through a traffic-driven objective that combines data size per transfer, spatial traffic patterns, and temporal iterations. Its representation captures these factors using allocated prime factors, tensor-dimension correlations, and permutation-level loop information.

3) Traffic-Driven Objective:

  • Communication traffic combines data size per transfer, spatial multicast or unicast factors, and temporal iterations to capture communication cost.The paper describes the total amount of network traffic as the product of these three factors.Source: 3) Traffic-Driven Objective · p. 6
  • CoSA derives data size per transfer from allocated prime factors in X and the dimension-tensor correlation matrix A.This calculation is analogous to the paper’s buffer-utilization expression.Source: 3) Traffic-Driven Objective · p. 6
  • Matrix A determines whether spatial mappings produce multicast, unicast, or reduction traffic, as illustrated by weight and output tensors.Mapping P spatially yields multicast weights when A_P,W = 0, while mapping C yields unicast weights when A_C,W = 1; for output tensor OA, mapping C with A_C,OA = 0 requires reduction, whereas mapping P with A_P,OA = 1 yields unicast.Source: 3) Traffic-Driven Objective · pp. 6–7, Figure 5 · p. 7
  • A traffic-iteration factor Y uses permutation-level information from X, A, and B to select NoC loop bounds and count data transfers.For each variable, a relevant factor inside the current loop level drives the traffic iteration even when it is unrelated to that variable’s tensor, supporting reuse optimization.Source: 3) Traffic-Driven Objective · p. 7

4) Overall Objective: … A. Evaluation Platforms

CoSA balances compute and memory objectives using user-selected weights, while targeting dense workloads and deterministic, explicitly managed-scratchpad systems. Its evaluation combines Timeloop estimates with cycle-exact NoC simulation to capture communication and concurrent hardware behavior.

4) Overall Objective:

  • Microbenchmarks can determine objective weights, which balance compute and memory terms and can be calibrated to cycles under double buffering.An alternative objective minimizes the difference between weighted traffic and compute terms.Source: 4) Overall Objective · p. 7

E. Limitation of CoSA

  • CoSA assumes dense CNN workloads and best targets deterministic systems with explicitly managed scratchpads, but iterative tuning can approximate unknown hardware models.The approach does not exploit data sparsity; iterative search can also prune invalid search-space points.Source: E. Limitation of CoSA · p. 7

IV. METHODOLOGY

  • The methodology evaluates CoSA schedules using Timeloop for cycle and energy estimates and a cycle-exact NoC simulator for overall latency.The NoC simulator more accurately captures communication overhead and concurrent hardware behavior on spatial architectures.Source: §IV · p. 7, A. Evaluation Platforms · p. 7

A. Evaluation Platforms

  • Timeloop estimates accelerator performance and energy, reporting maximum PE workload cycles under perfect latency hiding with double buffering.It estimates energy by multiplying component access counts inferred from the schedule by per-access energy values, then summing them.Source: A. Evaluation Platforms · pp. 7–8
  • The NoC simulator augments Timeloop’s analytical PE model with a synthesizable NoC to represent communication costs, especially for communication-bound schedules.Communication is identified as a key contributor to latency in NoC-based systems.Source: A. Evaluation Platforms · p. 8
  • The transaction-based, cycle-exact simulator models a resizable 2-D mesh with X-Y routing and unicast and multicast requests.It uses the synthesizable SystemC router design from Matchlib.Source: A. Evaluation Platforms · p. 8
  • Concurrent modeling of NoC transfers, PE execution, and off-chip DRAM accesses using DRAMSim2 captures the impact of NoC traffic congestion.Source: A. Evaluation Platforms · p. 8

B. Baseline Schedulers · C. Experiment Setup · V. EVALUATION

CoSA is evaluated against Random and Timeloop Hybrid schedulers on diverse DNN workloads using a Simba-like spatial-array baseline. The evaluation examines time-to-solution, performance, and energy across platforms, architectures, and layers.

B. Baseline Schedulers

  • Timeloop Hybrid randomly selects tiling factorizations, prunes superfluous permutations, and linearly explores each pruned mapping subspace.It runs 32 independent threads; each stops after 500 consecutive valid yet sub-optimal mappings, and the best schedule is chosen from 16,000+ valid schedules.Source: B. Baseline Schedulers · p. 8
  • The Random baseline searches five valid schedules and selects the one with the best result for the target metric.Source: B. Baseline Schedulers · p. 8

C. Experiment Setup

  • Benchmarks cover ResNet-50, ResNeXt-50 (32x4d), and Deepbench OCR and Face Recognition, using 8-bit inputs and weights and 24-bit partial sums.Dimensions are not padded to multiples of 2 because the added overhead outweighs the benefit of more scheduling options.Source: C. Experiment Setup · p. 8
  • The baseline is a Simba-like spatial-array architecture, and CoSA is described as generalizable across architecture parameters while producing high-performance schedules in one shot.Hardware construct specifications are summarized in Table V, but its contents are not supplied here.Source: C. Experiment Setup · p. 8

V. EVALUATION

  • CoSA is reported to improve time-to-solution, performance, and energy over baseline schedulers across evaluation platforms, DNN architectures, and diverse layers.The supplied evaluation passage states these comparison dimensions but gives no quantitative results.Source: §V · p. 8

A. Time to Solution … 2) Energy:

CoSA’s constrained-optimization schedules improve performance and energy efficiency over search-based baselines while reducing schedule-generation time. The evaluations compare four DNN workloads using Timeloop models and NoC simulation.

A. Time to Solution

  • CoSA generates schedules in 4.2s versus 379.9s for Timeloop Hybrid search, a more than 90× time-to-solution advantage.Timeloop Hybrid sampled 67 million schedules per layer and evaluated more than 16 thousand valid schedules; Random search found only five valid schedules among 20K samples in 4.6 seconds.Source: A. Time to Solution · p. 8, Table VI · p. 8

1) Performance:

  • Across four DNNs, CoSA schedules achieve 5.2× and 1.5× higher geomean speedup than Random and Timeloop Hybrid search, respectively.These speedups are relative to Random search on the baseline 4×4 NoC architecture.Source: Figure 6 · p. 9, 1) Performance · p. 9

2) Energy:

  • CoSA schedules are 22% more energy-efficient than the best Timeloop Hybrid solutions, as evaluated by the Timeloop energy model.The traffic objective targets energy efficiency because energy cost is highly correlated with hardware-component access counts; the figure uses the baseline 4×4 NoC and normalizes energy to Random search.Source: Figure 7 · p. 9, 2) Energy · p. 9

B. Evaluation on Timeloop Performance and Energy Models

1) Performance:

  • In a few layers, Timeloop Hybrid slightly outperforms CoSA because it uses more DRAM-level iterations; fine-tuning CoSA’s objective weights could further improve its schedules.The higher iteration count reduces DRAM transaction size and balances the pipeline.Source: 1) Performance · p. 9
  • A more exhaustive Timeloop Hybrid search with 32K valid schedules improves latency by only 7.5% while doubling runtime, and still does not match CoSA’s schedule efficiency.The comparison reports latency improvement and runtime increase for the exhaustive search.Source: 1) Performance · p. 9

2) Energy:

  • On NoC simulation, CoSA achieves 3.3× and 2.5× higher geomean speedup than Random and Timeloop Hybrid search, respectively, and generalizes across hardware architectures.The NoC comparison covers four DNN workloads; the architecture-generalization result concerns varying compute and on-chip storage resources.Source: Figure 10 · p. 10, Figure 9 · p. 10

3) Objective Breakdown: · 4) Different HW Architectures:

CoSA minimizes its weighted objective across utilization, compute cost, and traffic cost, while its speedups persist across hardware architectures with different PE-array and SRAM configurations.

3) Objective Breakdown:

  • CoSA attains the lowest total objective among the approaches and optimizes utilization, compute cost, and traffic cost simultaneously.The objective combines the three sub-objectives with a weighted sum.Source: 3) Objective Breakdown · p. 10
  • On ResNet50 layer 3_7_512_512_1, CoSA’s schedule runs 7× faster than schedules from Random and Timeloop Hybrid search.This objective breakdown aligns with the empirical result in Fig. 6.Source: 3) Objective Breakdown · p. 10

4) Different HW Architectures:

  • CoSA evaluates different hardware architectures, customizing objective weights for each architecture using a micro-benchmark.The evaluation uses the same weights for experiments on the same architecture.Source: 4) Different HW Architectures · p. 10
  • CoSA achieves 4.4× speedup over Random and 1.1× over Timeloop Hybrid search across four networks with a 4×-larger PE array.The larger array is paired with 2× on-chip communication and DRAM bandwidth; its larger spatial array makes communication costs more consequential.Source: 4) Different HW Architectures · pp. 10–11
  • The PE-array study scales PEs by 4× and on-chip communication and DRAM bandwidth by 2×, altering DNN layers’ compute and communication patterns.With more spatial arithmetic units, spatial and temporal mapping decisions become especially important for performance.Source: 4) Different HW Architectures · p. 10
  • With doubled local buffers and an 8×-larger global buffer, CoSA achieves 5.7× speedup over Random and 1.4× over Timeloop Hybrid search.The altered capacities can change optimal data reuse and NoC traffic reduction strategies.Source: 4) Different HW Architectures · p. 11
  • Doubling local accumulation, weight, and input buffers and increasing global buffer capacity 8× can change optimal data reuse and NoC traffic reduction.The study uses these memory-capacity changes to test schedule quality across architectures.Source: 4) Different HW Architectures · p. 11

C. Evaluation on NoC Simulator … VI. CONCLUSION

CoSA’s evaluations report performance gains from one-shot schedules on NoC-simulated DNN workloads and GPU workloads. The conclusion describes a unified constrained-optimization formulation for loop tiling, permutation, and spatial mapping.

C. Evaluation on NoC Simulator

  • CoSA outperforms baseline schedules on all four DNN workloads, with the largest gains on convolutional layers, while FC-layer performance shows no significant schedule differences.The NoC simulator captures communication overhead more accurately than Timeloop models; Timeloop Hybrid underperforms Random on the cited convolutional layers, while FC layers are memory-bound with low PE utilization.Source: C. Evaluation on NoC Simulator · p. 11
  • Across the four networks, CoSA achieves a geometric-average speedup of up to 3.3× versus the best Random search solutions and 2.5× versus Timeloop Hybrid search schedules.Unlike the iterative Random and Timeloop Hybrid searches, CoSA schedules are consistently performant with the one-shot solution.Source: C. Evaluation on NoC Simulator · p. 11

D. Evaluation on GPU

  • CoSA models GPU thread groups as spatial levels and constrains group sizes and shared/local-memory use to generate valid schedules.The evaluation targets an NVIDIA K80 with 2496 CUDA cores, 1.5MB L2 cache, and a maximum of 1024 threads per block.Source: Constraints · p. 11, D. Evaluation on GPU · p. 11

Constraints.

  • CoSA discounts GPU compute cycles by the total number of threads and adjusts other objective weights using a micro-benchmark.Source: Constraints · p. 11
  • 2,500× shorter time-to-solution (0.02s vs. 50s per layer) and 1.10× geomean speedup on ResNet50 distinguish CoSA schedules from TVM’s 50-trial-per-layer baseline.Source: Constraints · p. 11, D. Evaluation on GPU · p. 11

VI. CONCLUSION

  • CoSA’s conclusion reports up to 2.5× speedup, 22% better energy-efficiency, and 90× shorter time-to-solution than state-of-the-art schedules.Its single mathematical formulation jointly solves loop tiling, loop permutation, and spatial mapping using regularities in DNN workloads and accelerator designs.Source: §VI · p. 11

Related papers

Loading 2105.01898v1…