Source-linked AI summary
PolyCIM: Improving Data Reuse in Digital CIM Accelerators with Polyhedral-Based Compilation
Yingjie Qi, Cenlin Duan, Yiou Wang, Yikun Wang, Xiaolin He, Weisheng Zhao, Jianlei Yang
TL;DR
Modern DNN operators can underutilize digital CIM arrays because conventional mappings expose only axis-aligned reuse. PolyCIM applies polyhedral affine transformations to expose non-axial reuse, reporting up to 4× macro utilization improvement and 3.2× speedup across diverse workloads.
Problem
Rigid CIM arrays and conventional mappings can underutilize modern DNN operators because they expose only axis-aligned reuse, while reuse often lies along non-axial directions.
Source: §1 · pp. 1–2Method
PolyCIM uses polyhedral affine transformations to expose non-axial reuse and generate CIM mappings through reuse exposure, computation mapping, and data movement optimization.
Source: §1 · p. 2, §3.1 · p. 3Results
Up to 4× macro utilization improvement and 3.2× speedup across diverse DNN workloads and CIM configurations are reported for PolyCIM.
Source: §1 · p. 2Takeaways & Limitations
PolyCIM represents diverse DNN workloads and aligns their reuse opportunities with the rigid structure of CIM arrays.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Digital Compute-in-Memory (CIM) presents a promising solution for accelerating deep neural networks (DNNs) through the integration of computational logic directly within memory arrays. However, mapping modern DNN operators to CIM accelerators often results in severe array underutilization, due to the strict data reuse constraints imposed by the rigid CIM array structure. We observe that data reuse in modern DNNs forms hyperplane structures often oriented along non-axial directions, rendering them invisible to conventional mapping methods that only exploit axis-aligned reuse. In this work, we propose PolyCIM, a polyhedral-based compilation framework for CIM architectures that systematically exposes and realigns these hyperplanes through affine transformations. PolyCIM provides a unified abstraction capable of efficiently representing both diverse DNN workloads and digital CIM architectures. Through data reuse exposure, computation mapping, and data movement optimization, PolyCIM generates mappings for CIM architectures that achieve superior array utilization and performance. Experimental results show that PolyCIM delivers up to $4\times$ improvement in macro utilization and $3.2\times$ speedup, effectively bridging the gap between modern DNN operators and CIM architectures.
1 Introduction
Modern DNN operators can underutilize rigid digital CIM arrays because conventional mappings expose only axis-aligned reuse. PolyCIM uses affine transformations to expose non-axial reuse, reporting higher macro utilization and speedup across diverse workloads.
1 Introduction
- Conventional im2col mappings can leave columns activating without useful output when too few filters share an input, limiting support for depthwise and grouped convolutions.The rigid array activates each row atomically, while filters are mapped to columns.Source: §1 · p. 1
- DNN reuse often forms non-axial hyperplanes that fixed, operator-independent mappings cannot expose, but affine transformations can realign them with array axes.DNN operand accesses are affine functions of loop indices.Source: §1 · p. 2
- PolyCIM formulates non-axial reuse exposure as a polyhedral transformation problem and automatically generates high-utilization mappings through reuse exposure, computation mapping, and data movement optimization.The framework targets reuse patterns conventional mappings cannot capture.Source: §1 · p. 2
- Up to 4× macro utilization improvement and 3.2× speedup across diverse DNN workloads and CIM configurations are reported for PolyCIM.These results are compared with conventional mapping methods.Source: §1 · p. 2
2 Background
Existing CIM mappings typically align computation with hardware axes, while specialized non-axial approaches are limited in generality or array capacity. The polyhedral model represents affine loop programs and enables schedules that reshape iteration spaces while respecting dependences.
2.1 DNN Mappings on CIM Architectures
- Most CIM frameworks lower computations to fixed matrix multiplication primitives or use im2col, whose column utilization depends on how many filters share an input window.Columns can remain idle when that number is small.Source: §2.1 · p. 2
- SDK improves utilization for standard 2D convolutions by placing shifted copies of filters, but its manually designed layout does not generalize and can be constrained by digital SRAM array capacity.When the array cannot fit extra kernel copies, the mapping becomes equivalent to im2col.Source: §2.1 · p. 2
2.2 Polyhedral Model
- The polyhedral model represents valid loop iterations and affine memory accesses as integer polyhedra and relations, enabling dependence analysis and loop transformations.Iterations accessing the same element form reuse hyperplanes whose directions lie in the access matrix's null space.Source: §2.2 · p. 3
- Affine schedules assign timestamps to iterations and can rotate the iteration basis so reuse hyperplanes become coordinate directions that CIM arrays can exploit.Schedules reshape execution order subject to data-dependence constraints.Source: §2.2 · p. 3
3.1 Framework Overview
PolyCIM compiles DNN operators for CIM accelerators through a common polyhedral representation and three refinement stages: data reuse exposure, computation mapping, and data movement optimization.
3.1 Framework Overview
- PolyCIM takes an operator description and hardware configuration, builds a common polyhedral representation, and lowers it to executable CIM code.The representation is refined through data reuse exposure, computation mapping, and data movement optimization.Source: §3.1 · p. 3
- Data reuse exposure realigns operand reuse, computation mapping places iterations on the macro grid, and data movement optimization selects buffering and loop order to reduce redundant transfers.Code generation emits the executable CIM program from the resulting schedule.Source: §3.1 · p. 3
- In the Conv1D example, same-colored iteration points access the same input data.The figure illustrates data reuse exposure.Source: Figure 3 · p. 4
3.2 PolyCIM Abstractions
PolyCIM represents DNN operators and CIM hardware with polyhedral abstractions that encode iteration spaces and operand access relations, supporting systematic mapping between them.
3.2 PolyCIM Abstractions
- PolyCIM models MVM-based operators, including standard, depthwise, grouped, and dilated convolutions, as an iteration space with input, weight, and output access relations.The Conv1D example has two output channels, eight positions per channel, and a kernel width of three.Source: §3.2 · p. 4
- A CIM unit is modeled across macros, rows, and columns, with input broadcasting shared across each row and output accumulation shared down each column.Local memory retains a distinct location for every macro-row-column position.Source: §3.2 · p. 4
- The unified software and hardware abstractions support systematic transformations between operator iteration spaces and CIM computation spaces while preserving data dependencies and hardware constraints.The hardware model includes CIM units, auxiliary compute units, and supporting memory hierarchies.Source: §3.2 · p. 4
3.3 Data Reuse Exposure
PolyCIM exposes reuse patterns that lie along oblique iteration-space directions through pre-tiling and affine scheduling, then aligns them with CIM macro structure.
3.3 Data Reuse Exposure
- Oblique reuse directions are invisible to conventional axis-aligned approaches, so PolyCIM exposes them with pre-tiling followed by affine scheduling.The stated optimization objective is to maximize average macro utilization.Source: §3.3 · p. 4
3.3.1 Pre-Tiling Schedule.
- Pre-tiling decomposes iteration axes into factor-sized sub-coordinates before affine scheduling to limit iteration-space expansion.For the Conv1D example, direct affine scheduling without pre-tiling expands the space from 2 × 8 × 3 to 2 × 8 × 10, producing underutilized or invalid mapped operations.Source: §3.3.1 · pp. 4–5
- PolyCIM prunes pre-tiling candidates by excluding axes unused in affine transformations and deduplicating assignments across symmetric convolution axes.It selects a remaining candidate to minimize expansion while preserving reuse opportunities.Source: §3.3.1 · p. 5
3.3.2 Affine Schedule.
- Affine scheduling searches for basis vectors parallel to reuse hyperplanes, using direction and reuse-degree constraints to select bases that expose operand reuse.The selected independent vectors must include input- and output-reuse directions; candidate combinations are assessed by average macro utilization.Source: §3.3.2 · p. 5
- The selected basis vectors define an affine coordinate transformation that realigns reuse with new axes, making it exploitable by CIM macros.Scaling rows to integer coefficients preserves the transformation semantics.Source: §3.3.2 · pp. 5–6
3.4 Computation Mapping
PolyCIM maps reuse-aligned iterations onto CIM macros by coalescing and tiling dimensions, then distributing outer loops across multiple macros according to reuse patterns.
3.4 Computation Mapping
- PolyCIM uses a hierarchical mapping strategy that coalesces reuse dimensions, tiles them to physical macro dimensions, and distributes remaining iterations across macros.This strategy accommodates rigid constraints within a macro while exploiting parallelism across macros.Source: §3.4 · p. 6, §3.4.1 · p. 6
3.4.1 CIM Macro Mapping.
- The macro mapping coalesces axes that reuse the same data, then post-tiles dimensions exceeding the physical R × C bounds into inner macro positions and outer computation blocks.In the Conv1D example, the input-reuse coordinate maps to columns and the output-reuse coordinate to rows.Source: §3.4.1 · p. 6
3.4.2 Multi-macro Mapping.
- Across macros, PolyCIM prioritizes output-reuse axes, then input-reuse axes, non-reuse axes, and finally weight-reuse axes when storage permits.Output reuse is prioritized because partial sums typically have larger bit-widths than inputs; input distribution can share broadcasts across macros.Source: §3.4.2 · p. 6
3.5 Data Movement Optimization
PolyCIM addresses transformed memory accesses with an iteration-space-aligned layout and a schedule that chooses loop order and operand buffering points to reduce data movement.
3.5 Data Movement Optimization
- PolyCIM combines layout transformation for contiguous access with data-movement scheduling to reduce transfers across the memory hierarchy.The layout matches data organization to transformed access order; scheduling chooses loop ordering and where operands enter buffers.Source: §3.5 · p. 6
3.5.1 Iteration-Space-Aligned Layout.
- The iteration-space-aligned layout reorganizes each operand so consecutive transformed iterations access consecutive memory locations, with code generated offline before computation.The transformation is generated from the applied polyhedral schedules, avoiding manual reorganization.Source: §3.5.1 · pp. 6–7
3.5.2 Data Movement Scheduling.
- Data-movement scheduling jointly selects loop permutation and each operand’s buffering point, minimizing log-transformed transfer volumes across operands and memory levels under capacity constraints.The formulation uses binary decision variables; log-linearization turns multiplicative buffer-size and transfer-count expressions into sums.Source: §3.5.2 · p. 7
4 Evaluation
Evaluation compares PolyCIM with im2col-based CIMFlow mappings across operators and complete DNNs, then measures the impact of individual optimization components.
4.1 Experimental Setup
- Evaluation uses an 8-macro CIM architecture with 32 × 64 and 64 × 64 SRAM macros, compares CIMFlow’s im2col mapping, and cycle-simulates varied operators and three DNNs.The complete models are ConvNeXt-Tiny, EfficientNet-B0, and MobileNetV2; Table 1 lists the evaluated operators.Source: §4.1 · p. 7, Table 1 · p. 7
4.2.1 Operator-Level Evaluation.
- Across convolution operators, PolyCIM beats im2col, with 1.9×–8.2× speedups (4.9× average) and up to 75% utilization.The largest stated speedup is 8.2× on C2; grouped convolution and 3D convolution see smaller relative gains of roughly 2×.Source: §4.2.1 · p. 7, Figure 6 · p. 8
4.2.2 Model-Level Evaluation.
- Across three DNNs, PolyCIM averages 2.1× speedup; ConvNeXt-Tiny reaches 3.2×, while utilization rises from 20%–40% to 84%–89% on 32 × 64.The model-level averages exceed per-operator utilization because standard convolution layers are included and im2col maps them efficiently.Source: §4.2.2 · pp. 7–8, Figure 6 · p. 8
4.3 Ablation Study
- On C1, removing affine scheduling degrades latency 9.7×, disabling pre-tiling slows 6.2×, removing coalescing cuts utilization from 28% to below 10%, and data-movement optimization adds 1.4× speedup.The ablation evaluates pre-tiling, affine scheduling, coalescing, and data-movement optimization.Source: §4.3 · p. 8, Figure 7 · p. 8
5 Conclusion
PolyCIM uses polyhedral transformations to expose non-axial DNN reuse and reports improved macro utilization and speedup across diverse workloads.
5 Conclusion
- PolyCIM exposes non-axial DNN reuse through polyhedral transformations and achieves up to 4× macro utilization improvement and 3.2× speedup across diverse workloads.Its unified abstraction represents diverse DNN workloads and aligns reuse opportunities with rigid CIM array structures.Source: §5 · p. 8