Source-linked AI summary
CAP-RAM: A Charge-Domain In-Memory Computing 6T-SRAM for Accurate and Precision-Programmable CNN Inference
Zhiyu Chen, Zhanghao Yu, Qing Jin, Yan He, Jingyu Wang, Sheng Lin, Dai Li, Yanzhi Wang, Kaiyuan Yang
TL;DR
Limited computing and energy budgets make efficient edge CNN inference challenging, and data movement is a major energy bottleneck. CAP-RAM uses charge-domain, semi-parallel in-SRAM computing with configurable input and weight precision. A 65-nm prototype reports inference accuracy of 98.8% on MNIST and 89.0% on CIFAR-10, with 573.4-GOPS throughput and 49.3 TOPS/W energy efficiency.
Problem
Data movement rather than arithmetic is the energy bottleneck in convolution, complicating CNN inference on edge devices with limited computing resources and energy budgets.
Source: §I · p. 1Method
CAP-RAM combines charge-domain in-SRAM MAC computing with a semi-parallel architecture using standard 6T cells and configurable input and weight bitwidths.
Source: §I · p. 2, 3) Clustering Structure · p. 4Results
98.8% inference accuracy on MNIST and 89.0% on CIFAR-10; the prototype achieves 573.4-GOPS throughput and 49.3 TOPS/W energy efficiency.
Source: §V · p. 10Takeaways & Limitations
The macro is compatible with structured pruning and quantization training, and combining it with reduced CNN models promises fully on-chip weight storage and efficient inference.
Source: §I · p. 2
Abstract
from arXiv · showhide
A compact, accurate, and bitwidth-programmable in-memory computing (IMC) static random-access memory (SRAM) macro, named CAP-RAM, is presented for energy-efficient convolutional neural network (CNN) inference. It leverages a novel charge-domain multiply-and-accumulate (MAC) mechanism and circuitry to achieve superior linearity under process variations compared to conventional IMC designs. The adopted semi-parallel architecture efficiently stores filters from multiple CNN layers by sharing eight standard 6T SRAM cells with one charge-domain MAC circuit. Moreover, up to six levels of bit-width of weights with two encoding schemes and eight levels of input activations are supported. A 7-bit charge-injection SAR (ciSAR) analog-to-digital converter (ADC) getting rid of sample and hold (S&H) and input/reference buffers further improves the overall energy efficiency and throughput. A 65-nm prototype validates the excellent linearity and computing accuracy of CAP-RAM. A single 512x128 macro stores a complete pruned and quantized CNN model to achieve 98.8% inference accuracy on the MNIST data set and 89.0% on the CIFAR-10 data set, with a 573.4-giga operations per second (GOPS) peak throughput and a 49.4-tera operations per second (TOPS)/W energy efficiency.
I. INTRODUCTION
Edge CNN inference is constrained by limited computing and energy budgets, while data movement can dominate convolution energy. CAP-RAM targets these constraints with charge-domain, precision-configurable in-SRAM computing using standard 6T cells.
I. INTRODUCTION
- CNN inference hardware for edge devices must meet limited computing-resource and energy budgets while providing real-time, low-power operation.CNNs are described as successful across AI applications, with models commonly pretrained in the cloud and deployed to edge devices.Source: §I · p. 1
- Data movement, rather than arithmetic, is the energy bottleneck in convolution, with DRAM and SRAM accesses costing about 8×10^4 and 3 × 10^3 times an 8-bit addition, respectively.The passage attributes these figures to 45-nm technology and identifies the resulting memory wall as particularly severe for data-intensive computing.Source: §I · p. 1
- IMC reduces data movement by computing across multiple memory rows in parallel and amortizing read energy across parallel access.An early SRAM IMC implementation activates multiple standard 6T cells and accumulates current on the bitline for MAC computing.Source: §I · p. 1
- Existing IMC designs face accuracy limits from process variation, while precision-configurable architectures use bit-serial computation to support multi-bit operations.The introduction describes charge-domain IMC and reconfigurable in-SRAM computing as approaches developed to address accuracy and precision needs.Source: §I · p. 2
- CAP-RAM combines charge-domain in-SRAM computation using standard 6T cells with semi-parallel computing for eight input-activation levels and six weight levels.Its design also includes a charge-injection SAR ADC that avoids sample-and-hold circuits and reference-voltage drivers.Source: §I · p. 2
- CAP-RAM is presented as compatible with structured pruning and quantization training, with reduced CNN models promising on-chip weight storage and efficient inference.The introduction frames these techniques as a way to pair a high-density on-chip IMC macro with reduced CNN models.Source: §I · p. 2
A. Principles of the Core Circuits
CAP-RAM clusters eight standard 6T SRAM cells with a charge-domain MAC circuit, executing analog multiplication and accumulation through reset, DAC, multiplication, and accumulation phases. The design mitigates switching errors, with 1000 Monte Carlo simulations reporting a 0.6-mV standard deviation after offset cancellation.
A. Principles of the Core Circuits
- Each cluster combines eight standard 6T SRAM weight cells, switches, and a MOM capacitor for charge-domain MAC computing.The cluster also includes precharge and read/write circuitry for normal SRAM operations.Source: A. Principles of the Core Circuits · p. 2
- The four operating phases precharge the local bitline and capacitors, sample a 4-bit DAC input, multiply through the selected cell, then share charge for accumulation.The stored cell state determines whether the MOM capacitor is pulled to Vdd or retains VIN; leakage changes it by less than 0.3 mV in the stated worst case.Source: A. Principles of the Core Circuits · pp. 2–3
- Fig. 3 distinguishes current-domain computation with an 8T cell from charge-domain designs using wordline or bitline inputs.The figure identifies the three existing IMC cell-design categories.Source: Figure 3 · p. 3
CGS + CMOM
- Opposite-polarity coupling effects cancel the local MOM-cap coupling error, while output-line coupling partially mitigates charge-injection error.Rail-to-rail gate controls make the coupling terms constant; the output-line parasitic capacitance is CP.Source: CGS + CMOM · p. 3, A. Principles of the Core Circuits · p. 3
- 1000 Monte Carlo simulations report a 0.6-mV standard deviation after transistor sizing cancels the analog-computation offset.The 1.2-fF MOM capacitors reflect a tradeoff between energy and analog-output dynamic range.Source: CGS + CMOM · p. 3
B. Design Analysis and Related Work
Current-domain IMC is simple and compatible with standard SRAM cells, but transistor nonlinearity and process variation limit computing accuracy and parallelism. Charge-domain IMC computes on capacitors, avoiding transistor operating-region dependence and supporting better accuracy and higher parallelism.
1) Current-Domain Versus Charge-Domain Computing:
- Current-domain output is nonlinear and shows larger variations at lower output voltages under the simulated 65-nm conditions.The simulation uses 0.6-V input, 200-ps access time, and 64 activated rows to keep the access transistor in saturation.Source: 1) Current-Domain Versus Charge-Domain… · p. 3
- Nearly 2-LSB standard deviation characterizes the simulated MAC error distribution for the current-domain design quantized with an ideal 6-bit ADC.The inputs are grouped to cover the output range.Source: 1) Current-Domain Versus Charge-Domain… · p. 4
- Current-domain IMC trades accuracy against parallelism: higher input voltage reduces transistor variation but makes linear-region entry more likely, restricting active rows.Fewer parallel rows degrade throughput and efficiency.Source: 1) Current-Domain Versus Charge-Domain… · p. 4
- Charge-domain IMC offers better computing accuracy and higher parallelism because capacitor-based charge sharing is less affected by transistor variation and operating regions.CAP-RAM measurements show no significant linearity degradation.Source: 1) Current-Domain Versus Charge-Domain… · p. 4
2) Wordline Versus Bitline Inputs:
Charge-domain cells support both wordline and bitline inputs, but bitline input is adopted in CAP-RAM to support higher throughput. It samples multi-bit inputs on a local capacitor and performs multiplication before accumulation.
2) Wordline Versus Bitline Inputs:
- Fig. 7 compares a fully parallel structure with clustering, where CAP-RAM selects one cell per operation to amortize larger charge-domain bitcells.The larger bitcells otherwise reduce compute density and increase energy and delay relative to ideal fully parallel IMC.Source: Figure 7 · p. 4
- Bitline input samples the input signal on a local capacitor, supports multi-bit inputs, and is the approach CAP-RAM adopts for higher throughput.An h-bit input architecture achieves nearly h times the throughput of a pure bit-serial scheme requiring h loops.Source: 2) Wordline Versus Bitline Inputs · p. 4
- Wordline-input charge-domain cells perform an input–weight AND before charge sharing, a simple and efficient scheme for binary inputs.Multi-bit inputs require high-precision circuitry because the sampling capacitors are only a few femtofarads.Source: 2) Wordline Versus Bitline Inputs · p. 4
3) Clustering Structure:
CAP-RAM clusters standard 6T SRAM cells with a charge-domain MAC circuit, trading full row-level parallelism for amortized area and flexible weight storage. The 512 × 128 prototype uses 32 pairs of slices with programmable digital processing.
3) Clustering Structure:
- In an iso-area comparison, CAP-RAM is reported to provide the highest compute density, throughput, and weight storage density.The comparison uses a 4-bit input × 1-bit weight MAC and a cluster of three non-push-rule cells; the reported gap grows with push-rule layouts.Source: 3) Clustering Structure · pp. 4–5
- Eight-cell clusters let CAP-RAM store different CNN layers for layer-by-layer execution, tuning the tradeoff between weight storage density and compute density.The prototype uses eight-cell clusters to maximize on-chip storage density and support efficient layer-by-layer execution of LeNet and ResNet models.Source: 3) Clustering Structure · p. 5
III. IMPLEMENTATION
- The prototype implements a 512 × 128 memory divided into 32 pairs of slices, with each slice containing 128 clusters and performing one MAC computation.Each slice's output line connects to one differential input of an ADC.Source: §III · p. 5
- IMC and conversion run serially in one 70 MHz global clock cycle, while serial accumulators and programmable adder trees combine partial sums for input and weight precision.The adder trees support programmable weight bitwidths; the broader architecture groups several 6T cells with one analog computing module, selecting one cell per operation.Source: 3) Clustering Structure · p. 4, §III · p. 5
A. Reconfigurable Support of Two Data Encoding Schemes
CAP-RAM supports both 2’s complement and ternary weight encodings, using single-ended or differential ADC modes. Its charge-injection SAR ADC addresses area and driver requirements while omitting separate sample-and-hold circuits.
A. Reconfigurable Support of Two Data Encoding Schemes
- 2’s complement weights use one SRAM cell per bit and k neighboring slices for k-bit weights, with consecutive ADC outputs shifted and summed digitally.The most significant bit is negative, so its partial sum's sign is reversed; this encoding is preferred for storage efficiency.Source: A. Reconfigurable Support of Two Data… · p. 5
- Ternary encoding uses two SRAM cells per −1/0/+1 unit and differential ADC inputs to subtract positive and negative results without extra processing.For k-bit information, it requires 2k−2 bitcells and k−1 ADCs; ternary networks use the same memory footprint as 2-bit 2’s complement weights while saving one ADC per MAC.Source: A. Reconfigurable Support of Two Data… · p. 5
B. Compact and Driver-Less ciSAR ADC
- Conventional capacitor-based SAR ADCs can require ADC area comparable to the SRAM array, powerful reference buffers, and an input buffer with sample-and-hold circuits.Direct charge sharing to the capacitive DAC is also described as causing substantial dynamic-range loss.Source: B. Compact and Driver-Less ciSAR ADC · p. 6
- The ciSAR ADC replaces DAC capacitors with charge-injection transistors, achieving 6.85 ENOB in transient-noise simulations with a reported area of 429 μm2.Its conversion step is controlled by bias voltage VB rather than a voltage reference driver.Source: B. Compact and Driver-Less ciSAR ADC · p. 6
- CAP-RAM’s ciSAR ADC uses analog MAC outputs already computed and stored on 128 local CMOM and CP capacitors after charge sharing, so no extra sample-and-hold circuits are necessary.SAR control is shared among 32 ADCs, and single-input mode has 6-bit resolution.Source: B. Compact and Driver-Less ciSAR ADC · p. 6
C. Current-Steering DAC
The implementation uses a current-steering DAC and programmable digital accumulation around the ADCs. The 65-nm prototype's reported area is 0.179 mm2, with most area in the SRAM array.
C. Current-Steering DAC
- The current-steering DAC's simulated linearity is R2 = 0.9999, with input-line voltage standard deviation around 1.56 mV in the worst Monte Carlo scenario.Bias voltage and discharge time keep the bitline above 600 mV so the biased transistor remains in saturation.Source: C. Current-Steering DAC · p. 7
D. Digital Processing Periphery
- The digital adder tree offers four programmable output levels to support six weight bitwidths: 1, 2, 3, 4, 5, and 8 bits.Supporting up to 8-bit input activations takes two global cycles; inputs below 4 bits take one clock cycle.Source: D. Digital Processing Periphery · p. 7
IV. MEASUREMENT RESULTS
- The macro occupies 0.179 mm2, of which the 8KB SRAM array accounts for 62.6%, ciSAR ADCs 8.2%, and current DACs 3.6%.Digital processing and readout registers occupy 13.7% of the total macro area.Source: §IV · p. 7
A. Linearity
The paper evaluates linearity across the analog computing pipeline and proposes post-silicon digital calibration. Measured charge-domain MAC computing is nearly linear, while ADC slices exhibit offset and gain variations linked to component mismatch.
1) Linearity of Charge-Domain MAC Computing:
- Measured charge-domain MAC computing achieves an R2 value of 0.9999 across input patterns, with DAC inputs fixed at “0000” or “1111” to isolate DAC non-idealities.The ideal output code is the exact MAC result quantized by an ideal 6-bit ADC.Source: 1) Linearity of Charge-Domain MAC… · p. 7
2) Linearity of ADCs:
- ADC linearity is estimated indirectly by controlling MAC inputs; the 32 slices show offset and gain variations when operating in 7-bit differential mode.Only half of the ADC dynamic range is used because the selected weight pattern's computing results span half the full range.Source: 2) Linearity of ADCs · p. 7
- The paper attributes slice gain variations to mismatch in sampled charge, charge-sharing ratio, and ADC conversion step, and focuses on digital calibration.Sampled-charge variation arises from CMOM mismatch; charge-sharing ratio variation from CMOM-to-CP mismatch; and conversion-step variation from CI-cell and input-capacitor mismatch.Source: 2) Linearity of ADCs · p. 7
3) Calibration:
A two-step digital calibration mitigates slice-to-slice variations, while measurements show the complete analog computing chain remains within two LSBs of linearity. Computing errors are also assessed by comparing measured MAC outputs with expected outputs across randomized inputs.
3) Calibration:
- The two-step calibration first removes ADC offset and gain errors by linear fitting, then applies a master curve to alleviate systematic nonlinearity.The calibrated transfer curves are shown in Fig. 16(b).Source: 3) Calibration · pp. 7–8, Figure 16 · p. 8
4) Linearity of the Analog Computing Slices:
- The complete analog computing chain’s largest linearity error is expected to be less than two LSBs across 64 slices in two prototype chips.The measured nonlinearity includes contributions from the DAC, analog computation, and ADC; DAC nonideality does not significantly degrade system linearity.Source: 4) Linearity of the Analog Computing… · p. 8
B. Computing Accuracy
- Computing errors are assessed by feeding random input sets to the system and comparing measured MAC outputs with expected outputs.Fig. 17 presents error distributions after linear fitting and after linear fitting plus master-curve calibration with noise filtered out; Fig. 18 shows distributions across eight prototype chips.Source: B. Computing Accuracy · p. 8, Figure 17 · p. 8, Figure 18 · p. 8
1) Error Distribution:
The section evaluates error distributions and noise, then tests mapped CNNs on MNIST and CIFAR-10. CAP-RAM matches the software baseline on MNIST and achieves 89.0% accuracy on CIFAR-10, 1.6% below its baseline.
1) Error Distribution:
- Across 524,288 samples, the error spread is further reduced by averaging repeated inputs and applying master-curve calibration after linear fitting.The input sets use different distributions to reduce bias in the measured error distribution.Source: 1) Error Distribution · p. 8
2) Random Errors:
- The ADC’s average rms error is 0.35 LSB, with spikes near its transition threshold.The noise level is described as acceptable for low-power, compact ADCs, though it could be improved with relaxed area and power requirements.Source: 2) Random Errors · pp. 8–9, Figure 19 · p. 9
- The noise level could be further improved with relaxed area and power requirements.This is the stated boundary for improvement of the measured ADC thermal-noise level.Source: 2) Random Errors · p. 9
3) Inference on MNIST:
- 98.8% inference accuracy on MNIST matches the software baseline using compressed LeNet-5, with 500 images tested.The model is pruned by 95.6% and its 19,149 parameters fit in one CAP-RAM macro.Source: 3) Inference on MNIST · p. 9
4) Inference on CIFAR-10:
- 89.0% inference accuracy on CIFAR-10 is 1.6% below the software baseline with 500 images tested.The quantized ResNet-20 uses quantization-aware training to tolerate quantization errors; layers 15–19 occupy two macros.Source: 4) Inference on CIFAR-10 · p. 9
C. Energy and Throughput
CAP-RAM combines configurable CNN inference accuracy with 573.4-GOPS throughput and reported energy efficiency of 49.3 TOPS/W. Its throughput and compute-density comparisons depend on operation-width normalization and reflect a tradeoff between storage density and energy efficiency.
C. Energy and Throughput
- 27.2 TOPS/mm2 is CAP-RAM’s bitwise compute density, the best among the compared designs under the stated one-bit-operation definition.The prototype also reaches 573.4 GOPS peak throughput and 3.4 TOPS/mm2 for 4-bit inputs with binary or ternary weights.Source: C. Energy and Throughput · p. 9
- The authors frame CAP-RAM’s goal as compact, accurate, and programmable operation with competitive—not necessarily highest—energy efficiency, given a storage-density tradeoff.The system consumes 11.62 mW with random 4-bit inputs and 1-bit weights.Source: C. Energy and Throughput · pp. 9–10
V. CONCLUSION
- A 65-nm prototype maps pruned and quantized LeNet-5 and ResNet-20 to CAP-RAM macros, achieving 98.8% MNIST and 89.0% CIFAR-10 inference accuracy, 49.3 TOPS/W energy efficiency, and 573.4-GOPS throughput.The charge-sharing mechanism ensures good accuracy, while the semi-parallel architecture provides best-in-class weight storage density.Source: §V · p. 10