Source-linked AI summary
ThUnderVolt: Enabling Aggressive Voltage Underscaling and Timing Error Resilience for Energy Efficient Deep Neural Network Accelerators
Jeff Zhang, Kartheek Rangineni, Zahra Ghodsi, Siddharth Garg
TL;DR
Existing timing-error techniques offer limited voltage underscaling for high-performance DNN accelerators: TED can incur overhead, while TEP can harm accuracy. Thundervolt combines TE-Drop with dynamic per-layer underscaling, and reports energy savings with no performance overhead and at most 1% accuracy reduction.
Problem
TED recovery overhead and TEP’s accuracy drops limit voltage underscaling opportunities for high-performance DNN accelerators.
Source: §1 · p. 1Method
Thundervolt uses TE-Drop to recover timing errors without re-execution and dynamically sets voltage underscaling by layer to balance error rates.
Source: §1 · pp. 1–2Results
34.36%-57.75% energy savings over a TPU-modeled baseline enhanced with Zero-Skip, with no performance overhead and at most 1% classification-accuracy reduction.
Source: §1 · p. 2Takeaways & Limitations
Thundervolt can be combined synergistically with Zero-Skip, whose zero-input MAC energy savings are complemented by underscaling the remaining non-zero MACs.
Source: §1 · p. 2Global TED is impractical even for moderately sized designs, while distributed TED schemes incur higher performance overhead and perform no better than global TED.
Source: §2.2 · pp. 2–3
Abstract
from arXiv · showhide
Hardware accelerators are being increasingly deployed to boost the performance and energy efficiency of deep neural network (DNN) inference. In this paper we propose Thundervolt, a new framework that enables aggressive voltage underscaling of high-performance DNN accelerators without compromising classification accuracy even in the presence of high timing error rates. Using post-synthesis timing simulations of a DNN accelerator modeled on the Google TPU, we show that Thundervolt enables between 34%-57% energy savings on state-of-the-art speech and image recognition benchmarks with less than 1% loss in classification accuracy and no performance loss. Further, we show that Thundervolt is synergistic with and can further increase the energy efficiency of commonly used run-time DNN pruning techniques like Zero-Skip.
1 INTRODUCTION
Thundervolt addresses the limited voltage-underscaling opportunities of existing timing-error techniques for highly parallel DNN accelerators. It combines TE-Drop with dynamic per-layer underscaling and reports energy savings with negligible accuracy impact and no performance overhead.
1 INTRODUCTION
- Existing TED can impose significant overhead as global timing-error rates grow with MAC count, while TEP can sharply reduce accuracy at error rates as low as 0.1%.A TPU-like array with roughly 65K MACs has a 50% global timing-error rate when each MAC has 10^-5 timing errors per clock cycle.Source: §1 · p. 1
- TE-Drop permits execution at timing error rates greater than 10% without compromising accuracy, while avoiding re-execution’s performance impact.It drops the successor MAC operation and uses the extra cycle to correctly compute the erroneous MAC’s result.Source: §1 · p. 1
- 34.36%-57.75% energy savings over a TPU-modeled baseline enhanced with Zero-Skip, with no performance overhead and at most 1% classification-accuracy reduction.The reported savings are achieved by Thundervolt with Zero-Skip optimization.Source: §1 · p. 2
- Timing-error rates vary across DNN layers, motivating dynamic per-layer voltage underscaling that balances error rates and outperforms uniform underscaling.The scheme determines each layer’s underscaling ratio rather than applying the same amount everywhere.Source: §1 · pp. 1–2
- The paper investigates voltage-underscaling timing speculation for high-performance, highly parallel DNN accelerators such as the Google TPU.The authors describe this as the first such investigation to their knowledge.Source: §1 · p. 2
2 BACKGROUND AND MOTIVATION
This section motivates Thundervolt by explaining why conventional timing-speculation schemes limit energy savings in high-performance DNN accelerators.
2.1 Hardware Accelerators DNNs
- DNN layers repeatedly multiply weight and activation matrices, while a TPU-like systolic array reuses stationary weights and streams activations through MACs, with nonlinear activation handled separately.Source: §2.1 · p. 2
2.2 Timing Speculation for DNN Accelerators
- Global TED is impractical at moderate design sizes, while distributed recovery propagates stalls and performs no better than global TED.Source: §2.2 · pp. 2–3
- On a 256 × 256 MAC array, global timing errors surge below r=0.9, making deeper underscaling increase energy even with optimistic single-cycle recovery.Source: §2.2 · pp. 2–3, Figure 2 · p. 3
- At r=0.85, a 0.01% per-MAC timing-error rate yields activation RMSE 33.9x the golden-activation average, as errors can flip high-order partial-sum bits.Source: §2.2 · p. 3
3 THUNDERVOLT DESIGN
Thundervolt’s design combines TE-Drop, which resolves MAC timing errors without re-execution, with per-layer voltage underscaling that allocates a total timing-error budget across layers.
3.1 TE-Drop
- TE-Drop uses Razor flip-flops to detect errors, then borrows the successor MAC’s cycle to finish the erroneous update while dropping the successor’s update.The design relies on DNN weights being biased toward small values, and its error-dependent dropped connections resemble dropout and dropconnect.Source: §3.1 · p. 3
3.2 Per-layer Voltage Underscaling
- At r = 0.65, layer timing-error rates differ by 2× to 20× between highest- and lowest-error layers, with depth trends differing between smaller and larger networks.The reported depth trend increases in MNIST and Reuters, but decreases in TIMIT and AlexNet.Source: §3.2 · p. 4, Figure 4 · p. 4
- Per-layer underscaling distributes a total timing-error budget by sorting layers by error rate and assigning each the largest ratio within its current target.The chosen layer’s error probability is subtracted from the remaining budget before the algorithm proceeds to the next layer.Source: §3.2 · p. 4, Figure 5 · p. 4
- Online tracking sums MAC error signals by array column and then across the bottom row to measure batch error; each chip’s timing-error profile is characterized separately.The proposed tracking circuitry has less than 10% area overhead and no in-field power overhead on test data after ratios are determined.Source: §3.2 · p. 4
4 EMPIRICAL EVALUATION
The evaluation tests Thundervolt on four DNN benchmarks using a TPU-modeled accelerator and post-synthesis timing simulations. ThVolt-Dynamic provides substantial test-set energy savings, including when combined with Zero-Skip, while validation results favor per-layer dynamic underscaling over static underscaling.
4.1 Experimental Setup
- Evaluation covers MNIST, Reuters, TIMIT, and ImageNet, using MLPs for the first three benchmarks and a pretrained AlexNet CNN for ImageNet.The accelerator is modeled on the Google TPU with a 256 × 256 MAC array; timing errors are modeled using post-synthesis gate-level simulations, with ImageNet probabilities sampled from 32 columns.Source: §4.1 · pp. 4–5
4.2 Experimental Results
- ThVolt-Dynamic consistently outperforms ThVolt-Static on validation accuracy-energy trade-offs, while TEP accuracy falls sharply beyond 20% energy savings.The passage attributes the dynamic scheme’s advantage to balancing error probabilities across layers; TIMIT is described as more timing-error-resilient than ImageNet.Source: §4.2 · p. 5
- 73.27%-89.49% energy savings over the baseline are reported for Zero-Skip+ThVolt-Dynamic, the best-performing scheme.The experiments also report similar average energy savings across 50 chip samples with σ = 5% process-induced variations.Source: §4.2 · p. 5
- 36.3% on ImageNet, 50% on TIMIT, and 56% on MNIST and Reuters are the energy savings of ThVolt-Dynamic over the baseline.Applying ThVolt-Dynamic on top of Zero-Skip provides roughly the same energy savings over Zero-Skip alone.Source: §4.2 · p. 5
5 RELATED WORK
Thundervolt builds on timing-speculation approaches while addressing their limited energy-saving potential on highly parallel accelerators. The paper also relates it to approximate computation and DNN pruning, noting an unevaluated compressed-model setting.
5 RELATED WORK
- Baseline timing-error detection techniques offer limited energy savings on highly parallel, high-performance accelerators because global timing-error rates become high.Time borrowing and timing-error-aware synthesis can reduce baseline timing-error rates and can be leveraged by Thundervolt for more aggressive underscaling.Source: §5 · p. 5
- Unlike by-design approximate multipliers and reduced-precision arithmetic, Thundervolt approximates computation as a consequence of timing errors and could be applied on top of those techniques.The passage also notes that stochastic-logic DNN accelerator designs had not yet been adopted in commercial chips.Source: §5 · p. 5
- Thundervolt can also be applied to design-time-pruned DNNs, but quantitative evaluation on compressed models is identified as future work.The paper separately reports synergy with run-time pruning techniques.Source: §5 · p. 5
6 CONCLUSION AND FUTURE WORK
Thundervolt combines timing-error recovery with per-layer dynamic voltage underscaling and can be paired with dynamic DNN pruning. The authors report lower energy consumption without performance cost and with less than 1% accuracy loss, while identifying further combinations and training acceleration as future work.
6 CONCLUSION AND FUTURE WORK
- Thundervolt combines TE-Drop timing-error recovery and per-layer dynamic voltage underscaling, and can be synergistically applied with dynamic DNN pruning.The experiments report significant energy reductions with no performance cost and less than 1% accuracy loss.Source: §6 · p. 6