Source-linked AI summary
Faults That Fortify: CNN Adversarial Robustness via GPU Undervolting
Behnam Omidi, Ahmad Tahmasivand, Husam Alsyouri, Saba Al-Sayouri, Chongzhou Fang, Ihsen Alouani, Khaled N. Khasawneh
TL;DR
CNNs combine adversarial vulnerability with costly training, motivating methods that improve robustness without increasing energy demands. This paper characterizes real-GPU undervolting faults and evaluates undervolted standard and adversarial training across CNNs, datasets, and PGD attacks. Undervolting consistently improves adversarial robustness while reducing energy use, including when applied alongside adversarial training.
Problem
CNNs are vulnerable to adversarial attacks while their training requires substantial energy, creating a challenge for learning on resource-constrained edge platforms.
Method
The paper characterizes stochastic undervolting faults on a real GPU and compares nominal- and undervolted standard and adversarial training across three CNNs, two datasets, and PGD attacks.
Results
Undervolted models are consistently more robust than nominal-voltage counterparts in both training regimes while consuming less energy.
Takeaways & Limitations
GPU undervolting is presented as a practical, energy-efficient hardware-level robustness mechanism that requires no algorithmic change.
Abstract
from arXiv · showhide
Convolutional Neural Networks (CNNs) face a dual challenge: vulnerability to adversarial attacks and prohibitive training cost. Adversarial training is effective but expensive, a burden that grows as learning shifts to the energy-constrained edge. This paper addresses both through GPU undervolting during training. Reducing supply voltage introduces stochastic perturbations that act as implicit regularization, improving robustness while lowering power. We characterize undervolting-induced faults at the bit level, then train LeNet, VGG-6, and MobileNetV3 on MNIST and CIFAR-10 under two training regimes, standard and adversarial, each at nominal and undervolted voltage, and evaluate all models against adversarial attacks. In both regimes, the undervolted model consistently achieves higher adversarial accuracy than its nominal-voltage counterpart, showing that hardware-induced faults strengthen even adversarial training. Because dynamic power scales quadratically with supply voltage, these robustness gains arrive with substantial energy savings. GPU undervolting is therefore a readily deployable hardware-level defense requiring no algorithmic change, and opens a promising direction in which robustness and energy efficiency move together.
I. INTRODUCTION
CNN training is both computationally expensive and vulnerable to adversarial attacks, especially under edge-device power constraints. This work investigates physical-GPU undervolting as a hardware-level training intervention that injects stochastic faults, improves robustness, and reduces energy use.
- Training CNNs requires substantial computational resources and energy, creating particular challenges for fine-tuning on power-constrained edge platforms.
- Prior undervolting studies largely examined inference-time, software-emulated faults, leaving transient faults during physical-GPU training relatively unexplored.
- Undervolting-induced stochastic perturbations may regularize learning by steering networks toward flatter representations, while dynamic power falls approximately quadratically with supply voltage.
- The evaluation covers standard and adversarial training for LeNet, VGG-6, and MobileNetV3 on MNIST and CIFAR-10 under nominal and undervolted voltage, using PGD attacks.
- Across both training regimes, undervolted models are consistently more robust than nominal-voltage counterparts while consuming less energy.
- The paper characterizes undervolting faults on a real GPU and presents training-time undervolting as a practical hardware-generated-noise mechanism for CNN robustness.
II. BACKGROUND AND RELATED WORK
Undervolting improves energy efficiency by reducing voltage, but the resulting timing degradation can introduce transient computational faults. At mild levels these faults appear as stochastic arithmetic noise that may regularize learning, whereas excessive undervolting harms optimization.
- Undervolting reduces dynamic power approximately with the square of supply voltage but slows transistor switching and can produce timing faults.Lower voltage also reduces static power, at the cost of timing reliability.
- When propagation delay exceeds the clock period, timing violations produce incorrect outputs or transient faults, with likelihood increasing sharply as voltage margin shrinks.As Vdd approaches the minimum stable voltage, computation transitions from reliable operation to frequent soft errors.
- GPU undervolting faults mainly appear as small perturbations in floating-point arithmetic, represented as voltage-dependent noise whose variance grows as voltage decreases.Mild undervolting introduces stochastic noise, while aggressive undervolting causes substantial numerical instability.
- Low-level undervolting noise may act as regularization comparable to dropout or weight noise, potentially improving generalization and robustness.Excessive undervolting instead disrupts gradient-based optimization and degrades convergence and accuracy.
B. Randomization-based Defenses
Randomization-based defenses inject stochasticity into inputs, features, or parameters to make adversarial gradients less reliable. Randomized smoothing extends this idea by averaging noisy predictions to provide probabilistic norm-ball robustness guarantees.
- Randomization-based defenses counter adversarial perturbations by injecting stochasticity into inference or training, disrupting structured attack gradients.The paper distinguishes input, feature, and parameter randomization.
- Randomization levels: Input-level defenses use stochastic resizing, padding, and augmentation to randomize the model’s inputs.
- Randomization levels: Feature-level noise injection turns a single network into an implicit ensemble of stochastic models.
- Randomization levels: Parameter-level methods include Monte Carlo dropout, Bayesian uncertainty modeling, and stochastic quantization or weight noise.
- Randomized smoothing averages predictions under Gaussian noise to obtain probabilistic robustness guarantees within a norm-ball.Later variants improve certified robustness at comparable accuracy.
III. PROPOSED METHODOLOGY
The proposed methodology uses hardware-level GPU undervolting to inject stochastic noise during CNN training. It defines the threat model, models how undervolting faults enter learning, and specifies training and operating-point selection.
- The methodology exploits hardware-level undervolting to inject beneficial stochastic noise into CNN training.The section formalizes fault injection into learning and describes how training operating points are selected.
- The methodology covers the threat model, the formal entry of undervolting-induced faults into training, and the training procedure.
- Operating-point selection is included as part of the proposed training methodology.
A. Threat Model
The threat model evaluates white-box PGD attacks against nominal-voltage deployed models while controlling only training hardware. Undervolting perturbs every training operation, creating intrinsic noise that can regularize learning without algorithmic or computational modification.
- A. Threat Model: The adversary has full knowledge of model architecture and parameters and crafts Lp-bounded white-box perturbations using PGD across budgets ϵ.The attack targets the deployed model, which executes at nominal voltage.
- A. Threat Model: The defender controls only the GPU voltage configuration during training, assuming driver-supported voltage adjustment.
- B. Undervolting as a Training-Time Noise Source: Undervolting perturbs arithmetic operations throughout the forward pass, backward pass, and weight update, with perturbation variance increasing as voltage decreases.
- B. Undervolting as a Training-Time Noise Source: The effective update adds aggregated voltage-induced perturbations to the gradient, where δt captures noise accumulated during gradient computation.This noise arises across operations rather than at a selected software injection site.
- B. Undervolting as a Training-Time Noise Source: Treating undervolting perturbations as intrinsic noise regularization can preserve clean-accuracy convergence while shaping the learned solution.The perturbations concentrate in low-significance mantissa bits and are small enough for optimization to converge to full clean accuracy.
C. Training Procedure and Operating-Point Selection
The study selects the lowest stable undervoltage for full-run training and compares standard and adversarial regimes under matched conditions across architectures and datasets.
- Operating-point selection: The GPU voltage is swept downward from nominal, selecting the lowest setting at which training reliably completes.The GPU remains at this voltage from initialization through convergence, maximizing stable fault exposure.
- Training design: A 2×2 design compares standard and adversarial training at nominal and undervolted voltage with matched seeds, initializations, and hyperparameters.Within each training regime, voltage is the only systematic difference between runs.
- Evaluation models and data: Experiments use LeNet, VGG-6, and MobileNetV3 on MNIST and CIFAR-10, evaluating robustness with untargeted PGD attacks.The setup spans different model depths and computational demands.
V. EVALUATION
The evaluation characterizes GPU undervolting faults at the bit level and explains why their small, stochastic perturbations remain compatible with training while influencing robustness.
- Fault characterization: The fault experiment repeatedly multiplies tensors one billion times and compares nominal and lowest-stable-voltage outputs bit by bit.It tracks flip counts and transition directions across the 32-bit floating-point representation.
- Fault characterization: Most observed bit flips occur in mantissa bits, with far fewer in the exponent field.Mantissa corruption changes fine-grained precision rather than causing large numeric shifts.
- Fault impact: Inference accuracy remains nearly identical at nominal and undervolted voltage on the tested GPU.This supports the viability of undervolted training without materially disrupting clean model behavior.
- Training implications: Small, stochastic, unbiased perturbations are tolerated by gradient descent, yet their repetition across training iterations shapes optimization as implicit regularization.The cumulative effect is evaluated through adversarial robustness experiments.
B. Undervolted Training
Undervolted standard training is evaluated against PGD attacks and shows higher robustness than nominal-voltage training across architectures and datasets, with smaller gains in deeper models.
- Energy results: The power table reports nominal and undervolted voltage and frequency alongside power improvement across models.Frequency is unchanged, so the reported reduction is attributed to voltage alone.
- Robustness results: Undervolted models retain higher accuracy across all perturbation budgets at which either standard-trained model retains nonzero accuracy.The nominal baseline degrades rapidly as adversarial perturbation strength increases.
- Robustness results: The robustness trend holds across every architecture and dataset evaluated, consistent with hardware noise acting as an implicit regularizer.The proposed explanation includes flatter minima or exposure to fluctuation-like distortions.
- Limitations: Robustness gains are noticeably smaller for larger, deeper models such as MobileNetV3.The paper attributes this to fault masking through nonlinear and pooling stages, especially for low-significance-bit faults.
C. Undervolted Adversarial Training
Adversarial training is repeated under nominal and undervolted conditions using on-the-fly PGD examples, and undervolting further improves robustness across the tested settings.
- Adversarial-training procedure: Adversarial examples are generated from the model’s current parameters at each iteration and used to update the weights.Both nominal and undervolted runs keep the model exposed to perturbations throughout training.
- Robustness results: The undervolted adversarial-trained model is consistently more robust across every architecture and dataset.The margin widens at medium to high perturbation budgets, where the nominal baseline begins to deteriorate.
VI. DISCUSSION AND FUTURE WORK
The study presents GPU undervolting as a promising way to improve CNN robustness while reducing energy consumption, but emphasizes that the results are preliminary and require broader validation. Future work targets larger models, optimized models, voltage strategies, and bit-level fault characterization.
- Discussion: The current results are preliminary and motivate further research at the intersection of energy-efficient computing and AI robustness.
- Future Work: Future evaluations should test larger foundation models, quantized models, and other optimized AI models to assess generality and scalability.The authors specifically identify architectures with more linear layers as offering fewer masking opportunities.
- Future Work: Future work should explore voltage-frequency scaling to identify operating points that maximize robustness while preserving computational efficiency.
- Future Work: Characterizing undervolting-induced bit-flips could clarify fault propagation and support more targeted undervolting strategies.The proposed analysis includes bit-flip locations, frequencies, and effects on model behavior.
- Discussion: GPU undervolting enhances CNN robustness while simultaneously reducing GPU energy consumption.The approach uses hardware-induced perturbations as an implicit regularizer without significant degradation in model accuracy.