Source-linked AI summary
Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks
Sanghyun Hong, Pietro Frigo, Yiğitcan Kaya, Cristiano Giuffrida, Tudor Dumitraş
TL;DR
This paper investigates whether adversarial single-bit corruptions undermine the assumed graceful degradation of DNNs. Across 19 models, it analyzes vulnerability patterns and hardware-fault feasibility, finding widespread severe accuracy losses while evaluating mitigation directions.
Problem
The limits of DNN resilience to attacker-induced bitwise parameter corruptions remain insufficiently understood, despite evidence of robustness to accidental errors.
Method
The study evaluates single-bit flips across 19 DNN models, characterizes factors affecting vulnerability, uses heuristics to identify vulnerable parameters, and tests Rowhammer feasibility.
Results
Approximately 40-50% of parameters are vulnerable to single-bit flips causing indiscriminate damage, and all 19 models contain parameters producing accuracy drops above 90%.
Takeaways & Limitations
DNN parameter representations can permit graceless accuracy degradation under adversarial bitwise errors, exposing limits of their resilience to hardware fault attacks.
Takeaways & Limitations
The proposed mitigation mechanisms are promising but cannot deter practical hardware fault attacks in the general case and often require training the victim model.
Abstract
from arXiv · showhide
Deep neural networks (DNNs) have been shown to tolerate "brain damage": cumulative changes to the network's parameters (e.g., pruning, numerical perturbations) typically result in a graceful degradation of classification accuracy. However, the limits of this natural resilience are not well understood in the presence of small adversarial changes to the DNN parameters' underlying memory representation, such as bit-flips that may be induced by hardware fault attacks. We study the effects of bitwise corruptions on 19 DNN models---six architectures on three image classification tasks---and we show that most models have at least one parameter that, after a specific bit-flip in their bitwise representation, causes an accuracy loss of over 90%. We employ simple heuristics to efficiently identify the parameters likely to be vulnerable. We estimate that 40-50% of the parameters in a model might lead to an accuracy drop greater than 10% when individually subjected to such single-bit perturbations. To demonstrate how an adversary could take advantage of this vulnerability, we study the impact of an exemplary hardware fault attack, Rowhammer, on DNNs. Specifically, we show that a Rowhammer enabled attacker co-located in the same physical machine can inflict significant accuracy drops (up to 99%) even with single bit-flip corruptions and no knowledge of the model. Our results expose the limits of DNNs' resilience against parameter perturbations induced by real-world fault attacks. We conclude by discussing possible mitigations and future research directions towards fault attack-resilient DNNs.
1 Introduction
The paper challenges the assumed graceful degradation of DNNs by studying adversarial single-bit parameter corruptions and their practical exploitation through Rowhammer. Across models and tasks, these corruptions can cause severe accuracy loss, motivating targeted defenses.
- Motivation: Prior resilience studies mainly examined cumulative or accidental parameter changes, leaving attacker-induced worst-case bit corruptions insufficiently understood.This gap matters because DNNs are widely used in mission-critical systems.
- Method: 19 DNN models across six architectures and three image-classification tasks were tested by flipping each parameter bit and measuring validation-set misclassification.The study covers MNIST, CIFAR10, and ImageNet.
- Findings: ∼50% of model parameters were vulnerable to single-bit corruptions causing relative accuracy drops above 10%, and every model contained a parameter causing over 90% accuracy loss.These results indicate graceless rather than graceful degradation under adversarial bitwise errors.
- Findings: The vulnerability is associated with drastic parameter-value spikes and varies with bit position, direction, sign, width, activation, normalization, and architecture.Dropout and batch normalization did not prevent the massive spikes caused by bit-flips; heuristics were proposed to accelerate vulnerability analysis.
- Mitigations: Restricting activations or using low-precision parameters reduced vulnerable ratios in evaluated models, but these defenses cannot generally deter practical hardware attacks and often require retraining from scratch.For AlexNet, ReLU6 reduced the vulnerable ratio from 47% to 3%; quantization and binarization reduced the MNIST ratio from 50% to 1–2%.
- Practical attacks: Rowhammer attacks can inflict accuracy drops of up to 99% on victim models, including constrained corruptions without model knowledge and attacks against shared transfer-learning layers.The evaluated MLaaS setting places attacker and victim VMs on the same physical host.
2 Preliminaries
The preliminaries define feed-forward DNNs, their parameterized layer operations, floating-point representation, and Rowhammer’s single-bit memory-corruption primitive.
- Deep neural networks: A feed-forward DNN applies parameterized linear transformations, optional biases and structures, and nonlinear activations to produce predictions.Training updates weights iteratively through backpropagation.
- Floating-point representation: DNN parameters are commonly stored as IEEE754 32-bit floating-point numbers with 23 mantissa bits, 8 exponent bits, and 1 sign bit.Different bit positions have substantially different effects on represented values.
- Floating-point representation: Flipping a mantissa bit can create a negligible change, whereas flipping the highest exponent bit can increase a value dramatically.The paper uses this contrast to motivate adversarial analysis of individual bit positions.
- Rowhammer attacks: Rowhammer provides a software-induced single-bit corruption primitive at the DRAM level through specific memory-access patterns.Its direct, repeatable memory corruption makes it suitable for studying single-bit parameter perturbations.
3 Threat Model
The threat model considers hardware fault attacks against DNN parameters stored in shared cloud memory, focusing on minimal single-bit changes and attackers with different capabilities and knowledge.
- Adversarial setting: Earlier work established resilience to random or deliberate cumulative parameter changes, but provides limited insight into worst-case damage from minimal adversarial modifications.The threat model therefore targets the atomic single-bit corruption.
- Deployment setting: The victim DNN runs in a cloud VM or container, with its trained parameters loaded into shared memory during test-time inference.The parameters remain constant during normal operation.
- Attacker capabilities: The attacker is co-located on the same physical host and uses Rowhammer to corrupt the victim model stored in DRAM.Two attack capabilities are considered: surgical flips at intended locations and blind flips without location control.
- Attacker knowledge: Black-box attackers lack knowledge of the victim model and can only hope their bit-flips reduce accuracy, whereas white-box attackers know model details and memory placement.The knowledge distinction governs whether attack effects can be anticipated.
4 Single-Bit Corruptions on DNNs
Across 19 models and three image-classification datasets, single-bit corruption exposes substantial vulnerability despite DNNs’ expected resilience. Vulnerability is linked to bit-induced parameter spikes and varies with representation, activation, width, normalization, architecture, and attack knowledge.
- Experimental setup: 19 DNN models across MNIST, CIFAR10, and ImageNet were evaluated using a framework that flips parameter bits and measures validation-set misclassification.The study includes six architectures and model variants.
- Quantifying vulnerability: 40% to 99% of parameters were vulnerable to single-bit indiscriminate damage, defined as RAD > 0.1, depending on the model.The authors interpret consistency between exhaustive MNIST experiments and heuristic evaluations elsewhere as approximately half of parameters being vulnerable.
- Bitwise representation: Exponent-bit flips, especially the 31st bit, cause the strongest damage because they can drastically change parameter values, unlike mantissa flips.The heuristic evaluation therefore focuses on exponent bits, with ImageNet analysis restricted further to the most significant exponent bit as a conservative estimate.
- Bitwise representation: 0→1 exponent flips are damaging whereas 1→0 flips are not, because increasing a parameter can create extreme activations that override other activations during the forward pass.The reported explanation relies on typical parameter values being concentrated near zero.
- Activation function: PReLU increases vulnerability from 50.2% with ReLU to 99.2% because negative outputs remain active and negative parameters become vulnerable.Positive parameters are generally more vulnerable under ReLU, which suppresses negative activations.
- Normalization and architecture: Dropout and batch normalization reduce parameter magnitudes but leave vulnerability mostly persistent, with at most a 6.3% reduction in vulnerable-parameter ratio.Vulnerability ratios are also broadly consistent across architectures; InceptionV3 is lower at 40.8% versus 42.1%–48.9% for other ImageNet models.
- Attack scenarios: A black-box surgical attacker can target the 31st bit without model knowledge, achieving a 42.1% success rate on ImageNet VGG16; transfer-learning students also share vulnerable parameters with teachers.The transfer-learning setup freezes teacher layers in the student, leaving shared parameters exposed to surgical attacks.
5 Exploiting Using Rowhammer
Rowhammer is presented as a practical software-induced attack that can corrupt DNN parameters through single-bit memory faults. The paper evaluates surgical and blind attack scenarios, showing that vulnerable parameter locations and memory-layout techniques can produce severe accuracy damage.
- Rowhammer mechanism: Rowhammer provides a single-bit write primitive to specific physical memory locations through repeated DRAM access patterns.The attack induces persistent and repeatable bit corruptions from software.
- Rowhammer mechanism: Double-sided Rowhammer repeatedly accesses two aggressor rows surrounding a victim row, inducing spurious bit-flips in the victim’s DRAM cells.The victim’s data is stored in the enclosed row.
- Attack scenarios: The study considers surgical attackers that target specific locations and blind attackers that lack this capability.Surgical attacks are analyzed under both white-box and black-box knowledge assumptions.
- Experimental setup: The evaluation simulates co-located attacker and victim VMs sharing physical memory, focusing on models with a memory footprint realistically targetable by Rowhammer.The setup uses an ImageNet model in a PyTorch application and a database covering 12 susceptible DRAM chips.
- Surgical attack: A surgical attacker needs only one vulnerable template, with 1,024 possible offsets for a 4-byte parameter within a 4 KB page.Memory templating identifies exploitable offsets, while page-aligned allocations help locate target parameters.
- Experimental results: Across representative DRAM setups, successful indiscriminate attacks ranged from 1 to 24 of 25 experiments, averaging 15.6 effective attacks.In the least vulnerable setup, one single-bit flip produced RAD values of 0.9992 for Top-1 and 0.9959 for Top-5.
- Synopsis: The attacks demonstrate a large practical attack surface: Rowhammer can inflict terminal brain damage even when the ImageNet-VGG16 model is hidden from the attacker.The authors attribute this result to vulnerable parameters, memory-massaging primitives, and resilience to spurious flips in idle code regions.
6 Discussion
The discussion evaluates restricting activation magnitudes and using low-precision parameters as defenses against single-bit attacks, finding substantial vulnerability reductions but practical trade-offs.
- Mitigation directions: Restricting activation magnitudes and using low-precision numbers are proposed as two directions for making DNNs resilient to bit-flips.The discussion contrasts these model-level mechanisms with system-level Rowhammer defenses requiring infrastructure-wide changes.
- Restricting activation magnitudes: In MNIST models, Tanh and ReLU6 reduced vulnerable-parameter ratios from 50% to 1.4–2.4% without significant performance loss.ReLU6 substitution achieved a similar effect without retraining, although the supplied passage is truncated before its full qualification.
- Using low-precision numbers: Low-precision parameters reduced vulnerable-parameter ratios from 49% to 0–2%, with quantization producing 0% vulnerability in the reported cases.In binarized models, most vulnerable parameters were concentrated in the first convolutional and final classification layers.
- Trade-offs: 8-bit quantization mitigates vulnerability, but retraining large models can take a week on a supercomputing cluster, limiting practicality.Activation restriction offers a tunable trade-off and can provide ad-hoc protection, though residual vulnerable parameters may still permit Rowhammer damage at reduced success rates.
7 Related Work
Related work covers DNN resilience and attacks that corrupt models through data poisoning or hardware faults, positioning bit-flip damage as a distinct attack medium.
- DNN resilience: DNN parameter perturbations have supported quantization, efficiency improvements, and security techniques because accuracy often degrades gracefully.These prior uses establish the resilience property that hardware-fault attacks challenge.
- Indiscriminate poisoning attacks on DNNs: Indiscriminate poisoning attacks damage models by injecting malicious training data, but prior studies suggest they may require many poisonous instances.The paper frames parameter corruption as an alternative medium for indiscriminate damage.
- Hardware fault injection attacks: Hardware fault injection attacks use hardware glitches to corrupt victim data, increasingly including software-based attacks that provide single-bit writes to physical memory.Examples include CLKSCREW and other documented fault-injection attacks.
8 Conclusions
The paper concludes that single-bit parameter corruptions expose limits in DNN resilience, while activation restriction and low-precision representations provide two mitigation directions.
- Conclusions: Across 19 DNN models, six architectures, and three image-classification tasks, the authors estimate that 40–50% of parameters are vulnerable.A single-bit corruption of a vulnerable parameter can cause indiscriminate damage under the paper’s RAD > 0.1 criterion.
- Conclusions: The study characterizes vulnerability by bit position, flip direction, parameter sign, layer width, activation function, training techniques, and architecture.This analysis supports the paper’s broader characterization of how model properties shape bit-flip vulnerability.
- Conclusions: Rowhammer experiments found that attackers without knowledge of the victim’s deep-learning system could inflict indiscriminate damage without system crashes.The conclusion presents this as evidence of practical feasibility for software-induced fault injection.
- Conclusions: The paper identifies restricting activation magnitudes and using low-precision numbers as potential mitigation directions for this emerging threat.It frames the work as a step toward understanding and mitigating attacks that can compromise critical deep-learning systems.
A Network Architectures
The appendix describes the 19-model experimental set, including MNIST architectures and variants, a CIFAR10 base architecture with variations, and Internet-sourced ImageNet models.
- Model inventory: The experiments use 19 DNN models comprising six architectures and their variants.The appendix specifically details two MNIST architectures and six MNIST variations.
- ImageNet and auxiliary models: For ImageNet, the study uses DNN architectures available from the Internet.The appendix also notes community MNIST-L5 networks for the quantized and binarized experiments, with adjustments.
B The Vulnerability Using Different Criteria
Vulnerability varies with the severity of the targeted relative accuracy drop (RAD). In MNIST, CIFAR10, and two ImageNet models, vulnerable-parameter ratios generally decrease as the attacker targets more severe damage.
- 40% of MNIST-L5 parameters cause RAD > 0.5, estimating the upper bound for a blind attacker.The passage presents this as an example of the vulnerable-parameter ratio under a specific RAD criterion.
- Vulnerability decreases as the attacker aims to inflict more severe damage in MNIST, CIFAR10, and two ImageNet models.
C Hyper-parameters for Training
The experiments specify dataset- and architecture-dependent training hyper-parameters, including optimizer settings, learning-rate schedules, batch sizes, and transfer-learning choices. MNIST models use SGD for 40 epochs, while CIFAR10 and fine-tuned ImageNet models use distinct schedules and frozen layers where indicated.
- MNIST: MNIST models use SGD for 40 epochs with 0.01 learning rate, batch size 64, momentum 0.1, and ten-epoch learning-rate adjustments.
- CIFAR10: CIFAR10 Base models use 50 epochs, while AlexNet and VGG16 use 300 epochs with architecture-specific learning rates, batch sizes, momentum, and schedules.
- GTSRB: GTSRB fine-tunes ImageNet-pretrained VGG16 for 40 epochs and freezes the first 10 layers.
- Flower102: Flower102 fine-tunes ImageNet-pretrained ResNet50 for 40 epochs and freezes the first 10 layers.
- MNIST architectures: The MNIST architecture set contains Base and LeNet5 baselines with four and two variants, respectively.