Source-linked AI summary
Exploring the Trade-Off Between Structured Pruning and Fault Tolerance in Deep Neural Networks for Space Applications
Toon Vinck, Naïn Jonckers, Jaro De Roose, Jeffrey Prinzie, Peter Karsmakers
TL;DR
DNN redundancy can limit the effects of bit faults, but its memory and energy costs motivate testing whether pruning compromises robustness. The paper iteratively prunes models and evaluates simulated faults; per-inference sensitivity rises, while reduced operation counts lower the modeled probability of encountering an SEU. The authors report that these effects largely counterbalance in their hypothetical FIT analysis, with practical efficiency savings in memory, energy, and latency.
Problem
The paper asks whether removing DNN redundancy through pruning compromises fault tolerance, a concern for deploying DNNs in radiation-prone space environments.
Source: §1 · p. 1Method
The authors investigate global structured channel pruning using fault-injection simulation in a simple DNN accelerator that includes a high-bit-width accumulator.
Source: §1 · p. 1Results
Takeaways & Limitations
The authors conclude that efficiency optimization does not compromise reliability in their analysis and suggest prioritizing energy- and time-efficient models for space-borne AI systems.
Source: §5 · p. 4The impact of the observed SDC events on safety-critical applications remains unverified; future work should assess mission-level impact and distinguish benign anomalies from critical failures.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Deep Neural Networks (DNNs) inherently exhibit a degree of robustness to bit-level faults due to their distributed representation of information. As a model increases in width, this information becomes more dispersed, theoretically reducing the impact of any single bit fault. In this paper, we empirically investigate the relationship between model width and robustness to Single Event Upsets (SEUs). We conduct a comprehensive experiment in which baseline models undergo iterative structured pruning to reduce their width while preserving task performance as much as possible. At each pruning stage, we run a targeted fault-injection campaign to evaluate the model's performance under simulated bit-flip scenarios. Our results show that, although structured pruning increases per-inference sensitivity to faults by reducing redundancy, this effect is effectively counterbalanced by shorter execution time, which lowers the probability of encountering an SEU. These findings suggest that structured pruning can yield significant energy and latency savings without compromising overall reliability, providing useful guidance for designing robust AI systems for space applications.
1 Introduction
DNN redundancy can mask isolated bit faults, but it also increases memory and energy use, motivating the question of whether pruning compromises robustness. This paper examines global structured pruning, which removes channels, using fault-injection simulation that includes a high-bit-width accumulator.
1 Introduction
- DNN redundancy can limit the effect of individual neuron errors, but larger-than-needed networks consume memory and energy.The paper asks whether removing that redundancy compromises robustness.Source: §1 · p. 1
- Prior work found greater per-inference sensitivity after unstructured pruning, while accounting for shorter execution time led to a conclusion of improved reliability.Those studies did not alter network width, and edge accelerators may not exploit unstructured sparsity for practical speedups or energy savings.Source: §1 · p. 1
- Structured pruning removes channels, directly testing how reducing network width affects fault tolerance in a simple accelerator model with a high-bit-width accumulator.Earlier studies examined unstructured pruning, which changes sparsity rather than width, and did not consider faults in high-bit-width accumulator registers.Source: §1 · p. 1
2 Experiment
The experiments simulate bit faults in a simple processing element and compare models of different widths. Baselines are iteratively pruned, retrained, quantized, and evaluated in fault-injection campaigns.
2.1 Fault Injection Framework
- The fault-injection framework models a single processing element and assumes main memories are protected by error-correction codes, focusing the analysis on processing-element vulnerability.The framework excludes main memory because faults there could accumulate, while protecting processing elements typically incurs substantial area and power overhead.Source: §2.1 · p. 2
- Each simulated inference receives one randomly selected bit flip; multiple independent upsets and multi-bit upsets are excluded.The authors justify the single-bit assumption by the low probability of multiple independent SEUs within a millisecond-scale inference window in typical space environments.Source: §2.1 · p. 2
2.2 Methodology
- Models are trained to convergence, then pruned in 10 iterations by 14.9% of remaining channels per step, retrained, and quantization-aware trained for 8-bit arithmetic.The final model retains 20% of the original channels; the final layer remains unpruned to preserve output dimensionality.Source: §2.2 · p. 2
- ResNet-18 and MobileNetV2 are evaluated on EuroSAT, with 102,600 injected faults per model at every pruning iteration.The dataset is split into 80% training, 10% validation, and 10% test sets.Source: §2.2 · p. 3
3 Results
Across both architectures, pruning increases per-inference fault sensitivity, while the hypothetical FIT analysis combines that effect with reduced SEU exposure from fewer operations. The reported system-level FIT is relatively stable for ResNet-18 and decreases slightly for MobileNetV2 as models shrink.
3 Results
- For both architectures, SDC-1 rises as models shrink, indicating that narrower networks have less inherent fault tolerance.The second-row SDC-1 metric counts cases where the top-1 prediction under fault injection differs from fault-free execution, including destructive and constructive faults.Source: Figure 3 · p. 3, §3 · pp. 3–4
- The hypothetical FIT rate stays relatively stable for ResNet-18 and decreases slightly for MobileNetV2 as models shrink.The analysis combines reduced redundancy with a lower probability of encountering an SEU as operation count and utilization decrease; the most aggressively pruned model uses less than 20% of the original MAC budget.Source: §3 · pp. 3–4
- Destructive faults remain below 0.3%, and MobileNetV2 has a calculated FIT rate 2–3x lower than ResNet-18.The authors attribute MobileNetV2’s greater inherent fault tolerance to its inverted residual architecture and information distributed across many channels.Source: §3 · p. 4
4 Discussion
Structured pruning increases per-inference fault sensitivity by removing channel redundancy, but reduced computation shortens SEU exposure and largely counterbalances that sensitivity at the system level. Radiation-induced failures remain a distinct safety concern because they can cause high-confidence misclassifications even on trivial inputs.
4 Discussion
- Wider networks distribute classification information across more channels, so corruption in one neuron's computation has a smaller relative effect; pruning removes this redundancy.Remaining neurons become more critical to the output, increasing the likelihood that a bit flip causes a destructive fault.Source: §4 · p. 4
- Pruning increases per-inference sensitivity, but shorter SEU exposure largely counterbalances it, allowing memory, energy, and latency savings without substantially increasing system-level failure rates.Pruned models require fewer operations, shortening the time window for an SEU.Source: §4 · p. 4
- Radiation-induced SDCs can cause catastrophic, high-confidence misclassifications even on trivial inputs that fault-free models would classify correctly.Unlike epistemic uncertainty, these stochastic failures are unrelated to the data distribution.Source: §4 · p. 4
5 Conclusion
Structured pruning reduces fault tolerance by removing redundancy, while lower computational load mitigates the increased per-inference sensitivity. The paper concludes that efficiency can be prioritized for space-borne AI, while noting that safety-critical impacts still need mission-level assessment.
5 Conclusion
- Pruning reduces fault tolerance by removing redundancy, but lower computational load reduces SEU probability during inference and mitigates the increased per-inference sensitivity.The paper concludes that efficiency optimization does not compromise reliability.Source: §5 · p. 4
- The authors suggest prioritizing energy- and time-efficient models and leveraging DNN fault tolerance to relax strict radiation-hardening requirements in space accelerators.This could simplify accelerator design for space applications.Source: §5 · p. 4
- The low observed number of SDC events does not resolve their safety-critical impact; future work should assess mission-level effects and distinguish benign anomalies from critical failures.Source: §5 · p. 4