Source-linked AI summary
TreeFI: Value-Aware Statistical Fault Injection for Deep Neural Networks
Noam Bires, Marcello Traiola, Angeliki Kritikakou, Elisa Fromont
TL;DR
Modern DNN fault-injection campaigns are costly, and existing statistical methods do not explicitly account for value-dependent floating-point bit-flip effects. TreeFI partitions layer values into behaviorally similar intervals and allocates injections by interval relevance. Across evaluated CNN and Transformer models, it reduces injection requirements substantially while preserving target statistical accuracy, with more accurate estimates than a baseline on ResNet8.
Problem
Exhaustive fault injection is impractical for modern DNNs, while existing statistical methods overlook how the corrupted value affects floating-point bit-flip impact.
Method
TreeFI uses regression trees to partition each layer’s value distribution into intervals with similar bit-flip behavior, then weights and allocates injections across intervals for failure-rate estimation.
Results
Up to 72.1× fewer injections are required across evaluated models, while ResNet8 estimates show lower error than data-aware one-shot SFI and remain within the target margin against exhaustive reference results.
Takeaways & Limitations
Value-aware stratification reduces injection budgets while retaining reliable failure-rate estimates under the evaluated single-bit fault model and benchmarks.
Takeaways & Limitations
The evaluation instantiates TreeFI with FP32 single-bit flips during inference and uses software fault injection under a predefined single-fault model.
Abstract
from arXiv · showhide
Reliability evaluation of deep neural networks under hardware faults commonly relies on fault injection, but exhaustive campaigns are intractable for modern models and datasets. Statistical fault injection reduces this cost, yet existing approaches still require large injection budgets because they do not explicitly exploit a key property of floating-point faults: the effect of a bit flip depends strongly on the value being corrupted. We propose TreeFI, a value-aware statistical fault-injection methodology for FP32 single-bit faults in DNN activations and weights. TreeFI partitions each layer's value distribution into intervals with similar expected bit-flip behavior, learned using regression trees, and allocates injections across these intervals according to their relevance for failure-rate estimation. This stratified allocation preserves the target confidence and error margin while avoiding unnecessary injections in low-impact regions of the fault space. We validate TreeFI on CNN and Transformer models using CIFAR-10 and ImageNet. On ResNet8, where exhaustive activation fault injection is feasible, TreeFI provides more accurate estimates than state-of-the-art statistical FI baselines under the same campaign setting. Across the evaluated models, TreeFI reduces the required injection budget by up to 72.1x, with average reductions of 44.9x for activation faults and 11.2x for the executed weight campaigns.
1 Introduction
DNN reliability assessment requires fault injection, but exhaustive campaigns are impractical at modern scale and existing statistical methods overlook how corrupted values shape bit-flip effects. TreeFI addresses this gap with value-aware stratification and reports lower-cost, accurate reliability estimates.
- Exhaustive fault injection becomes impractical because modern models and datasets create massive spaces of locations, bit positions, and inputs.
- Existing statistical fault-injection methods reduce campaign cost but neglect the dependence of bit-flip effects on the corrupted value.
- TreeFI partitions each layer’s values into ranges with similar bit-flip behavior and allocates more injections to ranges most relevant to failure-rate estimation.
- TreeFI uses regression trees and stratified sampling to target a user-specified confidence and error margin with fewer injections than state-of-the-art statistical methods.
- 72.1× is the reported maximum reduction in required injections across the evaluated CNN and Transformer models.
- On ResNet8, TreeFI produces lower estimation error than data-aware one-shot statistical fault injection in a single campaign.
2 Background and related work
Statistical fault injection makes neural-network reliability analysis more scalable, but prior methods do not organize sampling by value range. TreeFI adds value-aware interval partitioning and allocation while retaining one-shot, upfront-budgeted campaigns.
- Statistical fault injection estimates full fault-population behavior from a selected subset under a desired confidence level and error margin.
- Prior methods include proxy-guided one-shot SFI and iterative IFI, but SFI is less favorable for heterogeneous, input-dependent activation values.
- IFI updates campaigns using intermediate results, whereas its iterative process can spend many injections.
- TreeFI explicitly exploits the dependence of fault effects on corrupted values by partitioning value space into intervals with similar bit-flip behavior.
- TreeFI allocates more injections to intervals important for failure-rate estimation while retaining one-shot, upfront-budgeted campaigns.
3 TreeFI: Value-aware statistical fault injection
TreeFI makes statistical fault injection value-aware by partitioning each layer's value distribution into intervals with similar bit-flip behavior, then allocating injections according to interval frequency and risk. Its two-phase workflow learns these intervals offline and uses stratified sampling to estimate failure rates while targeting a specified confidence level and error margin.
- Motivation: For a fixed bit position, bit-flip impact can vary substantially with the original floating-point value, motivating value-range-aware sampling.The same value-dependent behavior is also observed for weight faults, although the motivating example focuses on activations.
- Offline characterization: TreeFI first collects layer value distributions and partitions each layer–bit value axis into intervals with similar local bit-flip effects.Interval learning uses the numerical change caused by flipping a bit, without evaluating the resulting model output.
- Offline characterization: Regression trees learn interval boundaries from value histograms, placing splits where the local effect of flipping a selected bit changes significantly.Histogram counts support splits with sufficient observed values rather than rare bins; learned trees contain at most about 15 leaves per layer–bit pair in practice.
- Offline characterization: TreeFI assigns interval risk scores from average log-distance between original and bit-flipped values, mapped through a sigmoid.The resulting risk score is bounded between 0 and 1 and guides later injection allocation.
- Stratified campaign: Interval frequencies and risk scores determine stratified injection counts, concentrating faults in common intervals that are more difficult to estimate accurately.The total allocation for each layer–bit pair is chosen to satisfy the desired confidence level and error margin.
- Failure-rate estimation: TreeFI combines interval-level measured failure rates using golden-execution interval frequencies, making the final estimate reflect actual layer behavior.For activation faults, eligible locations are selected from current-input values inside the sampled interval; weight faults select parameters in the interval.
4 Experimental set-up and evaluation
TreeFI is evaluated on CNN and Transformer models for activation and weight faults, using exhaustive FI where feasible and statistical baselines elsewhere. Across these settings, it preserves failure-rate estimates while substantially reducing injection counts and runtime.
- Evaluation scope: TreeFI is evaluated against exhaustive FI and state-of-the-art statistical baselines on ResNet8, RepVGG-A0, and DeiT models.The experiments cover CIFAR-10 and ImageNet, with activation and weight campaigns.
- Activation validation: TreeFI-10× matches the main exhaustive ResNet8 failure-rate trends and stays within the 1% statistical error target.It reproduces the most vulnerable bits and highest-failure layers while achieving lower MAE than SFI and IFI in sensitive regions.
- Activation efficiency: 34,934 injections give TreeFI-10× about 21.2× and 28.0× theoretical reductions versus SFI and IFI on ResNet8 activations.The corresponding measured wall-clock reductions are about 15.5× versus SFI and 20.5× versus IFI.
- Scalability: 44.9× fewer injections are achieved on average by TreeFI-10× than SFI across evaluated activation campaigns, with reductions ranging from 5.9× to 72.1×.The reductions preserve the same statistical objective, and runtime scales approximately linearly with injection count.
- Scalability: TreeFI-10× preserves the main RepVGG and DeiT failure-rate trends while using substantially fewer injections.On the first DeiT-Tiny layer, TreeFI-10× requires 1,042 injections versus 124,089 for IFI and 139,778 for SFI; full TreeFI-Tiny analysis takes 6 h 42 min versus 111 h 30 min for only SFI's first three layers.
- Weight faults: TreeFI-20× reduces weight injections by 6.4×–17.9× versus SFI across evaluated models, while its largest MAE versus either baseline is 0.0437%.Weight campaigns covered all ResNet8 layers, 10 RepVGG-A0 layers, and 6 DeiT-Tiny layers; smaller gains reflect less variation in weight values.
5 Conclusions and discussion
TreeFI enables efficient neural-network reliability evaluation by allocating fault injections according to value-dependent bit-flip risk. Across evaluated benchmarks, it reduces injections while meeting target estimation margins and preserving key failure-rate trends.
- TreeFI partitions value distributions into intervals, scores interval risk from bit-flip severity, and allocates injections across intervals.The final reliability estimate uses measured fault-injection outcomes.
- TreeFI substantially reduces required injections compared with state-of-the-art statistical fault-injection approaches.The methodology avoids bit-level, value-uniform sampling while targeting a desired confidence level and error margin.
- TreeFI keeps estimation error within the target margin when validated against an exhaustive fault-injection reference.
- TreeFI preserves the main per-layer and per-bit failure-rate trends in larger activation-fault experiments.This supports its use when exhaustive validation is impractical.
- For weight faults, TreeFI is experimentally validated on CNN models and a 6-layer DeiT-Tiny subset.The experimental study focuses on FP32 bit flips.
6 Reproducibility
The TreeFI implementation and experimental artifacts are publicly available, with documentation for installation, examples, and reproducing the reported results.
- The TreeFI implementation and experimental artifacts are available in a public GitLab repository.
- The repository README provides installation instructions and ready-to-run activation and weight examples.
- The reproducibility documentation explains how to reproduce the paper’s results.