PaperlayerView exact revision on arXiv

Source-linked AI summary

Carry-Through Checksum: A Lightweight Fault-Detection for CNN Inference at the Edge

Kyrylo Nazarevych, Mohammad Hasan Ahmadilivani, Krister Kaldre, Davide Bertozzi, Jaan Raik

arXiv:2609.16742v1cs.ARcs.LG

TL;DR

Safety-critical CNN inference on resource-constrained embedded GPUs needs low-overhead protection against silent soft errors. Carry-through checksum embeds dedicated filters that propagate checksums through convolutional operations for single-output verification. Across multiple CNNs, it detects 95.86% of critical FP32 faults and 86.56% of critical FP16 faults, with 2.27% runtime overhead for re-execution.

  • Problem

    Safety-critical edge CNNs can suffer silent soft-error corruption, while existing ABFT methods impose substantial checksum-verification overhead on embedded GPUs.

  • Method

    Carry-through checksum replaces selected channels with dedicated filters that compute and propagate checksums through CNN operations for one-shot output verification.

  • Results

    95.86% of critical FP32 faults and 86.56% of critical FP16 faults are detected, with 2.27% runtime overhead for re-execution across the test set.

  • Takeaways & Limitations

    The scheme provides a lightweight approach for detecting critical faults in CNN inference on resource-constrained embedded GPUs.

Abstract

from arXiv · show

Convolutional Neural Networks (CNNs) are increasingly deployed in safety-critical edge applications, where soft errors can silently corrupt inference outputs and lead to unsafe decisions. Such applications typically rely on resource-constrained embedded GPUs, requiring fault detection and mitigation techniques that add minimal compute, memory, and latency overhead while integrating seamlessly with the standard GPU inference pipeline. Existing algorithm-based fault tolerance techniques rely on matrix augmentation and per-operation checksum verification, imposing substantial overhead that is prohibitive for CNN inference on embedded GPUs. In this work, we propose carry-through checksum, a fundamentally new scheme for soft-error detection in CNN inference on embedded GPUs. The method embeds dedicated carry-through filters into the convolutional layers, which compute a checksum from the CNN's own operations and propagate it through inference, enabling end-to-end error detection with a single output verification. Experimental results on multiple CNN architectures show that the proposed method detects 95.86% and 86.56% of critical faults for FP32 and FP16, respectively, at almost no additional per-image overhead. Detected faults are mitigated through re-execution, incurring only 2.27% run-time overhead across the entire test set on an NVIDIA Jetson Orin NX GPU.

I. INTRODUCTION

CNN inference on safety-critical edge systems is vulnerable to silent soft-error corruption, while existing checksum methods can exceed embedded-GPU resource constraints. Carry-through checksum addresses this gap with embedded checksum propagation and reports high fault coverage with low overhead.

  • Motivation: Embedded GPUs require protection schemes with limited compute, memory, and latency overhead because they serve real-time CNN workloads under tight power and thermal budgets.NVIDIA Jetson devices are identified as widely used platforms for edge safety-critical vision workloads.
  • Motivation: Classical ABFT and CNN-specific variants verify checksums around convolution operations, creating overhead that is problematic for embedded-GPU CNN inference.The supplied passage introduces this as the gap addressed by the proposed scheme.
  • Contribution: Carry-through checksum replaces the least important channel in each layer with dedicated filters that compute and propagate checksums through normal CNN operations.A final comparison against an expected threshold enables one-shot end-to-end detection.
  • Contribution: The method embeds checksum computation without changing CNN structure or operations and offers complementary FI-based and distribution-based threshold-selection methods.The contribution description characterizes the embedded filters as imposing zero CONV-layer overhead.
  • Evaluation: 95.86% of critical FP32 faults and 86.56% of critical FP16 faults are detected, with 2.27% runtime overhead after re-execution across the test set.The evaluation uses multiple CNNs on an NVIDIA Jetson Orin NX GPU.

II. CARRY-THROUGH CHECKSUM

The carry-through checksum accumulates convolutional input-feature-map values through the CNN and compares the propagated result with an expected threshold for fault detection.

  • II. CARRY-THROUGH CHECKSUM: The method accumulates Input Feature Maps from convolutional layers through the CNN and propagates them to the final output for threshold comparison.The checksum is computed through the CNN’s existing operations without additional operation overhead.

A. Carry-Through Checksum Scheme

Carry-through filters compute channel-wise feature-map sums inside convolutional layers and preserve them through architectural modifications and normalization. The design maintains layer dimensions and avoids additional convolutional compute or memory overhead.

  • A. Carry-Through Checksum Scheme: A 3×3 carry-through filter with a single center 1, repeated across input channels, computes the layer’s feature-map summation.The center placement is used with padding 1 and stride 1.
  • A. Carry-Through Checksum Scheme: The center-only kernel produces a one-to-one spatial summation across channels, avoiding the double counting caused by overlapping all-ones kernels.Each output element aligns with one input spatial location under the stated convolution settings.
  • A. Carry-Through Checksum Scheme: The scheme prunes the least important output channel, appends the carry-through filter, and zeros normal filters’ slices for the incoming checksum channel.These two structural changes separate checksum propagation from ordinary output channels.
  • A. Carry-Through Checksum Scheme: Because replaced channels and filter dimensions are preserved, convolutional layers incur no additional compute or memory overhead while carrying one checksum output channel.The carry-through filter spans all input channels.
  • A. Carry-Through Checksum Scheme: Residual architectures omit the checksum from the first layer of each residual block, while MobileNet-like architectures replace ReLU6 with ReLU to prevent checksum saturation.The residual treatment avoids accumulating incoming feature maps twice.
  • A. Carry-Through Checksum Scheme: Batch normalization is configured with E(x) = 0, Var[x] = 1, γ = 1, and β = 0 so it acts as an identity mapping on checksum channels.These settings prevent BN from modifying the accumulated checksum.

B. Fault Detection and Mitigation

Fault detection aggregates propagated checksum values at the CNN output and flags deviations beyond a threshold. A flagged inference is mitigated through re-execution under the stated consecutive-fault assumption.

  • B. Fault Detection and Mitigation: An added accumulated-checksum neuron sums all incoming activations with fixed weight 1 and provides the CNN-output indicator for error detection.The neuron is appended to the final fully connected layer.
  • B. Fault Detection and Mitigation: A detection flag is raised when the accumulated-checksum neuron deviates from its expected reference by more than threshold τ.The expected reference corresponds to a fault-free inference.
  • B. Fault Detection and Mitigation: Re-execution mitigates detected faults, with its result used despite the assumption that faults are unlikely in two consecutive inferences.The re-execution ignores the original detection result.

C. Detection Threshold Optimization

The study defines two ways to select the carry-through checksum detection threshold: fault-injection-based and distribution-based methods.

  • The FI-based method uses training-set fault-injection campaigns to assign τ while keeping the false positive rate below α%.It empirically evaluates injected faults and explores α to select the threshold.
  • The distribution-based method derives τ from accumulated-checksum values during fault-free training-set inferences, requiring no fault injection.The threshold is chosen so fewer than α% of fault-free executions are falsely flagged.

III. EXPERIMENTS

The experiments inject parameter bit flips across several CNN architectures, datasets, and numerical precisions, then evaluate detection and execution overhead on an embedded GPU.

  • Fault-injection campaigns inject random single bit flips into CNN parameters during inference until results reach 95% confidence with a 1% error margin.The injections are performed on the test set using PyTorch.
  • TPR measures correctly detected critical faults, while FPR measures non-critical faults incorrectly classified as critical.The study explores α values of 0.1%, 1%, and 5% and uses Youden’s J statistic, TPR − FPR, to select α.
  • The CNNs run on an NVIDIA Jetson Orin NX with TensorRT in FP32 and FP16, while the study measures modification overhead and projects re-execution-based mitigation cost.Inference uses CUDA 11.4 and a C++ implementation.

1) Impact of Carry-Through Filters on Baseline accuracy:

Carry-through filters can reduce CNN accuracy after channel pruning, but fine-tuning keeps the protected models within 1% of their corresponding baselines.

  • Final carry-through-protected CNN accuracy remains within 1% of the corresponding baselines across the evaluated models.Pruning has negligible effect on VGG variants, while ResNet and MobileNet variants require fine-tuning to recover accuracy.

2) Detection Ratio of Carry-Through Checksum:

Detection performance depends on the threshold trade-off, with α = 1% selected as the best balance and distribution-based optimization offering a practical implementation.

  • Youden’s J selects α = 1% as the operating point because α = 5% maximizes TPR with high FPR, whereas α = 0.1% minimizes FPR with reduced TPR.The selected value balances detection and false alarms.
  • At α = 1%, FI-based and distribution-based thresholding have marginal performance differences in most configurations.The distribution-based method avoids the extensive fault-injection campaigns required by FI-based optimization.
  • 95.86% average FP32 TPR and 86.56% average FP16 TPR are achieved with distribution-based thresholding at α = 1%.FP32 TPR ranges from 85.96% to 99.7%, while FP16 TPR ranges from 75.02% to 96.05%.
  • FP32 FPR averages 1.05%, while FP16 FPR averages 1.15% under the selected configuration.The reported FPR ranges are 0.68%–1.35% for FP32 and 0.67%–1.67% for FP16.
  • Average FP16 TPR is 9.3% lower than FP32 TPR because FP16’s smaller representable range and rounding error can leave faults below the detection threshold.Detection remains as high as 96.05% for FP16, and results show no noticeable dependence on CNN depth or parameter count.

3) Execution-Time Overhead Analysis:

Carry-through checksum adds negligible intrinsic latency and memory overhead, while fault detection with re-execution remains modest on the Jetson Orin NX.

  • Below 0.1% average intrinsic execution-time overhead is observed for modified CNNs, with measured variation from −0.5% to +0.5%.The added checksum neuron contributes less than 10^-3% memory overhead, and preceding layers remain unaltered.
  • 0.93% average overhead occurs during fault-free full-test-set inference, mainly from false-alarm re-executions.The reported range is 0.66%–1.35%.
  • 2.27% average overhead occurs under injected faults when detection triggers re-execution, spanning 1.2%–3.8%.The overhead reflects both critical and non-critical faults.
  • The resulting low-cost pipeline combines minimal execution overhead with effective critical-fault detection on resource-constrained embedded GPUs.

IV. CONCLUSIONS

The paper concludes that carry-through checksum detects CNN soft errors by propagating checksums through unchanged CNN operations and verifying them at the output. Across CNNs, it achieves high critical-fault detection with low overhead and supports lightweight threshold selection.

  • Carry-through checksum propagates checksums through dedicated filters without changing CNN structure or operations, enabling output-level critical-fault detection.
  • A lightweight distribution-based method identifies the detection threshold quickly and effectively.
  • 95.86% of FP32 and 86.56% of FP16 critical faults are detected on average, while re-execution mitigation incurs 2.27% test-set runtime overhead.
Loading 2609.16742v1…