FT-CNN: Algorithm-Based Fault Tolerance for Convolutional Neural Networks

1 INTRODUCTION

• Soft errors can threaten CNN inference reliability in long-running and safety-critical settings, while ECC cannot protect computational errors and instruction duplication incurs high overhead.Soft errors may arise from high-energy radiation, aging, wear, or voltage underscaling; cited studies report that bit flips can sharply affect prediction accuracy and safety-relevant failure rates.Source: §1 · pp. 1–2
• The paper proposes ABFT schemes that apply to any convolution implementation and analyzes their fault-protection ability and runtime.The schemes are designed to detect and correct errors at runtime.Source: §1 · p. 2
• A layerwise multischeme workflow combines the schemes to target high detection and correction ability with limited runtime overhead, and supports bias, grouped convolution, and back propagation.The paper focuses on convolutional layers because they consume a major portion of CNN computation time.Source: §1 · p. 2
• Less than 8% overhead across AlexNet, YOLO, VGG-19, and ResNet-18 in both error-free and erroneous cases is reported for FT-Caffe.The evaluation used ImageNet on the Bebop supercomputer with up to 128 nodes.Source: §1 · p. 2

2 BACKGROUND

• The background introduces convolution-layer concepts and summarizes the paper’s notation and symbols in Table 1.The table is identified as the notations-and-symbols reference.Source: §2 · p. 2, Table 1 · p. 2

2.1 Definition of Convolutional Layer

• A convolutional layer uses feature maps D and kernels W to produce output O, with bias B applied after convolution.The passage gives D as N×Ch×H×H, W as M×Ch×R×R, and O as N×M×E×E; bias protection is discussed separately.Source: §2.1 · p. 2

2.2 Implementation of Convolutional Layer

• Convolution implementations include MM-based, direct, FFT-based, and Winograd methods, with FFT suited to relatively large feature maps and kernels and inferior to Winograd for relatively small sizes.Frameworks and accelerators generally choose implementations automatically based on hardware resources and model structure.Source: §2.2 · pp. 2–3

2.3 ABFT for Matrix-Matrix Multiplication

• Traditional matrix-multiplication ABFT covers only MM-based convolution, incurs more than 50% overhead, and misses feature-map reorganization operations.The paper argues that new schemes are needed because CNN frameworks and accelerators may select other convolution implementations.Source: §2.3 · p. 3, §2.2 · p. 3

3 NOVEL ABFT SCHEMES FOR CONVOLUTION

• The four proposed ABFT schemes support any convolution implementation and protect the whole convolution process.A later workflow uses the schemes at different stages to maximize protection while minimizing performance overhead.Source: §3 · p. 3

3.1 Preliminary Analysis – Convolution

• At block level, convolution computes each output block from a feature-map block and a kernel block; its distributive property underpins checksum equivalence.The paper treats D, W, and O as collections of blocks and uses the distributive property to prove the checksum design’s correctness.Source: §3.1 · p. 3

3.2 Preliminary Analysis – CNN Checksums

• The four input checksums are treated as blocks, and output checksums are derived by convolving inputs and/or their checksums.The output contains seven defined checksums, Co1 through Co7; Table 2 summarizes the checksums used by each scheme.Source: §3.2 · p. 3, Table 2 · p. 3

3.3 Full Checksum Scheme (FC)

• The full checksum scheme (FC) uses checksums from both D and W, then compares output checksums Co1 and Co2 with output summations to detect, locate, and correct errors.The checksums Cd1 and Cw1 are computed before convolution, so memory errors striking D or W during convolution do not affect them.Source: §3.3 · p. 4

3.4 Row Checksum Scheme (RC)

• The row checksum scheme (RC) uses only output row checksums; a mismatch locates an error and supports correction of the affected block.The scheme compares Co1 with So1 and uses Co3 with So3 to locate the error.Source: §3.4 · p. 4

3.5 Column Checksum Scheme (ClC)

• The column checksum scheme (ClC) uses only output column checksums; a mismatch locates an error and supports correction of the affected block.It compares Co2 with So2 and uses Co4 with So4 to locate the error.Source: §3.5 · p. 4

3.6 Checksum-of-Checksum Scheme (CoC/CoC-D)

• The checksum-of-checksum scheme (CoC) derives output checksums from input checksums rather than D or W, then uses them to locate and correct an error.For detection only, CoC-D omits Co6 and Co7 but still requires input checksums for D and W.Source: §3.6 · p. 4

• The section discusses the fault-protection abilities and runtime of all four proposed schemes.Source: §4 · p. 4
• It then proposes a multischeme workflow powered by a calibrated arrangement of the schemes.Source: §4 · p. 4
• The workflow uses layerwise optimization.Source: §4 · p. 4

4.1.1 Fault Model

• The analysis considers transient computational faults and transient or persistent memory corruption, generally assuming at most one fault per convolution.Multiple faults can also be detected and recovered by recomputing the corrupted layer.Source: §4.1.1 · pp. 4–5

4.1.2 Analysis of Soft Error in D and W

• Errors in weights can be detected by comparing their checksum with Cw1 and corrected by reloading weights; input errors need no correction because the input is discarded.Output checksums detect and correct resulting output errors.Source: §4.1.2 · p. 5

4.1.3 Analysis of Soft Error in O

• A fault in the input or weights can corrupt at most one output row or column, respectively, which bounds the output-error patterns analyzed.Source: §4.1.3 · p. 5

4.1.4 Soft Error Protection Ability of CoC Scheme

• CoC detects multiple errors in inputs or outputs with Co5, but corrects only a single output error and cannot correct errors spanning multiple output blocks.Figure 2 illustrates these input/output error cases.Source: §4.1.4 · p. 5, Figure 2 · p. 5
• Corrupted checksums can create inconsistent output-checksum patterns; CoC detects this, discards the checksums, and finishes when the data and output remain clean.Figure 3 illustrates checksum corruption and detection.Source: §4.1.4 · p. 5, Figure 3 · p. 5

4.1.5 Soft Error Protection Ability of Row Checksum Scheme and Column Checksum Scheme

• Row checksums detect and correct errors in one row but only detect errors in one column; column checksums have the symmetric capability.Figure 4 contrasts the two orientations.Source: §4.1.5 · p. 5, Figure 4 · p. 5

4.1.6 Soft Error Protection Ability of Full Checksum Scheme

• The full checksum scheme corrects errors in either direction using both row and column checksums, and can use CoC checksums to locate errors when one checksum is corrupted.Figure 5 shows the correction cases.Source: §4.1.6 · pp. 5–6, Figure 5 · p. 6

4.1.7 Conclusion

• The full checksum scheme has the strongest correction ability, followed by row/column checksums and then CoC; CoC-D detects multiple errors but cannot correct them.This ranking guides the low-overhead, high-protection design in Section 4.3.Source: §4.1.7 · p. 6

• The full checksum scheme has the strongest correction ability but relatively long runtime, whereas CoC has lower correction ability and the shortest runtime.Kernel checksums Cw1 and Cw2 can be precalculated, at no cost for generating them in the row, column, and CoC schemes.Source: §4.2 · p. 6
• Experiments confirm that CoC and CoC-D have the shortest runtime, and the column checksum scheme runs much longer than the row checksum scheme.Smaller weight blocks and more efficient row-checksum calculation explain the difference.Source: §4.2 · p. 6, Figure 6 · p. 6
• Table 4 presents the theoretical runtime complexity of the checksum schemes.Source: Table 4 · p. 6, §4.2 · p. 6

• CoC-D performs low-overhead detection, followed by CoC and then full checksum correction, which has the greatest correction ability and overhead.The workflow integrates all four schemes to balance protection and runtime.Source: §4.3 · p. 7
• Detection runs on every execution, while correction starts only after an error is detected and escalates if a scheme cannot correct it.Source: §4.3 · p. 7
• Reusing checksums across workflow stages makes runtime lower than the sum of the individual scheme runtimes.For example, CoC can reuse Co5 and its summation after CoC-D has computed them.Source: §4.3 · p. 7
• Enable row checksums when their expected runtime savings for row errors exceed the runtime penalty for column errors.The decision uses per-layer runtimes and estimated row- and column-error probabilities; the YOLOv2 example shows why some layers disable row checksums.Source: §4.3 · p. 7

5 RESOLVING BIAS, GROUPED CONVOLUTION, AND BACK PROPAGATION

• The extensions cover bias, grouped convolution, and back propagation of convolutional layers.Source: §5 · p. 7

5.1 Bias

• For bias, output checksums omit the bias included in the convolution output, so the bias contribution is precomputed and subtracted from the output summation for comparison.Modifying every output element before verification and restoring it afterward is not feasible because of the overhead.Source: §5.1 · pp. 7–8

5.2 ABFT for Grouped Convolution

• Grouped convolution requires modified kernel checksums, while feature-map checksums and output-checksum definitions remain unchanged; the proposed schemes extend to grouped convolution.The kernel blocks are divided into groups, each operating on its corresponding feature-map channel group.Source: §5.2 · p. 8

5.3 ABFT for Convolution Back Propagation

• The schemes also protect convolution back propagation, using checksums for ∇O in the operations that calculate ∇W and ∇D.The paper focuses experiments on inference, noting that training is usually more stable and can tolerate some soft errors because of its iterative-convergent nature.Source: §5.3 · p. 8

6.1 Experimental Setup

• The evaluation tests AlexNet, VGG-19, ResNet-18, and YOLOv2 on 50,000 ImageNet validation images, with batch size 64 and injected faults simulated by randomly corrupting output rows or columns.Experiments use FT-Caffe on Bebop, with up to 128 nodes; the injected-error overhead averages runs across convolution layers.Source: §6.1 · pp. 8–9

6.2 Experimental Results with MKL-DNN

• Layerwise RC/ClC optimization cuts error-correction overhead by 40%–60%; RC is disabled in 30%–40% of ResNet-18 and YOLOv2 layers, while RC corrects nearly 90% of errors in AlexNet and VGG-19.Less than 5% of errors are corrected by CoC, and per-layer protection overhead varies with the shapes of D and W.Source: §6.2 · p. 9, Figure 11 · p. 9, Figure 12 · p. 10
• Error-free overhead is below 4% for AlexNet, 4.5% for VGG-19, 8% for YOLOv2, and 5% for ResNet-18, regardless of convolution implementation.ResNet-18's small 3×3 kernels have shorter computation time, making checksum computation and verification a larger share of runtime.Source: Figure 10 · p. 9, §6.2 · p. 9

6.3 Experimental Results with MM-Based Convolution

• Traditional MM-based ABFT incurs up to 60% overhead without injected errors and cannot protect the full MM-based convolution process or other convolution implementations.The authors conclude that this makes traditional MM-based ABFT unsuitable for CNN soft-error protection.Source: §6.3 · p. 10, Table 6 · p. 10
• For MM-based convolution, overhead is below 6% error-free and below 6.5% with injected errors across all four models; layerwise RC/ClC optimization reduces correction overhead by up to 77%.More layers use RC than under MKL-DNN because RC's relative runtime versus FC is lower in the MM-based implementation.Source: Figure 13 · p. 10, Figure 14 · p. 10, §6.3 · p. 10

6.4 Parallel Performance Evaluation

• 2%–6% erroneous-case overhead and 1%–4% error-free overhead do not increase with node count in the parallel evaluation.The evaluation uses up to 128 nodes on Bebop.Source: §6.4 · p. 10

System-level and hardware approaches

• Hardware hardening adds effort across varied CNN models and accelerators, while checkpoint/restart is considered a poor fit when a single inference runs quickly.The cited system-level work was developed to protect machine-learning training from soft errors.Source: §7 · pp. 10–11

Hardware approaches

• Partial hardware redundancy may avoid doubling power, but still requires hardware modification across varied CNN models and accelerators.Source: §7 · pp. 10–11

Software-level approaches

• Traditional ABFT applied to MM-based convolution was reported to correct 50%–60% of radiation-induced corruptions, but works only for that convolution implementation, unlike the proposed solutions.Source: §7 · p. 11

Conclusion

• The work proposes four ABFT schemes and a multischeme workflow for convolutional layers, extending the schemes to bias, grouped convolution, and convolution back propagation.Source: §8 · p. 11
• FT-Caffe incurs less than 8% overhead for AlexNet, YOLO, VGG-19, and ResNet-18 in both error-free and erroneous cases.Source: §8 · p. 11

Future Work

• Future work will extend the implementation to more CNN frameworks and develop architecture-specific optimizations for GPUs, FPGAs, and AI accelerators.Source: §8 · p. 11