Source-linked AI summary
Causality-based Neural Network Repair
Bing Sun, Jun Sun, Hong Long Pham, Jie Shi
TL;DR
Neural-network defects raise reliability concerns, and retraining can be costly without guaranteeing that unwanted behavior is removed. CARE uses causal fault localization to identify defect-contributing neurons and adjusts their parameters; experiments report improved specified properties across tasks while maintaining accuracy.
Problem
Neural networks can have hidden defects, while retraining to remove unwanted behavior can be costly and does not guarantee its elimination.
Source: §1 · p. 1Method
CARE uses causal attribution to locate neurons contributing to defects, then adjusts their associated weights to satisfy the specified property while maintaining accuracy.
Source: §1 · p. 2Results
CARE improves performance on specified properties across evaluated tasks and outperforms existing neural-network repair approaches while maintaining accuracy.
Source: §1 · p. 2, §6 · p. 11Takeaways & Limitations
CARE’s causality-based localization focuses repair on defect-contributing neurons, with repaired models reported to minimally disrupt existing correct behavior.
Source: §1 · p. 2, §6 · p. 11The paper’s demonstrated property scope is limited to fairness, backdoor absence, and safety properties.
Source: §2.1 · p. 2
Abstract
from arXiv · showhide
Neural networks have had discernible achievements in a wide range of applications. The wide-spread adoption also raises the concern of their dependability and reliability. Similar to traditional decision-making programs, neural networks can have defects that need to be repaired. The defects may cause unsafe behaviors, raise security concerns or unjust societal impacts. In this work, we address the problem of repairing a neural network for desirable properties such as fairness and the absence of backdoor. The goal is to construct a neural network that satisfies the property by (minimally) adjusting the given neural network's parameters (i.e., weights). Specifically, we propose CARE (\textbf{CA}usality-based \textbf{RE}pair), a causality-based neural network repair technique that 1) performs causality-based fault localization to identify the `guilty' neurons and 2) optimizes the parameters of the identified neurons to reduce the misbehavior. We have empirically evaluated CARE on various tasks such as backdoor removal, neural network repair for fairness and safety properties. Our experiment results show that CARE is able to repair all neural networks efficiently and effectively. For fairness repair tasks, CARE successfully improves fairness by $61.91\%$ on average. For backdoor removal tasks, CARE reduces the attack success rate from over $98\%$ to less than $1\%$. For safety property repair tasks, CARE reduces the property violation rate to less than $1\%$. Results also show that thanks to the causality-based fault localization, CARE's repair focuses on the misbehavior and preserves the accuracy of the neural networks.
1 INTRODUCTION
Neural networks’ broad adoption has heightened concerns about reliability, while their defects are difficult to diagnose because their decisions are comparatively opaque. CARE addresses causality-based localization and repair by making minor weight changes to remedy defects while maintaining accuracy.
1 INTRODUCTION
- Neural-network defects matter because they can undermine reliability in applications including medical diagnosis, facial recognition, fraud detection, and self-driving.Source: §1 · p. 1
- Neural-network repair seeks to fix unwanted behavior while retaining other functionality; retraining can be costly and does not guarantee the behavior is eliminated.Source: §1 · p. 1
- Existing causality analysis does not establish how to apply causality to neural-network repair, where a wrong decision may reflect many neurons’ joint contributions.Source: §1 · pp. 1–2
- CARE locates neurons that contribute most to defects, then adjusts their associated parameters until the specified property is satisfied while maintaining accuracy.Source: §1 · p. 2
- CARE is evaluated on fairness improvement, backdoor removal, and safety-property repair, across feed-forward and convolutional networks trained on benchmark datasets.Source: §1 · p. 2
2 PRELIMINARY
The paper considers fairness, backdoor absence, and safety as desirable neural-network properties, and frames repair as satisfying a property while minimizing the change from the original model. It illustrates the setup with a fairness example and measures model distance through accuracy difference.
2.1 Neural Network Properties
- Fairness requires the probability difference in predictions across protected-feature values to remain within threshold ξ.Source: §2.1 · p. 2
- Backdoor absence requires attack success rate to stay below a threshold; the rate is the share of triggered adversarial inputs classified as the target label.Source: §2.1 · p. 2
- Safety requires the property-violation rate to remain below a threshold, with violation rate measuring the proportion of inputs that violate the specified property.Source: §2.1 · pp. 2–3
- The running example repairs a five-hidden-layer Census Income model whose female–male favorable-prediction probability difference exceeds a 1% fairness threshold.Source: §2.1 · p. 3
2.2 Our Problem
- Repair constructs a model that satisfies the chosen property while minimizing semantic distance from the original, measured here by accuracy difference.Source: §2.2 · p. 3
3 OUR APPROACH
CARE verifies a desired property, localizes neurons causally implicated in misbehavior, then optimizes their weights to repair the network while limiting accuracy loss. Its approach models networks as structural causal models and uses particle swarm optimization for repair.
3 OUR APPROACH
- CARE verifies the target property, localizes neurons contributing to violations, and optimizes their weights to produce a repaired network that remains semantically close to the original.If the property already holds, CARE returns the network unchanged.Source: §3 · p. 3, §3.1 · p. 3, Figure 1 · p. 3
3.1 Property Verification
- If verification finds a violation, CARE proceeds to repair; its implementation uses existing verification approaches for fairness, backdoor success rate, and safety properties.Property verification itself is not the focus of the work.Source: §3.1 · p. 3
3.2 Causality Analysis
- CARE models neural networks as structural causal models and uses average causal effect to measure hidden-neuron influence on undesirable behavior, rather than relying on gradient-based correlations.The authors argue that causal influence better fits repair's goal of identifying neurons that cause the defect.Source: §3.2 · p. 4
3.3 Fault Localization
- CARE represents feed-forward and convolutional networks as structural causal models, estimates neuron attribution by intervening on a neuron and averaging undesirable behavior, then selects high-attribution neurons as repair candidates.It defines attribution for continuous-valued neurons and measures the defect through a property-specific output.Source: §3.3 · pp. 4–5
3.4 Network Repair
- Particle swarm optimization adjusts the identified neurons’ weights to balance property violation and accuracy, with the objective using a property-specific violation measure and an accuracy penalty.The particles represent candidate weights for the identified neurons, initialized from the original weights.Source: §3.4 · p. 6
- In the running fairness example, CARE improved fairness by 46.1%, with repaired-model accuracy of 0.86 versus 0.88 originally.The reported maximum probability difference after repair was 0.007.Source: §3.4 · p. 6
4 IMPLEMENTATION AND EVALUATION
CARE is evaluated on fairness, backdoor-removal, and safety-repair tasks, with experiments also testing parameter trade-offs and causality-based fault localization. Across the tasks, it repairs the reported misbehavior while generally preserving model accuracy.
4.1 Experiment Setup
- CARE was tested on three fairness datasets, three backdoor datasets, and three ACAS Xu networks paired with safety properties.The fairness datasets were Census Income, German Credit, and Bank Marketing; the backdoor datasets were GTSRB, MNIST, and Fashion-MNIST.Source: §4.1 · p. 7
- The PSO setup used 20 particles and at most 100 iterations, with search stopping when the property was satisfied or no better location appeared for 10 consecutive iterations.If accuracy dropped by more than 3%, CARE returned an error and required a larger α.Source: §4.1 · p. 7
4.2 Research Questions and Answers
- For fairness evaluation, the study set ξ=1% so all models failed the requirement before repair and limited repair to no more than 10% of network neurons.The fairness sampling assumes independent input features following a normal distribution.Source: §4.2 · p. 7
- CARE was compared with NNRepair on two CNNs trained on MNIST and CIFAR10; CARE reduced SR by 99.9% with 1.5% accuracy drop, while NNRepair reduced SR by 19.3% with 5.9% drop.Source: §4.2 · p. 8
- Increasing α gives model accuracy more weight in PSO, but smaller α often improves property repair at an accuracy cost, so users can choose α according to property importance.For NN1, α=0.2 reduced accuracy from 0.88 to 0.83; for NN3, unfairness was 0.001 at α=0.1 and 0.0368 at α=0.9.Source: §4.2 · p. 8, Figure 5 · p. 9
- Across fairness, backdoor, and safety tasks, CARE improved fairness 61.9% on average, reduced attack SR from over 98% to less than 1%, and lowered evaluation VR from 1.0 to <1%.For safety repair, drawdown-set performance was not affected.Source: §4.2 · pp. 7–8, Table 2 · p. 9, Table 3 · p. 9
- Against DTMC, CARE improved fairness by 61.9% on average with 1.7% accuracy drop, versus 45.1% improvement with 2.2% cost.Source: §4.2 · p. 7, Table 5 · p. 9
- Without fault localization, PSO failed to converge within the time limit, leaving performance improvement and repair cost unoptimized when it terminated.Randomly selected parameters yielded 47.6% average performance improvement but an 8.9% average accuracy drop. For fairness repairs, gradient-based localization improved fairness by 73.6% while average accuracy dropped by 17.8%.Source: §4.2 · pp. 9–10
5 RELATED WORK
Prior work spans neural-network verification, repair, and causal interpretation. CARE focuses on repair and uses causal attribution of hidden neurons to guide fault localization.
Neural network verification
- Verification methods use abstraction, SMT, optimization, and symbolic execution, while CARE focuses on repairing networks and uses existing verification methods for property checks.Fairness verification approaches include numerical integration, non-convex optimization, and probabilistic model checking.Source: §5 · p. 10
Machine learning model repair
- Prior repair approaches patch error regions, use SMT, or seek minimal verification-based fixes; NNRepair [65] and [59] use layer-wise repair despite faults potentially spanning layers.The verification-based approach is limited by its underlying verification method.Source: §5 · p. 10
Machine learning causal interpretation
- Causal interpretation work explains model decisions or measures feature effects [46] [8], while CARE uses SCMs and do-calculus to attribute undesirable behavior to hidden neurons for localization.The passage also describes fairness metrics based on counterfactuals and causal explanations.Source: §5 · p. 11
6 CONCLUSION
CARE is presented as a causality-based technique for repairing neural networks across properties. It localizes faults, adjusts identified-neuron weights with PSO, and reports improved properties with minimal disruption to correct behavior.
6 CONCLUSION
- CARE combines causal fault localization with PSO-based weight adjustment, improving specified properties while maintaining accuracy and minimally disrupting correct behavior.The evaluation covers multiple neural networks trained on benchmark datasets.Source: §6 · p. 11
7 DATA AVAILABILITY
The CARE proof-of-concept is implemented on SOCRATES, and its source code is publicly available.
7 DATA AVAILABILITY
- CARE's proof-of-concept repair engine is implemented on SOCRATES, and the source code is publicly available.The passage identifies this implementation as a proof of concept.Source: §7 · p. 11