Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead

I. INTRODUCTION

• Smart CPS architecture comprises edge, fog, and cloud layers, with some initial computation shifted to the edge to reduce transmission overhead or protect data privacy.The edge connects to the physical environment, fog generally handles most system computations, and cloud supports large-scale processing, storage, and communication.Source: §I · p. 1
• Security attacks and reliability threats can compromise ML accuracy and critically affect CPS applications across system layers.Threats arise at both cloud and edge; reliability failures can occur even without an explicit attacker.Source: §I · pp. 1–2
• The survey reviews vulnerabilities in modern ML systems, state-of-the-art attacks and defenses, solution directions, and research avenues.It also discusses verification and open challenges for secure and reliable systems.Source: §I · pp. 2–3
• The article is organized to cover ML concepts, security attacks and defenses, reliability concerns and mitigations, formal verification, and open challenges.Its roadmap assigns these topics to Sections II–VIII.Source: §I · pp. 2–3

II. MACHINE LEARNING: CONCEPTS AND TERMINOLOGY

• Neural networks connect input and output layers through hidden layers whose neurons use learned parameters and nonlinear activations.State-of-the-art ML commonly uses DNNs with two or more hidden layers.Source: §II · p. 3

A. Neural Network Taxonomy

• Feed-forward networks pass information in one direction, while recurrent neural networks contain feedback loops; LSTMs retain information for long-duration time-series prediction.Networks are fully connected when every neuron links to all neurons in the preceding layer; otherwise they are sparse.Source: A. Neural Network Taxonomy · p. 3
• First-generation networks include single- and multilayer perceptrons, generally built from fully connected layers with thresholding activations.The paper describes NN development across three generations.Source: A. Neural Network Taxonomy · p. 3
• Second-generation CNNs use convolutional filters for feature extraction and shift invariance, while pooling reduces parameters by retaining features at the cost of information loss.CapsNets are presented as a newer approach to address CNN information loss by learning spatial correlations through vectors.Source: A. Neural Network Taxonomy · pp. 3–4
• GANs pair a generator that produces synthetic inputs with a discriminator that learns to distinguish synthetic from real inputs.This interplay enables generation of synthetic outputs that are difficult to distinguish from real ones.Source: A. Neural Network Taxonomy · p. 4
• Third-generation spiking neural networks use spike trains to model biological neurons’ spatio-temporal characteristics rather than normalized firing frequencies.Source: A. Neural Network Taxonomy · p. 4

B. Neural Network Design Cycle

• Training uses forward and backward passes over dataset samples to predict outputs and update network parameters by minimizing loss.Repeated passes over all training samples form epochs; validation data can guide hyperparameter tuning and model selection.Source: B. Neural Network Design Cycle · p. 4
• Testing checks a finite set of inputs, whereas verification uses sound mathematical reasoning to provide guarantees beyond specific tested inputs.For complex ML systems, verification commonly targets bounded input regions rather than the entire input domain.Source: B. Neural Network Design Cycle · pp. 4–5, Figure 5 · p. 5, Figure 6 · p. 5

C. Robustness

• Robustness concerns output integrity under operating conditions and alterations, divided into security against explicit attackers and reliability without one.Reliability concerns changes caused by environmental conditions during fabrication and deployment.Source: C. Robustness · p. 5

III. SECURITY VULNERABILITIES OF ML SYSTEMS

• Security vulnerabilities in cloud and edge ML systems can produce discrepancies or lethal consequences depending on attack intensity and application.Source: §III · p. 5

A. Adversarial Attack

• Adversarial attacks exploit DNN parameters to add often-imperceptible input noise that can cause misclassification or reduce confidence.Attacks may be targeted, untargeted, or confidence-reducing; evasive attacks occur at inference, while poisoning attacks manipulate training data.Source: A. Adversarial Attack · pp. 5–6
• Poisoning can use tailored or random malicious training data, leaving a DNN correct on clean inputs but malicious on poisoned inputs.Dormant neurons can provide capacity for poisoned behavior; cited examples include hiding a person from an object-detection system.Source: A. Adversarial Attack · p. 6, Figure 8 · p. 6, Figure 9 · p. 6
• Many adversarial-attack analyses omit input preprocessing, whose averaging filters can smooth noise and undermine misclassification threats.Source: A. Adversarial Attack · pp. 6–7

B. Neural-Level Trojans

• Neural-level trojans add neurons to a pretrained DNN that trigger malicious behavior on specific inputs, but often require retraining and complex triggers.Source: B. Neural-Level Trojans · p. 7

C. Hardware Attacks

• Hardware trojans can be stealthily implanted during manufacturing to cause undesired behavior, leak information, or consume system resources.Side-channel leakage can reveal inputs or network architecture, while noise in the threat model reduces the chances of a successful attack.Source: C. Hardware Attacks · p. 7

D. IP Stealing

• Model- and dataset-stealing attacks infer a DNN or its training data from available descriptions, queries, and side channels; query timing can also reveal model depth and parameters.The attack objectives are shown in Fig. 11, and timing-based inference in Fig. 10.Source: D. IP Stealing · p. 7, Figure 11 · p. 7, Figure 10 · p. 7

A. Defending Against Adversarial Attacks

• Adversarial defenses either make malicious inputs more perceptible or reduce their impact by improving DNN robustness.The section describes gradient masking, adversarial training, input preprocessing, and training with randomly subsampled image pixels.Source: A. Defending Against Adversarial Attacks · p. 8, Figure 12 · p. 8
• These defenses may fail against stronger attacks: gradient masking can be bypassed with substitute-model gradients, preprocessing-aware attacks can defeat filtering, and adversarial training may overfit.The cited discussion says adversarial training does not necessarily make the network more robust.Source: A. Defending Against Adversarial Attacks · p. 8
• For poisoning attacks, local training avoids outsourcing but may be infeasible for large DNNs; encrypting training data is proposed when training is outsourced.Source: A. Defending Against Adversarial Attacks · p. 8
• Pruning dormant neurons or fine-tuning with clean inputs can reduce backdoor success, but pruning-aware attacks can undermine pruning and fine-tuning does not eliminate backdoors.Source: A. Defending Against Adversarial Attacks · p. 8
• Formal methods can estimate DNN robustness bounds, but this work is generally not scalable to practical DNNs and how to use the bounds to prevent attacks remains unresolved.Source: A. Defending Against Adversarial Attacks · p. 8

B. Defending Against Neural-Level Trojans

• Neural-level trojans may be countered by preprocessing or detecting anomalous inputs and prediction distributions, or by local training of the DNN.Source: B. Defending Against Neural-Level… · p. 9

C. Defending Against Hardware Attacks

• Hardware-trojan defenses include formal equivalence checking, but comparison requires a golden hardware reference; side-channel detection can fail when leakage is too small to detect.Randomizing operation order may also make useful side-channel inference more difficult.Source: C. Defending Against Hardware Attacks · p. 9

D. Defending Against IP Stealing

• IP protections span data blurring or obfuscation, encryption and other privacy measures, while rounding and runtime side-channel monitoring are proposed against piracy.The section notes that simple data-protection approaches may not be strong enough in practice.Source: D. Defending Against IP Stealing · p. 9

A. Hardware Faults

• Hardware faults are classified as transient, intermittent, or permanent; transient faults are temporary, intermittent faults recur, and permanent faults have lasting effects.Intermittent and transient faults may be removable, whereas permanent faults generally require replacing the faulty component.Source: A. Hardware Faults · p. 9

1) Transient Faults:

• Harsh edge conditions, higher clock frequencies, voltage reduction, and technology scaling contribute to transient faults, which may occur randomly or reproducibly.Source: 1) Transient Faults · p. 9
• Cosmic-ray protons or packaging impurities can cause charge-induced bit flips; soft errors can extend to multiple bits and lead to ML misclassification.The text identifies these effects as single-event upsets and multiple-bit upsets.Source: 1) Transient Faults · pp. 9–10, Figure 16 · p. 10

3) Permanent Faults:

• Process variation, aging, temperature, timing errors, Rowhammer, and DRAM retention failures can impair reliability; timing errors may reduce classification accuracy and raise security concerns.Permanent faults produce fixed or repetitive errors and may result from radiation, ESD, fabrication flaws, or recurring intermittent faults.Source: 1) Transient Faults · p. 10, 3) Permanent Faults · p. 10

B. Neural Network Anomalies

• Environmental noise such as fog, pollution, or sensor-acquisition changes can alter inputs and cause faulty inference, especially near overlapping DNN classification boundaries.The cited examples include changes in brightness, contrast, and camera angle.Source: B. Neural Network Anomalies · p. 10, Figure 17 · p. 10

1) Protection against Transient Faults:

• Transient-fault protection includes interleaving, error detection, scrubbing, redundancy with voting, and error-correction codes, but approaches trade detection or correction against area, power, and latency.The section notes that redundancy can incur large area overhead and cost.Source: 1) Protection against Transient Faults · p. 11
• Bit flips from 1 to 0 have a more drastic effect on DNN classification accuracy than flips from 0 to 1, suggesting stronger correction for more critical flips.Source: 1) Protection against Transient Faults · p. 11

2) Protection against Intermittent Faults:

• Different aging rates can make uniform chip-level guardbanding insufficient; dynamic protection, age-aware workload management, and profiling to disable weak memory cells are described.Probabilistic and other hardware or software techniques are also used against Rowhammer.Source: 2) Protection against Intermittent… · p. 11
• Razor flip-flops can detect timing errors; correction may add computation slack, skip a clock cycle, or scale voltage, while formal timing analysis checks behavior within defined bounds.Correction can delay execution as the correct result propagates to the output.Source: 2) Protection against Intermittent… · p. 11

3) Protection against Permanent Faults:

• Fault-aware training, pruning, mapping, and activation clipping address permanent faults, while chip replacement is effective but costly.Fault-aware training is computationally greedy; pruning uses hardware fault maps, mapping preserves salient parameters, and clipping avoids pruning or retraining.Source: 3) Protection against Permanent Faults · p. 11, Figure 18 · p. 11

B. Mitigation Techniques against Environmental Noise

• Filters, adversarial training, and synthetic datasets are proposed for environmental noise, but finite synthetic samples may not represent continuous input domains.Adversarial training may overfit without improving generalization, and obtaining a sufficiently large, diverse real dataset is not always possible.Source: B. Mitigation Techniques against… · pp. 11–12

VII. FORMAL VERIFICATION FOR ROBUST ML

• Formal verification offers mathematical guarantees as an alternative to testing, which is insufficient to ensure reliable inference on unseen inputs.Neural-network verification has been an active research area for about a decade, with SAT/SMT, linear programming, theorem proving, and incomplete verification as major categories.Source: §VII · p. 12

A. SAT/SMT

• SAT suits binarized networks, while SMT supports real or integer DNN parameters; SAT-based verification remains limited in network scale and tooling.CEGAR iteratively improves an over-approximated network model using counter-examples; existing DNN verification tools are not scalable to larger networks.Source: A. SAT/SMT · p. 12, Figure 20 · p. 13

B. Linear Programming

• Linear programming encodes system constraints and a verification objective, including the search for the smallest noise margin that causes misclassification.LP requires linear constraints, so approximating nonlinear activations can yield incomplete results; input bisection for ReLU increases problem size.Source: B. Linear Programming · p. 12
• Big-M encoding can provide reliable ReLU verification without significantly enlarging the problem, but LP scalability remains a challenge.Eliminating inactive neurons and exploiting practical DNN sparsity may help reduce the number of constraints.Source: B. Linear Programming · pp. 12–13

C. Interactive Theorem Proving

• For complex DNNs, theorem proving requires human guidance to model the system and establish why properties hold under sound mathematical assumptions.The cited perceptron-convergence verification concerns binary classifiers, a small subset that may not transfer readily to large state-of-the-art DNNs.Source: C. Interactive Theorem Proving · pp. 13–14

D. Incomplete Verification

• Incomplete verification formally checks an over-approximation of the system, enabling scalability but leaving open whether a property holds when no counter-example is found.Counter-examples indicate actual scenarios where the property fails; refinement or combining incomplete methods with SMT or LP may trade scalability for completeness, without eliminating incompleteness outright.Source: D. Incomplete Verification · p. 14

VIII. OPEN CHALLENGES AND DISCUSSION

• A single vulnerability may undermine accuracy, while the diversity of security and reliability threats makes strong robustness guarantees challenging.The authors expect new challenges as models, DNN depth, hardware technology, and attack models evolve.Source: §VIII · pp. 14–15
• Security challenges span outsourced training data, weak practical obfuscation, training fairness, hardware correctness, and side-channel leakage.Suggested directions include protecting datasets, switching among obfuscation techniques, using verification counterexamples to identify bias, verifying hardware under varied conditions, and limiting resource sharing or adding synthetic noise.Source: §VIII · pp. 14–15
• Reliability research needs operating-condition emulation, less costly fault-safe runtimes, and cross-layer measures to stop subsystem failures from propagating.Hardware and software redundancies can preserve overall performance after component malfunction, but are generally costly.Source: §VIII · p. 15
• Formal verification must model nonlinear, non-convex behavior without the constraint burden of complete methods or false positives from incomplete over-approximation.The discussion identifies Big-M as an example of a complete modeling approach whose generated clauses or constraints can be numerous.Source: §VIII · p. 15
• Verification also needs real-world uncertainty, broader input coverage, lower computational cost, better timing-completeness trade-offs, and scalability to practical DNNs.Current approaches often verify only seed inputs; larger systems increase formal-representation size, and the optimal timing-versus-completeness trade-off remains open.Source: §VIII · p. 15

IX. CONCLUSION

• The survey covers prominent ML-system security and reliability challenges, mitigation approaches, trained-NN verification methodologies, and open challenges to robustness.Its scope includes concerns associated with outsourced training, compromised foundries, stealthy attackers, system aging, and harsh operating environments at cloud and edge.Source: §IX · p. 15

• The NN overview labels input, hidden, and output layers alongside a pictorial representation.Source: APPENDIX · p. 22
• Feed-forward networks are cycle-free, and fully connected variants connect every neuron in one layer to every neuron in the next.Source: APPENDIX · p. 22
• RNN feedback loops let neurons affect successive and preceding layers, giving their internal values temporal characteristics.Source: APPENDIX · p. 22
• CNNs share weights through convolution, improving local spatial correlation and carrying prominent input features forward.Source: APPENDIX · p. 22
• GAN training pits a generator that produces synthetic inputs against a discriminator that distinguishes them from original data.The generator seeks to maximize error by producing more realistic inputs, while the discriminator minimizes error by learning to distinguish real from synthetic inputs.Source: APPENDIX · p. 22
• CapsNet layers represent feature presence with vector elements and instantiation probability with vector length; routing-by-agreement dynamically learns inter-layer connections during inference.Iterative coupling-coefficient updates let capsules interpret high-level features hierarchically.Source: APPENDIX · p. 22