Evasion Attacks on Cost-Utility-Based Adversarial Training for Online AutoML in IoT Networks

• Changing IoT traffic distributions can cause concept drift and degrade intrusion-detection performance.Traffic varies with user behavior, device configurations, network conditions, and attack patterns.Source: §I · p. 1
• Online AutoML adapts models and related pipeline choices to streaming data, but its security against adversarial attacks remains insufficiently explored.The paper identifies evasion attacks as a particularly severe risk to online ML systems.Source: §I · p. 1
• Prior evasion studies report attack success rates of 94% against flow classifiers and 100% in later malware-detection attack rounds.Source: §I · p. 1
• The study evaluates five online learners under black-box evasion attacks and compares naive with adversarially trained variants.Its contributions also include examining concept drift with EDDM and rolling-accuracy stability.Source: §I · pp. 1–2

• OnMAR assumes benign inputs, while Kireev et al. [12] describe cost-utility attacks under bounded costs and heterogeneous gains without training-data poisoning.The cited defense generalizes from utility-aware to cost-bounded threats, but not in the reverse direction.Source: §II · p. 2
• Existing Online AutoML studies emphasize adaptation and efficiency under benign streams, while adversarial-learning studies often use static or offline settings.Source: §II · p. 2
• The unresolved problem is maintaining cost-constrained evasion robustness while adapting to real-time concept drift in streaming Online AutoML.Source: §II · p. 2
• The paper extends Yang and Shami’s Online AutoML pipeline with cost-utility-aware adversarial training [12] and evaluates clean performance, robustness, and drift behavior.The attack is black-box and modifies inference-time inputs rather than training data.Source: §II · p. 2

• The pipeline preprocesses IoTID20 data through encoding, imputation, normalization, and feature engineering before online learning.Feature engineering uses information gain or Pearson correlation; performance measures include accuracy, precision, recall, and F1-score.Source: §III · p. 2
• The learner pool spans HT, HAT, LB, ARF, and SRP, representing incremental trees and online ensembles with differing drift adaptation and diversity mechanisms.SRP uses random feature and instance subspaces; HAT adds concept-drift adaptation to HT.Source: §III · pp. 2–3
• The attack search permits unit-cost feature modifications, but does not enforce categorical or integer validity, inter-feature consistency, or explicit utility preservation.The authors characterize this as a cost-limited feature-space enhancement rather than a complete utility-preserving traffic alteration.Source: §III · p. 3
• A limited attack search serves as a computational-feasibility benchmark rather than a comprehensive assessment; finalized models are also tested on records previously used for learning.Drift assessments compare EDDM, ADWIN, and DDM, with reported drift figures focused on EDDM because it produced clearer signals.Source: §III · p. 3
• The black-box attack queries predictions and perturbs test inputs at inference time under budgets from 0.00 to 1.00.The adversary does not access model parameters or alter training data.Source: §III · p. 3
• Adversarial training exposes learners to perturbed samples during training so they can adjust decision boundaries while evaluating detection under cost-constrained manipulation.Source: §III · p. 3

• Clean accuracy (CAcc) measures performance on unaltered test data, while adversarial accuracy (AdAcc) measures robustness to misclassification-inducing perturbations.Source: §IV · p. 3
• Reporting both CAcc and AdAcc captures the trade-off between standard prediction performance and adversarial robustness.Source: §IV · p. 3
• Before the cost-budget analysis, the pipeline equipped with adversarial training was first evaluated using accuracy.Source: §IV · p. 3

A. Impact of Evasion Attacks

• At cost budget 0.00, CAcc equaled AdAcc for every model in the study.For AT-HT, AT-LB, AT-SRP, AT-HAT, and AT-ARF, the supplied results also report matching CAcc and AdAcc at budget 0.05.Source: A. Impact of Evasion Attacks · pp. 3–4
• At budget 1.00, AT-LB and AT-SRP each reached AdAcc 98.5% and CAcc 99.3%, with ΔAcc 0.008.Table I is identified as comparing CAcc and AdAcc across AT models under different cost budgets.Source: Table I · p. 4, A. Impact of Evasion Attacks · p. 4
• At budget 1.00, AdAcc was 96.3% for AT-HT, 93.3% for AT-HAT, and 95.7% for AT-ARF.Each model’s accuracy decline began at cost budget 0.10 and remained stable at higher budgets.Source: A. Impact of Evasion Attacks · p. 4
• The accuracy plateau from budget 0.10 onward reflects repeated altered inputs in this deterministic evaluation, not proven resilience to stronger multi-feature attacks.The authors characterize the investigation as a computationally efficient foundation rather than a comprehensive attack evaluation.Source: A. Impact of Evasion Attacks · p. 4

B. Drift Detection using EDDM

• EDDM detects drift by observing increases in the distance between classification errors; it produced more drift occurrences than ADWIN and DDM in these experiments.The study used EDDM to compare drift sensitivity in naive and adversarially trained models.Source: B. Drift Detection using EDDM · p. 4
• For LB and SRP, adversarially trained models had fewer EDDM detections but lower or more variable online rolling accuracy than their naive counterparts.Naive-SRP had three detected drifts during the first 500 test samples, compared with two for AT-SRP.Source: B. Drift Detection using EDDM · pp. 4–5, Figure 2 · p. 5
• EDDM values fell for HT and LB but stayed constant for SRP and ARF; the varying detector responses did not establish improved drift resilience.ADWIN values dropped for HT and LB, stayed constant for SRP, and rose for ARF; DDM counts rose for LB, SRP, and ARF.Source: B. Drift Detection using EDDM · p. 5

V. DISCUSSION

• At cost budget 1.00, AT-LB and AT-SRP achieve the highest AdAcc (0.985) and CAcc (0.993), with only a 0.008 difference between them.The passage also reports a maximum accuracy decline of 0.8%.Source: §V · p. 5
• Naive LB has higher CAcc than naive SRP, 0.996 versus 0.994, despite SRP's smaller adversarial-accuracy decline.The comparison is between naive implementations.Source: §V · pp. 5–6, Table II · p. 6, Table III · p. 6
• At budgets of 0.10 and higher, naive LB has AdAcc 0.834 and ∆Acc 0.162, while naive SRP has AdAcc 0.914 and ∆Acc 0.080.The tables cover naive-LB clean and adversarial accuracies and naive-SRP CAcc, AdAcc, and their difference.Source: §V · p. 6, Table II · p. 6, Table III · p. 6

VI. CONCLUSION

• At cost budget 1.00, AT-LB and AT-SRP achieve the highest AdAcc (0.985) and CAcc (0.993), with an accuracy reduction of only 0.8%.The study reports both clean and adversarial accuracy to evaluate performance trade-offs under attack.Source: §VI · p. 6
• Among naive implementations, SRP performs better than LB under adversarial perturbations, suggesting SRP is preferable when resilience without AT matters.Source: §VI · p. 6
• Naive models often have high clean accuracy but larger drops under adversarial samples, while AT models reduce the clean-to-adversarial performance gap.Source: §VI · p. 6
• EDDM is used to detect drift in naive and AT versions of the best-performing models; future work will add datasets, repeated runs, and runtime/resource analysis.Source: §VI · p. 6