PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial RL for Port-Scan Evasion: Attacker Feature Visibility in Edge-Deployed IDS

Logan Andrew North, Priya Sanjay Kaluskar, Shasi Kumar Ramachandran Prabhu, Peilong Li, Suman Saha

arXiv:2610.08864v1cs.CRcs.LG

TL;DR

This paper addresses limited evidence about whether edge-deployed IDS withstand attackers adapting to detector feedback. It evaluates DQN-based port-scan evasion in a live Raspberry Pi pipeline across three models and visibility settings, finding high but model-dependent evasion that does not rise monotonically with feature visibility.

  • Problem

    IDS robustness is commonly evaluated against static attacks, leaving adaptive evasion against resource-constrained, edge-deployed IDS insufficiently understood.

    Source: §I · p. 1

  • Method

    A DQN agent learns port-scan probe timing, TCP flags, and payload size against three IDS models in a live Raspberry Pi 3B+ pipeline under black-, gray-, and white-box visibility.

    Source: §I · p. 1

  • Results

    61.9%–98.3% final-50-episode evasion rates were observed, with visibility effects non-monotonic and model-dependent: XGBoost was most vulnerable black-box, while 1D-CNN was most vulnerable white-box.

    Source: B. Adversarial Evasion Results · p. 4, §I · p. 8, §VI · p. 9

  • Takeaways & Limitations

    Limited feature visibility can still support effective adaptive evasion through repeated interaction with a static detector.

    Source: §I · p. 9, §VI · p. 9

    Experiments cover three IDS models on a Raspberry Pi 3B+ using TON IoT-derived features, and the scanning-class objective should not be assumed to transfer to other attack types.

    Source: §V · p. 9

Abstract

from arXiv · show

Machine learning-based intrusion detection systems (IDS) are increasingly used in resource-constrained Internet of Things (IoT) environments, yet their robustness is often evaluated against static attacks rather than adversaries that adapt to detection feedback. This paper investigates adaptive port-scan evasion against ML-based IDS models deployed on a Raspberry Pi 3B+. We implement a live Zeek-based IDS pipeline with XGBoost, a multi-layer perceptron, and a 1D convolutional neural network trained on TON_IoT telemetry, and use a Deep Q-Network (DQN) adversary to learn evasive combinations of probe timing, TCP flags, and payload size under black-box, gray-box, and white-box feature-visibility settings. Although the deployed IDS models detect conventional port scans at 91.1--99.8%, DQN final-50-episode evasion rates range from 61.9% to 98.3% across feature-visibility settings. Greater feature visibility does not monotonically improve evasion, and its effect is model-dependent: against XGBoost, the black-box agent achieves 92.9% evasion, compared with 61.9% and 76.9% for gray-box and white-box agents, respectively, whereas 1D-CNN is most vulnerable under white-box access at 98.1%. Because standard DQN can overestimate action values, we additionally spot-check representative conditions using Double DQN. The gray-box condition remains unstable in this check, providing no evidence that overestimation bias alone explains the observed instability. These results show that limited feature knowledge can still enable effective adaptive evasion against static edge-deployed IDS models, motivating more robust defenses for IoT edge environments.

I. INTRODUCTION

Edge-deployed IDS research has established the feasibility of lightweight detection, but its robustness against attackers adapting through detector feedback remains insufficiently understood. This study evaluates that gap with a live Raspberry Pi pipeline and finds substantial, model-dependent adaptive evasion, while stopping short of testing defenses.

I. INTRODUCTION

  • The study tests DQN-learned combinations of probe timing, TCP flags, and payload size against three IDS models on a live Raspberry Pi 3B+ pipeline.The models are XGBoost, an MLP, and a 1D-CNN trained on TON IoT telemetry and monitored with Zeek.Source: §I · p. 1
  • Final-50-episode evasion rates span 61.9%–98.3%, with XGBoost most vulnerable black-box and 1D-CNN most vulnerable white-box.MLP remains highly vulnerable across visibility settings; added feature visibility does not monotonically improve evasion.Source: §I · p. 1
  • The paper characterizes vulnerability rather than evaluating defenses; it proposes or tests no countermeasures.Adaptive retraining, threshold randomization, and co-adaptive evaluation are discussed as future work.Source: §I · p. 2

A. Intrusion Detection on Resource-Constrained Devices

  • Prior edge IDS work established lightweight deployment feasibility, but did not examine detectors responding to attackers adapting through repeated feedback.The gap is especially relevant to port scanning, where probing behavior can vary over time.Source: §I · p. 1, A. Intrusion Detection on… · p. 2

B. Adversarial Attacks Against ML-Based Network IDS

Prior work documents adversarial vulnerabilities and realistic attacker assumptions, while RL-based detector evasion has also been studied for botnet samples. This paper instead examines adaptive port scans through live edge-deployed IDS interaction and compares attacker feature visibility.

B. Adversarial Attacks Against ML-Based Network IDS

  • Prior work highlights practical black-box and gray-box threat models, functionality-preserving attacks, and adversarial network-traffic validity.The cited literature also reports ML-based network IDS vulnerability to adversarial manipulation.Source: B. Adversarial Attacks Against… · p. 2

III. SYSTEM AND METHODOLOGY

  • The evaluation uses RL to adapt port-scanning behavior from feedback in a live edge-device pipeline, rather than relying only on offline classifiers or fixed adversarial examples.The IDS remains fixed while the agent updates its policy over delay, TCP flag, and payload size.Source: B. Adversarial Attacks Against… · p. 2, §III · p. 2

C. Reinforcement Learning for IDS Evasion and Defense

  • DReLAB applies deep RL to botnet evasion and benchmark-data generation; this study focuses on interactive port-scan evasion on Raspberry Pi and feature-visibility settings.DReLAB includes DDQN and Deep Sarsa.Source: C. Reinforcement Learning for IDS… · p. 2
  • The study characterizes edge-deployed IDS vulnerability to adaptive attackers rather than proposing a new defense.Prior work has explored detector hardening and adversarial training.Source: C. Reinforcement Learning for IDS… · p. 2

A. System Overview

The system tests whether a fixed edge-deployed IDS withstands an attacker learning from repeated interaction. It uses a shared 154-dimensional feature representation, with temporal aggregates, for offline training and live inference across three model architectures.

A. System Overview

  • The deployed IDS stays fixed while the attacker learns from its predictions returned through a reward interface.The Pi's resource constraint applies to IDS inference, not to the learning process.Source: A. System Overview · p. 2

B. Dataset and Feature Construction

  • TON IoT telemetry is transformed into a 154-dimensional representation with per-source sliding-window aggregates over 3/5/10/30/60-second intervals.The random stratified split can place temporally adjacent rows with overlapping windows in different partitions, so offline F1 is treated as an upper bound.Source: B. Dataset and Feature Construction · pp. 2–3

C. Edge IDS Models

  • XGBoost, a 1D-CNN, and an MLP share the same representation and are all deployed for live Raspberry Pi inference.The models use class weighting and the neural models use early stopping; Table I summarizes configurations.Source: C. Edge IDS Models · p. 3, Table I · p. 3

D. Live IDS Deployment

The live inference pipeline reconstructs IDS features from Zeek traffic and returns probe classifications to a DQN agent. The agent's sequential MDP uses visibility-dependent observations, discrete probe actions, live transitions, and rewards based on classification and delay.

D. Live IDS Deployment

  • Live Zeek inference reconstructs the same 154-dimensional representation, and source-IP and source-port matching routes the agent's probe predictions to its reward socket.Background traffic is classified but does not contribute to the agent's reward.Source: D. Live IDS Deployment · p. 3

E. RL-Based Port Scanning Agent

  • The MDP uses 154-dimensional observations that are all-zero, partially masked, or complete according to the feature-visibility setting.Each probe and response updates sliding-window features that affect subsequent observations.Source: E. RL-Based Port Scanning Agent · p. 3
  • The agent selects among 1,250 combinations of inter-probe delay, TCP flag, and payload size.Delays span 0.1–5.0 seconds; the flag and payload choices are specified in the action space.Source: E. RL-Based Port Scanning Agent · p. 3
  • +1 minus a delay penalty rewards probes classified normal, −1 applies to probes classified scanning, and 0 applies otherwise.Episodes last 20 probes, with discount factor 0.99; the maximum delay is 5.0 seconds.Source: E. RL-Based Port Scanning Agent · p. 3

F. Reward Sensitivity Analysis

The study selects a reward weight based on convergence and breakeven behavior, then compares three levels of attacker access to the IDS feature representation. The same DQN architecture and 10,000-timestep budget are used across target-model and visibility combinations.

F. Reward Sensitivity Analysis

  • At ω = 1.0, maximum-delay stalling yields zero reward; ω = 1.0 and ω = 2.0 converge faster and more stably than ω = 0.5 with comparable final evasion, so ω = 1.0 is used.The sensitivity runs use XGBoost black-box access, 300 episodes, and 6,000 timesteps each; the main experiments use 10,000 timesteps (500 episodes) per target-model and visibility setting.Source: F. Reward Sensitivity Analysis · p. 3

G. Attacker Feature Visibility

  • The visibility settings vary access to the IDS feature representation, not access to target-model parameters or architecture.All combinations use the same DQN architecture and 10,000-timestep main training budget.Source: G. Attacker Feature Visibility · pp. 3–4
  • Black-box observations are all-zero at every timestep, making the agent’s policy stateless while the IDS environment still evolves through sliding-window statistics.The setting is treated as bandit-like from the agent’s perspective, while retaining DQN for consistency across settings.Source: G. Attacker Feature Visibility · p. 3
  • Gray-box agents observe only the top 20 features, while white-box agents receive the complete 154-dimensional feature vector.Top-20 features are selected per model using XGBoost gain importance or mean absolute SHAP values for the 1D-CNN and MLP.Source: G. Attacker Feature Visibility · pp. 3–4

IV. EXPERIMENTAL RESULTS

All three deployed models detect conventional port scans effectively, but adaptive DQN evasion varies by model and feature visibility. XGBoost is most vulnerable in black-box access, whereas 1D-CNN is most vulnerable in white-box access; gray-box XGBoost also shows substantial instability.

A. Baseline IDS Performance

  • 91.1–99.8% live detection rates show that all three models detect conventional port scans effectively on the Raspberry Pi 3B+ deployment.Measured throughput is 10.7–24.4 flows/s and model sizes range from 52.8 KB to 14.5 MB.Source: A. Baseline IDS Performance · p. 4

B. Adversarial Evasion Results

  • 92.9% ± 2.8% XGBoost black-box evasion contrasts with 61.9% ± 19.0% gray-box; 1D-CNN peaks at 98.1% ± 0.4% white-box, while MLP stays 96.3–98.3%.Greater feature visibility does not monotonically improve evasion; Table IV reports average last-50-episode values as mean ± standard deviation across available seeds.Source: Table IV · p. 4, B. Adversarial Evasion Results · p. 4
  • XGBoost gray-box has the largest cross-seed instability among the reported convergence trajectories.Figure 2 shows trajectories across available seeds.Source: B. Adversarial Evasion Results · p. 4

C. Learned Evasion Strategies

Black-box DQN runs converge on shared probe strategies across the three models, while richer feature visibility yields model-dependent strategies. Some high-evasion flags do not retain observed reconnaissance value, and XGBoost’s shift toward SYN-ACK under richer visibility coincides with lower evasion and no observed port-state discrimination.

C. Learned Evasion Strategies

  • Figure 2 shows evasion-rate convergence across target models and attacker feature-visibility settings, using a 20-episode moving average and available seeds to characterize the plotted trajectories.The dashed black line shows the mean across available seeds when multiple runs are present.Source: Figure 2 · p. 5
  • In the black-box condition, DQN does not diverge across target models, unlike the bandit; accordingly, black-box dominant strategies are reported as a single, largely shared action sequence evaluated against three classifiers.Source: C. Learned Evasion Strategies · p. 5
  • With richer visibility, XGBoost favors SYN-ACK in gray-box and white-box (73%, 61%), 1D-CNN favors it in white-box (72%), and MLP favors it in gray-box but reverts to SYN in white-box.Gray-box strategies diverge across models at 10–12% action agreement, making it the strongest evidence here for independent, model-specific learning.Source: C. Learned Evasion Strategies · p. 5
  • SYN-ACK probes return a uniform RST across 5,940 instances regardless of logged port state, providing no evidence of retained reconnaissance utility.By contrast, SYN, FIN, NULL, and Xmas show differential responses associated with port state; NULL’s bandit strategy is only plausibly reconnaissance-retaining because its own response fields were not logged.Source: C. Learned Evasion Strategies · p. 6
  • For XGBoost, shifting from SYN toward SYN-ACK under richer visibility coincides with lower evasion and no observed port-state discrimination; this trade-off does not hold for 1D-CNN or MLP.The port-state conclusions rely on the environment’s logged field rather than independently audited ground truth for every probe.Source: C. Learned Evasion Strategies · pp. 5–6

D. DQN versus a Multi-Armed Bandit Baseline

A stateless ϵ-greedy bandit provides a baseline against DQN in the zero-observation setting. It achieves high evasion in a single-seed run and diverges across models, unlike the largely shared black-box DQN action sequence.

D. DQN versus a Multi-Armed Bandit Baseline

  • The bandit treats each of 1,250 actions as an independent arm, updates its value by incremental mean, and selects the highest-value arm except during ϵ-greedy exploration.It ignores observations and uses no replay; ϵ decays from 1.0 to 0.05 over the first 30% of timesteps.Source: D. DQN versus a Multi-Armed Bandit… · p. 6
  • 97.4% (XGBoost), 99.8% (1D-CNN), and 98.1% (MLP) are the bandit’s final-50-episode evasion rates in single-seed runs, not statistically comparable to multiseed DQN figures.The bandit favors NULL for XGBoost and MLP and SYN-ACK for 1D-CNN, whereas DQN favors SYN across all three black-box targets.Source: D. DQN versus a Multi-Armed Bandit… · p. 6
  • Under the shared-seed setup, bandit runs diverge across models with 8.5% action agreement, unlike the evaluated DQN configuration’s shared black-box behavior.The comparison does not isolate whether update rule, bootstrapping, function approximation, or another difference explains the contrast.Source: D. DQN versus a Multi-Armed Bandit… · p. 6
  • The ablation shows DQN is not necessary for high evasion in the zero-observation setting, but the authors retain DQN for a common learning framework across visibility settings.The authors do not claim to have isolated which learner difference causes their observed divergence patterns.Source: D. DQN versus a Multi-Armed Bandit… · p. 6

E. Convergence Stability and Seed Sensitivity

XGBoost gray-box learning is unusually seed-sensitive, and a single-seed Double DQN check did not resolve its instability. The check leaves open whether an unfavorable seed or changing masked observations explains the behavior.

E. Convergence Stability and Seed Sensitivity

  • XGBoost gray-box had the largest cross-seed variability (19.0%; range 37.8–84.2%), while all remaining conditions were at or below 3.9%.Its seeds followed divergent trajectories: seed 42 stabilized above 80% by episode 100, while seeds 67 and 320 degraded for extended periods before partial recovery.Source: E. Convergence Stability and Seed… · p. 6

F. Double DQN Spot Check: Is Overestimation Bias a Factor?

  • Under Double DQN, XGBoost gray-box evasion fell to 46.2% from 84.2% under standard DQN and oscillated rather than plateauing.The authors interpret this as evidence against overestimation bias being the dominant explanation, not as a controlled isolation of that bias.Source: F. Double DQN Spot Check: Is… · p. 7
  • Double DQN’s gray-box policy became less decisive: SYN-ACK use fell to 36.5%, and no delay value exceeded 10.2%.The authors suggest shifting masked observations as a candidate explanation, but do not isolate the mechanism.Source: F. Double DQN Spot Check: Is… · p. 7
  • The gray-box check used one Double DQN seed, so an unfavorable trajectory remains possible and the comparison does not isolate overestimation bias.The authors leave a multiseed Double DQN comparison to future work.Source: F. Double DQN Spot Check: Is… · p. 7

G. Time-to-First-Evasion

First-step evasion was frequent for MLP and 1D-CNN, while XGBoost’s timing varied more by feature-visibility setting. Transfer results also depended strongly on the source and target IDS models.

G. Time-to-First-Evasion

  • MLP reached step-1 evasion in 95.8–97.7% of episodes across settings, and 1D-CNN did so in 79.8–90.9%.XGBoost step-1 rates were 83.2% black-box, 46.4% gray-box, and 16.9% white-box; its mean first-evasion steps were 1.32, 2.72, and 2.33, respectively.Source: G. Time-to-First-Evasion · p. 7, Figure 6 · p. 8

H. Cross-Model Transferability

  • The transfer evaluation covered a small set of source–target pairs and seed-42 policies, using the destination model’s top-20 feature mask for gray-box tests.The two gray-box masks shared only 6 of 20 indices.Source: H. Cross-Model Transferability · p. 8
  • XGBoost policies achieved 99.8%, 99.8%, and 96.8% evasion against 1D-CNN, while the 1D-CNN white-box policy achieved 65.0% and the MLP gray-box policy 0.4% against XGBoost.The transfer test used five frozen seed-42 policies; the XGBoost black-box result carries a shared-seed caveat.Source: H. Cross-Model Transferability · p. 8

I. Discussion: Model-Dependent Effects of Attacker Feature Visibility

Feature visibility affected evasion differently across IDS models, and the conclusion reports evasion rates of 61.9%–98.3% despite strong conventional scan detection. The experiments are limited to specific models, data, edge hardware, and port scanning.

I. Discussion: Model-Dependent Effects of Attacker Feature Visibility

  • In a seed-42 run, XGBoost shifted from SYN under black-box to SYN-ACK under gray- and white-box access, though this is a single-run illustration.The reward did not explicitly favor particular flags.Source: §I · p. 8
  • The findings caution against assuming limited feature access prevents adaptive evasion; the paper recommends evaluating adaptive attacker–detector interaction alongside conventional detection.The conclusion’s scope is evasion against a static detector.Source: §I · p. 9, §VI · p. 9

V. LIMITATIONS AND FUTURE WORK

  • The study covers three IDS models on Raspberry Pi 3B+ using TON_IoT-derived features, and targets port scanning rather than other attack classes.Reconnaissance validity relies mainly on logged fields; only one instance was directly cross-checked against the live host.Source: §V · p. 9
  • Future work should test adaptive defenses and co-adaptive attacker–defender experiments; transfer tests also need broader model pairs, seeds, datasets, attack classes, and platforms.Suggested defenses include threshold randomization and adaptive or periodic retraining.Source: §V · p. 9

VI. CONCLUSION

  • Evasion rates ranged from 61.9% to 98.3%; XGBoost was most vulnerable black-box, 1D-CNN white-box, and MLP remained highly evadable across settings.More feature information did not necessarily produce stronger evasion.Source: §I · p. 8, §VI · p. 9
Loading 2610.08864v1…