A Feature-Rich Embedded NIDS with eBPF/XDP: Detector and Architecture Trade-offs

1 Introduction

• DDoS volumes exceed 30 Tbps, telecommunications networks are a primary target, and NIDS feature sets and software architecture remain underexamined.The paper identifies packet-count-only detection and the limited study of architecture alongside detection effectiveness as two key challenges.Source: §1 · p. 2

1.1 Research questions

• The study asks how eBPF filtering and adaptive machine learning can improve transport-network DDoS detection, and how monolithic, Kafka, and gRPC architectures affect NIDS performance.Source: §1.1 · p. 2
• The proposed NIDS combines eBPF/XDP filtering with an adaptive Isolation Forest trained on GoFlowMeter flow features and compares three deployments on CIC-DDoS2019 using a Raspberry Pi 5.Source: §1.1 · p. 2

1.2 Contribution

• The system filters real interface traffic with eBPF/XDP, compares three architectures, and uses GoFlowMeter’s 80-feature flow vectors to fine-tune Isolation Forest.GoFlowMeter is an open-source Go reimplementation of CICFlowMeter; a standalone sequential replayer sends more than 100 GB of CIC-DDoS2019 traffic over a real network interface.Source: §1.2 · p. 3

1.3 Paper outline

• The paper’s remaining sections cover background, related work, the system and experimental setup, results, discussion, and conclusion.Source: §1.3 · p. 3

2.1 DDoS Attacks and the CIC-DDoS2019 Dataset

• CIC-DDoS2019 contains two days of labelled reflection and exploitation attacks with benign traffic, but its traffic is synthetically generated.The dataset includes 80-feature flow analysis extracted by CICFlowMeter.Source: §2.1 · pp. 3–4

2.2 Intrusion Detection Systems

• A NIDS inspects traffic at network vantage points, where aggregated DDoS signals can be detected upstream and linked to ingress mitigation.Source: §2.2 · p. 4

Network Flows and Sliding Time Windows

• A flow groups packets sharing a 5-tuple, while fixed sliding windows are classified independently and spanning flows contribute a sub-flow to each window.Source: Network Flows and Sliding Time Windows · p. 4

CICFlowMeter

• CICFlowMeter assembles packets into bidirectional flows and extracts 80 statistical features that define this paper’s flow-feature space.Source: CICFlowMeter · p. 4

2.3 Isolation Forest

• Isolation Forest is an unsupervised anomaly detector that isolates points through random partitions; shorter average tree paths indicate higher anomaly likelihood.It needs no labelled training data and is lightweight, with the paper noting its suitability for zero-day attacks absent from training data.Source: §2.3 · pp. 4–5

2.4 Fast filtering with eBPF/XDP

• eBPF/XDP runs programs at kernel events and hooks them into the network driver, enabling packets to be dropped, passed, or redirected before memory allocation.The passage reports processing rates exceeding 20 million packets per second.Source: §2.4 · p. 5

2.5 Microservice Communication Frameworks

• Microservices communicate through asynchronous Kafka messaging or synchronous gRPC calls, trading buffering and persistence against broker overhead and request-response latency.Kafka adds serialisation, broker round-trips, and consumer fetch cycles; gRPC avoids broker persistence and buffering.Source: §2.5 · p. 5, Apache Kafka · p. 5

3.1 NIDS for DDoS Defence

• Wickman and Rygaard’s single-feature Isolation Forest reached 73.63% detection accuracy versus 96.09% for their statistical baseline, which this study adopts as its baseline.Their detector used packet counts per time window, motivating the present study’s use of more features.Source: §3.1 · p. 6
• Supervised CNN, LSTM, and RF studies report high detection results, but require labelled data; Bhawsar et al.’s RF reached 99.92% accuracy and took around ten times longer than SVM.The reviewed CNN and recurrent approaches report results on simulated or standard datasets, while the text notes that CNN reshaping can impose artificial spatial structure on sequential traffic.Source: §3.1 · pp. 6–7
• Unsupervised approaches, including an autoencoder-XGBoost scheme and Isolation Forest, model normal traffic or isolate deviations without prior attack labels.Source: §3.1 · p. 7

3.2 Fast Packet Processing with eBPF/XDP

• A prior eBPF/XDP NIDS evaluated multiple classifiers; its BiLSTM achieved 99.3% accuracy, 99.3% F1, and 99.9% ROC AUC on a generated dataset.That framework extracted CICFlowMeter-V4.0 features and selected its top 20 with SelectKBest.Source: §3.2 · p. 7

3.3 Gap Analysis

• The identified gap is an unsupervised detector using multiple network features, evaluated with operational performance in view because labels are difficult to obtain and computational cost and latency matter.Frequent network changes and emerging attack vectors make static labelled data impractical in the target operational context.Source: §3.3 · pp. 7–8

Sequential Replayer

• The sequential replayer streams CIC-DDoS2019 over a real interface, preloading files and preserving capture timing to limit memory pressure and avoid replay pauses.A warmup filter forwards known-benign traffic during startup, and a frame rewriter redirects frames to the victim interface while preserving IP addresses and ports.Source: Sequential Replayer · p. 8, §4.1 · p. 8

Network Feature Analysis Using GoFlowMeter

• GoFlowMeter provides a native Go implementation of CICFlowMeter’s 80 statistical features, keeping its output compatible with existing datasets and models.It was developed because the Java implementation could not be embedded naturally in the Go-based NIDS.Source: Network Feature Analysis Using… · p. 9
• For each window, the engine averages the 80-dimensional feature vectors of the bidirectional flows observed during that window.Packets buffered during the completed window are assembled into flows before their features are computed.Source: Network Feature Analysis Using… · p. 9
• Non-finite feature components become zero, empty windows use an 80-dimensional zero vector, and per-IP vectors average only that IP’s flows.Source: Network Feature Analysis Using… · p. 9

• Flow labels are merged into malicious time intervals, and a time duration is malicious when any flow within it is malicious.Source: Ground Truth Determination · p. 9
• Packets are labeled malicious when they fall within a malicious interval and normal otherwise.Source: Ground Truth Determination · p. 9
• A sliding window is malicious if it overlaps any malicious interval; only a window entirely within a normal period is labeled normal.Figure 1 illustrates malicious windows WM1–WM6 and normal window WN1; partial attack overlap is treated as malicious.Source: Ground Truth Determination · p. 9, Figure 1 · p. 10

4.2 Fine-Tuning of Isolation Forest

• The offline probe parses each PCAP once into one-second, warmup-filtered windows and caches packet counts and aggregated feature vectors for hyperparameter selection.Source: §4.2 · p. 10
• The grid varies tree count, sample size, anomaly fraction, and count-only versus full 80-dimensional input; its top-ranked configuration is used in all live deployments.Source: §4.2 · p. 10

4.3 Software Architecture

• The engine coordinates capture, window management, detection, online updates, and output in one process, while architectures differ only in engine-to-detector communication.The monolithic engine consumes extracted features directly in memory without serialization.Source: §4.3 · p. 10

Monolithic Architecture

• The monolithic NIDS integrates ingestion, aggregation, detection, mitigation, logging, and visualization in one Go process, providing a baseline with in-memory communication.The design minimizes orchestration overhead.Source: Monolithic Architecture · p. 10
• The monolithic pipeline classifies completed windows and per-IP behavior, supports IQR thresholds or Isolation Forest modes, and updates its model online with non-malicious observations.During detected attacks, the per-IP threshold is frozen to avoid instability.Source: Monolithic Architecture · p. 11

Microservice Architectures

• Kafka places a broker between feature extraction and detection: the engine publishes serialized windows, and the detector returns outcomes asynchronously through topics.The downstream detection behavior is otherwise the same as in the monolithic version.Source: Microservice Architectures · p. 12
• gRPC uses a synchronous unary RPC between the engine client and detector server, returning the decision and related detection outputs in the same call.All variants share the same detector code; Figure 2 depicts the three architectures and their differing transports.Source: Figure 2 · p. 11, Microservice Architectures · p. 12

4.4 eBPF/XDP

• The system captures and filters real traffic through eBPF/XDP, unlike the previous NIDS, where replay and processing ran inside one program without traffic crossing a network interface.The existing rate limiter remains the indicator of malicious sources.Source: §4.4 · p. 12

Kernel Integration and Filtering Logic

• At each window's end, the Isolation Forest sets a rate threshold; sources exceeding it enter the BPF blocklist and their subsequent traffic is dropped by the kernel.The XDP hook checks each arriving packet's source address: blocklist hits trigger XDP_DROP, while misses pass packet metadata to user space.Source: Kernel Integration and Filtering Logic · pp. 12–13, §4.4 · p. 12

4.5 Experimental Setup and Evaluation

• All experiments use the same Raspberry Pi 5, with processes communicating over loopback to isolate framework overhead from external network conditions.The board has a 2.4 GHz quad-core Arm Cortex-A76 CPU and 16 GB RAM.Source: §4.5 · p. 13

Evaluation Metrics

• Detection quality is measured using accuracy, precision, recall, and F1 over window-level ground truth.Source: Evaluation Metrics · p. 13
• Software performance is assessed through processing, detector, and transport time, plus engine memory consumption.Metric definitions depend on the software architecture.Source: Evaluation Metrics · p. 13

Overview of Experiments

• The experiments tune Isolation Forest hyperparameters, compare baseline and Isolation Forest detection across three architectures, and compare time and memory across six architecture–detector combinations.Source: Overview of Experiments · p. 13

5.1 Hyperparameter Tuning Results

• With anomaly_fraction 0.2, all nine feature-vector configurations tie at mean F1 0.932 and accuracy 0.873; the best packet-count configuration reaches F1 0.929 and accuracy 0.868.The anomaly fraction dominates, while num_trees and sample_size have almost no measurable effect once it is set appropriately; the selected 200-tree, sample-size-128 configuration is fixed for later experiments.Source: §5.1 · p. 14, Table 1 · p. 14

5.2 Detection Accuracy Results

• The Isolation Forest variants outperform the baseline on every detection metric except precision, where both are near perfect, across 1,533 windows.Source: §5.2 · p. 14, Table 2 · p. 15

5.3 Software Performance Results

• gRPC transport stays below 2 ms on average, while Kafka transport exceeds 27 ms, an order-of-magnitude difference.Processing time excludes packet capture and window accumulation, and is post-window detector-side latency rather than full end-to-end NIDS latency; monolithic timing also includes feature extraction, unlike Kafka and gRPC.Source: §5.3 · p. 15, Table 3 · p. 16
• Kafka's engine consumes the most memory, gRPC the least, and monolithic falls between them; engine memory remains far below the platform's 16 GB.The reported heap allocation covers only the engine process, excluding the separate detector and, for Kafka, the JVM broker.Source: §5.3 · pp. 15–16, Table 4 · p. 16

6.1 Detection Accuracy

• With a shared 75-window warm-up, all Isolation Forest variants outperform the tightly clustered baselines; precision remains 0.996–0.999 while recall and F1 rise.The baseline's precision of 1 accompanies low recall and missed low-volume attack windows.Source: §6.1 · pp. 16–17

6.2 Software Performance Comparison

• gRPC averages less than 2 milliseconds of transport time, compared with more than 27 ms for Kafka on the same Raspberry Pi 5 host.The paper attributes the difference to Kafka's batching, broker log and acknowledgement, and consumer-fetch mechanisms; gRPC avoids these mechanisms.Source: §6.2 · p. 17

6.3 Switching from Baseline to Isolation Forest

• Switching from baseline to Isolation Forest raises accuracy but degrades processing time across all three architectures.Figure 4 identifies Kafka as having the largest processing-time degradation and monolithic as having the smallest.Source: Figure 4 · p. 18, §6.3 · p. 18

6.4 Effectiveness of Sequential Replayer

• The independent replayer reached an average 3 Gbps without artificial inter-packet delays, roughly three times CIC-DDoS2019's peak attack rate, so it did not constrain evaluation.In this setup, the replayer's speed bounds the attack.Source: §6.4 · p. 19

Feature Selection

• Only packet count and the full 80-dimensional vector were evaluated; Isolation Forest's sensitivity to irrelevant or redundant dimensions leaves smaller, informative feature subsets unexplored.Feature selection could also reduce scoring, serialisation, and transport costs in microservice variants.Source: Feature Selection · p. 19

Validity Threats

• Results are bounded by a non-isolated Linux system, one Raspberry Pi 5 configuration, synthetic 2019 CIC-DDoS2019 data, and hyperparameters tuned and evaluated on that dataset.Window labels may hide short bursts; mitigation is per-IP and may be weaker against attacks distributed across many low-rate sources.Source: Validity Threats · p. 19

6.6 Future Work

• Future work proposes feature selection, evaluating sequence models on constrained hardware, and moving eBPF/XDP blocks to time-bounded mitigation.The proposed time-bounded blocks would allow falsely flagged addresses to regain access.Source: §6.6 · p. 19

6.7 Ethics

• The evaluation used only the public, synthetically generated CIC-DDoS2019 dataset, confined replay to an isolated testbed, and made no attempt to identify represented hosts or individuals.The paper describes its purpose as defensive and its released GoFlowMeter artefact as a general-purpose flow-analysis tool.Source: §6.7 · p. 20

7 Conclusion

• The NIDS extension combines eBPF/XDP integration, GoFlowMeter feature extraction, a standalone attacker program, a fine-tuned Isolation Forest, and three deployment architectures.The research addresses detection effectiveness and the effect of architecture on performance.Source: §7 · p. 20
• 0.9335 live accuracy and 0.9650 F1: the fine-tuned Isolation Forest improves on the statistical baseline, whose recall suffers on low-volume attack windows.The model uses an 80-dimensional feature vector, 200 trees, sample size 128, and anomaly fraction 0.2.Source: §7 · p. 20
• Below 2 ms average transport time lets gRPC preserve near-monolithic detection quality, whereas Kafka adds over 27 ms and a measurable detection penalty.The monolithic deployment remains most favourable on this dataset, though the workload may not expose single-process bottlenecks.Source: §7 · p. 20
• Slow-growing attack detection remains to be demonstrated on datasets containing more such attacks; future work includes feature selection, LSTM evaluation, and time-bounded eBPF/XDP blocking.The proposed future evaluation of sequence models would use the same constrained hardware.Source: §7 · p. 20