Source-linked AI summary
A Cyber Threat Intelligence Sharing Scheme based on Federated Learning for Network Intrusion Detection
Mohanad Sarhan, Siamak Layeghy, Nour Moustafa, Marius Portmann
TL;DR
Organisations need NIDS models that can learn across heterogeneous network traffic, but direct data sharing raises privacy concerns and local training has limited exposure. The paper proposes federated CTI sharing with local training and aggregation of model updates; it reports a robust model for classifying traffic from multiple organisations without exchanging local data.
Problem
NIDS trained on individual networks may not generalise across distinct non-IID networks, while centralised learning requires sharing sensitive network samples.
Source: §1 · p. 2, §3.1 · p. 4Method
Results
The paper reports a robust NIDS capable of classifying benign and intrusive traffic from multiple organisations without exchanging local data.
Source: §1 · p. 3, §5 · p. 12Takeaways & Limitations
The authors conclude that organisations can collaborate on NIDS using network intelligence from multiple sources while keeping their local data private.
Source: §1 · p. 3The evaluation uses two datasets collected from different networks, so its scope is limited to those datasets and their shared NetFlow v9 feature set.
Source: §4 · pp. 6–7
Abstract
from arXiv · showhide
The uses of Machine Learning (ML) in detection of network attacks have been effective when designed and evaluated in a single organisation. However, it has been very challenging to design an ML-based detection system by utilising heterogeneous network data samples originating from several sources. This is mainly due to privacy concerns and the lack of a universal format of datasets. In this paper, we propose a collaborative federated learning scheme to address these issues. The proposed framework allows multiple organisations to join forces in the design, training, and evaluation of a robust ML-based network intrusion detection system. The threat intelligence scheme utilises two critical aspects for its application; the availability of network data traffic in a common format to allow for the extraction of meaningful patterns across data sources. Secondly, the adoption of a federated learning mechanism to avoid the necessity of sharing sensitive users' information between organisations. As a result, each organisation benefits from other organisations cyber threat intelligence while maintaining the privacy of its data internally. The model is trained locally and only the updated weights are shared with the remaining participants in the federated averaging process. The framework has been designed and evaluated in this paper by using two key datasets in a NetFlow format known as NF-UNSW-NB15-v2 and NF-BoT-IoT-v2. Two other common scenarios are considered in the evaluation process; a centralised training method where the local data samples are shared with other organisations and a localised training method where no threat intelligence is shared. The results demonstrate the efficiency and effectiveness of the proposed framework by designing a universal ML model effectively classifying benign and intrusive traffic originating from multiple organisations without the need for local data exchange.
1 Introduction
ML-based NIDS can learn traffic patterns beyond known indicators of compromise, but designing models across organisations must balance heterogeneous data exposure with privacy. The paper proposes federated collaboration so organisations can share threat intelligence without exchanging local network samples.
1 Introduction
- Localised training learns from one endpoint's traffic and can perform well on similar IID samples, but its limited exposure constrains generalisation across heterogeneous networks.Network traffic varies with safe applications, services, threats, and intrusions.Source: §1 · p. 2
- Centralised training can learn from heterogeneous network traffic, but requires transferring sensitive user and endpoint data to a central server.Network data may reveal browsing sessions, applications, services, and endpoint details.Source: §1 · p. 2
- Federated learning lets organisations train a shared NIDS on heterogeneous traffic while keeping local samples private and exchanging model parameters instead.Each organisation trains locally; a global organisation aggregates updated parameters and returns the improved model.Source: §1 · p. 3
- The paper proposes a privacy-preserving CTI scheme and evaluates it on two NIDS datasets against centralised and localised learning scenarios.Its intended outcome is a robust model trained on heterogeneous, non-IID networks without exchanging local samples.Source: §1 · p. 3
2 Related Works
Prior federated NIDS studies addressed model design and other implementation aspects, but generally used data split from one organisational network. This paper identifies a gap in evaluating collaboration across distinct non-IID networks.
2 Related Works
- Earlier studies generally used endpoints whose data came from one organisational network, leaving multi-organisation, heterogeneous non-IID NIDS design unaddressed to the authors' knowledge.The proposed setting aims to share threat intelligence without directly exchanging sensitive data samples.Source: §2 · pp. 3–4
- Abdul Rahman et al. reported federated learning outperformed on-device learning and achieved similar detection performance to centralised learning while preserving privacy.Their comparison used safe and malicious samples from NSL-KDD.Source: §2 · pp. 3–4
- Mothukuri et al. reported their federated IoT anomaly detector outperformed centralised ML with minimized error detection and fewer false alarms.They evaluated LSTM and GRU models with a Random Forest ensemble on a Modbus dataset.Source: §2 · p. 4
- Popoola et al. compared federated zero-day botnet detection with centralised and localised methods, reporting similar performance to centralised learning while preserving privacy.Their approach used FedAvg [34] to aggregate local model parameters.Source: §2 · p. 4
3 Cyber Threat Intelligence Sharing
The paper motivates cross-organisation CTI sharing because NIDS trained on one network may not generalise to others, while centralised data collection raises privacy concerns. Its federated scheme keeps network samples local and aggregates model updates across organisations.
3 Cyber Threat Intelligence Sharing
- The motivation is to combine network intelligence across organisations because each network has distinct benign traffic and attack distributions.The paper introduces ML-based NIDS as learning patterns from captured traffic to distinguish benign from intrusive activity.Source: §3 · p. 4, §3.1 · p. 4
3.1 Motivation
- Local NIDS may not generalise across networks with distinct SOEs and threats, while collecting every scenario locally limits scalability and requires many labelled samples.The paper uses training on one network and evaluation on another to measure cross-network generalisation; it identifies labelled-data scarcity as a challenge for small organisations.Source: §3.1 · pp. 4–5
- Centralised training enables learning from multiple organisations' traffic, but sharing sensitive network data creates privacy, security, and transactional risks.The paper notes that such data can reveal user activity, applications, and endpoint details.Source: §3.1 · p. 5
3.2 Collaborative Federated Learning
- Federated CTI collaboration is proposed to expose NIDS models to more benign and attack variants while keeping network samples within participating organisations.The authors also state that organisations with little local traffic could collaborate rather than collect a large training set.Source: §3.2 · p. 5
- The scheme requires participating organisations to use a common network logging format and feature set so the global model can learn patterns across sources.The global server coordinates local training; in this paper it is hosted within a participant organisation, while an external trusted mediator or blockchain is future work.Source: §3.2 · p. 6
- Each organisation trains a shared model locally, sends only updated weights to the global server, and receives an aggregated model using FedAvg.Federated rounds can be repeated to improve detection across participating environments.Source: §3.2 · p. 6
- The authors suggest broader training across networks may lower false alarms after benign-traffic changes and may improve detection of advanced or zero-day attacks.These are presented as possible benefits, not as established evaluation results.Source: §3.2 · p. 6
4 Experimental Setup
The section evaluates collaborative federated NIDS training across two NetFlow datasets against centralized and localized approaches. Federated learning achieves performance similar to centralized learning while avoiding local-data exchange.
4 Experimental Setup
- The two datasets represent distinct organizational networks, share a NetFlow v9 feature set, and remain internally held during federated training and testing.Experiments use balanced dataset subsets, 70/30 training-test splits, and DNN and LSTM models; the paper notes that full hyperparameter exploration is not covered.Source: §4 · pp. 6–7, §4.1 · pp. 8–9
4.1 Evaluation
- The evaluation compares federated training, which averages locally trained model parameters, with centralized training on shared data and localized training without collaboration.Federated results in the tables are reported after the tenth round.Source: §4.1 · pp. 7–8
4.2 Results
- DNN performance stabilizes after three federated rounds, while LSTM improves rapidly between rounds four and seven; FAR falls from 100% to nearly 8% over ten rounds.DNN’s remaining metrics rise by around 30% on NF-UNSW-NB15-v2 and 15% on NF-BoT-IoT-v2.Source: §4.2 · p. 9, Figure 2 · p. 9, Figure 3 · p. 10
- On NF-UNSW-NB15-v2 binary detection, federated versus centralized accuracy is 90.83% versus 99.38% with DNN and 89.10% versus 95.80% with LSTM.The paper also reports localized cross-dataset detection rates of 4.17% with DNN and 5.78% with LSTM after training on NF-BoT-IoT-v2.Source: Table 4 · p. 10, §4.2 · p. 10
- On NF-BoT-IoT-v2 binary detection, federated attack DR is slightly higher than centralized attack DR for both DNN and LSTM.The cited prose gives no reliable numerical comparison for both models.Source: Table 5 · p. 10, §4.2 · pp. 10–11
- On NF-UNSW-NB15-v2 multi-class detection, centralized average DR is 99.41% with DNN and 96.48% with LSTM; federated learning averages around 85%.Localized learning averages 6.70% on this dataset.Source: Table 6 · p. 11, §4.2 · p. 11
- On NF-BoT-IoT-v2 multi-class detection, federated average attack DR is 93.40% with DNN and 94.61% with LSTM, exceeding the centralized and localized results.The paper attributes the other methods’ lower averages to weaker reconnaissance detection.Source: Table 7 · p. 11, §4.2 · p. 11
- Federated learning often approaches centralized performance while preserving user privacy by avoiding sensitive data sharing.The paper presents this as a more feasible option for organizations than centralized learning, which requires sharing data with third parties.Source: §4.2 · p. 12
5 Conclusion
The proposed federated scheme shares cyber threat intelligence across organisations and performs better than localized learning, with performance similar to centralized learning. It uses heterogeneous network traffic to expose the model to more organisational environments and attack scenarios; future work will improve federated averaging.
5 Conclusion
- Federated learning outperforms localized learning and performs similarly to centralized learning, while avoiding centralized data privacy and security breaches.The paper describes centralized learning as unfeasible in the real world and identifies improving the federated averaging algorithm as future work.Source: §5 · pp. 12–13
- Heterogeneous traffic exposes the model to a wider variety of organisational operating environments and attack scenarios.The paper notes that network distributions vary across organisations, which may limit model generalisation.Source: §5 · p. 12
- Future work will enhance the federated averaging algorithm to improve ML-based NIDS detection performance.Source: §5 · p. 13