PaperlayerView exact revision on arXiv

Source-linked AI summary

DÏoT: A Federated Self-learning Anomaly Detection System for IoT

Thien Duc Nguyen, Samuel Marchal, Markus Miettinen, Hossein Fereidooni, N. Asokan, Ahmad-Reza Sadeghi

arXiv:1804.07474v5cs.CR

TL;DR

Vulnerable IoT devices and sparse communications make comprehensive anomaly models difficult to train. DÏoT addresses this with device-type-specific communication models and federated learning, achieving 95.6% true positive rate, detection in approximately 257 ms, and zero false alarms.

  • Problem

    Vulnerable IoT devices can be exploited, while sparse communications make it challenging to train comprehensive models covering their full behavior.

  • Method

    DÏoT learns device-type-specific anomaly models from packet-symbol sequences and uses federated learning to aggregate model updates from multiple Security Gateways without sharing training data.

  • Results

    95.6% true positive rate, detection in approximately 257 ms, and zero false alarms were achieved in experiments with more than 30 off-the-shelf IoT devices.

  • Takeaways & Limitations

    DÏoT provides a self-learning distributed system for detecting compromised IoT devices through deviations from learned benign communication patterns.

Abstract

from arXiv · show

IoT devices are increasingly deployed in daily life. Many of these devices are, however, vulnerable due to insecure design, implementation, and configuration. As a result, many networks already have vulnerable IoT devices that are easy to compromise. This has led to a new category of malware specifically targeting IoT devices. However, existing intrusion detection techniques are not effective in detecting compromised IoT devices given the massive scale of the problem in terms of the number of different types of devices and manufacturers involved. In this paper, we present DÏoT, an autonomous self-learning distributed system for detecting compromised IoT devices effectively. In contrast to prior work, DÏoT uses a novel self-learning approach to classify devices into device types and build normal communication profiles for each of these that can subsequently be used to detect anomalous deviations in communication patterns. DÏoT utilizes a federated learning approach for aggregating behavior profiles efficiently. To the best of our knowledge, it is the first system to employ a federated learning approach to anomaly-detection-based intrusion detection. Consequently, DÏoT can cope with emerging new and unknown attacks. We systematically and extensively evaluated more than 30 off-the-shelf IoT devices over a long term and show that DÏoT is highly effective (95.6% detection rate) and fast (~257 ms) at detecting devices compromised by, for instance, the infamous Mirai malware. DÏoT reported no false alarms when evaluated in a real-world smart home deployment setting.

I. INTRODUCTION

DÏoT addresses IoT intrusion detection challenges caused by vulnerable, heterogeneous devices, scarce communications, and signature-based methods' inability to detect novel attacks. It introduces autonomous device-type-specific anomaly detection with federated learning and reports fast, effective detection with no false alarms in deployment.

  • Motivation: IoT devices' insecure design and delayed patching leave networks vulnerable to malware targeting these devices.Reactive intrusion detection is needed when automated updates are unavailable or delayed.
  • Motivation: Signature-based IDSs cannot detect novel attacks until vendors release corresponding attack signatures.Anomaly detection profiles normal device behavior instead, but often suffers from high false alarm rates.
  • Motivation: Heterogeneous device behaviors and scarce communications make comprehensive, precise IoT anomaly models difficult to train and maintain.An all-encompassing model can be too broad to detect subtle anomalies without many false alarms.
  • Goals and Contributions: DÏoT is a self-learning distributed system that detects compromised IoT devices using device-type-specific anomaly detection models.Its design targets the challenges of learning normal behavior across diverse IoT devices.
  • Goals and Contributions: DÏoT represents network packets as symbols, applies language analysis, and uses federated learning to aggregate anomaly-detection profiles for intrusion detection.The system's contributions include packet-sequence anomaly analysis and federated aggregation across participating gateways.
  • Goals and Contributions: 95.6% true positive rate, 0% false positive rate, and ≈257 ms detection were reported in experiments using more than 30 off-the-shelf IoT devices.The evaluation included real IoT malware traffic, including Mirai, and an Attack dataset intended for research use.

II. SYSTEM MODEL

DÏoT monitors IoT communications at a Security Gateway and uses an IoT Security Service to maintain and aggregate device-type-specific anomaly detection models. The architecture supports local monitoring alongside distributed model aggregation.

  • System Architecture: DÏoT consists of a Security Gateway and an IoT Security Service supporting IoT device monitoring and model aggregation.The Security Gateway performs anomaly detection, while the service aggregates models trained by gateways.
  • Security Gateway: The Security Gateway monitors identified IoT devices and detects abnormal communication behavior using locally trained anomaly detection models.It acts as the local Internet access gateway for devices connected over WiFi or Ethernet.
  • IoT Security Service: The IoT Security Service maintains a repository of device-type-specific models and aggregates updates supplied by Security Gateways.A gateway retrieves the model corresponding to a newly identified device type from the service.

B. Adversary Model and Assumptions

DÏoT models an IoT malware adversary and relies on assumptions about gateway integrity, device identification, and initially uncompromised devices. Its scope targets dynamic, heterogeneous IoT environments with limited traffic and false-alarm concerns.

  • Adversary Model: The adversary is IoT malware that attacks vulnerable SOHO devices or launches attacks from them.The considered malware actions include discovering, infecting, and exploiting vulnerable devices.
  • Defense Goals: DÏoT aims to detect IoT attacks early enough to support countermeasures such as preventing compromise or isolating compromised devices.The stated goal is detection preferably before successful infection.
  • Assumptions: The model assumes devices are not compromised when first released and that the Security Gateway itself is not compromised.The first assumption provides time to learn benign behavior; the second is treated as necessary for network protection.
  • Assumptions: DÏoT assumes automated identification and labeling of IoT device types is available in the local Security Gateway.The paper selects AuDI as an existing approach for autonomous device identification.
  • IoT Challenges: IoT anomaly detection must address dynamic threats, device heterogeneity, resource limitations, and scarce communications.These conditions complicate on-device detection, model training, and false-alarm control.

E. Design Choices

DÏoT places monitoring at the Security Gateway, builds separate models for each device type, aggregates locally learned models federally, and autonomously labels training data. These choices target resource limits, behavioral heterogeneity, scarce traffic, and changing threats.

  • Resource-Constrained Monitoring: Monitoring occurs at the Security Gateway because on-device monitoring is rarely feasible for resource-limited IoT devices.The gateway performs communication monitoring on behalf of connected devices.
  • Device-Specific Models: DÏoT assigns each device type a dedicated behavior model to avoid combining highly heterogeneous device behaviors.Single-use appliances generally have relatively static and limited behavior patterns, reducing false-alarm risk.
  • Federated Learning: Federated learning aggregates local models from Security Gateways to exploit scarce communication data across the system.The approach improves anomaly-model accuracy while avoiding direct sharing of local training data.
  • Autonomous Self-Learning: DÏoT autonomously identifies device types and labels training data, requiring no human intervention during learning or operation.The system begins without an anomaly detection model and improves as gateways aggregate more data.

5) Modeling techniques requiring little data:

DÏoT uses lightweight packet representations and GRU models to learn device-type-specific communication patterns from limited data. Packets are mapped using seven discrete characteristics, then modeled as symbol sequences for anomaly detection.

  • 5) Modeling techniques requiring little data:: GRU models are selected because they can be trained efficiently with few data while providing similar accuracy to other recurrent neural networks.This design addresses the scarcity of IoT communications.
  • 5) Modeling techniques requiring little data:: The resulting symbol sequence is input to a pretrained GRU, which estimates each symbol’s probability from preceding symbols.The model evaluates whether observed communication is consistent with the learned benign behavior of that device type.
  • 5) Modeling techniques requiring little data:: Packets are converted into symbols using device-type-specific mappings based on seven discrete packet characteristics.The characteristics include direction, local and remote port type, packet length, TCP flags, encapsulated protocol, and inter-arrival-time bins.
  • 5) Modeling techniques requiring little data:: The seven characteristics capture communication differences involving packet direction, ports, lengths, protocol behavior, TCP flags, and traffic timing.These features represent patterns that may differ between normal device communication and attacks.

B. Detection Process

DÏoT detects anomalies by modeling the conditional probability of packet symbols in device-specific communication sequences. It combines probability thresholds with window-level aggregation to distinguish attack traffic while reducing false alarms.

  • B. Detection Process: Individual packet-symbol frequencies are insufficient because benign and Mirai attack traffic can contain symbols mapped to the same packet types.Detection therefore relies on the likelihood of a symbol given its preceding sequence.
  • B. Detection Process: The GRU estimates pi = P(si|<si−k, si−k+1, . . . , si−1>) for each symbol using a lookback history of preceding symbols.The parameter k is determined by the GRU network and specifies how many preceding symbols are considered.
  • B. Detection Process: δ = 10−2 provides a good separation between benign and attack traffic in the analyzed probability estimates.Malicious packets in the Edimax Plug example receive probability estimates below 10−4.
  • B. Detection Process: In the Mirai standby example, malicious packets marked red as ‘#0’ receive low occurrence probabilities, while preceding malicious packets can also depress later benign-packet estimates.The latter effect occurs because the detection model lacks the expected preceding sequence.
  • B. Detection Process: An anomaly alarm is raised only when the fraction of anomalous packets in a consecutive window exceeds the anomaly triggering threshold γ.This window-level condition avoids triggering an alarm for every isolated low-probability packet.

IV. FEDERATED LEARNING APPROACH

DÏoT trains device-type-specific GRU anomaly models across Security Gateways using federated learning. Local models are trained on gateway traffic, aggregated by the IoT Security Service, and redistributed for detection.

  • Federated model training: Each Security Gateway trains a device-type-specific GRU model on locally collected IoT traffic and sends model updates to a centralized service.Clients retain their training data and report updates rather than sharing raw data.
  • Federated training process: The training cycle requests an initial device-type profile, performs local retraining, aggregates updates, and pushes the updated model back to gateways for anomaly detection.The process begins with a random model when DÏoT has not yet trained one.
  • Global aggregation: Federated learning aggregates local model updates into a global model for each device type, using client data-sample counts in the aggregation.The global model is defined from local weights W1, …, Wn and associated sample counts s1, …, sn.
  • Communication efficiency: Local training for several epochs reduces the number of updates sent to the IoT Security Service, limiting communication overhead.The approach follows a method introduced by McMahan et al.
  • Implementation: The implementation uses Flask and SocketIO for federated communication and Keras with a TensorFlow backend for the GRU network.The client-side application uses the gevent asynchronous framework.

V. EXPERIMENTAL SETUP

The evaluation combines laboratory and real-world smart-home traffic from consumer IoT devices with Mirai attack traces. It covers device activity, deployment behavior, and multiple attack stages under constrained experimental conditions.

  • Device and dataset coverage: The study used 33 consumer IoT devices mapped to 23 device types, including cameras, power plugs, light bulbs, and sensors.The devices and their type assignments are summarized in Table II.
  • Laboratory dataset: Laboratory traffic was collected through a Kali Linux gateway that connected devices over WiFi or Ethernet and captured packets with tcpdump.The laboratory network setup is shown in Fig. 6.
  • Activity dataset: The activity dataset repeatedly invoked the distinct user actions available on each device to capture communication patterns associated with interactions.Each listed action was repeated 20 times.
  • Deployment dataset: The deployment dataset contained realistic communication traces from 14 smart-home devices used by real users in domestic scenarios without an attack server.The setup otherwise matched the laboratory network.
  • Attack dataset: The Attack dataset included benign and malicious traffic from Mirai-infected devices across pre-infection, infection, scanning, DoS, and standby stages.Mirai was selected because its public codebase and attack stages represent related IoT malware behavior.
  • Attack traffic collection: The attack collection covered five vulnerable devices, with scanning producing more than 446,000 packets and DoS experiments producing more than 20 million packets.The DoS experiments used all ten attack vectors available in the Mirai source code.

B. Parameter Selection

DÏoT uses device-type-specific GRU models and evaluates them with false-positive and true-positive rates. Thresholds were selected using fixed 250-packet windows and ROC analysis.

  • Model configuration: DÏoT learned 23 anomaly detection models using GRU networks with a 20-symbol lookback history and three hidden layers of 128 neurons each.Input and output sizes are device-type-specific.
  • Evaluation metrics: False positive rate measures benign communications incorrectly classified as anomalous, while true positive rate measures attacks correctly reported as anomalous.The evaluation seeks to minimize FPR and maximize TPR.
  • False-positive testing: False-positive testing used four-fold cross-validation on Activity and Deployment data, dividing test data into windows of w = 250 packets.Because the testing data were benign, an alarm in a window indicated a false positive.
  • True-positive testing: True-positive testing trained on Activity and Deployment data and tested on Attack data using the same 250-packet window setting.The evaluation also measured packets required until an alarm was triggered.

VI. EXPERIMENTAL RESULTS

DÏoT achieved high attack-detection performance with no false positives in a one-week deployment evaluation, while detection varied by attack intensity. Increasing federated clients slightly reduced TPR but reached zero FPR.

  • Accuracy: 95.60% TPR and 0% FPR were achieved on the Deployment and Attack datasets during one week of evaluation.The selected thresholds were δ = 0.01 and γ = 0.5 with w = 250.
  • Detection latency: Average detection time varied across Mirai attack scenarios according to attack traffic intensity.Table V summarizes the average detection times.
  • Attack-stage performance: 88.96% TPR was achieved for DoS attacks, with all DoS attacks eventually detected because their high throughput filled multiple analyzed windows.The reported DoS throughput was 1,412.94 packets/s.
  • Attack-stage performance: 33.33% of infected devices in standby mode were detected, despite standby activity generating only 0.05 packets/s.Standby mode was described as very stealthy.
  • Federated learning evaluation: Using 15 federated clients decreased TPR by 3% and reduced FPR to 0%, a 21% decrease.The comparison varied the number of clients from 2 to 15.
  • Federated learning evaluation: Federated models showed a small TPR drop compared with centralized training while FPR remained constant at 0.00%.The comparison used models trained with data divided among 5, 9, or 15 clients.

C. Data Needed for Training

DÏoT’s device-type-specific models improve the balance between detection sensitivity and false alarms, while training data quantity affects false-positive performance. The evaluation also discusses adaptation to firmware changes and the scope of malware tested.

  • Training Data: The false-positive rate decreases noticeably as the training dataset grows for Edimax smart plug models evaluated with progressively larger training samples.The experiment divided seven days of deployment data into one-hour chunks and sampled different numbers of chunks for training.
  • Device-Type-Specific Models: Device-type-specific models focus on one device type, producing more specific behavioral models regardless of how many device types the system handles.
  • Device-Type-Specific Models: 0% FPR and 95.6% TPR were achieved with device-type-specific models, whereas a single model raised FPR to 0.67% and TPR to 97.21%.In the deployment setup, 0.67% FPR would produce around eight alarms per day, potentially yielding hundreds of false alarms daily across a smart home.
  • Evaluation Scope: The evaluation focused on Mirai, while the authors argue that deviations caused by infected-device behavior should also expose other botnet malware.
  • Adaptation: DÏoT can adapt to firmware-driven behavior changes by correlating anomaly reports across Security Gateways, canceling widespread alarms, and triggering model re-learning.A human expert can confirm alarm cancellation to distinguish firmware updates from sudden malware outbreaks.

C. Mimicking Legitimate Communication

DÏoT restricts behavioral models to individual device types and uses packet-sequence features to make mimicry difficult. The paper also identifies adversarial-example and federated-poisoning risks, while contrasting its GRU-based modeling with prior approaches.

  • Mimicking Legitimate Communication: Device-type-specific models make it difficult for attackers to mimic legitimate communication while simultaneously performing malicious actions such as scanning or flooding.The models use packet protocol, size, port, and ordering characteristics, and attackers would need knowledge of device-specific communication patterns.
  • Adversarial Machine Learning: DÏoT’s processed symbolic packet features are difficult to modify into useful adversarial examples that preserve malicious functionality.The paper contrasts these features with raw image and audio inputs, where adversarial-example objectives are easier to define.
  • Adversarial Machine Learning: Federated retraining remains vulnerable to poisoning attacks in which adversarial examples corrupt anomaly models so malicious traffic is accepted as benign.The paper cites local outlier-detection filtering as a possible mitigation for adversarial examples.
  • Adversarial Machine Learning: The paper assumes Security Gateways are not compromised, leaving gateway compromise as a possible route for poisoning attacks that local filtering cannot prevent.The authors are investigating robust learning and detection of malicious model updates at the IoT Security Service.
  • Related Approaches: GRU models longer packet sequences than finite-state automatons, enabling detection of stealthy attacks, while finer-grained features are harder to forge without reducing malicious utility.
  • Related Approaches: DÏoT uses GRUs rather than the LSTMs common in earlier anomaly-detection applications because GRUs can be learned with less training data.
  • Summary: In a real-world deployment, DÏoT detected 95.6% of Mirai attacks in 257 milliseconds on average without raising false alarms.
Loading 1804.07474v5…