PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Network-based Intrusion Detection Data Sets

Markus Ring, Sarah Wunderlich, Deniz Scheuring, Dieter Landes, Andreas Hotho

arXiv:1903.02460v2cs.CR

TL;DR

Network intrusion detection needs representative labeled data sets, but publicly available examples are limited and difficult to track. The paper surveys network-based data sets and their packet- and flow-based data, then organizes 15 suitability properties into five categories for structured comparison. It produces a comprehensive overview of 34 data sets, highlights their peculiarities and attack scenarios, and offers observations and recommendations for selecting and creating data sets.

  • Problem

    Representative publicly available labeled data sets are limited, hindering systematic training, evaluation, and comparison of network intrusion detection methods.

  • Method

    The paper surveys network-based data sets, analyzes packet- and flow-based data, and groups 15 evaluation properties into five categories for structured comparison.

  • Results

    The survey provides a comprehensive overview of 34 data sets, assessing each against the categorization scheme and highlighting attack scenarios and relationships.

  • Takeaways & Limitations

    Readers can use the property framework and data-set overview to identify data sets suited to specific evaluation scenarios.

Abstract

from arXiv · show

Labeled data sets are necessary to train and evaluate anomaly-based network intrusion detection systems. This work provides a focused literature survey of data sets for network-based intrusion detection and describes the underlying packet- and flow-based network data in detail. The paper identifies 15 different properties to assess the suitability of individual data sets for specific evaluation scenarios. These properties cover a wide range of criteria and are grouped into five categories such as data volume or recording environment for offering a structured search. Based on these properties, a comprehensive overview of existing data sets is given. This overview also highlights the peculiarities of each data set. Furthermore, this work briefly touches upon other sources for network-based data such as traffic generators and traffic repositories. Finally, we discuss our observations and provide some recommendations for the use and creation of network-based data sets.

I. INTRODUCTION

Network-based intrusion detection research depends on representative labeled data sets, yet publicly available examples remain scarce and difficult to track. This survey addresses that gap by organizing existing data sets, their formats, properties, and attack scenarios for evaluation.

  • Motivation: Representative network-based data sets are required to develop and compare network intrusion detection systems.Labeled normal and attack data support evaluation through detected attacks and false alarms.
  • Motivation: The shortage of representative public data sets remains a major challenge for anomaly-based intrusion detection.New data sets continue to appear, but no overall index makes recent developments difficult to track.
  • Contribution: The survey reviews network-based data sets, distinguishes packet- and flow-based formats, and analyzes properties used to assess their quality.Packet-based data include payload, whereas flow-based data contain connection metadata.
  • Contribution: Its main contribution is an exhaustive overview that maps data sets to properties and highlights attack scenarios and relations among data sets.The paper also discusses traffic generators and repositories as additional sources of network traffic.
  • Related work: Earlier reviews often examined fewer data sets or pursued different primary objectives, whereas this work provides a neutral overview of existing network-based data sets.The survey continues earlier reviews while emphasizing comparison rather than proposing another data set.
  • Network data: Packet-based traffic contains complete protocol headers and payload, while IP, TCP, UDP, and ICMP provide different network and transport metadata.The cited protocol headers include addressing, transport, control, and checksum-related information.

B. Flow-based data

Flow-based data condense packets into connection-level records containing metadata rather than payload. The survey describes flow formats, conversion options, mixed data sets, and common properties for systematic comparison.

  • Flow-based data: Flow-based data aggregate packets sharing connection properties within a time window and usually exclude payload.The widely used five-tuple comprises source and destination IP addresses and ports plus the transport protocol.
  • Flow direction: Flows may be unidirectional or bidirectional, depending on whether traffic directions are represented separately or together.Unidirectional flows separate A-to-B and B-to-A traffic; bidirectional flows summarize both directions.
  • Flow formats: NetFlow, IPFIX, sFlow, and OpenFlow are typical flow formats, with exporters able to provide additional derived attributes.Examples include bytes per second, bytes per packet, TCP flags, and payload entropy.
  • Conversion: Packet-based data can be converted into flow-based data with tools such as nfdump or YAF, but the reverse conversion is not possible.Flow exporters can affect downstream analyses such as botnet classification.
  • Other data: Other data sets combine flow attributes with packet-derived or host-based information and therefore require individual attribute analysis.KDD CUP 1999 combines network attributes with host-based features such as failed-login counts.
  • Comparison framework: The survey defines common properties and groups them into five categories to support side-by-side comparison without assigning universal importance scores.Property importance is treated as dependent on the evaluation scenario.

A. General Information

The general-information properties describe when data were created, how they can be accessed, and whether they include normal or malicious traffic. These criteria help determine suitability for particular evaluation settings.

  • General information: The general-information category covers creation year, availability, and the presence of normal and malicious network traffic.These properties provide basic information for comparing data sets.
  • Age: The capture year is more relevant than publication year for judging whether a data set reflects current network conditions.This matters because network traffic undergoes concept drift and new attack scenarios emerge over time.
  • Availability: Public availability enables comparison of intrusion detection methods and allows third parties to inspect data-set quality.Availability is categorized as yes, on request, or no.
  • Normal user behavior: Normal user behavior is indispensable for evaluating detection and false-alarm rates, although its absence does not make a data set unusable.Such data sets may be merged with other data or real-world traffic through overlaying or salting.
  • Malicious traffic: The malicious-traffic property records whether a data set contains at least one attack and provides additional attack-type information.The survey uses attack presence as a basic indicator and reports specific attack types separately.

B. Nature of Data

Nature-of-data properties characterize format, metadata, anonymization, volume, and recording duration. Together, they describe what information a data set contains and the temporal and practical scope of its observations.

  • Nature of data: Nature-of-data properties describe data-set format and the presence of metadata.The category distinguishes what is recorded from contextual information supplied with the data.
  • Metadata: Metadata provides information about network structure, IP addresses, and attack scenarios that can make packet- and flow-based data easier to interpret.The survey treats metadata presence as an explicit property.
  • Format: The survey distinguishes packet-based traffic with payload, flow-based traffic with connection metadata, and other data sets with additional attributes.Other data sets may combine flow traces with packet-derived or host-log information.
  • Anonymization: Anonymization records whether attributes such as IP addresses or payload have been removed or anonymized, which can affect data analysis.The survey differentiates no anonymization, IP anonymization, and payload anonymization.
  • Volume and duration: Volume and duration characterize the amount of data and the length of the recording period.Count may be expressed as packets, flows, points, or gigabytes, while duration records capture time.
  • Duration: Long recording periods help capture periodic effects such as differences between daytime and nighttime or weekdays and weekends.Duration is therefore relevant when evaluation depends on temporal variation.

2) Duration:

The survey classifies data-set properties by recording environment and evaluation conditions, including traffic origin, network setting, completeness, predefined subsets, balance, and labels.

  • Recording environment: Traffic can be real, emulated, or synthetic, depending on whether it was captured in a productive network, test bed, or generated artificially.
  • Recording environment: The underlying network environment matters because small and medium-sized companies differ fundamentally from internet service providers, requiring environment-specific evaluation data sets.
  • Recording environment: A completeness property distinguishes data sets containing an entire multi-host network from those limited to one host or selected protocols.
  • Evaluation: Predefined training and evaluation subsets help clarify whether IDS comparisons use the same data partitions.
  • Evaluation: Balance concerns whether normal and attack classes contain equal numbers of data points, although real-world traffic is typically imbalanced.

3) Labeled:

Labels are a central data-set property because they support supervised training and evaluation of both supervised and unsupervised intrusion-detection methods.

  • Labeled: Labeled data sets are needed to train supervised methods and evaluate supervised and unsupervised intrusion-detection methods.
  • Labeled: A data set is labeled when it includes at least normal and attack classes, with possible values including yes, yes with background, yes (IDS), indirect, and no.
  • Labeled: The yes with background value indicates a third background class in addition to normal and attack traffic.

V. DATA SETS

The survey organizes network-based data sets around properties that enable side-by-side comparison and selection for specific evaluation scenarios, emphasizing labels, format, and attack coverage.

  • V. DATA SETS: Labeled and Format are identified as the most decisive properties when searching for suitable network-based data sets.The appropriate choice depends on whether the intrusion-detection method is supervised or unsupervised and whether it requires packet, flow, or other data.
  • V. DATA SETS: Table II classifies investigated data sets by labeling and format, while Table III provides a more detailed comparison across the surveyed properties.
  • V. DATA SETS: Attack-scenario presence is recorded in Table III, and Table IV gives details about specific attacks within each data set.
  • V. DATA SETS: Some data sets are modifications or combinations of others, with Figure 3 showing interrelationships among several well-known data sets.

Network-based data sets in alphabetical order

The survey lists diverse network-based data sets alphabetically and records their formats, environments, attack scenarios, availability, and relationships, highlighting distinct design choices and limitations.

  • AWID: AWID captures labeled 802.11 WLAN traffic from an 11-client emulated network, with 37 million packets recorded in one hour and separate training and test subsets.It contains 156 attributes per packet and 16 specific attacks.
  • Booters: Booters contains more than 250GB of packet-based traffic from nine booter attacks, with attacks separated into files rather than individual packets labeled.
  • Botnet: Botnet combines parts of ISOT, ISCX 2012, and CTU-13 into packet-based training and test subsets of 5.3 GB and 8.5 GB.
  • CIC DoS: CIC DoS provides 24 hours of packet-based traffic containing eight application-layer DoS attacks combined with attack-free ISCX 2012 traffic.
  • CICIDS 2017: CICIDS 2017 spans five days in an emulated environment and provides packet-based and bidirectional flow-based traffic with more than 80 attributes per flow.Its scenarios include SSH brute force, heartbleed, botnet, DoS, DDoS, web, and infiltration attacks.
  • CIDDS-002: CIDDS-002 contains two weeks of unidirectional flow-based traffic from an emulated small-business environment, covering normal behavior and varied port scans.
  • CDX: CDX records four days of packet-based traffic from a 2009 network-warfare competition, including normal behavior and several attack types.
  • DARPA: DARPA 1998/99 provides seven and five weeks of packet-based traffic from an emulated network, but is criticized for artificial attack injections and redundancy.

VI. OTHER DATA SOURCES

Beyond fixed data sets, network intrusion detection research can draw on repositories and traffic generators as additional sources of packet- and flow-based traffic.

  • Repositories and traffic generators provide additional sources of packet- and flow-based network traffic.

A. Data Repositories

Network traffic repositories differ substantially in content, format, availability, access requirements, recency, and maintenance. They range from specialized archives and competition traces to general data-sharing platforms.

  • Repositories differ greatly in type and structure, so the survey uses a textual overview rather than a tabular comparison.The repositories were checked for actuality on 26 February 2019.
  • AZSecure provides labeled and unlabeled network data in formats including pcap and arff, with examples such as CTU-13 and the Unified Host and Network Data Set.
  • CAIDA offers diverse data sets with varying availability, while some access requires forms or an IMPACT login.
  • Competition archives such as DEF CON CTF and MACCDC contain predominantly attack traffic and little normal user behavior.DEF CON data are typically available in packet-based format; MACCDC’s latest competition took place in 2018.
  • MAWILab supplies daily 15-minute packet traces since 2007, but anonymizes IP addresses and omits payloads for privacy.
  • Other sources include restricted community data, public trace archives, security-data lists, machine-learning platforms, and repositories containing packet or flow traffic.Examples include MWS, NETRECSEC, OpenML, RIPE, SecRepo, Simple Web, UMassTraceRepository, VAST Challenge, Kaggle, and Malware Traffic Analysis.

B. Traffic Generators

Traffic generators create synthetic network traffic that can be adapted to particular network structures, unlike fixed data sets and repositories. Approaches use real traffic, extracted templates, generative models, replayable scenarios, or labeling frameworks.

  • Traffic generators use user-defined parameters or properties extracted from real traffic to create adaptable synthetic network traffic.
  • FLAME and ID2T use real traffic as a normal-behavior baseline, then add malicious traffic by editing values or injecting synthetic flows.
  • Graph-based generators extract traffic templates from real traffic, while Improved WGAN-GP generates synthetic flow-based network traffic.
  • Moirai addresses privacy concerns by sharing complete attack scenarios for replay in virtual machines, allowing data generation on the fly.
  • INSecS-DCS captures or imports traffic, divides it into time windows, extracts attributes, and labels the resulting data points.

VII. OBSERVATIONS AND RECOMMENDATIONS

The survey recommends matching data-set choice and evaluation design to the application domain, while recognizing that no single perfect data set exists. It emphasizes multiple, public, realistically formatted, well-documented data sets with meaningful splits and careful anonymization.

  • Labeled network data must be representative to train supervised methods and evaluate supervised or unsupervised data-mining methods and NIDS.
  • The survey identifies data sets, repositories, and traffic generators as sources and establishes properties for matching data sets to evaluation scenarios.
  • Perfect data set: No perfect data set exists because current, realistic, fully labeled, public traffic with attacks, normal behavior, payload, and long duration is difficult to provide simultaneously.
  • Use of several data sets: Using several data sets can reduce over-fitting and data-set-specific artifacts, while combining synthetic or emulated traffic with real-world traffic addresses differing environments.
  • At least one publicly available data set supports reproducibility, while CICIDS 2017, CIDDS-001, UGR’16, and UNSW-NB15 are suggested for general evaluation settings.The recommendation is explicitly presented as personal views, and other data sets may suit specific scenarios better.
  • Predefined Subsets: Ten-fold cross-validation can leak flows from the same port scan into training and testing, facilitating attack detection and weakening generalization assessment.
  • Predefined Subsets: Meaningful predefined training and test splits should reflect the IT-security application domain and facilitate comparisons on identical data.
  • Standard formats: Standard packet- or flow-based formats should be published alongside transformation scripts so researchers can reuse data and apply the scripts to future sets.

VIII. SUMMARY

The paper surveys labeled network-based intrusion detection data sets using a 15-property categorization scheme and provides a comprehensive overview of 34 data sets. It also discusses additional traffic sources and recommendations for evaluating and creating data sets.

  • 15 properties are grouped into five categories to assess the suitability of network-based intrusion detection data sets.The categories are General Information, Nature of the Data, Data Volume, Recording Environment, and Evaluation.
  • 34 data sets are comprehensively reviewed, with attention to each data set’s peculiarities and attack-scenario interrelationships.
  • The detailed investigation is intended to help readers identify data sets suited to their purposes.
  • Traffic generators and data repositories are discussed as additional sources, while users are encouraged to evaluate methods across several data sets.The paper also advocates standard formats with predefined training and test subsets and notes that no perfect data set is likely to exist.
Loading 1903.02460v2…