PaperlayerView exact revision on arXiv

Source-linked AI summary

Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data

Alina Oprea, Zhou Li, Ting-Fang Yen, Sang Chin, Sumayah Alrwais

arXiv:1411.5005v2cs.CR

TL;DR

The paper addresses early-stage enterprise infection detection when sophisticated malware can evade traditional defenses and enterprise traffic is large but locally limited. It combines belief propagation with seeded or seedless C&C detection, achieving strong performance on LANL simulations and finding previously overlooked suspicious domains in large-enterprise logs. The approach is intended to complement existing security tools, with effectiveness bounded by attacker evasion strategies such as alternate channels and randomized timing.

  • Problem

    The paper asks how to detect early-stage enterprise infections when sophisticated malware evades traditional defenses and enterprise observations are large but limited to one organization.

  • Method

    The method combines graph-based belief propagation from compromised-host or malicious-domain seeds with an enterprise-specific C&C detector that can operate without seeds.

  • Results

    98.33% true detection, 1.67% false detection, and 6.25% false negative rates were achieved on LANL simulations, while enterprise logs yielded hundreds of suspicious domains overlooked by existing products.

  • Takeaways & Limitations

    The approach complements existing security products by identifying new suspicious activities and related domains or hosts for SOC investigation.

  • Takeaways & Limitations

    Attackers may evade detection through non-HTTP channels, compromised popular sites, or completely randomized C&C timing, which require different models or remain difficult to detect.

Abstract

from arXiv · show

Recent years have seen the rise of more sophisticated attacks including advanced persistent threats (APTs) which pose severe risks to organizations and governments by targeting confidential proprietary information. Additionally, new malware strains are appearing at a higher rate than ever before. Since many of these malware are designed to evade existing security products, traditional defenses deployed by most enterprises today, e.g., anti-virus, firewalls, intrusion detection systems, often fail at detecting infections at an early stage. We address the problem of detecting early-stage infection in an enterprise setting by proposing a new framework based on belief propagation inspired from graph theory. Belief propagation can be used either with "seeds" of compromised hosts or malicious domains (provided by the enterprise security operation center -- SOC) or without any seeds. In the latter case we develop a detector of C&C communication particularly tailored to enterprises which can detect a stealthy compromise of only a single host communicating with the C&C server. We demonstrate that our techniques perform well on detecting enterprise infections. We achieve high accuracy with low false detection and false negative rates on two months of anonymized DNS logs released by Los Alamos National Lab (LANL), which include APT infection attacks simulated by LANL domain experts. We also apply our algorithms to 38TB of real-world web proxy logs collected at the border of a large enterprise. Through careful manual investigation in collaboration with the enterprise SOC, we show that our techniques identified hundreds of malicious domains overlooked by state-of-the-art security products.

I. INTRODUCTION

The paper targets early-stage enterprise infection detection amid stealthy attacks, massive enterprise traffic, and limited enterprise-local visibility. It proposes belief propagation with seeded or seedless detection and reports strong results on simulated and real-world data.

  • Motivation: Advanced attacks and malware variants increasingly evade traditional defenses, making early infection detection difficult.The motivation includes APTs, rapidly appearing malware strains, and evasion of anti-virus, firewalls, and intrusion detection systems.
  • Motivation: Enterprise-scale monitoring must analyze terabytes of daily traffic while relying only on traffic observed within one organization.This limits the applicability of approaches based on ISP-level data.
  • Approach: The framework uses belief propagation to identify small communities of related rare domains associated with early-stage malware infections.Rare domains are new to the organization and contacted by few internal hosts; iterative scoring combines features and similarity to previously detected domains.
  • Approach: The method supports SOC-provided seeds or a no-seed mode that first detects automated C&C communication.The C&C detector uses enterprise-specific and prior-work features and can detect communication from a single compromised host.
  • Results: 98.33% true detection, 1.67% false detection, and 6.25% false negative rates were achieved on the LANL simulated attacks.The LANL evaluation used two months of anonymized DNS records containing 20 simulated APT-like campaigns.

A. Enterprise Infections

Enterprise infections often leave small, stealthy traces despite large volumes of security data and diverse attack vectors. The paper therefore targets common infection patterns and develops detection that can operate without malware samples, including single-host C&C activity.

  • Enterprise infection patterns: Targeted and mainstream attacks use social engineering, compromised legitimate sites, spam, USB drives, and web-based techniques.These vectors may deliver malicious attachments, hyperlinks, or payloads to enterprise users.
  • Enterprise infection patterns: Attackers favor small-scale infections and uncommon domains, including cases where one host communicates with a C&C center.Stealth motivates focusing on infections involving few hosts and destinations contacted by few internal systems.
  • Operational setting: SOC analysts commonly investigate network activity using commercial blacklists or external indicators of compromise.They then manually investigate suspected malicious domains and infected hosts.
  • Prior limitations: Existing systems may require malware samples or multiple synchronized compromised hosts, limiting their applicability to large networks and stealthy infections.The paper contrasts these requirements with its enterprise-focused approach.
  • Design challenges: The system addresses large heterogeneous logs by focusing on general infection patterns across DNS, web proxy, NetFlow, and packet-capture data.The evaluation uses distinct DNS and web proxy datasets, with additional web-proxy context features.
  • Proposed response: The C&C detector identifies suspicious domains even when only one host communicates with them and without prior malware-sample knowledge.This capability is intended for stealthy targeted infections that leave limited evidence in logs.

A. Datasets

The study evaluates enterprise infection detection on anonymized LANL DNS logs and richer large-enterprise web proxy logs. Its workflow uses rare destinations, SOC hints or no-hint detection, and a host-domain bipartite graph for belief propagation.

  • Datasets: The LANL dataset contains anonymized DNS events, while the AC dataset contains richer HTTP/HTTPS proxy logs from an enterprise with over 100,000 hosts.The AC logs include URLs, timestamps, source and destination IPs, HTTP methods, status codes, user agents, and referers.
  • Datasets: Both datasets span two months and are extremely large, measuring 1.15 TB for LANL and 38.14 TB for AC.LANL provides simulated infection campaigns and labels for validation, whereas AC supplies the main real-world evaluation setting.
  • Candidate selection: Rare destinations are new to internal hosts and visited by only a small number of them, reducing the search space for suspicious domains.The enterprise study contains roughly 50,000 rare destinations daily.
  • Modes of operation: SOC-hints mode starts from confirmed malicious hosts or domains and expands to related suspicious domains and previously unknown compromised hosts.Seeds may come from incidents investigated by the enterprise SOC.
  • Modes of operation: No-hint mode detects C&C communication using timing, registration, and enterprise-specific web-proxy features, including single-host communication.Detected C&C domains and contacting hosts can seed the belief-propagation process.
  • Belief propagation: The method models hosts and domains as a bipartite graph and iteratively passes beliefs to label domains malicious or benign and hosts compromised or clean.Incremental graph construction adds hosts and domains when their compromise confidence is high.
  • Belief propagation: Domain scores combine C&C-like behavior with similarity to previously labeled suspicious domains based on shared hosts, timing, and IP proximity.This supports iterative expansion from known or newly identified suspicious entities.

D. Detecting C&C communication

The C&C detector distinguishes automated communication from legitimate regular activity using timing and contextual features. It feeds high-scoring potential C&C domains into the broader daily detection pipeline.

  • C&C detection: Automated malware communication exhibits regular connection timing, whereas normal user activity is generally more variable.Regular timing alone is insufficient because legitimate site refreshes and automatic updates can also be periodic.
  • C&C detection: The detector compares host-domain inter-connection histograms with a periodic distribution and labels communication automated below a statistical-distance threshold.The comparison operationalizes regularity in connection timing.
  • C&C detection: Rare-domain filtering is combined with domain age, registration validity, user-agent popularity, and web-referer features to identify suspicious automated connections.Enterprise-specific features help reduce false suspicion from legitimate periodic requests.
  • C&C detection: A linear regression model assigns each automated domain a weighted feature score, and domains above a threshold become potential C&C domains.The threshold reflects tradeoffs between accuracy and coverage.
  • System operation: The system trains during a one-month bootstrapping period and operates daily using normalized logs, profiles, C&C scores, and belief propagation.The operation output is an ordered list of suspicious domains for SOC investigation.

IV. SYSTEM DETAILS

The system reduces enterprise traffic to rare external destinations, profiles destination history, and iteratively expands seeded infection findings through belief propagation. It supports both SOC-seeded and no-hint operation, where potential C&C domains and contacting hosts initialize the algorithm.

  • A. Data Normalization and Reduction: The system normalizes timestamps and host identities, extracts web-traffic fields, and excludes destinations represented only by IP addresses.Normalization converts timestamps to UTC and maps DHCP/VPN addresses to hostnames before extracting analysis fields.
  • B. Belief Propagation Algorithm: Belief propagation can start from SOC-provided compromised hosts and malicious domains or from potential C&C domains and their contacting hosts.The no-hint mode runs the C&C detector first and uses its outputs as seeds.
  • B. Belief Propagation Algorithm: Each iteration detects C&C-like domains, scores remaining rare domains, adds domains above threshold Ts, and expands the compromised-host set.The process stops when no new domains qualify or the maximum iteration count is reached, returning expanded host and malicious-domain sets.

C. Detection of C&C communication

The C&C detector identifies automated, periodic communications while tolerating timing outliers and attacker randomization. It combines dynamic interval histograms with enterprise-specific traffic features and a trained scoring model.

  • C. Detection of C&C communication: Dynamic histogram binning clusters inter-connection intervals within width W, allowing periodicity detection despite timing randomization.Each cluster becomes a histogram bin, rather than relying on statically defined intervals.
  • C. Detection of C&C communication: Communications are labeled automated when their interval histogram is sufficiently close to a periodic distribution under Jeffrey divergence threshold JT.The comparison period is set by the hub of the highest-frequency cluster.
  • C. Detection of C&C communication: The detector exploits enterprise traffic regularities, including user-agent popularity and web-referrer information, alongside domain features.Rare user-agent strings are identified from a maintained history and can indicate unpopular software on a host.
  • C. Detection of C&C communication: A linear regression model scores rare automated domains, labeling a domain as C&C when automated connections exist and its score exceeds Tc.Training uses six extracted features and VirusTotal-based reported or legitimate labels.

D. Domain similarity

Domain similarity ranks rare domains against previously labeled malicious domains using connectivity, timing, IP-space, and additional suspiciousness features. A linear regression model supplies the feature weights used during operation.

  • D. Domain similarity: Similarity scoring compares a candidate domain with labeled malicious domains using connectivity, timing correlations, and IP-space proximity.Shorter visit-time differences and shared IP/16 or IP/24 ranges are treated as more suspicious signals.
  • D. Domain similarity: The similarity model uses eight features trained on non-automated rare domains collected over two weeks.Additional C&C-scoring features include NoRef, RareUA, DomAge, and DomValidity.
  • D. Domain similarity: During operation, ComputeSimScore applies the learned feature weights relative to domains labeled malicious in previous belief-propagation iterations.The score supports iterative ranking of candidate domains.

V. EVALUATION ON THE LANL DATASET

The LANL evaluation adapts the enterprise-infection techniques to four challenge cases with less domain and connection information. Despite using fewer features, the belief-propagation framework achieves excellent challenge results.

  • V. EVALUATION ON THE LANL DATASET: The LANL evaluation covers four challenge cases and adapts the enterprise-infection methods to reduced domain and connection information.The authors report excellent results despite using fewer features.

A. The LANL Challenge Problem

The LANL challenge evaluates detection of 20 simulated infection campaigns, using training-derived parameters and limited DNS features. The framework combines timing, domain, and IP-space signals to identify malicious domains, with or without host hints.

  • The LANL dataset contains 20 independent simulated infection campaigns across four cases, with host hints in cases 1–3 but none in case 4.
  • Parameters are selected on one half of the attacks and evaluated on the other, with cases represented in both sets where possible.
  • Thresholds for dynamic histograms: 10- and 20-second histogram bins capture all 33 labeled malicious automated pairs at threshold 0.06, whereas 5-second bins require threshold 0.35.
  • The LANL setting provides fewer features than the enterprise setting because anonymized DNS records lack domain-registration and HTTP-connection information.
  • C&C communication features: The method uses rare domains and combines automated-communication timing with domain connectivity and IP-space similarity to detect C&C activity.
  • C&C communication features: Connection intervals between two malicious domains are much shorter than intervals between a malicious and legitimate domain for the same host.

C. Results

Results show belief propagation can expand from a hint host or detected C&C communication to identify malicious domains in LANL campaigns. On enterprise proxy data, feature-based scoring distinguishes reported from legitimate automated domains and supports discovery of additional suspicious domains.

  • LANL results: Starting from hint host 74.92.144.170, belief propagation labels a detected C&C domain and expands through three further iterations to identify additional malicious domains.
  • LANL results: All 12 labeled malicious domains are detected in training days without false positives, while testing days contain all 12 with one false positive on 3/21.
  • LANL results: Case 4 identifies five malicious domains without host hints and produces no false positives.
  • LANL results: Overall TDR is 98.33% (97.06% testing), FDR is 1.67% (2.94% testing), and FNR is 6.35% (2.94% testing).
  • Enterprise results: On enterprise data, 841 automated domains are scored using VirusTotal-reported versus legitimate labels and learned feature relevance.
  • Enterprise results: At threshold 0.4, automated-domain scoring achieves 57.18% true detection rate and 10.59% false positive rate on training, versus 54.95% and 11.52% on testing.
  • Enterprise results: The most relevant enterprise features include RareUA, DomInterval, IP24, and DomAge, while reported domains receive higher scores than legitimate domains.

B. Validation methodology

The detector is evaluated in no-hints and SOC-hints modes using threshold sweeps, external validation, and manual investigation. Results show tradeoffs between detection accuracy and coverage, while uncovering malicious, suspicious, and previously unknown domains.

  • No-hints validation: Thresholding automated-domain scores from 0.4 to 0.48 reduced C&C labels from 114 to 19 while increasing TDR from 85.08% to 94.7%.The authors fixed the threshold at 0.4 because it detected more malicious domains, including 23 unknown to VirusTotal or SOC.
  • No-hints validation: At belief-propagation thresholds from 0.33 to 0.85, detected domains ranged from 265 to 114 and TDR from 76.2% to 85.1%.At threshold 0.33, 202 malicious or suspicious domains associated with 945 hosts were detected, including 70 new domains.
  • No-hints validation: The no-hints mode identified new malicious and suspicious clusters, including 10 DGA domains unreported by VirusTotal and SOC.Other findings included five domains associated with Sality and 15 domains sharing a URL pattern reported by VirusTotal.
  • No-hints validation: The evaluation labeled 63 detected domains legitimate, including ad-network, gaming, toolbar, and torrent-tracker domains exhibiting suspicious features.Some were considered policy violations despite lacking discovered malicious activity.
  • SOC-hints validation: With 28 IOC domains as seeds, SOC-hints detection returned 137 to 73 domains as similarity thresholds increased, with TDR ranging from 78.8% to 94.6%.At threshold 0.33, 108 detected domains were malicious or suspicious, about four times the seed set.
  • Validation and comparison: Among 108 malicious or suspicious SOC-hints detections, 79 were confirmed by SOC or VirusTotal and 29 were new findings, including DGA domains detected before registration.The validation combined VirusTotal and IOC checks with URL crawling, SiteAdvisor, and analyst investigation.
  • Validation and comparison: The two operating modes overlapped on only 21 domains, motivating deployment of both modes for broader coverage.Thresholds can be selected according to SOC investigation capacity and the tradeoff between accuracy and coverage.

VII. RELATED WORK

The paper situates its enterprise infection detector among approaches for C&C and malicious-domain detection. It emphasizes enterprise-specific traffic features, graph-based relationships among campaign domains, and limitations imposed by attacker evasion and early-stage focus.

  • Positioning: The framework targets early-stage enterprise infections by exploiting relationships among malicious domains associated with the same attack campaign.The authors present this graph-theoretic belief-propagation approach as distinct from prior work.
  • C&C detection: Unlike DISCLOSURE, the C&C detector uses enterprise-specific HTTP features rather than relying on NetFlow features and external intelligence.The comparison highlights a difference in input signals and operational setting.
  • C&C detection: The approach differs from ExecScent because it does not require malware samples to extract templates representing malicious C&C connections.ExecScent is identified as a close prior approach for large-enterprise C&C detection.
  • Malicious-domain detection: Related malicious-domain research detects malicious activity through characteristics of resilient infrastructures and patterns in DNS traffic, including DNS tunneling.The paper places its domain-community approach alongside systems such as EXPOSURE, Notos, Kopis, and Antonakakis et al.
  • Detection scope: Although targeted attacks are difficult to detect, many campaigns exhibit recurring infection-stage patterns that the enterprise detector leverages.The paper reports potential for detecting infections from targeted attacks based on LANL challenge results.
  • Limitations: Attackers may evade the detectors through non-HTTP channels, compromised popular websites, or randomized C&C timing, although some alternatives are uncommon or require different models.The dynamic histogram method is described as resilient to small timing randomization, while completely randomized timing remains challenging.
  • Discussion: The approach complements rather than replaces existing tools, detecting both new enterprise appearances of known campaigns and entirely new campaigns.Because the methods focus on initial infection stages, the paper leaves longer-term monitoring and cross-source correlation for future work.

APPENDIX

The appendix presents example communities of malicious domains and compromised hosts detected by the proposed techniques. One example is shown for no-hint detection, and another illustrates SOC-hints detection with domain and host annotations.

  • Case studies: The appendix reports two example communities of malicious domains and compromised hosts detected by the techniques.These examples are used to illustrate detected communities rather than provide aggregate evaluation results.
  • No-hint case: Figure 7 illustrates a malicious-domain community detected in no-hint mode on 2/13.The passage identifies the operating mode and date of the example.
  • SOC-hints case: Figure 8 depicts a SOC-hints community in which a seed domain, VirusTotal detections, SOC confirmations, and unconfirmed domains are distinguished by shape and color.The figure also marks SOC-confirmed and other compromised hosts while anonymizing hostnames.
Loading 1411.5005v2…