PaperlayerView exact revision on arXiv

Source-linked AI summary

RuleAutoPilot: Synthesizing Deployable Suricata Rules from Network Traffic

Mughees Ur Rehman, Aritran Piplai, Murat Kantarcioglu

arXiv:2609.16231v1cs.CR

TL;DR

Existing LLM-based rule generation often starts from processed threat artifacts, while manual rule authoring requires specialist effort. RuleAutoPilot generates deployable Suricata rules from malware traffic, filters benign background flows, and verifies candidates through execution. With Claude Opus 5 as its backbone, it reaches 0.656 F1 against Claude Code’s 0.623 using 40× fewer tokens.

  • Problem

    Existing automated rule-generation approaches rely on processed artifacts, while manual rule authoring requires expertise in traffic analysis and rule construction and struggles to scale.

    Source: §1 · p. 1

  • Method

    RuleAutoPilot generates Suricata rules directly from malware PCAPs without CTI input and filters recurring benign flows before LLM processing.

    Source: §1 · p. 2

  • Results

    0.656 F1 against Claude Code’s 0.623 with the same Claude Opus 5 backbone, while using 40× fewer tokens.

    Source: §1 · p. 3

  • Takeaways & Limitations

    At a fixed backbone, the results show that RULEAUTOPILOT’s scaffold contributes independently to rule quality relative to Claude Code.

    Source: §1 · p. 3

    Without a live intelligence feed, the generator cannot determine whether an unseen JA3/JA3S fingerprint is malicious.

    Source: §8 · p. 12

Abstract

from arXiv · show

Rule-based Intrusion Detection Systems (IDS) such as Suricata are central to network security, yet crafting effective detection rules demands deep expert knowledge and cannot keep pace with emerging threats. Existing LLM-based approaches can reduce analyst effort, but they either rely on curated threat intelligence that is produced only after the underlying traffic artifacts already exist, or they require costly LLM use without sufficient quality control. We present RuleAutoPilot, an end-to-end agentic framework that generates deployable Suricata rules directly from malware network traffic, with no prior threat intelligence required. A key challenge is noise: network traffic captures often contain a small amount of security-relevant traffic mixed with large volumes of background traffic, which reduces LLM reasoning quality and increases cost. RuleAutoPilot addresses this challenge with a Benign Traffic Fingerprinting stage that removes known benign background flows before LLM processing. Rules that fail syntax checks, do not trigger on the source traffic, or generate false positives on a benign corpus are automatically repaired using structured feedback. Across 1,296 malware PCAPs, execution-grounded verification raises rule quality (F1) from 0.443 to 0.539. On a stratified 200-PCAP subset, RuleAutoPilot on the open-weight gpt-oss-120b reaches near-frontier quality, 0.524 F1 against Claude Opus 5 under Claude Code's 0.623, at 52x lower billed-token cost. Swapping only the backbone to Claude Opus 5, RuleAutoPilot surpasses Claude Code outright, 0.656 F1 against 0.623, at 40x fewer tokens. A stronger backbone raises RuleAutoPilot's own ceiling, but at the same backbone, our scaffold still outperforms Claude Code's, showing the scaffold contributes independently of the backbone.

1 Introduction

This paper targets the rule-authoring bottleneck by generating deployable Suricata rules directly from malware traffic, without curated threat intelligence or analyst-labeled flows. It introduces traffic filtering and execution-grounded verification, and reports that its scaffold exceeds Claude Code’s F1 with the same backbone while using fewer tokens.

1 Introduction

  • Existing LLM rule-generation approaches start from processed artifacts such as CTI reports, PoC write-ups, or analyst-labeled traffic, rather than raw malware traces.The paper frames this as leaving raw-evidence analysis and identification of relevant behavior outside those systems.Source: §1 · pp. 1–2
  • RULEAUTOPILOT generates Suricata rules directly from malware PCAPs without CTI reports, CVE descriptions, or analyst-labeled flows, then verifies syntax, malware triggering, and benign-traffic behavior.The work evaluates 1,296 malware PCAPs and frames rule quality around deployability.Source: §1 · p. 2
  • Benign Traffic Fingerprinting removes 88.8% of background flows while retaining 98.0% of security-relevant flows, reducing average LLM token usage by 1.99×.The filter is built from 981 benign PCAPs.Source: §1 · p. 2
  • Flow-alignment F1 rises from 0.443 to 0.539, while benign false-positive rate falls from 0.128% to 0.006% after execution-grounded validation and repair.Flow-alignment F1 measures how closely generated-rule alerts match flows flagged by expert-authored rules.Source: §1 · pp. 2–3
  • At a fixed gpt-oss-120b backbone, RULEAUTOPILOT reaches 0.524 F1 versus 0.416 for OpenClaw and 0.400 for Hermes, under the same prompt and benign false-positive control.This comparison varies the scaffold while holding the model fixed.Source: §1 · p. 3
  • 0.656 F1 against Claude Code’s 0.623, with 40× fewer tokens, when both use Claude Opus 5.Holding the backbone fixed isolates the scaffold as the source of the gap.Source: §1 · p. 3

2 Background

This section introduces Suricata rules and the distinction between content-based and stateful matching that shapes what a rule can detect. It also situates ET Open as a reference for high-quality signatures.

2 Background

  • The section provides background on Suricata rule structure and community-maintained rulesets used in operational deployments.These rulesets provide the operational context for the paper’s discussion of signature-based detection.Source: §2 · p. 3

2.1 Rule Structure

  • Suricata rules combine an action and header with options that specify matching conditions and metadata for identification and maintenance.The illustrated DNS rule matches a domain in the DNS query field.Source: §2.1 · p. 3

2.2 Deep Packet Inspection and Stateful Matching

  • Content rules inspect packet bytes or parsed protocol fields, while stateful rules express conditions over accumulated network behavior.Encryption hides data above the TLS record layer but can leave handshake information such as the server name visible; threshold, detection_filter, and flowbits support stateful matching.Source: §2.2 · p. 3

2.3 Rule Maintenance and Community Rulesets

  • Indicator feeds match specific URLs, domains, IP addresses, or IP:port pairs, while ET Open also contains signatures for protocol fields, payload content, and malware-specific behavior.The paper uses ET Open as its ground truth for studying how high-quality signatures capture malicious network behavior.Source: §2.3 · p. 3

3 Problem Statement and Threat Model

The problem statement defines deployability as more than plausible rule text: a rule must run, detect source traffic, and remain quiet on benign traffic. The threat model assumes network-visible, rule-expressible malware behavior and excludes adaptive evasion.

3 Problem Statement and Threat Model

  • Deployable rules must be syntactically valid, trigger on source traffic, and stay quiet on benign traffic.The system assumes access to a malware traffic capture or a binary that can be executed in a controlled environment to obtain one.Source: §3 · pp. 3–4
  • RULEAUTOPILOT requires a Suricata validation environment and benign traffic corpus, but not CTI reports, CVE write-ups, attacker-IP lists, PoC descriptions, or malware-family labels as input.Rule generation is assumed to occur in a trusted environment.Source: §3 · p. 4
  • The threat model covers network-visible malware behavior that is at least partly expressible in Suricata rules, but not adversaries that adapt traffic specifically to evade generated rules.Analyzing adaptive evasion is beyond the stated scope.Source: §3 · p. 4

4 RULEAUTOPILOT Architecture

RULEAUTOPILOT prepares malware traces, synthesizes candidate rules, and uses Suricata execution to verify and repair them. Its stages filter recurring benign activity, identify rule-expressible evidence, and use structured verifier feedback for failures.

4 RULEAUTOPILOT Architecture

  • The architecture decomposes PCAPs into enriched flows, removes recurring benign patterns, synthesizes rules from remaining candidates, and validates and repairs them through execution.The four components are Benign Traffic Fingerprinting, the Rule Synthesis Agent, the Rule Verification Agent, and the Rule Repair Agent.Source: §4 · p. 4, Figure 2 · p. 5

4.1 Benign Traffic Fingerprinting

  • Benign Traffic Fingerprinting extracts recurring protocol-level patterns from 981 benign PCAPs and removes matching flows from captures before synthesis.It removes 88.8% of background flows while discarding 2.0% of security-relevant flows.Source: §4.1 · p. 4

4.2 Rule Synthesis Agent

  • The Rule Synthesis Agent selects surviving flows with stable, discriminative evidence expressible in Suricata, including specific protocol indicators or behavior visible across multiple flows.Triage routes specific-indicator evidence to content rules, cross-flow behavioral evidence to stateful rules, and captures with both to each; stateful rules typically pair a rate condition with an anchor.Source: §4.2 · pp. 4–5

4.3 Rule Verification Agent

  • The Rule Verification Agent accepts a candidate only after it passes syntax, source-malware trigger, and benign false-positive checks in sequence.For stateful flowbits, the verifier checks setter and tester rules as a pair rather than testing the setter alone.Source: §4.3 · p. 5

4.4 Rule Repair Agent

  • The Rule Repair Agent uses deterministic rewrites for precise failure patterns, otherwise retrieving failure-specific guidance and prompting from structured verifier feedback.For stateful bundles, it uses seven behavioral failure patterns and withholds stripping a threshold clause because that would remove the condition the rule exists to enforce; repair attempts are capped at K = 3.Source: §4.4 · pp. 5–6

5 Dataset and Ground Truth Construction

The benchmark combines malware and benign traffic corpora, labels security-relevant flows using ET Open rule triggers, and selects a stratified subset for cross-system comparison.

5 Dataset and Ground Truth Construction

  • The malware benchmark contains 1,296 PCAPs across 192 families, spanning samples from 2014 to 2026.Fifteen unlabeled PCAPs are treated as singletons.Source: §5 · p. 6
  • The two disjoint benign corpora contain 981 DIKE-derived PCAPs and 191 AsiaCCS benchmark PCAPs.DIKE traffic builds the fingerprint library and supports benign replay and false-positive testing; the AsiaCCS corpus is VirusTotal-verified benign.Source: §5 · p. 6
  • Each malware PCAP triggers at least one ET Open rule, while no benign PCAP triggers any; flows are labeled by whether they trigger an ET Open rule.Each security-relevant flow maps to its triggered rule or rules.Source: §5 · p. 6
  • The cross-system evaluation uses a 200-capture subset stratified by ground-truth rule requirements, with the stateful stratum oversampled from 81/1,296 to 40/200 captures.Stateful cases require a rate or ordered sequence that no single-packet rule expresses; encrypted cases draw from only 15 families.Source: §5 · p. 6, Table 1 · p. 6

6 Evaluation

Evaluation measures flow-level rule alignment and deployability across malware and benign traffic, then compares systems on a stratified 200-capture subset. It tests end-to-end performance, component effects, and the contributions of the scaffold and backbone.

6.3 RQ1: End-to-End Rule Generation

  • Across 1,296 malware PCAPs, RULEAUTOPILOT reaches FAS-F1 0.54, with an 85.3% syntax pass rate, an 81.1% malware trigger rate, and a 0.006% benign FPR.The three tested prompt configurations change FAS-F1 by only 0.018.Source: §6.3 · p. 8, Table 3 · p. 9
  • On post-cutoff malware, RULEAUTOPILOT reaches FAS-F1 0.533; on 191 benign captures, it emits no malware-style rule and reaches 99% flow-classification accuracy.Fingerprinting removes 98.0% of flows on the benign benchmark and filters 126 captures entirely, requiring no LLM call.Source: §6.3 · p. 8
  • On same-family targets, generated rules reach FAS-F1 0.67 versus 0.75 for expert-authored rules; on other-family targets, scores fall to 0.06 versus 0.16.Generated rules trigger slightly more often on same-family targets (0.58 versus 0.55), while cross-family trigger rates are 0.09 versus 0.07.Source: §6.3 · p. 8, Table 4 · p. 9

6.4.1 Isolating RULEAUTOPILOT’s Own Components

  • Benign fingerprinting removes 88.8% of flows across 1,296 PCAPs while discarding 2.0% of security-relevant flows, cutting median flows per capture from 65 to 12.Filtering reduces mean tokens per PCAP from 40.16k to 20.20k and leaves overall FAS-F1 nearly unchanged, from 0.53 to 0.54.Source: §6.4.1 · p. 8, Figure 4 · p. 9
  • Across 1,296 PCAPs, cumulative verification raises FAS-F1 from 0.443 to 0.539; malware replay raises detection quality, while benign replay cuts FPR from 0.128% to 0.006%.Syntax validation raises the syntax pass rate from 81.8% to 85.2%, but barely changes FAS-F1.Source: §6.4.1 · p. 9, Table 5 · p. 9
  • On a family-stratified 100-PCAP subset, swapping gpt-oss-120b for GPT-5.5 raises FAS-F1 from 0.561 to 0.701, while verification stages retain their distinct roles.GPT-5.5 raises syntax pass rate from 85.4% to 90.0%; benign replay lowers gpt-oss-120b’s FPR from 2.09% to 0.05%.Source: §6.4.1 · pp. 9–10, Figure 5 · p. 11

6.4.2 Our Scaffold vs. Other Scaffolds, Same Backbone

  • On the shared gpt-oss-120b backbone, RULEAUTOPILOT reaches FAS-F1 0.524 versus 0.416 for OpenClaw and 0.400 for Hermes, using 37× fewer tokens per capture than OpenClaw.RULEAUTOPILOT also emits half as many rules per capture and alerts on 1.17× the reference flow count, versus 2.1–2.5× for the general-purpose agents.Source: §6.4.2 · p. 10

6.4.3 Backbone and Baseline: From Regex to a Frontier Model

  • With an identical backbone, RULEAUTOPILOT reaches FAS-F1 0.656 versus Claude Code’s 0.623, surpassing it with a 40× reduction in tokens per capture.It trades recall (0.830 versus 0.967) and coverage (95% versus 100%) for higher FAS-precision (0.618 versus 0.536) and a lower over-fire ratio (1.49× versus 10.99×).Source: §6.4.3 · p. 11

7 RQ3: Rule Generation Under Encryption

RuleAutoPilot maintains detection under encryption, though FAS-F1 and precision fall as encrypted traffic increases. Its encrypted-trace rules use visible TLS SNI and DNS query names, as well as stateful rules.

Performance across the encryption spectrum

  • FAS-F1 falls from 0.614 on cleartext to 0.355 on encrypted captures, while coverage rises from 80.4% to 88.2%.As encryption increases, precision drops faster than recall, while rules that are written fire more reliably.Source: §7 · p. 11, Table 7 · p. 12
  • The three bands contain 331 cleartext, 889 mixed, and 76 encrypted captures, classified by the share of flow records dissected as TLS or QUIC.Protocol dissection captures encrypted traffic even when it uses a non-standard port.Source: §7 · p. 11, Table 7 · p. 12

What RuleAutoPilot writes when the payload is encrypted

  • On 40 fully encrypted samples whose ground truth requires a content match, RuleAutoPilot answers 34 and writes 89 rules, including 26 matching tls.sni and 38 matching dns.query.Of 27 stateful rules on these samples, 13 combine a threshold with tls.sni, compared with 5 of 28 on cleartext samples.Source: §7 · p. 12

8 Discussion

The discussion argues that a task-specific scaffold can make local rule generation viable with open-weight models, including when traffic cannot be sent to a third-party API. It also identifies limits in attribution, rule alignment, and encrypted-traffic visibility.

Scaffolding and local deployment

  • On an open-weight backbone, RuleAutoPilot’s scaffold comes within 0.099 F1 of the frontier system at a fraction of the cost, and at equal backbone surpasses that system outright.Source: §8 · p. 12

What happens when more fields are encrypted

  • Wider adoption of Encrypted Client Hello or DNS over HTTPS/TLS would hide the SNI and DNS query fields current rules use, increasing reliance on stateful rules based on network behavior.The cited examples are connection timing and volume.Source: §8 · p. 12

Where FAS goes to zero, and why

  • Of 1,296 PCAPs, 306 score zero FAS-F1; missing attribution for uninterpretable JA3/JA3S fingerprints and over-generation are identified as structural causes.Another 207 PCAPs have partial alignment (0 < FAS-F1 < 0.5), with mean precision 0.28 and recall 0.77.Source: §8 · p. 12

What FAS does not measure

  • FAS measures agreement with the ET Open rule firing on a capture, so a different but valid indicator can score as a false positive; the corpus cannot establish live-deployment detection quality.The authors also report over-fire because FAS alone cannot distinguish rules that fire twice too often from those that fire fifty times too often.Source: §8 · p. 12

9 Related Work

Related work spans traditional signature extraction, LLM-based rule generation, and LLMs for traffic analysis. The discussion distinguishes generating a rule from structured intelligence or prefiltered traffic from discovering relevant behavior in noisy captures and validating executable rules.

9.1 Traditional Signature Generation

  • Honeycomb, Autograph, and Polygraph extract reusable traffic invariants, but produce byte-level signatures rather than protocol-aware Suricata rules.The paper frames its non-LLM baselines as modern indicator extractors and suggests encryption shifts the challenge from invariant extraction to invariant selection.Source: §9.1 · pp. 12–13

9.2 LLM-Based Rule Generation Systems

  • RuleMaster+ [27] learns rule-writing conventions from exploit descriptions, while FALCON [31] generates Snort and YARA rules from CTI reports; both start from structured intelligence rather than traffic.The paper says these approaches do not address identifying security-relevant flows in a noisy capture before rule writing.Source: §9.2 · p. 13
  • Moreno et al. generate Suricata rules from industrial captures prefiltered by attacker IP, whereas RuleAutoPilot must identify relevant flows in noisy sandbox PCAPs.RuleAutoPilot’s execution-feedback repair adds +0.083 FAS-F1 over syntax-only verification, from 0.456 to 0.539.Source: §9.2 · p. 13
  • GRIDAI [26] generates or repairs rules for Web-attack samples after deciding whether each is new or already covered, unlike RuleAutoPilot’s noisy multiprotocol discovery setting.In RuleAutoPilot’s setting, relevant flows must be identified before a rule is written.Source: §9.2 · p. 13
  • RuleXploit, Hex2Sign, GenTI [19], and [29] are evaluated with BERTScore or an LLM judge rather than execution; the paper notes that similarity does not establish deployability.The passage says practitioners treat execution evidence as a precondition for adoption [16].Source: §9.2 · p. 13
  • General-purpose agents serve as baselines using the same prompt and evidence, and the paper says prior work had not isolated scaffold effects by holding the backbone fixed.These agents are not built specifically for rule generation.Source: §9.2 · p. 13

9.3 LLMs for Network Traffic Analysis

  • TrafficLLM [14] and NetGPT [30] study traffic understanding or synthesis, while RuleAutoPilot specifically transforms raw PCAPs into validated Suricata signatures.The cited traffic-modeling systems use raw traffic or packet- and flow-level representations.Source: §9.3 · p. 13

10 Conclusion

The conclusion presents RuleAutoPilot as a framework for synthesizing deployable Suricata rules from network traffic and reports that its scaffold can improve results while reducing token use. It also describes encrypted-traffic dependencies and a boundary around indicators that cannot be interpreted from traffic alone.

Conclusion

  • With Claude Opus 5 as the backbone, RuleAutoPilot outperforms Claude Code while using 40× fewer tokens per capture.The conclusion identifies the scaffold, rather than the backbone, as the central finding.Source: §10 · p. 13
  • Across 1,296 malware PCAPs, the conclusion says execution-grounded verification and benign traffic fingerprinting raise rule quality while reducing cost and false positives.It describes the evaluation as showing the pipeline scales to a large, real-world corpus.Source: §10 · p. 13
  • Under encryption, precision falls as payload disappears, and generated rules rely more on stateful rules and visible fields such as TLS SNI and DNS queries.The conclusion notes that ET Open continues to publish plaintext-targeting signatures.Source: §10 · p. 13
  • The proposed future extension is live threat intelligence for indicators such as JA3, JA3S, or JA4 fingerprints, which have no interpretable pattern in the traffic itself.A reputation feed could supply evidence about whether an unseen fingerprint is malicious.Source: §10 · p. 13

Ethical Considerations

The study used malware traces from sandboxed or directly ingested PCAPs; no malware binaries were distributed or tested outside the sandbox, and the corpus was not released.

Ethical Considerations

  • Malware binaries were not distributed or tested outside isolated sandbox environments; Triage PCAPs were ingested directly as traffic traces.The authors characterize the generated rules as comparable to public ET Open signatures and report no novel dual-use risks beyond those of existing public rulesets.Source: Ethical Considerations · p. 16

Open Science

The artifact release includes the pipeline and evaluation materials, but not the malware corpus or binaries. The appendix describes dataset coverage and family-label distribution.

Open Science

  • The release includes the pipeline, evaluation scripts, and baseline harness, but the malware corpus and binaries are withheld for ethical reasons.Source: Open Science · p. 16

A Dataset and Ground-Truth Details

A.1 Temporal Coverage of Malware PCAPs

A.2 Temporal Coverage of Triggered ET Open Rules

A.3 Malware Family Distribution

  • Normalized malware-family labels are grouped by assigned PCAP count, with unlabeled PCAPs treated as SINGLETON instances.Source: §A.3 · p. 17

B Experimental Setup Details

The appendix documents experimental settings, flow and prompt representations, repair procedures, and supplementary analyses. Its family-generalization analysis reports near-ground-truth fidelity for same-family targets, with deviations concentrated in some family pairs.

B Experimental Setup Details

  • The primary backbone is gpt-oss-120b; prompting variants include zero-shot, chain-of-thought, few-shot, and combined chain-of-thought plus few-shot.The setup also describes protocol-spanning examples and separate benign fingerprinting, verification, and false-positive test corpora.Source: B Experimental Setup Details · p. 17

E Baseline Harness Prompt

C Flow Representation and Benign Fingerprinting

  • The synthesis agent receives protocol-aware flow records, and the benign fingerprint library covers recurring identifiers such as DNS names, HTTP hosts, destination IPs, and local resolution artifacts.Source: §C.1 · p. 18, §C.2 · p. 18

C.3 Filtering Impact

  • Benign fingerprinting removes most background flows while preserving nearly all security-relevant flows across the 1,296-PCAP evaluation corpus.Source: §C.3 · p. 18, Table 13 · p. 20

D Prompt Templates

  • Prompt variants range from task instructions and flows alone to structured reasoning steps or worked flow-to-rule examples.The chain-of-thought prompt asks for protocol evidence, suspicious indicators, traffic direction, and rule anchors.Source: §D.1 · p. 18, Figure 6 · p. 18, §D.2 · p. 19

G Repair Knowledge Base

I Rule Family Generalization

  • RULEAUTOPILOT-generated rules reach same-family target FAS-F1 of 0.67 versus 0.75 for ET Open, with comparable overall generalization behavior.Source: I Rule Family Generalization · p. 26
  • Across top family pairs, trigger-rate deviations include asyncrat → sfuzuan at +0.47 and vidar → asyncrat at −0.66; other deviations are concentrated in specific pairs.Figure 13 encodes over-triggering in red, under-triggering in blue, and matches in white.Source: I Rule Family Generalization · p. 26, Figure 13 · p. 26
Loading 2609.16231v1…