PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoDP-LLM: Automating Data Pre-processing for Intrusion Detection Systems using Large Language Models

Bao-Phong Nguyen, Gia-Khanh Pham, Thai-Duong Do, Mai Xuan Trang, Minh-Tuan Le, Xuan-Nam Tran, Huan Vu, Tien-Cuong Nguyen, Vu-Duc Ngo, Thien Van Luong

arXiv:2610.05369v1cs.CRcs.AI

TL;DR

IDS preprocessing for high-dimensional, heterogeneous traffic data is labor-intensive and often involves repeated pipeline exploration. AutoDP-LLM uses deterministic planning and LLM specialist agents to generate and validate compact preprocessing pipelines; on two benchmark datasets, it reports the highest weighted-F1 value among compared baselines, with runtime advantages varying by dataset.

  • Problem

    Designing effective preprocessing for high-dimensional, heterogeneous IDS datasets requires labor-intensive, dataset-dependent choices and repeated trial-and-error exploration.

    Source: §I · p. 1

  • Method

    AutoDP-LLM combines deterministic host-side planning with LLM specialist agents to generate executable preprocessing and feature-reduction pipelines using semantic and training-derived statistical evidence.

    Source: §I · p. 2, 5) Summary and Discussion · p. 13

  • Results

    AutoDP-LLM reports the highest weighted-F1 value among compared baselines on both datasets, exceeding the strongest listed baseline point estimate by 0.24 percentage points on UNSW-NB15 and 2.79 on NSL-KDD.

    Source: 5) Summary and Discussion · p. 13

  • Takeaways & Limitations

    The framework produced compact, executable pipelines across UNSW-NB15 and NSL-KDD, though practical benefits depend on dataset, downstream classifier, and LLM backbone.

    Source: 5) Summary and Discussion · p. 13

Abstract

from arXiv · show

The increasing complexity and scale of modern cyber-attacks demand intelligent and computationally efficient Intrusion Detection Systems (IDS). However, designing effective data pre-processing pipelines traditionally involves substantial trial-and-error effort and repeated evaluation of alternative configurations. For large, high-dimensional network traffic data, this process can create a significant computational burden. In this work, we propose AutoDP-LLM, an automated pre-processing framework designed to reduce manual pipeline development and computational overhead. Specifically, AutoDP-LLM leverages Large Language Models (LLMs) to autonomously generate and validate executable data pre-processing pipelines. The framework combines deterministic host-side planning with LLM-based specialist agents to formulate data-processing strategies, synthesize executable code, and adaptively determine retained feature sets using semantic reasoning and training-derived statistical evidence, without requiring a predefined feature budget. Focusing on multiclass intrusion detection, we evaluate AutoDP-LLM on the UNSW-NB15 and NSL-KDD benchmark datasets using multiple downstream classifiers. Comparative experiments against conventional feature-selection methods show that AutoDP-LLM achieves competitive detection performance while automating the generation of compact and executable pre-processing pipelines. Component-level ablation experiments further demonstrate the complementary contributions of the semantic and statistical feature-reduction components. The repeated generation, validation, execution, and assessment of candidate pipelines are amenable to parallel execution, highlighting the potential of scalable computing environments, including high-performance computing (HPC) systems, to support automated IDS pipeline development.

I. INTRODUCTION

IDS data preparation is consequential but labor-intensive: high-dimensional, heterogeneous datasets require dataset-dependent preprocessing and repeated pipeline exploration. AutoDP-LLM automates pipeline generation and validation, while the related work describes conventional feature-selection approaches and their manual design burden.

I. INTRODUCTION

  • High-dimensional, heterogeneous IDS datasets make preprocessing important, yet pipeline design often requires repeated trial and error and dataset-specific redesign.Repeated candidate generation, execution, and evaluation can add computational overhead and scalability challenges.Source: §I · p. 1
  • AutoDP-LLM combines deterministic host-side planning with specialist agents to generate and validate executable data-transformation, resampling, and feature-reduction pipelines.Its adaptive feature-reduction strategy combines semantic recommendations with training-derived Random Forest and mutual-information evidence, without a predefined feature budget.Source: §I · p. 2
  • The paper evaluates AutoDP-LLM on UNSW-NB15 and NSL-KDD for multiclass intrusion detection and reports higher weighted F1-scores than reimplemented baselines with competitive execution time.The evaluation covers multiple classifiers, repeated generation runs, LLM backbones, and measures including performance variability, API latency, and computational overhead.Source: §I · p. 2

for IDS

  • Conventional filter, wrapper, and embedded feature-selection methods use statistical criteria, classifier performance, or model-derived importance, respectively.Their criteria, thresholds, and model-specific configurations generally require manual design and iterative evaluation.Source: A. Conventional Data Pre-processing… · p. 2

B. LLM-based approaches in cybersecurity

Prior cybersecurity LLM work emphasizes analysis and interpretation, while automated construction of complete IDS preprocessing pipelines remains little explored. AutoDP-LLM uses a validated, three-stage workflow, with dataset ingestion and sanitization preceding its automated preprocessing and downstream evaluation.

B. LLM-based approaches in cybersecurity

III. PROPOSED AUTODP-LLM FRAMEWORK

  • AutoDP-LLM divides preprocessing into analysis and host planning, transformation and resampling, and feature reduction, using isolated specialist agents and structured validation.The overall IDS workflow places ingestion and sanitization before automated preprocessing and downstream evaluation.Source: §III · p. 3, A. Overall Pipeline Architecture · p. 3

A. Overall Pipeline Architecture

  • The modular workflow is designed to accommodate datasets with different schemas, categorical cardinalities, class distributions, and sample sizes.Dataset-specific feature descriptions can inform semantic reasoning, but are not required for statistical feature completion.Source: A. Overall Pipeline Architecture · p. 3

B. Data Ingestion and Sanitization Phase

  • Data ingestion identifies the dataset schema and target, then deterministic sanitization normalizes the schema and constructs training and held-out test partitions.Known benchmark conventions guide operations such as resolving target definitions and removing record identifiers or explicit leakage variables.Source: B. Data Ingestion and Sanitization Phase · p. 4

C. Automated Data Preprocessing Phase

AutoDP-LLM profiles training data before planning preprocessing, and its host constructs a validated plan with conservative removal rules and bounded resampling candidates. The resampling decision is restricted to one family choice and does not use held-out test information.

C. Automated Data Preprocessing Phase

  • Train-only profiling summarizes feature statistics and types, missingness, constants, categorical cardinalities, class counts, and selected feature associations before the analysis agent is called.Feature-target associations use one-way ANOVA for numeric features and Cramér’s V for categorical features; feature-feature numeric correlations at or above 0.85 are included.Source: C. Automated Data Preprocessing Phase · p. 4

1) Analysis and Host Planning:

  • The analysis agent chooses exactly one resampling family using training imbalance, class support, encoding policy, computational cost, and workflow compatibility.It cannot access validation or test labels or test distributions, and it does not select a winning candidate or set resampling intensity.Source: 1) Analysis and Host Planning · p. 5
  • The host constructs a validated plan that limits conservative feature removal to all-missing, constant, exact-duplicate, or metadata-verified identifier and leakage features.It builds only a none candidate when no resampling is selected; otherwise, it builds none plus mild and moderate candidates, each preserving the full original training class set.Source: 1) Analysis and Host Planning · p. 5
  • Oversampling and hybrid candidates target classes below half the median training-class count, with mild and moderate expansion bounded to 1.05 and 1.10 times original training size.Synthetic-neighbour methods require at least two original observations in every targeted class; unsupported candidates are rejected.Source: 1) Analysis and Host Planning · p. 5
  • For undersampling and cleaning, mild candidates target largest-count classes and moderate candidates target classes above the training median, with removal policies bounded by the plan.Tomek links permits at most one mild or two moderate passes; edited nearest neighbours uses three neighbours for both candidates.Source: 1) Analysis and Host Planning · p. 5

2) Transformation & Resampling Specialist Agent:

The Transformation & Resampling Agent applies planned preprocessing while learning transformations from training data and resampling only training rows. Feature reduction combines semantic feature lineage and pruning with statistical evidence to select a compact subset without a fixed feature budget.

2) Transformation & Resampling Specialist Agent:

  • Transformations are fitted on training data and applied to held-out features, while the approved resampling candidate affects training data only.The agent performs guarded drops and imputation, transformation, encoding and scaling, then bounded candidate resampling.Source: 2) Transformation & Resampling… · pp. 5–6
  • Log transformation eligibility is assessed on the original training data; categorical sources are routed to binary, one-hot, or target encoding by cardinality.Numeric features require finite, non-negative usable values, at least 20 distinct values, skewness above 2.0, and the specified upper-tail condition for log(1 + x).Source: 2) Transformation & Resampling… · pp. 5–6

3) Feature-Reduction Specialist Agent:

  • Semantic reduction selects feature groups using training-only Random Forest evidence, retaining at most 15 seed representatives after the importance threshold.Candidates use exact schema names, receive semantic roles and priority tiers, and protected status prioritizes—but does not exempt—a group from the importance threshold.Source: 3) Feature-Reduction Specialist Agent · pp. 6–7
  • Statistical completion combines RF and mutual-information percentiles, admits candidates with S_j ≥ 0.55, and rejects candidates correlated at least 0.90 with selected columns.The resulting feature count is determined after filtering, without a predefined feature budget; thresholds are fixed across generation runs and both datasets.Source: 3) Feature-Reduction Specialist Agent · p. 7

4) Contract Validation & Repair:

Contract validation checks generated decisions and programs before execution, tests their behavior in isolated fixtures, and uses bounded repair attempts when validation fails.

4) Contract Validation & Repair:

  • Static validation checks the host-built plan and verifies generated programs’ structure, interfaces, metadata, policy constants, and approved sampler families.Stage 3 checks also enforce candidate-name, role, tier, and protected-candidate limits.Source: 4) Contract Validation & Repair · p. 7
  • Isolated runtime validation tests generated functions on compact fixtures, including training-only fitting, schema and lineage checks, deterministic replay, and resampling constraints.Stage 3 checks also inspect model configurations, the mutual-information mask, test isolation, and feature-selection audits.Source: 4) Contract Validation & Repair · p. 7
  • A failed response triggers diagnostic feedback and a fresh isolated call to regenerate the complete artifact, with one correction attempt for Stage 1 and two for Stages 2 and 3.The accepted programs are combined and checked for syntax validity before the module is saved.Source: 4) Contract Validation & Repair · p. 7

5) Data Pipeline Execution:

The execution controller fixes an adaptive feature set and selects among resampling candidates using validation data before downstream evaluation. The study identifies task-level parallelism but evaluates the workflow on a single computing platform.

5) Data Pipeline Execution:

  • A stratified 20% validation split is fixed, then the feature set is selected on the unresampled candidate and locked before resampling candidates are compared.LightGBM weighted F1 selects among none, mild, and moderate resampling when applicable; ties within 0.003 favor lower intensity.Source: 5) Data Pipeline Execution · p. 8

D. Downstream Evaluation Phase

  • Six downstream classifiers use the same locked feature set and resampling winner, and each is evaluated once on held-out test data.Weighted F1 is primary, with accuracy, weighted precision, and weighted recall reported as complementary measures; held-out test results do not guide selection.Source: D. Downstream Evaluation Phase · p. 8

E. Scalability Considerations

  • The workflow was evaluated on a single computing platform, though independently generated pipelines and downstream models offer task-level parallelism.The paper identifies multicore, distributed, and HPC-enabled environments as possible settings for future investigation.Source: E. Scalability Considerations · p. 8

IV. EXPERIMENTAL RESULTS AND DISCUSSION

The experiments assess multiclass intrusion detection on UNSW-NB15 and NSL-KDD using predefined splits, weighted evaluation metrics, and six classifiers. Preprocessing is fitted only on training data.

A. Overview of Benchmark Datasets

1) UNSW-NB15 Dataset:

  • UNSW-NB15 contributes 175,341 training and 82,332 test records across ten retained classes, with 42 predictor features after removing id and label.The dataset is highly imbalanced, particularly for Worms and Backdoor.Source: 1) UNSW-NB15 Dataset · p. 8

2) NSL-KDD Dataset:

  • NSL-KDD uses 125,973 training and 22,544 test samples, with 40 predictor features and five grouped attack classes.Difficulty score and the zero-variance num - outbound cmds feature are removed.Source: 2) NSL-KDD Dataset · p. 8

1) Evaluation Metrics:

2) Classification Models:

  • Six classifiers are considered, using their software-library default hyperparameters unless otherwise specified.The listed models include Random Forest, MLP, XGBoost, LightGBM, and CatBoost.Source: 2) Classification Models · p. 9

3) Baseline Methods:

The evaluation compares four preprocessing or feature-selection baselines, specifies its workstation environment, and reports results using repeated pipeline-generation runs.

3) Baseline Methods:

  • Four comparison methods are LiteShield-MI-FS, XGBoost-FS, IGRF-RFE, and DSSTE.They are described respectively as mutual-information selection, XGBoost feature-importance selection, a hybrid Information-Gain Random Forest/RFE method, and a domain-specific statistical transformation and encoding baseline.Source: 3) Baseline Methods · p. 9

4) Experimental Environment:

  • Experiments ran on an Intel Core i5-13500 workstation with an RTX 3060 GPU, Ubuntu 24.04.4 LTS, Python 3.12.3, and the listed machine-learning libraries.The listed libraries are scikit-learn, xgboost, lightgbm, and catboost.Source: 4) Experimental Environment · p. 9

C. Results and Discussion

  • Each configuration used 30 separately generated, validated, executed, and evaluated pipelines; reported means and standard deviations summarize run-to-run dispersion under fixed data and downstream random-state settings.Classification-metric standard deviations are expressed in percentage points; execution-time and operational-measure units follow their table headings.Source: C. Results and Discussion · p. 9
  • Bold marks best performance in the designated performance, baseline, ablation, and cross-LLM tables; retained-feature counts are reported for comparison, not as a best-performing result.For the operational comparison, lower output-token counts, API latency, monetary cost, and training time are considered better.Source: C. Results and Discussion · pp. 9–10

1) Performance on Benchmark Datasets:

On both datasets, AutoDP-LLM reports the highest weighted-F1 value among the listed baseline point estimates, but the runtime comparison is not uniformly favorable. Across classifiers, the strongest weighted-F1 and runtime results differ by dataset and metric.

1) Performance on Benchmark Datasets:

  • 78.41±0.21% weighted F1 is shared by Random Forest and XGBoost on UNSW-NB15; XGBoost has the lowest mean execution time at 106.38 ± 9.77 s.MLP has the highest mean accuracy (77.51 ± 0.64%), but a lower weighted F1-score of 77.76 ± 0.49%.Source: 1) Performance on Benchmark Datasets · p. 10
  • 76.38±0.99% weighted F1 is highest for MLP on NSL-KDD, while CatBoost has the highest mean accuracy at 79.24 ± 0.26%.CatBoost’s weighted F1-score is 76.13±0.33%; its mean execution time is 99.94±14.61 s versus MLP’s 112.65±16.28 s, and its observed dispersion is lower.Source: 1) Performance on Benchmark Datasets · p. 10
  • Across runs, retained feature counts stay within 26–28 for UNSW-NB15 and 24–26 for NSL-KDD.Source: 1) Performance on Benchmark Datasets · p. 10

2) Comparison with Baseline Methods:

  • 78.41 ± 0.22% vs 78.17% for LiteShield-MI-FS (UNSW-NB15), and 76.38±0.99% vs 73.59% for DSSTE (NSL-KDD): AutoDP-LLM leads weighted-F1 point estimates but runs slower than baselines on NSL-KDD.Comparisons use XGBoost for UNSW-NB15 and MLP for NSL-KDD, and are interpreted descriptively.Source: Table III · p. 10, 2) Comparison with Baseline Methods · p. 10

3) Ablation Study on Feature-Reduction Components:

Ablations show that the statistical and semantic feature-reduction components contribute differently across datasets. GPT-5.4 provided the strongest fixed-classifier results and the lowest API latency among the evaluated backbones.

3) Ablation Study on Feature-Reduction Components:

  • Removing RF-MI Completion reduced weighted F1 from 78.41±0.22% to 77.28±0.17% on UNSW-NB15 and from 76.38 ± 0.99% to 75.03 ± 0.47% on NSL-KDD.The respective decreases were 1.13 and 1.35 percentage points.Source: 3) Ablation Study on Feature-Reduction… · p. 11
  • Removing Semantic Pruning lowered weighted F1 to 77.90±0.10% on UNSW-NB15 and 76.26±1.14% on NSL-KDD, decreases of 0.51 and 0.12 percentage points.Source: 3) Ablation Study on Feature-Reduction… · p. 11
  • The full configuration reached weighted F1-scores of 78.41 ± 0.22% on UNSW-NB15 and 76.38 ± 0.99% on NSL-KDD; semantic guidance mattered more on NSL-KDD.Removing the entire semantic branch left UNSW-NB15 only 0.08 points lower, but reduced NSL-KDD by 1.85 points.Source: 3) Ablation Study on Feature-Reduction… · p. 11

4) Ablation Study on LLM Backbones:

5) Summary and Discussion:

AutoDP-LLM produced compact preprocessing pipelines with competitive multiclass detection results across UNSW-NB15 and NSL-KDD. The discussion reports descriptive baseline comparisons, backbone-dependent trade-offs, and remaining research directions.

5) Summary and Discussion:

  • AutoDP-LLM retained 26–28 features on UNSW-NB15 and 24–26 on NSL-KDD while maintaining competitive multiclass classification performance.RF–MI Completion contributed to performance on both datasets; the observed benefit of semantic guidance was more pronounced on NSL-KDD.Source: 5) Summary and Discussion · p. 13
  • 0.24 percentage points on UNSW-NB15 and 2.79 on NSL-KDD: AutoDP-LLM's mean weighted-F1 exceeded the strongest listed baseline point estimate, descriptively.No inferential comparison of paired test-set predictions was performed, and runtime advantage was not consistent across datasets.Source: 5) Summary and Discussion · p. 13
  • All three evaluated LLM backbones generated executable pipelines for both datasets, but performance, latency, token use, training-time, and cost trade-offs differed.GPT-5.4 had the fewest output tokens and lowest mean API latency under representative-classifier configurations, while DeepSeek-V4-Pro had the lowest monetary cost.Source: 5) Summary and Discussion · p. 13, §V · p. 14

V. CONCLUSION AND FUTURE WORK

  • AutoDP-LLM automates IDS preprocessing and feature reduction, but its practical benefits depend on dataset, downstream classifier, and selected LLM backbone.The conclusion characterizes this as a practical approach within the evaluated scope.Source: 5) Summary and Discussion · p. 13, §V · p. 14
  • 78.41% on UNSW-NB15 and 76.38% on NSL-KDD: these were the highest mean weighted F1-scores in the representative GPT-5.4 experiments.The same core framework was applied to both datasets without changing its multi-agent architecture, providing initial evidence of adaptability to different schemas and class distributions.Source: §V · p. 13
  • Applicability to raw packet captures and heterogeneous system logs, HPC-enabled implementations, locally deployed smaller open-weight LLMs, and continuous data streams remain future research directions.The paper proposes investigating these extensions rather than reporting them as evaluated capabilities.Source: §V · p. 14

APPENDIX A EXECUTION CONFIGURATIONS, HYPERPARAMETERS, AND

The appendix documents execution configurations, prompts, and input formats, while describing the staged agent workflow and its validated plan handoff. It also reports settings intended to reduce output variation without guaranteeing identical hosted-model responses.

SYSTEM PROMPT TEMPLATES

  • Each agent runs as an independent LLM call, making the workflow multi-agent rather than one extended conversation.The appendix documents model identifiers and generation settings; Table VIII summarizes hyperparameter configurations and design rationales.Source: Appendix A · p. 14, Table VIII · p. 14

1) Shared Base Context (All Agents):

  • Every agent receives a shared base context, with the dataset name substituted at runtime and an optional prior preprocessing block appended when applicable.That prior block represents the fixed, hand-written Data Ingestion and Profiling stage and supplies the Sanitized Dataset.Source: 1) Shared Base Context (All Agents) · p. 14

2) Analysis and Host Planning (Step 1):

  • Step 1 combines train-only profiling, resampling-family decision, and deterministic plan construction; the host validates the agent's resampling decision.The host then synthesizes the complete Validated Preprocessing Plan as Stage 1's output for downstream agents.Source: 2) Analysis and Host Planning (Step 1) · p. 14, Appendix B · p. 14

3) Transformation and Resampling Specialist Agent (Step 2):

  • The Transformation and Resampling Specialist Agent consumes the Validated Preprocessing Plan to generate transformation and resampling code.Its listed functions include guarded drops and imputation, transformation, encoding and scaling, and bounded candidate resampling.Source: 3) Transformation and Resampling… · p. 14

4) Feature-Reduction Specialist Agent (Step 3):

  • The Feature-Reduction Specialist Agent implements semantic guard and feature lineage, evidence-guided semantic pruning, and threshold-based RF-MI completion.This stage generates the feature-reduction function.Source: 4) Feature-Reduction Specialist Agent… · p. 14

APPENDIX B JSON SCHEMA FOR INTER-AGENT PRE-PROCESSING PLAN

  • A strict single-field JSON decision specifies the resampling family, while a schema defines the complete data-exchange contract for the Validated Preprocessing Plan.Validated plan fields and authoritative execution constants are passed downstream within fixed delimiters to preserve state integrity; Fig. 7 shows the plan schema.Source: Appendix B · p. 14, Figure 7 · p. 16
Loading 2610.05369v1…