PaperlayerView exact revision on arXiv

Source-linked AI summary

Are We Shooting Flies with Cannons? Trade-off Analysis for AI-based 5G Intrusion Detection

Federica Uccello, Simin Nadjm-Tehrani

arXiv:2608.26844v1cs.CR

TL;DR

The paper asks whether computationally expensive LLMs are justified for tabular 5G intrusion detection, where the relevant trade-off includes detection performance and resource use. It compares XGBoost, TabNet, and an LLM under zero-shot and few-shot prompting, finding that traditional ML—especially XGBoost—offers the better supported performance-cost trade-off. The conclusions are bounded by evaluation of a single dataset and single LLM architecture, with prompt sensitivity adding variability.

  • Problem

    The paper examines whether LLMs have an advantage over traditional ML models for tabular 5G intrusion detection when performance, inference time, and energy-efficiency proxies are considered together.

  • Method

    The study compares XGBoost, TabNet, and Open-Mistral-7B1 using zero-shot and few-shot prompting on common preprocessed 5G telemetry, measuring detection performance, inference time, and CPU time.

  • Results

    Traditional ML models significantly outperform LLM-based approaches in detection performance, inference time, and computational cost, while few-shot prompting increases CPU time without consistent detection improvements.

  • Takeaways & Limitations

    For tabular 5G intrusion detection, tree ensemble models are a more practical and effective choice than DNNs and LLMs.

  • Takeaways & Limitations

    The study is limited to a single dataset and a single LLM architecture, and LLM performance is highly sensitive to prompt design.

Abstract

from arXiv · show

The increasing adoption of Artificial Intelligence (AI) in network intrusion detection raises the question of whether complex and computationally expensive models are justified for this task. In this work, we investigate the trade-off between detection performance and computational cost for intrusion detection in 5G network telemetry. We compare traditional machine learning (ML) models, including XGBoost as a representative of tree ensemble, and TabNet for tabular deep neural network (DNN), with a large language model (LLM) used as a general-purpose intrusion detector. The LLM is evaluated under both zero-shot and few-shot prompting configurations. We evaluate the models in terms of detection performance, inference time, and CPU time as a proxy for energy efficiency. Using a relatively large available 5G dataset, we show that traditional ML models consistently achieve near-perfect detection performance with negligible inference time, while LLM-based approaches perform significantly worse and incur orders-of-magnitude higher CPU usage. Few-shot prompting improves recall, but at the cost of lower accuracy and further increased CPU time, without closing the performance gap. These findings indicate that, for tabular intrusion detection in 5G networks, XGBoost offers a substantially better performance-cost trade-off than DNNs and LLMs, highlighting the importance of selecting models based on task suitability rather than increasing complexity.

I. INTRODUCTION

The expansion of 5G increases the attack surface and generates complex, high-volume tabular telemetry that motivates ML-based security solutions. This work tests whether LLMs justify their computational cost compared with traditional ML and DNN approaches for 5G intrusion detection.

  • 5G adoption has increased the attack surface, while telecommunications remains a primary target of cyberattacks.
  • ML techniques are increasingly considered for analyzing high-volume, complex network telemetry represented in tabular form.
  • LLMs have attracted interest in cybersecurity, but their training and execution require substantial computational resources.
  • The study hypothesizes that LLMs provide a less favorable performance-cost trade-off than tree ensembles and DNNs for tabular 5G intrusion detection.
  • The paper compares tree ensembles, DNNs, and general-purpose LLMs, including zero-shot and few-shot prompting, using detection, inference-time, and CPU-time measures.

II. RELATED WORK

Prior work reports strong and efficient tree-based intrusion detection, while LLM research highlights broader cybersecurity potential alongside substantial deployment costs. This study addresses the lack of a joint comparison of detection performance and computational consumption for tabular 5G intrusion detection.

  • Tree ensemble methods have achieved high detection performance with low training and inference complexity across intrusion-detection benchmarks.
  • Prior comparisons found tree-based models faster than several neural-network approaches and consistently stronger than DNNs for anomaly detection.
  • LLMs have been explored for contextual, adaptive, and privacy-preserving cybersecurity and 5G anomaly-detection applications.
  • LLM deployment remains challenging because model size, architecture, batch size, and quantization affect inference time and energy consumption.
  • Existing work had not jointly evaluated detection performance and computational consumption across traditional ML, DNN, and LLM approaches for tabular 5G intrusion detection.

A. Tree Ensemble Methods

The study compares model families suited to different data and deployment settings, focusing on whether LLMs offer an advantage over traditional ML for network telemetry. XGBoost represents tree ensembles, TabNet provides a tabular deep-learning baseline, and Open-Mistral-7B1 represents LLM-based detection.

  • Tree Ensemble Methods: Tree ensemble methods combine multiple decision trees to improve generalization, accuracy, and robustness while mitigating overfitting on tabular classification tasks.
  • Deep Neural Networks: DNN performance on tabular tasks often depends heavily on feature preprocessing, architecture design, and training strategy.
  • Large Language Models: LLMs encode structured or semi-structured data into text but face high computational requirements, limited interpretability, and task-specific adaptation challenges.
  • Evaluation Focus: The study examines performance, inference-time, and energy-efficiency trade-offs while comparing zero-shot and few-shot LLM prompting.
  • Compared Models: XGBoost represents tree ensembles, TabNet provides a tabular neural-network baseline, and Open-Mistral-7B1 provides the LLM-based approach.

V. EXPERIMENTAL SETUP

Experiments use the 5G-NIDD tabular intrusion-detection dataset and a common preprocessing pipeline for three model families. Because LLM inference is computationally demanding, its evaluation uses a representative stratified subset of the test data.

  • Execution Environment: The experiments ran in a Debian-based virtual machine with an Intel Core Ultra 7 155U CPU and 32 GB RAM.
  • Dataset: The 5G-NIDD dataset contains 1.2 million samples and 92 features labeled as benign or malicious for binary classification.
  • Data Split: Data were split into training, validation, and test sets using 72%/8%/20% stratified sampling.
  • Preprocessing: Preprocessing reduced the feature space to 20 selected numerical features, and the same processed data were used for all three models.
  • LLM Evaluation: The LLM was evaluated on a stratified subset of 300 test samples preserving the original class distribution.

B. Intrusion Detectors

The study compares XGBoost, TabNet, and Mistral-based intrusion detection using standardized telemetry evaluation, zero-shot or few-shot prompting, inference time, and CPU time.

  • Model training: XGBoost and TabNet use shared preprocessed data, validation-based tuning, and held-out test evaluation.XGBoost handles class imbalance with scale_pos_weight, while TabNet uses validation-based early stopping.
  • LLM detector: Mistral receives feature-value pairs and must classify each 5G telemetry sample as Benign or Malicious.The prompt requires exactly one label without additional explanation.
  • Prompting strategies: Zero-shot prompting uses one sample per prompt, whereas few-shot prompting adds one benign and one malicious labeled example before the test sample.Few-shot examples are sampled from the validation set.
  • Evaluation dimensions: Detection performance is measured with accuracy, precision, recall, F1-score, balanced accuracy, and confusion-matrix components.Balanced accuracy averages recall across classes to mitigate class-imbalance effects.
  • Evaluation dimensions: Inference time measures average per-sample alert-generation speed, while CPU time measures total processor time as a proxy for energy efficiency.CPU time includes accumulated work across multiple CPU cores and covers training and inference where applicable.

VI. RESULT

XGBoost and TabNet achieve near-perfect performance, whereas the LLM performs substantially worse and becomes strongly biased toward malicious predictions under few-shot prompting.

  • Detection performance: XGBoost and TabNet achieve near-perfect performance across all reported detection metrics.
  • LLM prompting: 0.67 balanced accuracy is achieved by zero-shot prompting, compared with 0.50 for few-shot prompting.Few-shot prompting reaches recall of 1.00 but flags every sample as malicious, producing a high false-positive rate.

B. Inference Time

Traditional ML models infer rapidly, while the LLM requires approximately 10 seconds per sample, excluding prompt-construction overhead.

  • Traditional ML: Microsecond-scale per-sample inference times characterize the traditional ML models.
  • LLM: Approximately 10 seconds per sample is required by the LLM-based approach.This excludes time for prompt formulation and example selection; LLM training time is also absent from the reported wall-clock comparison.

C. CPU Time

XGBoost and TabNet consume modest CPU resources, whereas the LLM consumes substantially more, with few-shot prompting increasing CPU time without corresponding performance gains.

  • CPU accounting: Most CPU time for XGBoost and TabNet is spent during training, with train and test CPU times reported separately.Values are reported in seconds.
  • Model comparison: The LLM consumes significantly higher CPU time on both server and client than XGBoost and TabNet.
  • Trade-off: Few-shot prompting increases CPU time without yielding corresponding performance improvements.The performance-cost trade-off is visualized using balanced accuracy, average inference time, and total CPU time.

D. Impact of Migration

The study evaluates API-based Open-Mistral-7B deployment to assess whether remote GPU infrastructure changes the model’s latency and efficiency trade-offs. Migration reduces end-to-end latency, but computational resource use remains unobservable.

  • Measurement scope: Average latency includes both model inference and network overhead in the API-based deployment.The study also reports total elapsed prompting time and average tokens processed per request as deployment-cost indicators.
  • Migration effects: API-based deployment reduced end-to-end latency compared with local CPU-based deployment.The experiment was designed to assess whether local deployment constraints explained the LLM’s weak results.
  • Prompting configurations: API-based few-shot prompting added minimal overhead and slightly outperformed local few-shot prompting, but remained below local zero-shot performance.API-based zero-shot prompting performed worse than both local approaches.
  • Measurement scope: The API-based setting leaves GPU utilization, power consumption, and hardware configuration unobserved, so true computational and energy costs remain unknown.This limits direct comparison with the local CPU-based experiments on resource consumption.

VII. DISCUSSION

Traditional ML models substantially outperform LLM-based approaches for tabular 5G intrusion detection across detection performance, inference time, and computational cost. Additional experiments support XGBoost as the strongest overall trade-off, while deployment setting and supervised-data assumptions constrain the conclusions.

  • Overall findings: Traditional ML models significantly outperform LLM-based approaches in detection performance, inference time, and computational cost.Few-shot prompting increases CPU time without consistent detection improvements, and the LLM systematically flags every sample as malicious.
  • Detection performance: Both XGBoost and TabNet achieve near-perfect detection performance on the same stratified 300-sample subset used for LLM experiments.Their subset results are consistent with results on the full test set.
  • Practical implications: XGBoost provides the best overall trade-off, combining near-perfect detection performance with minimal inference time and low computational demand.The paper frames this result as especially relevant for real-time intrusion detection, where timely responses are essential.
  • Computational cost: 943.14 total CPU seconds for XGBoost contrasts with 5,319.27 for TabNet, 41,720.50 for Mistral zero-shot, and 44,168.18 for Mistral few-shot.The corresponding test CPU times are 1.68, 26.69, 20,859.85, and 22,083.72 seconds, respectively.
  • Scope and limitations: The study is limited to a single dataset and LLM architecture, and LLM performance is highly sensitive to prompt design.It focuses on supervised detection with labeled malicious samples, which may be less suitable for previously unseen threats.

VIII. CONCLUSION AND FUTURE DIRECTIONS

The study finds that XGBoost and TabNet substantially outperform Open-Mistral-7B for tabular 5G intrusion detection in computational efficiency, while maintaining stronger detection performance. It identifies model suitability, interpretability, and evaluation scope as important considerations for future work.

  • Conclusion: XGBoost and TabNet were compared with Open-Mistral-7B under zero-shot and few-shot prompting for labeled tabular 5G intrusion detection.The comparison focused on specialized tabular models versus a general-purpose LLM.
  • Conclusion: Traditional ML models achieved near-perfect detection performance with negligible inference time and substantially lower computational cost than the LLM.The LLM performed significantly worse and required several orders of magnitude more inference time and CPU usage.
  • Conclusion: Few-shot prompting increased computational cost without improving the overall performance-cost trade-off.Few-shot prompting improved recall but did not close the gap with traditional models.
  • Conclusion: The findings identify tree ensemble models as a more practical and effective choice than DNNs and LLMs for 5G intrusion detection.The conclusion supports selecting models according to task suitability rather than complexity trends.
  • Future Directions: LLM discrimination between samples and few-shot bias toward malicious traffic remain unclear, creating an interpretability challenge in cybersecurity.The black-box issue also affects deep learning models and is presented as an open challenge for high-risk applications.
  • Future Directions: Future studies should examine domain-specific fine-tuning, specialized knowledge bases, unsupervised or semi-supervised settings, and tabular LLMs or fine-tuned SLMs.These directions are proposed to broaden evaluation and potentially improve LLM effectiveness while reducing computational requirements.
Loading 2608.26844v1…