PaperlayerView exact revision on arXiv

Source-linked AI summary

PMANet: Malicious URL detection via post-trained language model guided multi-level feature attention network

Ruitong Liu, Yanbin Wang, Haitao Xu, Zhan Qin, Fan Zhang, Yiwei Liu, Zheng Cao

arXiv:2311.12372v2cs.CR

TL;DR

Malicious URL detection remains difficult because pre-trained language models do not fully capture URL-domain structure, character-level cues, multi-order encoding, or local anomalies. PMANet adapts a pre-trained model through self-supervised post-training and multi-level feature attention, and it outperforms prior methods across challenging evaluation settings. The paper concludes that URL analysis benefits from combining character- and subword-level information with local and global features.

  • Problem

    Pre-trained language models for malicious URL detection struggle with domain adaptation, character-level information, multi-order encoding, and local-feature extraction.

    Source: §1 · p. 2

  • Method

    PMANet uses masked, noisy, and domain-discrimination post-training with dynamic multi-level feature attention and spatial pyramid pooling.

    Source: §1 · pp. 2–3, §4.1 · pp. 7–8

  • Results

    PMANet outperforms previously leading pre-trained-model and conventional deep-learning methods across small-data, imbalanced, multi-class, cross-dataset, and adversarial scenarios.

    Source: §1 · p. 3, §6 · p. 15, §7 · p. 16

  • Takeaways & Limitations

    Combining character- and subword-level representations with multi-level, local, and global URL features supports broad generalization and robustness.

    Source: §1 · p. 3, §7 · p. 16

    Further URL-only information extraction may be approaching its limits, motivating future integration of webpage content, visual elements, HTML, and JavaScript.

    Source: §6 · p. 16

Abstract

from arXiv · show

The proliferation of malicious URLs has made their detection crucial for enhancing network security. While pre-trained language models offer promise, existing methods struggle with domain-specific adaptability, character-level information, and local-global encoding integration. To address these challenges, we propose PMANet, a pre-trained Language Model-Guided multi-level feature attention network. PMANet employs a post-training process with three self-supervised objectives: masked language modeling, noisy language modeling, and domain discrimination, effectively capturing subword and character-level information. It also includes a hierarchical representation module and a dynamic layer-wise attention mechanism for extracting features from low to high levels. Additionally, spatial pyramid pooling integrates local and global features. Experiments on diverse scenarios, including small-scale data, class imbalance, and adversarial attacks, demonstrate PMANet's superiority over state-of-the-art models, achieving a 0.9941 AUC and correctly detecting all 20 malicious URLs in a case study. Code and data are available at https://github.com/Alixyvtte/Malicious-URL-Detection-PMANet.

1. Introduction

Malicious URLs pose escalating security risks, while existing detection and pre-trained language-model methods struggle with evolving threats, URL-specific structure, character cues, and local features. PMANet addresses these gaps through domain-adaptive post-training and multi-level feature attention, with evaluations showing broad performance advantages.

Motivation

  • PMANet targets domain adaptation, character-level information, multi-order encoding, and local-feature extraction challenges in pre-trained language models for malicious URL detection.These limitations arise because malicious URLs contain critical character variations and local anomalies that general text models may not capture.Source: §1 · p. 2

Method

  • Its feature-processing modules dynamically extract and weight low- to high-order representations while spatial pyramid pooling captures local details and global context.The architecture includes layer-wise attention and modules designed to enrich feature granularity and local information.Source: §1 · pp. 2–3

Results

  • Across small-data, imbalanced, multi-class, cross-dataset, and adversarial settings, PMANet outperforms previously leading pre-trained-model and conventional deep-learning methods.The reported evaluations emphasize generalizability, robustness, learnability, scalability, and adaptability.Source: §1 · p. 3

Method

  • PMANet post-trains a pre-trained language model with masked language modeling, noisy language modeling, and domain discrimination to learn token- and character-level URL contexts.The post-training strategy adapts pre-trained linguistic knowledge to the URL domain while keeping learning overhead low.Source: §1 · pp. 2–3

2. Related Work

Prior malicious-URL detectors progressed from manually engineered features toward CNN, recurrent, hybrid, and pre-trained language-model approaches. PMANet is positioned as addressing remaining domain-alignment, character-level, and feature-extraction gaps in those methods.

2. Related Work

  • Conventional URL detection traditionally relies on manually extracted URL elements, whereas deep-learning methods reduce but do not eliminate manual representation design.CNNs, RNNs, LSTMs, and hybrid models improve local and sequential feature modeling, but often require manual character, word, or n-gram initialization.Source: §2 · p. 3, §2.1 · p. 3

2.1. Conventional DL-based Methods

  • CNN-based systems such as URLNet [23] learn character and word embeddings through dual paths, while hybrid models combine local-feature extraction with sequential-context modeling.Grambeddings further combines CNNs, LSTMs, and attention for n-gram-based URL insights.Source: §2.1 · p. 3

2.2. Pre-trained LMs-based Methods

  • Pre-trained language models automate the data-processing pipeline, but direct transfer from English text can suffer domain misalignment for malicious URL detection.Building a domain-adapted BERT model from scratch can instead require large datasets, substantial computation, and long training periods.Source: §2.2 · p. 4
  • PMANet addresses prior gaps involving domain relevance, contextual understanding, character-level discrepancies, and feature-extraction intricacies through model adaptation and training techniques.The paper presents this as its bridge between pre-trained language models and URL-specific analysis.Source: §2.2 · p. 4

3. Malicious URL Data

The study evaluates malicious-URL detection across four public datasets that vary substantially in size, class balance, task structure, and top-level-domain distributions. This diversity supports testing across multiple real-world data conditions.

Dataset Overview

  • The four datasets differ in sample size and benign–malicious top-level-domain distributions, providing varied evaluation conditions.The collection includes balanced, imbalanced, and multi-class settings.Source: §3 · p. 4

GramBeddings Dataset

  • The GramBeddings dataset contains 800,000 balanced samples collected from phishing repositories and well-known benign websites.It includes 400,000 malicious and 400,000 benign URLs, with distinct .com, ccTLD, and other-gTLD distributions.Source: §3 · p. 4

Mendeley Dataset

  • The Mendeley dataset contains 1,561,934 URLs with approximately 1:43 malicious-to-benign imbalance and substantial .com representation.It includes 35,315 malicious and 1,526,619 benign URLs validated through the Google Safe Browsing API.Source: §3 · p. 4

Kaggle Datasets

  • Kaggle 1 contains 632,503 balanced malicious and benign URLs, whereas Kaggle 2 is multi-class with benign, defacement, phishing, and malicious categories.Kaggle 2 contains 428,079 benign samples and three negative categories totaling separate class distributions.Source: §3 · p. 4

4. Method of PMANet

PMANet combines post-trained CharBERT with multi-level feature processing to represent URLs across character, subword, local, and global scales. Its two-stage training uses three unsupervised objectives before supervised fine-tuning for URL-domain adaptation.

4. Method of PMANet

  • PMANet uses a 12-layer CharBERT backbone with character and subword channels, followed by supervised fine-tuning after unsupervised post-training.The backbone includes character embedding and heterogeneous interaction modules for combining both channels.Source: §4 · pp. 4–5, §4.1 · p. 5

4.1. Network Structure of PMANet

  • The character module applies a two-layer bidirectional GRU to produce context-aware character embeddings across the input sequence.Forward and backward hidden states are concatenated to form the character representation.Source: §4.1 · pp. 5–6
  • The heterogeneous interaction module fuses subword and character representations with separate transformations and convolution, then restores two channels using GELU and residual connections.Fusion enriches mutual representations while division preserves distinct token and character information.Source: §4.1 · pp. 6–7
  • MoFE stacks merged outputs from all 12 layers, exploiting hierarchical representations from low-level phrase information to higher-level semantics and long-range dependencies.The resulting tensor is rearranged before entering the attention module.Source: §4.1 · pp. 7–8
  • Layer-aware attention assigns differentiated weights to feature maps from different layers using average- and max-pooled descriptors processed by a shared MLP.The attention maps are multiplied with the input feature map; the model uses 12 channels and r=3.Source: §4.1 · p. 8
  • Spatial pyramid pooling applies a three-level pyramid to weighted features, aggregating local regions before mean pooling, dropout, and binary classification.This combines fine-to-coarse local information for URL prediction.Source: §4.1 · pp. 8–9

4.2. Post-training Method

  • Post-training optimizes MLM, noisy LM, and domain discrimination objectives for subword context, character morphology, and URL-domain adaptation.MLM masks 10% of URL tokens, noisy LM reconstructs words after character perturbations, and domain discrimination uses adversarial training with a gradient reversal layer.Source: §4.2 · pp. 9–10

5. Experiments

PMANet is evaluated across multi-layer feature extraction, data-scale variation, binary and multi-class classification, cross-dataset transfer, adversarial attacks, and real-world phishing URLs. Across these settings, it generally outperforms the compared baselines while maintaining robustness under imbalance and adversarial perturbations.

5. Experiments

  • The evaluation covers multi-layer feature extraction, varying training sizes, binary and multi-class tasks, cross-dataset testing, adversarial samples, and case studies against CNN- and Transformer-based baselines.Baselines include URLNet, TextCNN, Grambeddings, BERT, and RoBERTa, with binary-task code used without structural or hyperparameter modifications.Source: §5 · pp. 10–11, §5.2 · p. 11, §5.2.1 · p. 11

5.1. Evaluation of Multi-Layer Feature

  • Using embeddings from all 12 Transformer layers produces the highest performance, with results improving incrementally as more layers are included.The experiment trains on 128,000 URLs, validates on 32,000, and tests on another 32,000 from GramBeddings.Source: §5.1 · p. 11

5.2.1. Binary classification

  • Using only 1% of the balanced Grambeddings training data, PMANet achieves 93.52% accuracy and 91.34% recall, above baseline ranges of 81.25%–91.35% and up to 90.83%.With 80% of training data, it reaches 98.40% accuracy and 98.39% F1, compared with the best baseline’s 97.36% and 97.33%.Source: §5.2.1 · pp. 12–13
  • On the imbalanced Mendeley dataset, PMANet reaches 70.27% F1 and 95.06% AUC, exceeding the baseline F1 peak of 1.37%.The dataset contains about 43 benign URLs for every malicious URL, making false positives and class bias salient evaluation concerns.Source: Table 4 · p. 12, §5.2.1 · p. 13

5.2.2. Multiple classification

  • PMANet achieves the highest performance across all four multi-class URL categories and metrics, including 98.38% accuracy.Grambeddings reaches about 50% F1 on defacement and phishing, while URLNet reaches 97.07% overall accuracy.Source: §5.2.2 · p. 13

5.3. Cross-Dataset Testing

  • Cross-dataset testing shows PMANet leading across metrics, with an F1 score near 91% and a 4.59% false-positive rate.The closest baselines, BERT and RoBERTa, reach a highest AUC of 96.62%, while Grambeddings has an 86.67% FPR.Source: §5.3 · p. 13, Table 5 · p. 14

5.4. Adversarial Evaluation

  • AUC reaches 99.41% under adversarial attacks, about 20 percentage points above URLNet, while PMANet remains stable as baselines decline.At a fixed FPR of 0.1%, PMANet achieves nearly 80% TPR, whereas all baselines remain below 70%.Source: §5.4 · p. 14

5.5. Case Study

  • In the PhishTank case study, PMANet correctly identifies all 20 active phishing URLs, whereas Grambeddings reaches 75% and URLNet and TextCNN reach 95%.BERT and RoBERTa also achieve 100% accuracy in this case study.Source: §5.5 · p. 14

6. Discussion

The discussion presents PMANet as a broadly robust URL detector with simplified feature engineering, multi-level representation, efficient training, and theoretical contributions to domain adaptation and feature weighting. It also identifies webpage-level analysis as a future direction beyond URL-only information.

6. Discussion

  • PMANet is reported to generalize across small datasets, imbalanced data, multi-classification, cross-dataset validation, and adversarial attacks.The discussion frames these results as evidence of generalization and robustness across challenging scenarios.Source: §6 · p. 15
  • A pretrained tokenizer converts raw URL strings into numerical embeddings, reducing manual feature engineering while supporting semantic representation.The approach is described as end-to-end and uses only raw URL strings as input.Source: §6 · p. 15
  • PMANet’s interactive dual-channel architecture is described as improving interaction between character-level and subword-level representations.This is presented as a response to limitations in earlier dual-path networks such as URLNet.Source: §6 · p. 15
  • The model reportedly reaches optimal performance within five epochs and can operate on a single GPU with more than 20G of memory.The discussion attributes the training efficiency to transfer from the pretrained model to URL contexts.Source: §6 · p. 15
  • Domain-specific post-training is presented as a framework for adapting pretrained language models from general language understanding to specialized URL analysis.The discussion positions this as a contribution to domain-adaptive transfer learning.Source: §6 · p. 16
  • Layer-wise attention is presented as a way to assess the relative importance of feature representations at different abstraction levels.This contribution concerns dynamic feature weighting in layered neural architectures such as BERT applied to structured URLs.Source: §6 · p. 16
  • Future work should extract and fuse webpage content, visual elements, HTML structure, and JavaScript because URL-only information may be nearing its extraction limits.The proposed scope includes icons, page layouts, design patterns, and other webpage elements.Source: §6 · p. 16

7. Conclusion

The conclusion presents PMANet as an end-to-end malicious-URL detector that combines pretrained language modeling with character-aware, multi-level, and multi-scale feature learning. Across varied datasets and attacks, it reports stronger generalization and robustness than the baselines, including perfect identification in the case study.

7. Conclusion

  • PMANet combines pretrained language models with three integrated feature-learning modules for enhanced URL feature extraction.The conclusion emphasizes end-to-end learning from raw URL strings and advanced Transformer networks.Source: §7 · p. 16
  • The architecture jointly learns interactive subword- and character-level representations together with multi-level and multi-scale URL features.The conclusion identifies these representations as central benefits of the method.Source: §7 · p. 16
  • Across varied URL datasets, PMANet consistently outperforms state-of-the-art baselines and produces stable decisions across scenarios.The reported scope includes cross-dataset detection and adversarial sample attacks.Source: §7 · p. 16
  • In the case study, PMANet accurately identifies all active malicious webpages.This is presented as further evidence of efficacy in practical applications.Source: §7 · p. 16

CRediT authorship contribution statement

The contribution statement assigns conceptualization, methodology, and software to Ruitong Liu, and data curation and original-draft preparation to Yanbin Wang.

CRediT authorship contribution statement

Related papers

Loading 2311.12372v2…