PaperlayerView exact revision on arXiv

Source-linked AI summary

ReCAST: Restoration-aware Cascaded Stage-wise Training for Obfuscated SMS Risk Classification

Jieyun Huang, Yi Shen, Kaikai Zhao, Jiangze Yan, Wenjing Zhang, Ping Chen, Ning Wang, Zhaoxiang Liu, Kai Wang, Shiguo Lian

arXiv:2609.04878v1cs.CRcs.AI

TL;DR

Obfuscated Chinese SMS can evade cost-conscious classifiers by hiding risk-bearing phrases while remaining readable. ReCAST distills restoration-oriented supervision from a large teacher into a smaller student, then trains it for direct risk classification. On real-world obfuscated Chinese SMS, it improves robustness over direct classification while preserving production-oriented efficiency, though broader validation remains necessary.

  • Problem

    Obfuscated Chinese SMS challenges cost-conscious risk classifiers because transformed or fragmented risk indicators are no longer directly observable.

  • Method

    ReCAST distills obfuscated-span, obfuscation-type, restored-text, and risk-label supervision into a student, then fine-tunes it for downstream classification.

  • Results

    ReCAST outperforms direct classification, data augmentation, and explicit restoration pipelines while preserving single-pass inference efficiency comparable to same-backbone classifiers.

  • Takeaways & Limitations

    Restoration-aware distillation provides a practical way to improve robustness against diverse real-world SMS obfuscation patterns in deployable classifiers.

  • Takeaways & Limitations

    Experiments focus on Chinese SMS risk classification with a 9B-scale student, leaving effectiveness on other languages, platforms, domains, and smaller models insufficiently validated.

Abstract

from arXiv · show

Fraudulent messages sent via Short Message Service (SMS) are increasingly obfuscated to evade cost-conscious classifiers in production systems. In Chinese SMS, attackers can exploit a wide range of carefully crafted obfuscation strategies to hide risk-bearing phrases while preserving human readability, making direct classification brittle under real-world latency and throughput constraints. We propose ReCAST, a Restoration-aware Cascaded Stage-wise Training framework for robust obfuscated Chinese SMS classification. ReCAST distills a large teacher model's de-obfuscation ability into a smaller deployable student model by supervising obfuscated span detection, obfuscation type prediction, and text restoration, and then uses the restoration-aware student for downstream risk classification. Experiments on an internally constructed real-world Chinese SMS benchmark show that ReCAST substantially improves classification performance over directly trained baselines under obfuscation. The results suggest that restoration-aware distillation offers a practical path toward robust SMS risk classification with smaller deployable models under production-oriented constraints.

1 Introduction

Obfuscated Chinese SMS hides risk-bearing phrases while preserving human readability, exposing cost-conscious classifiers to a practical robustness gap. ReCAST transfers large-model de-obfuscation ability into smaller deployable classifiers through restoration-aware, stage-wise training.

  • Obfuscated Chinese SMS uses language-specific and generic transformations that preserve readability while hiding or distorting classifier-visible risk indicators.
  • Rule-based methods and directly trained cost-conscious classifiers can fail when key risk indicators are transformed or fragmented.
  • Large language models can restore intended meanings, but serving them for every SMS conflicts with production latency, cost, and concurrency requirements.
  • ReCAST distills de-obfuscation into a smaller student using obfuscated-span detection, obfuscation-type prediction, text restoration, and downstream risk classification.
  • ReCAST improves robustness under obfuscation while maintaining latency close to same-backbone single-pass classification baselines in a controlled serving setup.

2 Related Work

Related work treats SMS abuse detection as text classification, but industrial deployment emphasizes lightweight models because massive streams impose stringent latency, throughput, and cost constraints. Adaptive obfuscation challenges these classifiers.

  • SMS abuse detection commonly uses BERT-style classifiers and LLM-based detectors for text classification.
  • Lightweight LLM-based classifiers are more practical than directly serving large LLMs for industrial systems processing massive SMS streams.
  • Adaptive obfuscation challenges the robustness of these practical SMS risk-control approaches.

3 Methodology

ReCAST trains a student with restoration-oriented supervision before adapting it to direct SMS risk classification. Its unified prompt interface preserves restoration knowledge while online inference uses a single classification pass.

  • 3.1 Task Formulation: ReCAST formulates obfuscated SMS classification around risk labels, obfuscated spans, obfuscation types, and restored text.
  • 3.2 Overview of ReCAST: The large teacher is used offline to distill interpretation and restoration ability into a smaller deployable student.
  • Stage 1: Restoration-oriented Distillation: Stage 1 trains structured FULL outputs covering spans, types, restored SMS, and risk labels, providing intermediate links between obfuscated forms and risk semantics.
  • 3.2 Overview of ReCAST: The unified prompt interface changes task tags and output schemas across FULL, CLS, and NORM tasks while keeping the system instruction shared.
  • Stage 2: Classification-oriented Training: Stage 2 initializes classification from the restoration-aware student and trains it to output only the final risk label from the original obfuscated SMS.
  • Stage 2: Classification-oriented Training: Stage 2 uses 90% CLS examples and 10% FULL rehearsal examples to focus on classification while mitigating catastrophic forgetting of restoration ability.
  • Stage 2: Classification-oriented Training: Unlike explicit restoration-then-classification pipelines, ReCAST internalizes restoration knowledge and directly maps incoming SMS to risk categories.
  • Stage 2: Classification-oriented Training: Online inference uses only the Stage 2 student with the CLS tag, without the teacher, a separate restoration module, or full restoration generation.

4 Experiments

On a real-world obfuscated Chinese SMS test set, ReCAST improves risk classification over direct, augmented, and explicit-restoration baselines while retaining substantially lower serving cost than explicit restoration.

  • Evaluation Setup: The evaluation uses production Chinese SMS with four labels—fraud, gambling, pornography, and benign—and temporally separated validation and test sets.The validation and test sets contain 500 and 1,000 manually annotated messages collected from different months; the test set is approximately category-balanced.
  • Evaluation Setup: The baselines cover direct classification, synthetic obfuscation augmentation, explicit restoration pipelines, prompt-only inference, published methods, and direct teacher inference.Trainable baselines generally use the same backbone as ReCAST, while direct teacher inference is a non-deployable offline reference.
  • Main Results: 86.6 ± 0.17% Accuracy and 89.5 ± 0.15% Risk Recall make ReCAST the best-performing method on the obfuscated Chinese SMS test set.Against Direct-CLS, ReCAST improves Accuracy and Risk Recall by 10.8 and 10.3 points, respectively.
  • Main Results: ReCAST exceeds Aug-CLS by 6.0 Accuracy points and 4.4 Risk Recall points, indicating gains beyond simply adding synthetic obfuscated examples.Aug-CLS uses 50K examples but only classification labels, whereas ReCAST adds restoration-aware intermediate supervision.
  • Main Results: ReCAST improves over same-backbone Pipeline-CLS by 3.2 mean ACC points and 3.3 mean RR points, while avoiding its explicit restoration serving cost.Pipeline-CLS reports 1.42s average latency, 2.52s P95 latency, and 6.66s total test time.
  • Efficiency and Comparisons: ReCAST achieves 0.71s average latency, 1.20s P95 latency, and 3.03s total test time, close to single-pass Direct-CLS and Aug-CLS.It also outperforms published methods and improves over same-backbone CA-CoT by 5.1 ACC and 5.2 RR points.
  • Ablation Study: Ablations show that stage-wise ordering, classification adaptation, localized span-and-type supervision, and restored-text supervision each contribute to performance.Joint-Shuffled is 2.3 ACC and 2.9 RR points below ReCAST; removing span&type supervision reduces Accuracy to 73.8% and Risk Recall to 72.1%.

5 Conclusion

ReCAST distills de-obfuscation ability into a smaller student through structured restoration and task-aware supervision, then adapts it for classification. On real-world Chinese SMS, it outperforms several baselines while preserving single-pass efficiency, although broader validation remains future work.

  • Conclusion: ReCAST distills de-obfuscation ability from a large teacher through obfuscated-span, restored-text, and task-aware label supervision before classification fine-tuning.The teacher is used during training, while the resulting smaller model supports deployment without online teacher inference.
  • Conclusion: Experiments show that ReCAST outperforms direct classification, data augmentation, and explicit restoration pipelines while preserving efficiency comparable to same-backbone classifiers.Ablations identify localized obfuscation and restored-text supervision as key contributors.
  • Conclusion: Although evaluated on Chinese SMS, ReCAST is presented as conceptually applicable to other harmful-text detection settings involving adversarial obfuscation, with broader validation left for future work.The stated scope boundary is the lack of broader validation beyond Chinese SMS.

Limitations

The study identifies limitations in language, model-size, teacher dependence, fairness evidence, and deployment setting. These boundaries leave generalization to other settings and production use insufficiently validated.

  • Evaluation focuses on Chinese SMS risk classification with a 9B-scale student, leaving other languages, platforms, domains, and smaller models insufficiently validated.The authors specifically propose future testing under stricter latency and resource constraints.
  • Training-data construction and structured supervision rely on a single DeepSeek-V4-Pro teacher, which may bias synthetic obfuscation coverage and styles.The sensitivity of ReCAST to teacher choice remains an open question.
  • The offline deployment evaluation uses controlled settings rather than full production traffic with evolving templates, sender behaviors, and system-level constraints.The authors call for shadow deployment or online monitoring before high-impact operational use.

Ethics Statement

The study describes data governance, privacy protection, controlled teacher deployment, and safeguards for practical use. It also acknowledges dual-use risks from analyzing SMS obfuscation strategies.

  • Sensitive SMS data were processed under internal governance, with personally identifiable information and sensitive fields removed or replaced before training and evaluation.Examples include phone numbers, account identifiers, verification codes, URLs, names, and addresses.
  • The teacher model was privately deployed from released weights, and only final structured outputs—not intermediate reasoning traces—supervised student training.The teacher was not accessed through a third-party or public API.
  • ReCAST is designed as an assistive risk-detection component rather than an autonomous production decision system because false positives may affect legitimate communication.Recommended safeguards include trusted sender signals, threshold control, manual review, and appeal or correction mechanisms.
  • The work presents potential dual-use risks from analyzing evasion strategies and mitigates them by withholding operationally sensitive artifacts and using sanitized examples.The stated purpose is defensive improvement of anti-abuse systems.
  • Annotation followed internal security and data-governance procedures, with sensitive fields removed and review restricted to authorized personnel in an access-controlled system.Annotators were bound by confidentiality agreements and provided task and quality-control guidelines.

A Dataset Details

The dataset combines human-labeled and LLM-assisted synthetic Chinese SMS with structured restoration supervision, while validation and test data come from manually annotated production messages separated across months.

  • The validation and test sets contain 500 and 1,000 real production SMS messages, respectively, sampled from different months to reduce temporal leakage.Each message receives one of four labels: fraud, gambling, pornography, or benign.
  • The final training set contains 30K approximately category-balanced examples combining human-labeled and LLM-assisted synthetic data.A subset includes manually annotated risk categories and restored texts.
  • Synthetic generation first creates diverse clean SMS base texts with balanced categories, varied topics, sentence patterns, and lengths, while limiting near duplicates.These clean messages are later used for obfuscation.
  • The generation pipeline creates obfuscated variants spanning the taxonomy’s single and mixed strategies, including homophones, character variants, inserted symbols, URL splitting, noise, abbreviations, and slang.The variants are produced from clean messages using human-labeled obfuscation examples.
  • Hard-example mining selects incorrectly predicted obfuscated messages for manual annotation, enriching training data with cases likely to confuse direct classifiers.Human annotators provide corrected labels and restored texts for these examples.
  • Clean–obfuscated pairs are converted into structured instances containing the message, restored text, risk category, obfuscated spans, and obfuscation types.These labels supervise ReCAST’s restoration stage.
  • Automatic consistency checks and manual review remove or correct problematic generated instances before inclusion in the training set.Checks compare restored text, spans, obfuscation types, and category labels.
  • Diversity controls and variant limits reduce duplication and template overfitting, while final validation and test results use real manually annotated production SMS.The controls target sentence patterns, lengths, topics, and the number of variants per clean message.

B Implementation Details

Implementation uses a 9B Qwen backbone, full-parameter training on H100 GPUs, controlled vLLM serving, and a substantially larger privately deployed teacher for offline data construction.

  • Qwen3.5-9B is fine-tuned with SWIFT on eight NVIDIA H100 80GB GPUs using BF16 precision.
  • Training uses AdamW with a 1 × 10^-5 learning rate, 0.1 weight decay, a 0.05 warmup ratio, five epochs, and 4096-token sequences.Per-device batch size is 2 with 16 gradient-accumulation steps, yielding an effective batch size of 256.
  • Classification uses constrained label decoding over the predefined risk-label set, with generation capped at 256 tokens.This produces deterministic, valid category predictions.
  • A privately deployed DeepSeek-V4-Pro MoE teacher with 1.6T total parameters generates synthetic data, mines hard examples, and supplies restoration annotations only during training-data construction.Reasoning mode is enabled during teacher-side generation for difficult obfuscation cases.

C Benign False Positive Analysis

ReCAST evaluates benign false positives alongside risky-message recall, showing strong overall classification while limiting over-prediction of risky categories. Remaining errors are concentrated in legitimate financial notifications whose surface cues resemble fraud.

  • Benign FPR measures benign messages incorrectly classified as fraud, gambling, or pornography, complementing Risk Recall.
  • ReCAST reduces Benign FPR from 16.0% with Prompt-CLS to 2.8% while achieving the best overall classification performance.
  • ReCAST matches Pipeline-CLS on Benign FPR while substantially outperforming it in ACC and Risk Recall.
  • Restoration-aware supervision helps distinguish intentional obfuscation in risky messages from naturally occurring noise in benign SMS.
  • Most remaining benign errors involve legitimate banking, financial, or securities notifications containing URLs, account terms, amounts, or stock-code-like strings.Trusted-sender signals, whitelists, and lightweight business rules are suggested as complementary ways to reduce these false positives.

D Class-wise Analysis

Class-wise results show that ReCAST performs best on pornography and faces its greatest difficulty on fraud, with residual errors dominated by risky messages classified as benign. The reported confusion matrix and per-class metrics provide complementary views of these category-specific behaviors.

  • Class-wise performance: Pornography achieves the highest class-wise F1 at 91.1%, while Fraud is the most challenging category at 82.6%.
  • Class-wise performance: Fraud has 38 of 258 messages predicted as Benign, making risky-to-benign misclassification the dominant residual error.
  • Class-wise performance: Benign messages achieve 97.2% recall, with only seven incorrectly predicted as risky.Most such false positives are legitimate financial or securities notifications whose surface cues resemble fraudulent SMS.
Loading 2609.04878v1…