Source-linked AI summary
UAV Individual Identification via Distilled RF Fingerprints-Based LLM in ISAC Networks
Haolin Zheng, Ning Gao, Donghong Cai, Shi Jin, Michail Matthaiou
TL;DR
Traditional physical surveillance can locate UAVs but does not identify them, complicating authentication in non-cooperative low-altitude settings. The paper proposes a modified GPT-2 RFF-LLM and PPO-controlled dynamic distillation to transfer knowledge to Lite-HRNet. The resulting model achieves 98.38% UAV ID identification accuracy with 0.15 million parameters and 2.74 ms response time.
Problem
Traditional physical surveillance can perceive UAV physical state but cannot recognize identity, making authentication difficult in complex non-cooperative low-altitude scenarios.
Source: §I · p. 1Method
A modified GPT-2 RFF-LLM transfers knowledge to lightweight Lite-HRNet through PPO-based dynamic distillation that adjusts distillation temperature.
Source: §I · p. 2, B. Dynamic Distillation With PPO · p. 3Results
98.38% UAV ID identification accuracy is achieved with 0.15 million parameters and 2.74 ms response time.
Source: §I · p. 2Takeaways & Limitations
Lite-HRNet-KD's 2.74 ms inference latency and lowest RAM usage support real-time security surveillance on resource-constrained devices.
Source: C. Comparison of Different Models · p. 4
Abstract
from arXiv · showhide
Unmanned aerial vehicle (UAV) individual (ID) identification is a critical security surveillance strategy in low-altitude integrated sensing and communication (ISAC) networks. In this paper, we propose a novel dynamic knowledge distillation (KD)-enabled wireless radio frequency fingerprint large language model (RFF-LLM) framework for UAV ID identification. First, we propose an RFF-LLM framework based on the modified GPT-2 model to improve the identification accuracy in complex outdoor environments. Then, considering the parameter overhead of the RFF-LLM, we design a dynamic KD strategy to compress the model. Specifically, the proximal policy optimization (PPO) algorithm is employed to dynamically adjust the distillation temperature, overcoming the local optimum dilemma inherent in static KD. As a next step, the knowledge of the RFF-LLM is adequately transferred to the lightweight Lite-HRNet model. Finally, our experiments are conducted based on the self-built drone RFF dataset of Release one, namely DRFF-R1, by collecting the I/Q signals of 20 commercial UAVs in channel 149. The experiment results show that the proposed framework achieves 98.38% ID identification accuracy with merely 0.15 million parameters and 2.74 ms response time, which outperforms the benchmarks.
I. INTRODUCTION
The paper addresses the difficulty of identifying UAVs in complex, non-cooperative low-altitude settings and the scarcity of UAV ID identification work. It proposes a dynamic-distillation RFF-LLM framework and reports a lightweight model with high accuracy and fast response.
I. INTRODUCTION
- Traditional physical surveillance detects UAV physical state but not identity, limiting authentication in complex non-cooperative low-altitude scenarios.The introduction notes that most consumer UAVs have not been registered online.Source: §I · p. 1
- Prior work largely focused on UAV type classification, while UAV ID identification remained scarce amid efficiency, cost, and dataset challenges.Existing feature engineering had high overhead, and fixed-temperature distillation was described as limiting lightweight-model efficiency.Source: §I · p. 1
- The modified GPT-2 RFF-LLM extracts high-dimensional temporal features and long-range dependencies from time-frequency spectra for UAV ID identification.The framework is designed to improve identification accuracy in complex outdoor environments.Source: §I · pp. 1–2
- PPO-based dynamic distillation transfers RFF-LLM knowledge to lightweight Lite-HRNet and addresses the local-optimum dilemma of static distillation.The stated motivation is deployment cost and efficiency.Source: §I · p. 2
- 98.38% UAV ID identification accuracy is achieved with 0.15 million parameters and 2.74 ms response time.The dynamically distilled Lite-HRNet converges within 61 iterations on the collected UAV RFF dataset.Source: §I · p. 2
A. Signal Reception
The signal-reception model represents an outdoor UAV channel and the transmitter's hardware-impaired I/Q signal. Reception samples the signal into discrete in-phase and quadrature components for subsequent data formation.
A. Signal Reception
- The outdoor UAV link includes LoS and NLoS components and is modeled using Ricean fading.The model introduces the Ricean factor and direct and scattered link components.Source: A. Signal Reception · p. 2
- The transmitter signal model accounts for hardware imperfections through gain imbalance α and phase imbalance φ.The ideal baseband I/Q signal is revised to include these transmitter effects.Source: A. Signal Reception · p. 2
- Reception samples s(t) at intervals Ts to form S[n] = I[n] + jQ[n], which is transformed into labeled raw data.I[n] and Q[n] are the in-phase and quadrature components, respectively.Source: A. Signal Reception · p. 2
B. Problem and Framework Formulation
UAV ID identification is formulated as classification from augmented signal samples. The framework trains a student Lite-HRNet using a teacher RFF-LLM, a distillation controller, and a classifier, with PPO dynamically adjusting distillation temperature.
B. Problem and Framework Formulation
- UAV ID identification is a multi-class classification task that maps augmented signal samples to labels by minimizing cross-entropy loss.The objective optimizes model parameters against predicted and true labels.Source: B. Problem and Framework Formulation · p. 2
- The framework combines an RFF-LLM teacher, Lite-HRNet student, distillation controller, and classifier in a two-phase optimization strategy.With teacher parameters fixed, the student is trained using a weighted cross-entropy and KL-divergence loss; PPO adjusts temperature from state and reward.Source: B. Problem and Framework Formulation · p. 2
- The optimized student parameters are passed to the classifier, which returns the final classification result.Source: B. Problem and Framework Formulation · p. 2
III. METHODOLOGY
The RFF-LLM modifies GPT-2 to address limitations of absolute positional encoding for STFT spectra and overfitting with insufficient training data. It combines BiLSTM temporal features with Transformer layers and a classification head adapted for UAV identification.
A. RFF-LLM Framework
- The modified GPT-2 replaces absolute positional encoding with BiLSTM temporal modeling because it struggles with STFT dependencies, while the original deep Transformer can overfit limited data.The paper describes the BiLSTM as generating dynamic position-aware representations for non-stationary signals.Source: A. RFF-LLM Framework · p. 3
- The architecture sends STFT spectra through two stacked BiLSTM layers, three Transformer layers with residual connections, and fully connected classification layers.The Transformer layers model long-range dependencies.Source: A. RFF-LLM Framework · p. 3
- Forward and backward LSTM hidden states are concatenated into features, which the classifier aggregates across time using global average pooling and regularizes with weight penalties and stochastic dropout.The pooling replaces projection of only the last hidden state to the vocabulary space.Source: A. RFF-LLM Framework · p. 3
B. Dynamic Distillation With PPO
The method treats distillation temperature adjustment as a PPO-controlled decision process, using training statistics to choose actions and a reward that considers accuracy and teacher–student divergence. PPO is selected for stable policy improvement with low computational complexity.
B. Dynamic Distillation With PPO
- PPO models temperature adjustment as an MDP because temperature depends only on the current state-action pair, with transitions represented through experience replay.The process uses a fixed discount factor.Source: B. Dynamic Distillation With PPO · p. 3
- The state uses k-step accuracy and KL-divergence statistics plus training progress, while the actor samples Gaussian actions that are mapped to the allowed temperature range.The actor outputs the Gaussian mean and variance; the action is transformed into [τmin, τmax].Source: B. Dynamic Distillation With PPO · p. 3
- The reward combines batch-wise accuracy relative to a threshold with KL divergence relative to a target divergence.The passage defines ξt as batch-wise accuracy, ξbase as its threshold, kt as network KL divergence, and ktarget as target divergence.Source: B. Dynamic Distillation With PPO · p. 3
- PPO is used for temperature control to provide stable policy improvement while keeping computational complexity low, with a clipped policy objective and generalized-advantage-estimator advantage.The method also limits temperature fluctuations based on a validation-accuracy threshold and exploration-rate clipping.Source: B. Dynamic Distillation With PPO · p. 3
A. Experimental Setup
Experiments use a self-built real-world drone dataset collected outdoors from commercial UAVs at multiple altitudes. The time-frequency data are segmented into training, validation, and test sets.
A. Experimental Setup
- The self-built dataset captures uplink I/Q signals from 20 commercial drones across seven models at five outdoor altitudes, with interference-free reference signals acquired at 0 m under wave-absorbing cotton.A USRP B210 is used for signal capture.Source: A. Experimental Setup · p. 3
- Experiments run on NVIDIA Tesla V100 GPUs and Intel Xeon E5-2698 CPUs with 128 GB RAM, using TensorFlow 2.2.Source: A. Experimental Setup · p. 3
- The time-frequency diagrams are segmented into 20,000 samples, divided into training, validation, and test sets at a 6:2:2 ratio.The dataset is publicly available on GitHub.Source: A. Experimental Setup · p. 4
B. Dynamic Distillation Feasible Analysis
Dynamic temperature distillation outperforms fixed-temperature and no-KD Lite-HRNet, while class-level gains improve both weak and already-strong classes.
B. Dynamic Distillation Feasible Analysis
- 98.38% test accuracy for dynamic distillation exceeds fixed τ = 4 at 96.62% and no-KD Lite-HRNet at 95.28%.The comparison includes fixed temperatures τ = 2/4/6/8; dynamic distillation is the best-performing configuration described.Source: B. Dynamic Distillation Feasible… · p. 4
- Validation accuracy rises from 89.9% to 99.00% in the first 5 epochs, then stabilizes at 99.60% ± 0.10% after 61 epochs as τ settles at 3.90 ± 0.10.The authors say this convergence characteristic confirms that dynamic distillation overcomes the local-optimum dilemma in static distillation.Source: B. Dynamic Distillation Feasible… · p. 4
- Lite-HRNet-NKD drops to 0.76 accuracy in Class 17, whereas the teacher network maintains high accuracy across classes.The comparison uses confusion matrices across the models.Source: B. Dynamic Distillation Feasible… · p. 4
C. Comparison of Different Models
Lite-HRNet-KD leads the reported model comparison in accuracy and resource indicators; its confusion matrices and feature visualization further characterize its performance.
C. Comparison of Different Models
- The PCA feature visualization shows Lite-HRNet-KD with tighter intra-class clustering and greater between-class distance than the compared models.The paper attributes this pattern to transferring high-order feature knowledge from RFF-LLM to Lite-HRNet through dynamic distillation.Source: C. Comparison of Different Models · p. 4, Figure 5 · p. 5
- 98.38% accuracy for Lite-HRNet-KD exceeds 97.17% for RFF-LLM and 95.28% for Lite-HRNet-NKD in the confusion-matrix comparison.Source: Figure 4 · p. 5
- 98.38% accuracy with the fewest parameters; Lite-HRNet-KD inference latency is 2.74 ms and RAM usage is lowest, 24.13% below ShuffleNet-v2.The paper presents these as evidence of real-time security surveillance performance on resource-constrained devices.Source: C. Comparison of Different Models · p. 4, Table I · p. 5
V. CONCLUSION
The conclusion presents PPO-based dynamic distillation as a way to transfer RFF-LLM feature-extraction capability to lightweight Lite-HRNet. On a self-built real-world dataset, the distilled model outperforms baseline methods across reported indicators.
V. CONCLUSION
- PPO-based dynamic knowledge distillation transfers RFF-LLM feature-extraction capability to the lightweight Lite-HRNet model.Source: §V · p. 5
- Within a few PPO iterations, the distillation temperature converges to a stable optimal range, overcoming the local-optimum dilemma in static distillation.Source: §V · p. 5
- On the self-built real-world dataset, dynamic-distilled Lite-HRNet achieves better performance than baseline methods across different indicators.Source: §V · p. 5