PaperlayerView exact revision on arXiv

Source-linked AI summary

Temporal transformer CAN encoder with federated lightweight heads for anomaly detection

Konstantinos Gyftodimos, Kyriakos Chiotis, Elena Politi, George Dimitrakopoulos, Eirini Liotou

arXiv:2610.10613v1cs.LGcs.CR

TL;DR

CAN anomaly detection is difficult because subtle temporal and contextual irregularities occur in high-rate traffic, while centralized training raises privacy and data-sharing risks. The paper combines a lightweight temporal Transformer, XGBoost, and federated learning; on real CAN datasets (OTIDS), it reports improved detection of timing and payload anomalies.

  • Problem

    High-rate CAN traffic contains subtle temporal and contextual irregularities, while centralized training raises privacy and data-sharing risks.

    Source: §1 · p. 1

  • Method

    The framework encodes fixed-length CAN windows with a lightweight Transformer, classifies window embeddings with XGBoost, and optionally uses federated learning.

    Source: §1 · p. 2

  • Results

    On real CAN datasets (OTIDS), the framework demonstrated improved detection of timing and payload anomalies.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Federated learning supports collaborative model improvement without exchanging raw CAN data, while the lightweight classifier is intended for resource-constrained ECUs.

    Source: §1 · p. 2

Abstract

from arXiv · show

Modern vehicles rely on large numbers of Electronic Control Units (ECUs) that constantly exchange information over the Controller Area Network (CAN) bus. Due to the rapidity, structure, and repetition of this communication, even slight variations in timing, payload values, or message patterns can point to unusual activity. Whether due to errors, malfunctions, or deliberate interference, these anomalies are frequently subtle and challenging to identify with conventional methods that handle messages separately or rely on manually created rules. Motivated by this gap, we present a privacy-preserving framework for anomaly detection in in-vehicle networks, based on a Temporal Transformer CAN Encoder with Federated Lightweight Heads, to better capture these irregularities. The detection of subtle temporal and contextual anomalies is made possible by a lightweight Transformer encoder that learns how these signals evolve over time, while a federated learning mechanism enables several vehicles or ECUs to work together to improve a shared model without exchanging raw CAN data. This combination of federated learning and temporal sequence modeling provides robust anomaly detection performance while maintaining efficiency and privacy, according to experiments conducted on open-source datasets.

1. Introduction

The introduction motivates CAN anomaly detection as a challenge involving subtle temporal and contextual irregularities, high-rate traffic, and privacy risks from centralized training. It presents a temporal Transformer, lightweight XGBoost classifier, and federated learning framework, and reports improved detection of timing and payload anomalies on OTIDS.

1. Introduction

  • High-rate CAN messages, ignored timing structure, and computationally heavy centralized approaches make temporal, contextual anomaly detection challenging and raise privacy and data-sharing risks.Source: §1 · p. 1
  • The method processes raw CAN logs in fixed-length windows, extracts temporal representations with a lightweight Transformer, and classifies window embeddings with XGBoost.Source: §1 · p. 2
  • The lightweight local XGBoost classifier and federated learning are presented as suitable for resource-constrained ECUs while respecting vehicular privacy requirements.Source: §1 · p. 2
  • The authors present the framework as an end-to-end pipeline covering preprocessing through federated learning.Source: §1 · p. 2
  • The framework was evaluated on real CAN datasets (OTIDS), demonstrating improved detection of timing and payload anomalies.Source: §1 · p. 2

2. Related Work

Prior CAN anomaly-detection work ranges from manually engineered features to temporal deep-learning methods and federated or spatiotemporal approaches. The authors distinguish their work by combining a lightweight temporal Transformer with XGBoost and fusing timing, payload, and identifier information.

2. Related Work

  • Traditional methods such as Isolation Forest and OC-SVM use features selected with human intervention, while deep-learning approaches reduce reliance on manual feature selection.Source: §2 · p. 2
  • Temporal approaches include CAN frame prediction with an LSTM, sequential autoencoding, and time-frequency reconstruction with AE-AnoWGAN.Source: §2 · p. 2
  • Prior work includes manually constructed per-message characteristics, frequency and arbitration analysis, federated IDS training, spatiotemporal attention, and temporal-structural embeddings.Source: §2 · p. 3
  • The authors distinguish their approach by combining a lightweight Temporal Transformer with XGBoost and fusing inter-arrival times, payload bytes, and CAN identifiers in one temporal representation.Source: §2 · p. 3

3. Proposed Framework

The framework turns CAN time series into fixed-length windows, extracts temporal embeddings, and uses XGBoost for window-level anomaly predictions. Federated residual ensembling combines local models without centralizing raw data.

3. Proposed Framework

  • CAN time series are partitioned into fixed-length frames, with inter-reading intervals added to represent temporal irregularity.The window length is use-case specific and should be tuned accordingly.Source: §3 · p. 3
  • A Transformer extracts temporal positional patterns from each frame and outputs a compact fixed-length embedding for XGBoost.Source: §3 · p. 3
  • XGBoost uses embeddings to predict label counts within each frame, while Figure 1 depicts the downstream components and multistage federated clients.The framework is described as balancing representational depth, statistical expressiveness, and deployment practicality.Source: §3 · pp. 3–4, Figure 1 · p. 4
  • Federated clients train local models and contribute their best-performing models to a global ensemble, without centralizing raw data.The ensemble summarizes the clients’ distributional predictions.Source: §3 · p. 4

4. Dataset Processing & Model Development

The experiments use labeled vehicle CAN messages, which are sorted, featurized, and grouped into fixed-length windows. A lightweight Transformer maps each window to a 64-dimensional representation.

A. Dataset Overview and Pre-Processing Approach

B. Temporal Embedding Model for Feature Extraction

  • A lightweight Transformer encodes each window into a 64-dimensional representation, using input projection, learnable positional embeddings, two self-attention layers, and adaptive mean pooling.The output preserves short-range payload structure and long-range temporal dynamics; the per-message embeddings are averaged across 15 messages.Source: B. Temporal Embedding Model for… · pp. 4–5

C. Multi-Output XGBoost

The downstream XGBoost component maps dense temporal embeddings to predictions of the anomaly-label distribution within each CAN message window.

C. Multi-Output XGBoost

  • A multi-output XGBoost regressor uses dense 64-feature window vectors to predict the distribution of anomaly labels within each window.Source: C. Multi-Output XGBoost · p. 5
  • The four-class setup uses four XGBoost regressors with squared-error loss to predict each class’s empirical proportion in a window.Source: C. Multi-Output XGBoost · p. 5
  • Predicting a full label distribution rather than a single label supports finer-grained detection of partially anomalous windows.The paper also notes that the distribution can support downstream anomaly-scoring strategies.Source: C. Multi-Output XGBoost · p. 5

A. Objective of Federated Learning Integration

Federated learning keeps raw CAN traffic local while training across distributed data silos. Client-side windows are assigned by ECU and split locally for training, validation, and testing.

A. Objective of Federated Learning Integration

B. Client-Side Dataset Creation

  • Each window represents a short message sequence with payload and inter-message timing, preserving sequential structure while reducing inference time.Source: B. Client-Side Dataset Creation · p. 5
  • Client silos receive exclusive ECU windows and independently stratify them into train, validation, and test sets; the OTIDS federated subset is held out from centralized training.The held-out subset is CAN_HCRL_OTIDS_UB, used to evaluate federated generalization.Source: B. Client-Side Dataset Creation · p. 5

C. Local Model Training Logic

Clients train gradient-boosted components on residuals from the current global ensemble, then send components and validation metrics to a server that selects updates for the global model. The server monitors validation performance and evaluates the final model on each client's test windows.

C. Local Model Training Logic

  • Clients fit new gradient-boosted components to local residuals, improving the ensemble without modifying previously learned components.They first predict local training windows with the current ensemble, then compare predictions with true labels to calculate residuals.Source: C. Local Model Training Logic · p. 6
  • Clients share only newly trained components and validation metrics, allowing federated boosting without averaging tree-model weights.The additive procedure focuses on window-level embeddings to reduce computational load for edge inference.Source: C. Local Model Training Logic · p. 6

D. Federated Server and Aggregation Strategy

  • The server distributes the current ensemble each round and sequentially incorporates client components selected by validation performance.Tree-based gradient-boosted models cannot be averaged in the traditional sense.Source: D. Federated Server and Aggregation… · p. 6
  • The server monitors validation performance across clients and rounds, then evaluates the final global model on each client's test windows.The select-best strategy and compact window-level representations support collaborative learning without access to raw CAN traffic.Source: D. Federated Server and Aggregation… · p. 6

Ε. Execution Flow

Each client extracts fixed-size embeddings from temporal CAN windows, trains local gradient-boosted updates, and exchanges updates with the server over multiple rounds. Clients then test the final global ensemble on their own windows; operating on windows reduces computational cost and inference time.

Ε. Execution Flow

  • Clients select temporal CAN windows and compute fixed-size embeddings that capture payload and timing information.The framework operates on window-level features rather than individual CAN frames.Source: Ε. Execution Flow · p. 6
  • Clients train local gradient-boosted models on residuals from the current global ensemble and send updated components and validation metrics to the server.The server incorporates new components into the global ensemble.Source: Ε. Execution Flow · p. 6
  • Local and global updates continue across federated rounds, after which each client evaluates the final ensemble on its test windows.Window-level processing reduces computational cost and inference time for edge devices.Source: Ε. Execution Flow · pp. 6–7

6. Evaluation

The evaluation reports window-level classification and regression performance, with a centralized model trained on the full embedding dataset serving as a baseline. Its reported results include strong detection metrics for critical attack classes, while the per-class error plot indicates where performance may need further improvement.

Α. Centralized Global Model

  • The centralized model is trained on the full embedding dataset before federated partitioning and evaluated for classification and regression across train, validation, and test splits.Table 1 covers accuracy, macro- and weighted precision, recall, F1-score, and regression MSE.Source: Α. Centralized Global Model · p. 7, Table 1 · p. 7
  • High recall and F1 values for critical attack classes accompany regression MSE indicating accurate attack-probability estimates.The per-class error distribution highlights attack categories that may need further improvement or fine-tuning.Source: Α. Centralized Global Model · p. 7
  • Figure 2 reports test-set per-class error, defined as one minus recall, to show which attack types are more challenging.The evaluation reports window-level results and frames embeddings as a way to reduce inference time on edge devices.Source: Α. Centralized Global Model · p. 7, Figure 2 · p. 8

B. Federated Model

In a two-client simulated federation, the global ensemble incorporates client knowledge while data remains siloed. The authors report high accuracy and low regression error for both clients, with round-based validation and test evaluations tracking model performance.

B. Federated Model

  • The federated model is trained across two simulated clients with distinct ECU subsets from CAN_HCRL_OTIDS_UB, while embeddings and residual XGBoost components are computed locally.The setup emulates decentralized data that cannot be shared directly.Source: B. Federated Model · p. 8
  • Table 2 summarizes average validation MSE by round, and the reported ensemble improves incrementally as knowledge from both clients is incorporated.Figure 3 visualizes per-client validation MSE across rounds, including convergence trends and relative contributions.Source: Table 2 · p. 8, Validation Metrics Across Federated… · p. 8, Figure 3 · p. 8
  • The final federated model achieves high accuracy and low regression error for both clients despite siloed data.Table 3 reports test accuracy and regression MSE per client.Source: Table 3 · p. 9, Final Test Metrics per Client · p. 9
  • Figure 4 tracks each client's test MSE across rounds; the text describes improving prediction accuracy on unseen windows and differences linked to heterogeneous local data.The final global ensemble is evaluated on each client's test windows.Source: Final Test Metrics per Client · p. 8, Figure 4 · p. 9

7. Conclusions & Future Work

The framework combines a lightweight Temporal Transformer encoder, an XGBoost anomaly classifier, and privacy-preserving federated learning for CAN-based in-vehicle anomaly detection. The conclusion identifies online learning as future work.

  • The CAN anomaly-detection framework combines a lightweight Temporal Transformer encoder, an XGBoost-based classifier, and privacy-preserving federated learning.Source: §7 · p. 9
  • The lightweight downstream classifier is intended to keep the system suitable for resource-constrained ECUs.Source: §7 · p. 9
  • Federated learning enables collaborative model improvement without exposing raw CAN data.Source: §7 · p. 9
  • Future work would integrate online learning to update the model gradually for dynamic environments.Source: §7 · p. 9
Loading 2610.10613v1…