Temporal Heterogeneous Graph Transformer for Credit Card Fraud Detection

1 Introduction

• Fraud is rare and can resemble legitimate activity, while risk patterns may emerge only when repeated transaction attributes are considered.This makes leveraging relational signals in high-dimensional transaction data an important fraud-detection problem.Source: §1 · p. 1
• Tabular gradient-boosting models capture nonlinear interactions but mostly predict from individual records, representing relational meaning only indirectly.Explicit relational information generally requires manually engineered aggregation features.Source: §1 · p. 1
• Graph methods model relational structure, but ambiguous repeated values and the cost of complete transaction graphs complicate their use on anonymized data.Temporal modeling also faces changing fraud risk and transaction activity pace.Source: §1 · p. 1
• THGT-FD represents each transaction with one transaction token and six relation tokens, using Time2Vec and a Transformer to model their interactions.The study evaluates fraud-risk ranking on IEEE-CIS against a class-weighted histogram-based gradient-boosting baseline.Source: §1 · p. 1

2.1 Data Processing and Relation Tokens

• Data are temporally divided into train, validation, and test sets, and preprocessing parameters are estimated from training data.High-missingness features are removed; numeric features are median-imputed and standardized, while other categorical features are frequency-encoded and relative day and hour features are derived from TransactionDT.Source: §2.1 · pp. 1–2
• Six relation types cover card, address, product, purchaser email, recipient email, and device attributes, with each relation value represented by a relation-specific embedding and type embedding.Card, address, and device relations are each jointly defined by multiple fields.Source: §2.1 · p. 2

2.2 Model Architecture and Training

• The transaction token combines processed transaction features with a Time2Vec encoding of the normalized timestamp, using one linear and m periodic components.The combined representation is projected into the hidden space.Source: §2.2 · p. 2
• The model input is a fixed-order sequence of one transaction token and six relation tokens, processed by a two-layer Transformer with four attention heads per layer.The first-position representation is passed through a multilayer perceptron to estimate fraud probability.Source: §2.2 · p. 2
• Training uses AdamW and class-weighted focal loss, with checkpoints selected by validation AUC-ROC and the classification threshold calibrated on validation data.The positive-class weight is based on the training-set ratio of legitimate to fraudulent transactions.Source: §2.2 · p. 2

3 Experiment

• 150,000 chronologically ordered IEEE-CIS transactions preserved the original 3.499% fraud ratio, with later transactions reserved for testing.The reported metrics covered global ranking quality and fraud coverage under constrained review budgets.Source: §3 · p. 2, Table 1 · p. 2
• 0.8536 test AUC-ROC and 0.4164 test Average Precision accompanied THGT-FD’s 0.8506 validation AUC-ROC and 0.4475 validation Average Precision.The test Average Precision was above the 3.499% positive-class prevalence, and validation and test AUC-ROC were nearly identical.Source: §3 · p. 2
• 0.4708 test Recall@5% meant that a review queue containing 5% of transactions captured nearly half of all fraud cases.Recall@1% was 0.2414, corresponding to 24.14% of fraud cases in the highest-risk 1% of transactions.Source: §3 · p. 2
• 0.8722 test AUC-ROC for HistGradientBoosting exceeded THGT-FD by 0.0186, while high-risk Recall@1% and Recall@5% each differed by 0.0265.Table 2 reports validation and test results for the tabular baseline and THGT-FD.Source: §3 · p. 2, Table 2 · p. 3
• Relation-specific embeddings, self-attention, and Time2Vec represent heterogeneous relations and temporal patterns without constructing a full transaction graph.The model’s tokens capture interactions among cards, devices, addresses, email domains, products, and temporal signals.Source: §3 · pp. 3–4
• Validation-based threshold selection can align binary predictions with review capacity rather than applying a fixed 0.5 threshold.The passage identifies probability calibration and threshold selection as opportunities to strengthen threshold-dependent performance.Source: §3 · p. 3
• Validation AUC-ROC rose from 0.3623 before optimization to 0.8206 after epoch one, peaked at 0.8506 at epoch 13, then declined.The decline supported retaining the epoch-13 checkpoint rather than the final training state.Source: Figure 2 · p. 3, §3 · p. 4
• Entity-level historical aggregation is not yet incorporated, and the pilot sample may capture only a subset of recurrence patterns in the full dataset.Larger-scale training could provide more observations for low-frequency relation values and support more stable entity embeddings.Source: §3 · p. 4

• 0.8536 test AUC-ROC; THGT-FD captured 47.08% of fraudulent transactions within the highest-risk 5%.The conclusion reports these pilot-experiment results on the IEEE-CIS dataset.Source: §4 · p. 4
• THGT-FD remained competitive with a strong class-weighted gradient-boosting baseline.Source: §4 · p. 4
• THGT-FD combines a feature token, a Time2Vec representation, and relation tokens for shared entities in a compact sequence.The paper says this formulation incorporates relational context while retaining efficient token-based processing.Source: §4 · p. 4
• Future work will add entity histories and rolling temporal statistics to relation tokens, with multi-hop propagation between transactions linked through shared entities.Source: §4 · p. 4
• Full-data training, targeted component ablations, and probability calibration are proposed to broaden relational coverage, measure component contributions, and align thresholds with review capacity.Source: §4 · p. 4