Source-linked AI summary
Towards automated feature engineering for credit card fraud detection using multi-perspective HMMs
Yvan Lucas, Pierre-Edouard Portier, Léa Laporte, Liyun He-Guelton, Olivier Caelen, Michael Granitzer, Sylvie Calabretto
TL;DR
Credit-card fraud features often aggregate cardholder history without modeling fine-grained transaction dependencies or seller history. The paper uses multiperspective HMM features to add sequential information, reporting higher PR-AUC for both face-to-face and e-commerce transactions.
Problem
Cardholder aggregates omit fine-grained transaction dependencies and seller history, limiting how transaction sequences are represented for fraud detection.
Source: Introduction · p. 2, §2 · p. 7Method
The paper engineers HMM-based features from transaction sequences so a non-sequential classifier can use sequential information.
Source: Conclusion · p. 20Results
Precision-recall AUC increases by 18.1% for face-to-face transactions and 9.3% for e-commerce transactions.
Source: Conclusion · p. 21Takeaways & Limitations
The feature strategy is reported as relevant across Random Forest, logistic regression, and Adaboost, and robust to feature-construction hyperparameter choices.
Source: Conclusion · p. 21Insufficient transaction history prevents HMM-feature calculation; the reported experiments omit about 20% of transactions in section 4.1 and around 40% in section 4.3.
Source: §5 · p. 19
Abstract
from arXiv · showhide
Machine learning and data mining techniques have been used extensively in order to detect credit card frauds. However, most studies consider credit card transactions as isolated events and not as a sequence of transactions. In this framework, we model a sequence of credit card transactions from three different perspectives, namely (i) The sequence contains or doesn't contain a fraud (ii) The sequence is obtained by fixing the card-holder or the payment terminal (iii) It is a sequence of spent amount or of elapsed time between the current and previous transactions. Combinations of the three binary perspectives give eight sets of sequences from the (training) set of transactions. Each one of these sequences is modelled with a Hidden Markov Model (HMM). Each HMM associates a likelihood to a transaction given its sequence of previous transactions. These likelihoods are used as additional features in a Random Forest classifier for fraud detection. Our multiple perspectives HMM-based approach offers automated feature engineering to model temporal correlations so as to improve the effectiveness of the classification task and allows for an increase in the detection of fraudulent transactions when combined with the state of the art expert based feature engineering strategy for credit card fraud detection. In extension to previous works, we show that this approach goes beyond ecommerce transactions and provides a robust feature engineering over different datasets, hyperparameters and classifiers. Moreover, we compare strategies to deal with structural missing values.
Introduction
The paper addresses limits of aggregated cardholder histories with multiperspective HMM features, reporting higher fraud detection and robustness across transaction types, classifiers, and hyperparameters.
Introduction
- Cardholder aggregates miss fine-grained temporal dependencies, rely on fixed time windows, and omit terminal histories and fraudulent sequences.The paper notes that transaction counts within a fixed period can vary substantially across cardholders.Source: Introduction · p. 2
- HMMs generate history-based features that quantify similarity between observed sequences and past fraudulent or genuine sequences for cardholders or terminals.Source: Introduction · p. 2
- The authors expect genuine-only sequences to model common behavior, while features from sequences containing fraud should capture risky behavior and reduce false positives.The other perspectives distinguish cardholder from terminal histories and transaction amount from elapsed time.Source: Introduction · pp. 2–3
- The evaluation uses a real-world transaction dataset provided and labeled by a large European card processing company.Source: Introduction · p. 3
- Adding HMM-based features to transactional data with state-of-the-art aggregation significantly increases the detection of fraudulent transactions.The authors use Precision-Recall AUC to quantify the impact.Source: Introduction · p. 3
- The approach improves detection for both e-commerce and face-to-face transactions and is reported as relevant across classifiers and robust to feature-construction hyperparameters.For some short-history transactions, HMM features cannot be calculated; missing-value strategies later yield steady improvement across all transactions.Source: Introduction · pp. 3–4
1 Related Work
Prior fraud-detection work uses raw, aggregated, network, and sequential approaches; the paper positions its feature evaluation against cardholder aggregates and reviews sequence-modeling alternatives.
1.1 Credit card fraud detection
- Prior fraud systems range from raw-feature models to sequence models; Jurgovsky et al. (2018) report LSTM improvement for face-to-face transactions but not e-commerce.Source: §1.1 · pp. 4–5, §1.2 · p. 7
- Saia and Carta (2019) use Fourier and wavelet transforms for a frequency-domain view, whereas this paper creates likelihood scores across sequential views.Source: §1.1 · p. 5
- Whitrow et al. (2008) aggregate cardholder histories; later work adds periodic features or transaction-network suspiciousness.Source: §1.1 · pp. 5–6
- The paper treats Whitrow’s cardholder aggregates as the state-of-the-art baseline and uses Random Forests to evaluate feature effects and calculate feature importance.Feature importance is defined as the decrease in Gini impurity weighted by the share of dataset elements passing through a node.Source: §1.1 · pp. 5–6
1.2 Sequence Classification
- Sliding-window sequence methods omit dependencies between consecutive events, while Strivastava et al. (2008) use HMM likelihoods to classify recent transaction sequences.Jurgovsky et al. (2018) also study LSTMs for variable-length cardholder time series.Source: §1.2 · pp. 6–7
2 Multiple perspectives Hidden Markov Model based feature engineering
The method adds eight HMM-derived likelihood features to expert-engineered transaction aggregates, using supervised sequence groups to represent fraud status, transaction actor, and observed signal. These features compare each transaction’s recent history with learned behavior distributions.
2 Multiple perspectives Hidden Markov Model based feature engineering
- Existing aggregated features omit seller histories and within-sequence dependencies, and their design relies on expert knowledge.Aggregates also summarize behavior over fixed periods rather than fine-grained temporal dependencies.Source: §2 · p. 7
- Eight HMM-based features quantify how closely transaction histories match distributions learned from supervised sequence sets.They are added to descriptive aggregated features.Source: §2 · p. 7
- Three binary perspectives—fraud status, card-holder or terminal, and amount or time-delta—produce eight HMMs.The models represent genuine and fraudulent histories for both actors and both observed signals.Source: §2 · pp. 7, 10
- Training sequences are separated into genuine and compromised histories for both card-holders and terminals.The HMM transition and emission probabilities are optimized with the Baum-Welch algorithm.Source: §2 · pp. 8–9
- Each feature is the likelihood that the current transaction and its two predecessors were generated by one of the trained HMMs.The most probable hidden-state sequence is computed with the Viterbi algorithm.Source: §2 · p. 10
3 Experimental Setup
The experiments use anonymized Belgian card transactions to predict whether each transaction is genuine or fraudulent. Feature comparisons include card-holder- and terminal-centered aggregates, with temporal partitioning that accounts for delayed fraud labels.
3.1 Dataset Description
- The dataset contains anonymized transactions from Belgian credit cards dated March 1 through May 31, 2015, and the task is transaction-level fraud classification.The study also examines face-to-face transactions, whose properties differ from e-commerce transactions.Source: §3.1 · p. 11
3.2 Feature Engineering and Dataset Partitioning
- Terminal-centered aggregated features were calculated alongside card-holder-centered aggregates to make them comparable with the HMM-based features.Transaction vectors include card-holder, transaction, and terminal characteristics.Source: §3.2 · p. 11, §3.1 · p. 11
- The data were split temporally into training, validation, and testing periods, with a seven-day gap before testing to reflect delayed ground truth.Transactions in the gap contribute to test-set feature calculation but not classifier training.Source: §3.2 · p. 12
4 Results
Adding HMM-based sequence features improved fraud detection across e-commerce and face-to-face transactions, including when added to existing transaction aggregation. The improvement was reported across classifiers and as relatively stable across HMM hyperparameters.
4.1 Improvement in fraud detection when using HMM-based features
- Adding sequence descriptors, including HMM-based features or Whitrow aggregates, to raw features improved precision-recall AUC by 3.6%–40.5% for e-commerce and 6.0%–85.4% for face-to-face transactions.For each dataset, the range ran from the best feature set without HMM features (raw+all-agg) to the worst (raw features).Source: §4.1 · pp. 14–15
- 18.1% (face-to-face) and 9.3% (e-commerce) PR-AUC increases resulted from adding HMM features to the state-of-the-art raw+aggCH strategy.The comparison uses raw features plus card-holder aggregated features as the baseline.Source: §4.1 · p. 15
4.2 Robustness over different machine learning algorithms
- The feature-engineering improvement was significant and reliable across classifiers and datasets, and was larger for weaker classifiers than for Random Forests.Logistic regression performed worse than Random Forests; the authors note label encoding of categorical features as a possible issue.Source: §4.2 · pp. 15–16
- Adaboost trees of depth 4 detected fraud significantly better than depth-1 trees, while deeper trees carry greater overfitting risk.The authors suggest monitoring training and validation PR-AUC to track when overfitting begins.Source: §4.2 · p. 16
4.3 Robustness to hyperparameters changes
- The HMM feature strategy was relatively stable across hyperparameters, but no tested combination was confidently better than the others.Window size 3 with 5 hidden states had the best average AUC; combining all nine feature sets improved face-to-face results by 13.3% but decreased e-commerce results by 8.5%.Source: §4.3 · pp. 17–18
5 Handling the missing value limitations
The authors compare three ways to handle HMM features that cannot be calculated because transaction histories are too short. Replacing unavailable values with zero gave the best reported PR-AUC and stability, while short histories limited which transactions could be included.
5 Handling the missing value limitations
- Insufficient history prevented HMM feature calculation, excluding about 20% of transactions in section 4.1 and around 40% in the hyperparameter experiments.The authors identify integrating transactions with short histories as a remaining challenge.Source: §5 · p. 19
- The comparison tested nine HMM feature sets across 16 combinations of terminal and card-holder history constraints.The tested window sizes were 3, 5, and 7 for each history perspective.Source: §5 · p. 19
- Replacing uncalculable HMM features with zero gave the best PR-AUC and stability on both datasets, and required training only one Random Forest.The weighted PR and stacked RF approaches required training 16 and 17 Random Forests, respectively.Source: Table 9 · p. 20, §5 · p. 20
- The weighted PR strategy was unsatisfactory, while stacked RF gave good PR-AUCs and could combine different classifier types.The three strategies were default-zero replacement, a weighted sum of predictions, and stacked Random Forests.Source: §5 · pp. 19–20
- Handling structural missing values could broaden the framework’s application and let different window sizes represent short- and long-term history.The authors describe smaller windows as characterizing short-term history and larger windows as characterizing long-term history.Source: §5 · p. 20
Conclusion
The conclusion presents multiperspective HMM features as a way to incorporate sequential knowledge into transaction classification, with reported gains across transaction settings, classifiers, and feature hyperparameters. It also highlights structural missing values as a limitation and benchmarks ways to address them.
Conclusion
- HMM-based features let a non-sequential classifier, Random Forest, use sequential information in transaction classification.The conclusion suggests similar features may be useful in supervised tasks involving sequential data.Source: Conclusion · pp. 20–21
- The features model genuine and fraudulent behavior for merchants and card-holders using transaction amount and timing, with supervised construction that lowers the need for expert knowledge.The terminal perspective is reported to help detection for both face-to-face and e-commerce transactions.Source: Conclusion · pp. 20–21
- Precision-recall AUC increased by 18.1% for face-to-face transactions and 9.3% for e-commerce transactions.These results extend the framework beyond e-commerce transactions.Source: Conclusion · p. 21
- The strategy was relevant across random forest, logistic regression, and Adaboost, and robust to feature-construction hyperparameter choices.The conclusion also reports benchmarking several solutions to structural missing values.Source: Conclusion · p. 21
- The authors propose combining LSTM predictions with predictions from an HMM-feature-enhanced Random Forest because the classifiers have not detected the same face-to-face frauds.This is presented as future work.Source: Conclusion · p. 21