PaperlayerView exact revision on arXiv

Source-linked AI summary

FinFraudBench: A Heterogeneous Graph Benchmark for Financial Fraud Detection

Yixuan Chen, Hongyu Zhan, Jie Sheng, Weiyu Han, Shuai Chen, Tianyi Zhang, Xiao Tan, Jun Xia

arXiv:2608.15177v1cs.LGcs.AI

TL;DR

Existing fraud benchmarks often simplify financial ecosystems and omit realistic heterogeneous, imbalanced, limited-label settings. FinFraudBench constructs heterogeneous datasets and a reproducible evaluation protocol, finding that heterogeneous GNNs perform best overall while multi-relation and fraud-oriented methods remain competitive.

  • Problem

    Existing public fraud benchmarks often simplify financial ecosystems and lack large-scale heterogeneous graphs under extreme imbalance and limited labels.

  • Method

    FinFraudBench constructs two transaction-centered heterogeneous datasets preserving typed entities, semantic relations, type-specific attributes, limited labels, and realistic imbalance.

  • Results

    Heterogeneous GNNs perform best overall, while multi-relation and fraud-oriented methods remain competitive but less consistent.

  • Takeaways & Limitations

    Preserving heterogeneous financial structure is valuable for modeling fraud signals, while extending fraud-oriented designs to heterogeneous architectures remains promising.

  • Takeaways & Limitations

    Because the benchmark is built from public transaction-level tables, its graph signals may not fully reflect richer real-world fraud contexts.

Abstract

from arXiv · show

The increasing complexity of digital financial systems has reshaped financial fraud detection from isolated transaction classification into relational risk reasoning over interconnected financial entities. This shift has motivated graph-based fraud detection, where models identify fraudulent nodes by exploiting dependencies among customers, cards, merchants, categories, and locations. However, despite rapid progress in graph-based methods, existing public benchmarks remain misaligned with real-world financial systems in two important aspects. First, they often simplify financial ecosystems into homogeneous or single-node-type multi-relational graphs, failing to preserve the multi-entity and multi-relational nature of financial data. Second, they rarely provide large-scale heterogeneous financial graph datasets with realistic operating conditions such as extreme class imbalance and limited label availability, making it difficult to assess the practical effectiveness of current methods. To address these gaps, we present FinFraudBench, a heterogeneous graph benchmark for financial fraud detection. FinFraudBench contains two heterogeneous graph datasets (CreditCard-Fraud and BankTrans-Fraud) with up to 8.99M nodes and 89.23M directed typed edges. Each dataset preserves six financial entity types, fourteen directed edge types, and natural fraud rates that mirror deployment constraints. With these datasets, we establish a standardized evaluation protocol covering both ranking and imbalance-sensitive classification metrics, and evaluate representative baselines. Extensive experiments yield empirical insights into current methods' limitations and suggest promising avenues for future research. FinFraudBench is available at https://anonymous.4open.science/r/FinFraudBench-B002.

1 INTRODUCTION

FinFraudBench addresses the mismatch between simplified public fraud benchmarks and heterogeneous financial ecosystems by preserving typed entities, relations, and attributes. It also provides reproducible evaluation resources for limited-label, highly imbalanced settings and reports insights favoring heterogeneous fraud modeling.

  • Financial fraud is relational: suspicious transactions become easier to identify through interactions among customers, payment instruments, merchants, and other entities.
  • Existing benchmarks often simplify financial ecosystems into homogeneous or single-target multi-relational graphs, obscuring entity- and relation-aware fraud modeling.
  • The benchmark provides labeled and unlabeled training transactions, validation and test splits, balanced mini-test subsets, and standardized metrics for limited labels and extreme class imbalance.
  • FinFraudBench constructs two heterogeneous financial fraud datasets preserving multiple entity types, relation types, and type-specific attributes, while supporting deterministic simplified views.
  • Experiments indicate that typed financial entities and relations preserve essential fraud signals, motivating heterogeneous architectures that combine structural information with fraud-specific cues.

2 PRELIMINARIES AND RELATED WORK

This section distinguishes homogeneous, multi-relational, and heterogeneous graph formulations, then frames financial fraud detection as binary classification over target nodes. It positions FinFraudBench within graph-based fraud detection and contrasts its heterogeneous setting with commonly used single-node-type benchmarks.

  • Graph formulations: Homogeneous graphs use one node type, one edge type, and a shared node feature matrix.They are represented as G = (V, E, X), with X ∈ R|V|×d.
  • Graph formulations: Multi-relational graphs retain one node type while allowing multiple relation types assigned to edges.They are represented as G = (V, E, R, X, ϕ), where ϕ maps edges to relation types.
  • Graph formulations: Heterogeneous graphs model multiple node types and relation types, with type-specific node sets and feature matrices.FinFraudBench uses this stricter setting to capture both relation-level and entity-level heterogeneity.
  • Fraud detection task: Financial fraud detection is formulated as binary node classification over a target node type, with unlabeled targets retained as relational context but excluded from supervised loss.In FinFraudBench, transactions are targets, while customers, cards, merchants, categories, and locations provide relational context.
  • Fraud datasets and benchmarks: Prior graph-fraud benchmarks such as YelpChi and Amazon commonly use single-node-type multi-relational graphs to test noisy and suspicious relational patterns.These datasets are widely used in graph-based fraud detection but are not financial datasets.

3 BENCHMARK CONSTRUCTION

FinFraudBench constructs CreditCard-Fraud and BankTrans-Fraud from public transaction-level fraud tables as transaction-centered heterogeneous graphs. The construction preserves typed financial entities and semantic relations while restricting fraud labels to transaction nodes.

  • Data Sources: CreditCard-Fraud and BankTrans-Fraud are constructed from two public transaction-level fraud datasets, whose rows contain fraud labels and transaction, customer, payment-instrument, merchant, category, and geographic information.Each source-table row represents one transaction.
  • Construction Principles: The construction represents recurring financial objects as typed nodes, preserves semantic entity associations as distinct edge types, and keeps non-transaction nodes unlabeled for limited-label fraud detection.Labels remain defined only on transaction nodes.
  • Heterogeneous Schema: Both datasets use a transaction-centered heterogeneous schema with six node types and seven semantic edge types, targeting transactions while using customer, card, merchant, category, and location nodes as context.Location denotes state or location fields in the source records.
  • Heterogeneous Schema: Adding reverse edges to the seven semantic edge types yields fourteen directed edge types, including transaction-context and context-context associations.The context-context associations include customer–card ownership and customer–location association.
  • Graph Representation: Rows map to transaction nodes, entity-valued fields are deduplicated by node type, intermediate entities remain explicit, and type-specific attributes range from 7 to 21 dimensions.Edges are not projected into direct transaction–transaction links; attributes use non-label fields and leakage-free deterministic features.

4 DATASET STATISTICS AND ANALYSIS

FinFraudBench provides two heterogeneous fraud graphs that share a six-node-type, fourteen-edge-type schema while differing substantially in scale. Their transaction splits use only about 2% labeled training data and include balanced mini-test subsets for stable evaluation under extreme class imbalance.

  • Graph Statistics: 8.99M nodes and 89.23M directed typed edges make BankTrans-Fraud substantially larger than CreditCard-Fraud, which has 1.86M nodes and 18.53M edges.Both datasets preserve six node types and fourteen directed edge types; transaction nodes dominate graph size, while context nodes support relational structure.
  • Transaction Splits: About 2% of transactions are labeled for training in each dataset, with 37.0K labeled and 1.22M unlabeled transactions in CreditCard-Fraud.BankTrans-Fraud contains 178.3K labeled and 6.78M unlabeled training transactions.
  • Transaction Splits: Each dataset includes labeled and unlabeled training transactions, validation transactions, a full test split, and five balanced mini-test subsets.The balanced subsets contain equal numbers of fraudulent and non-fraudulent transactions for broader, more stable comparison under extreme class imbalance.

5 BENCHMARK TASKS AND EVALUATION PROTOCOL

FinFraudBench evaluates binary fraud classification on transaction nodes using the full heterogeneous graph or deterministic feature-only, homogeneous, and multi-relational projections. Its protocol separates ranking from threshold-dependent classification and uses limited training labels, validation AUPRC selection, and balanced mini-test reporting.

  • Benchmark task: Deterministic projections support methods requiring feature-only, homogeneous, or multi-relational inputs.All evaluations use the transaction-level splits defined in Section 4.2.
  • Benchmark task: The benchmark performs binary node classification on transaction nodes, with models using structure and attributes from the full graph or an input projection.Supervised labels are provided only for labeled training transactions, while validation labels support model selection and early stopping.
  • Metrics: Six complementary metrics evaluate ranking and classification behavior on balanced mini-test subsets with equal fraud/non-fraud weight.The main experiments use the balanced subsets defined in Section 4.2.
  • Metrics: AUROC and AUPRC assess score ranking before threshold selection, with AUPRC especially important under class imbalance.AUROC measures fraud-versus-legitimate score ordering, while AUPRC captures the fraud-class precision–recall trade-off.
  • Metrics: Accuracy, Macro-F1, Fraud F1, and Fraud Recall provide threshold-dependent classification views of balanced performance, class-specific errors, false positives, and missed fraud.Accuracy is informative on balanced mini-test subsets; Macro-F1 covers both classes, Fraud F1 penalizes excessive false positives, and Fraud Recall captures missed-fraud risk.
  • Reporting protocol: Main benchmark tables report mean performance over five balanced mini-test subsets, with validation AUPRC used for model selection unless otherwise stated.AUROC and AUPRC are ranking metrics, whereas accuracy, Macro-F1, Fraud F1, and Fraud Recall are threshold-dependent classification metrics.

6 EXPERIMENTS

Experiments compare non-GNN, homogeneous, multi-relation, fraud-oriented, and heterogeneous methods on balanced mini-test subsets using ranking and imbalance-sensitive classification metrics. Heterogeneous models are strongest overall, while scalability and stability remain important practical considerations.

  • Experimental setup: The evaluation groups representative baselines into five families: non-GNN, homogeneous GNN, multi-relation, fraud-oriented, and heterogeneous methods.MLP and LLM prompting are non-GNN baselines; GCN and GraphSAGE use collapsed homogeneous graphs; R-GCN, ConsisGAD, HAN, SeHGNN, HGT, and fraud-oriented methods cover the remaining families.
  • Overall results: Heterogeneous methods are strongest overall, with HGT leading most CreditCard-Fraud metrics and achieving the best AUROC and AUPRC on BankTrans-Fraud.GAAP leads CreditCard-Fraud AUROC and AUPRC, while SeHGNN leads BankTrans-Fraud classification metrics.
  • Overall results: Relation-aware and fraud-oriented methods remain competitive but do not consistently match heterogeneous models on threshold-dependent classification metrics.GAAP leads CreditCard-Fraud AUROC and AUPRC, while ConsisGAD and PMP remain competitive on BankTrans-Fraud.
  • Practical considerations: 18.53M directed typed edges in CreditCard-Fraud and 89.23M in BankTrans-Fraud make scalability, sampling, and batching central evaluation considerations.Graph baselines can use unlabeled training transactions as context but cannot rely on full-batch propagation.
  • Practical considerations: LLM prompting probes show larger standard deviations than graph-learning methods, whereas the balanced mini-test protocol provides stable estimates for most trained baselines.The comparison uses Qwen2-7B-Instruct for zero-shot and 4-shot prompting probes.

7 DISCUSSION

The discussion finds that FinFraudBench supports both baseline ranking and diagnosis of which schema components methods exploit, with heterogeneous graph design providing the strongest overall signal. It also identifies limitations from public transaction fields and motivates richer operational signals and fraud-oriented models for realistic settings.

  • FinFraudBench supports ranking baselines and diagnosing how methods exploit transaction features, relation channels, entity types, type-specific attributes, and fraud-oriented assumptions.These components provide complementary signals for multi-entity financial fraud detection.
  • Heterogeneous methods provide the most consistent gains because entity types, relation semantics, and type-specific attributes preserve useful fraud information.Homogeneous or single-node-type multi-relational projections can discard part of the original entity-relation structure.
  • Multi-relation and fraud-oriented methods remain competitive on several metrics, but their gap to the strongest heterogeneous methods favors combining these cues with entity types and type-specific attributes.The discussion presents relation channels and fraud-oriented inductive biases as useful components of heterogeneous financial graphs.
  • Public transaction-level tables limit the available graph signals, motivating richer operational data and models designed for heterogeneous fraud graphs under limited labels and extreme class imbalance.The released fields may not fully reflect richer real-world fraud contexts.

8 CONCLUSION

FinFraudBench is a benchmark of two heterogeneous graph datasets for financial fraud detection, constructed from public transaction-level fraud sources and designed to preserve realistic financial-data properties. Baseline reproduction results provide family-level insights and show that heterogeneous GNNs perform best.

  • FinFraudBench comprises two heterogeneous graph datasets for financial fraud detection constructed from public transaction-level fraud sources.
  • The graphs preserve multiple financial entity types, relation types, type-specific attributes, limited labels, and realistic fraud imbalance.
  • Baseline reproduction provides family-level insights into how model designs use heterogeneous fraud signals, with heterogeneous GNNs performing best.

ETHICS AND REPRODUCIBILITY STATEMENT

FinFraudBench uses public transaction-level data without human-subject experiments or new data collection, and is intended for research rather than unvalidated operational decision making. Its preprocessing, splits, experiments, code, and fixed-split datasets are openly documented or released to support reproduction.

  • FinFraudBench uses public transaction-level data and involves neither human-subject experiments nor new data collection.
  • The benchmark targets fraud detection research rather than operational decision making without additional validation.
  • Preprocessing, split construction, and reported experiments are described in the paper and appendix.
  • The code repository is openly released, and constructed datasets are available on Hugging Face with fixed random splits for reproducing main results.

AI USE STATEMENT

AI-assisted tools supported literature search, manuscript preparation, language polishing, and formatting, but not the benchmark’s design, datasets, protocol, conclusions, or final scientific approval.

  • AI USE STATEMENT: AI-assisted tools supported literature search, survey, manuscript writing, language polishing, and formatting during paper preparation.The authors used these tools for preparation-related tasks.
  • AI USE STATEMENT: The tools were not used to design the benchmark, construct datasets, define the experimental protocol, or draw scientific conclusions.The authors retained responsibility for the paper’s scientific development.
  • AI USE STATEMENT: The authors reviewed and approved all scientific claims, dataset construction, experimental analysis, and final wording.This review covered both the scientific content and the final text.

A ADDITIONAL DATASET DETAILS … A.4 MINI-TEST STABILITY

The appendix details dataset feature construction, transaction splits, complete balanced mini-test results, and stability analysis. It emphasizes schema consistency, avoidance of label leakage, and stable estimates across mini-test subsets.

  • A.1 TYPE-SPECIFIC FEATURE FIELDS: Both datasets share node and relation schemas, while same-type feature dimensions differ because their source tables expose different raw fields.Feature construction uses representative non-label feature groups and avoids label leakage in both graph and feature construction.
  • A.1 TYPE-SPECIFIC FEATURE FIELDS: Table 3 summarizes representative non-label feature groups used to construct node attributes by node type.The table organizes the feature groups underlying the dataset-specific node representations.
  • A.1 TYPE-SPECIFIC FEATURE FIELDS: Table 4 provides detailed statistics for the transaction splits used in the benchmark.These split statistics document how transactions are partitioned for evaluation.
  • A.3 COMPLETE BALANCED MINI-TEST RESULTS: Table 5 reports balanced mini-test results for CreditCard-Fraud as means and standard deviations across five mini-test subsets.The results provide variability information alongside average performance.
  • A.3 COMPLETE BALANCED MINI-TEST RESULTS: Table 6 reports balanced mini-test results for BankTrans-Fraud as means and standard deviations across five mini-test subsets.The results provide variability information alongside average performance.
  • A.4 MINI-TEST STABILITY: Figure 7 measures coefficient of variation across five balanced mini-test subsets for each metric and trained non-LLM baseline.The heatmaps complement averaged main-paper results, with lighter cells indicating more stable estimates.
  • A.4 MINI-TEST STABILITY: The balanced mini-test protocol gives stable estimates for most graph-learning baselines.This conclusion is drawn from the coefficient-of-variation analysis across the five subsets.
Loading 2608.15177v1…