PaperlayerView exact revision on arXiv

Source-linked AI summary

CryptoFlowBench: Benchmarking Multi-Hop Reasoning on Blockchains for Cryptocurrency Fund Tracing

Runang He, Tongya Zheng, Jing Cao, Yucheng Ying, Shunyu Liu, Bingde Hu, Yang Gao, Zeyuan Hu, Jianrong Zhang, Mingli Song

arXiv:2610.16449v1cs.CRcs.AIcs.LG

TL;DR

Public blockchains expose transfers but do not directly bind accounts to real-world identities, complicating analysis of illicit flows. CryptoFlowBench provides four case-grounded tracing tasks, and ChainSleuth uses tools to retrieve evidence and revise decisions. Conventional methods remain competitive on single-step decisions, while ChainSleuth performs better on sequential and seed-uncertain tracing.

  • Problem

    Public blockchains provide transparent transaction records without directly binding accounts to real-world identities, complicating analysis of illicit cryptocurrency flows.

    Source: Introduction · p. 1

  • Method

    CryptoFlowBench aligns incident reports with locally verified transactions for four tracing tasks, while ChainSleuth retrieves on-chain evidence through tools and revises tracing decisions.

    Source: Introduction · p. 2

  • Results

    Conventional methods remain competitive on single-step decisions, while ChainSleuth performs better on sequential and seed-uncertain tracing, improving Path F1 by 70.0% over the strongest specialized baseline for that metric.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    The results highlight the potential of evidence-grounded agents for practical and reproducible cryptocurrency investigations.

    Source: Conclusion · p. 7

    Exact transaction-existence claims are limited to the indexed chain, block window, and supported transfer types, and the verified reference graph is not an exhaustive reconstruction of illicit activity.

    Source: Blockchain Ledger · p. 3, Transaction-Graph Construction · p. 3

Abstract

from arXiv · show

Pseudonymous accounts and complex smart-contract ecosystems substantially hinder the tracing of illicit cryptocurrency fund flows. Existing Graph Neural Networks (GNNs) have achieved promising performance in blockchain fraud detection, yet their static node- or edge-level predictions do not naturally generalize to multi-hop fund tracing. Meanwhile, existing tracing methods mainly rely on predefined heuristics to capture multi-hop relationships, limiting their adaptive reasoning capabilities. To bridge this gap, we introduce CryptoFlowBench, a benchmark constructed from authoritative reports on cryptocurrency-related illicit incidents, with four tasks spanning local to multi-hop fund tracing. These tasks pose substantial challenges to existing learning- and heuristic-based methods, particularly in complex multi-hop scenarios. We therefore propose ChainSleuth, a graph-tool-augmented agent that iteratively reasons, invokes graph tools, and updates its tracing trajectory, enabling adaptive information acquisition and recovery from erroneous paths. Experiments show that no baseline consistently dominates all four tasks, revealing substantial gaps between local fraud detection and multi-hop fund tracing. ChainSleuth achieves notable improvements on multi-hop tasks, demonstrating the potential of agentic graph reasoning for cryptocurrency fund investigation.

Introduction

CryptoFlowBench turns heterogeneous incident reports into a case-grounded benchmark for local and multi-hop fund tracing. ChainSleuth uses retrieved on-chain evidence to revise tracing decisions, and its strongest reported gains are on multi-hop metrics.

Introduction

  • Fixed propagation policies and independent local predictions offer limited support for revising earlier decisions, while multi-hop tracing also faces error accumulation and diversion through irrelevant service nodes.The paper identifies case-grounded evaluation, error accumulation, and adaptive exploration with boundary control as three challenges.Source: Introduction · p. 2
  • CryptoFlowBench aligns incident reports with locally verified transactions and derives four tasks spanning factual verification, seeded decisions, multi-hop reconstruction, and seedless investigation.The benchmark represents incidents with address roles, tracing actions, path stages, hard negatives, evidential boundaries, and source provenance.Source: Introduction · p. 2
  • ChainSleuth maintains an evolving investigation state, retrieves transaction-level evidence through tools, and can revisit hypotheses, suppress unsupported branches, and stop at evidential boundaries.It is presented as an evidence-grounded, tool-augmented LLM agent operating within a bounded ReAct loop.Source: Introduction · p. 2
  • 70.0% higher Path F1, 26.6% lower over-expansion, and 21.4% higher predicted-seed F1 than the strongest specialized baseline for each metric.The evaluation compares task-appropriate heuristic, machine-learning, GNN, and LLM baselines under matched case splits and tracing budgets.Source: Introduction · p. 2

Related Work

Prior work spans illicit-activity detection, fund-flow tracing, and LLM agents for security and analysis. The paper identifies evidence-grounded investigative decisions and multi-hop tracing as gaps across these lines of work.

Cryptocurrency Illicit-Activity Detection

  • Detection methods range from feature-based classifiers and GNNs to general graph-anomaly methods, but remain focused on node-level detection rather than evidence-grounded, multi-hop fund tracing.The paper notes trade-offs involving handcrafted features, computational cost, explainability, and handling heterophily, label scarcity, or class imbalance.Source: Cryptocurrency Illicit-Activity… · p. 2

Cryptocurrency Fund-Flow Tracing

  • Taint rules, classifiers, GCNs, relevance-guided search, and dense-flow objectives support transaction classification, account prioritization, or path reconstruction, but do not explicitly model evidence-grounded decisions or revision after new observations.Deterministic taint methods can produce different results for the same source transaction under different fixed value-allocation assumptions.Source: Cryptocurrency Fund-Flow Tracing · p. 3

LLM Agents for Blockchain Security and Analysis

  • LLM agents iteratively gather evidence and refine actions in external environments and structured graphs, while blockchain-security applications have largely focused on contract code rather than multi-step fund-flow tracing.The paper connects agentic methods to querying large transaction graphs and verifying intermediate decisions with external evidence.Source: LLM Agents for Blockchain Security and… · p. 3

Preliminaries

The preliminaries define the indexed blockchain-transfer setting and the graph objects used for tracing. They also bound transaction-existence claims and clarify that the verified reference graph is not an exhaustive reconstruction of illicit activity.

Blockchain Ledger

  • Transfers are indexed within a block window and record transaction, block, time, source, destination, asset, amount, and transfer-type information; exact existence claims are limited to the indexed chain, window, and supported types.Table 1 summarizes the benchmark cases by incident type and primary source.Source: Blockchain Ledger · p. 3, Table 1 · p. 3

Transaction-Graph Construction

  • Raw transfers form a directed address-relation graph, with relation edges retaining transfer records and aggregating amounts only across records sharing asset and transfer type.The graph's nodes are addresses, and its edges represent ordered address pairs with transfers.Source: Transaction-Graph Construction · p. 3
  • Given a case graph, public incident evidence, optional seeds, and an exploration budget, tracing returns seeds, a selected subgraph, and paths contained in that subgraph.The objective is to recover relevant edges under the budget while limiting irrelevant expansion; the reference graph represents verified case evidence, not all illicit activity.Source: Transaction-Graph Construction · p. 3

CryptoFlowBench

CryptoFlowBench grounds cryptocurrency fund-tracing tasks in public incident reports aligned with on-chain evidence. It defines four complementary tasks that assess investigation from factual verification through seedless subgraph tracing.

CryptoFlowBench

  • The benchmark defines transaction existence, next-step action, path completion, and subgraph filtering to evaluate tracing from factual verification to seedless subgraph-level reasoning.The tasks respectively assess factual verification, local decisions, sequential reasoning under accumulated errors, and seedless subgraph tracing.Source: CryptoFlowBench · p. 3

Authoritative Reports Integration

  • Public incident reports are consolidated into structured case summaries, aligned with indexed on-chain transactions, and converted into grounded tasks with task-specific verification.LLM-driven agents collect and summarize reports, while graph alignment, task generation, and verification are deterministic.Source: Figure 2 · p. 4, CryptoFlowBench · p. 4

On-chain Case Graph Construction

  • The case-evidence specification records entities, transfers, roles, stages, provenance, temporal constraints, transaction identifiers, and verification status without asserting that reported relations exist on-chain.It supplies candidate anchors and alignment constraints for grounding reports against indexed blockchain evidence.Source: On-chain Case Graph Construction · p. 4

Cryptocurrency Fund Tracing Tasks

  • Investigation actions distinguish following the reference flow, inspecting relevant but insufficient evidence, stopping at a boundary, and ignoring irrelevant relations.This extends binary edge classification into an investigative interface that represents evidential boundaries.Source: Cryptocurrency Fund Tracing Tasks · p. 5
  • The four tasks share a graph-first generation process, with Action evaluating one seeded decision, Path evaluating sequential decisions, and Subgraph requiring seed inference before tracing.Path exposes error accumulation across hops; Subgraph removes the trusted starting node and tests tracing in a distractor-rich subgraph.Source: Cryptocurrency Fund Tracing Tasks · p. 5

ChainSleuth

ChainSleuth is a ReAct-based, retrieval-augmented LLM agent for evidence-grounded fund tracing. It iteratively gathers evidence and updates its tracing trajectory within a bounded tool-interaction loop.

  • ChainSleuth uses a ReAct-based agent to retrieve evidence and revise tracing hypotheses, addressing the limited revision support of fixed rules and independent local decisions.It can use a local blockchain index, blockchain explorers, and optional pretrained detectors.Source: ChainSleuth · p. 5
  • At each bounded interaction step, ChainSleuth either makes an allowed tool call and receives a sanitized observation or returns a structured answer.Execution stops when an answer is produced, evidence coverage is reached, progress ceases, or the tool budget is exhausted.Source: ChainSleuth · p. 5
  • For Action, Path, and Subgraph, the agent retrieves transaction evidence and expands only relations whose follow probability exceeds a declared threshold.Action applies the decision once, Path repeats it along a bounded trajectory, and Subgraph first estimates a seed distribution.Source: ChainSleuth · p. 5

Experiments

The evaluation distinguishes fundamental investigation capabilities from multi-hop tracing. Path and Subgraph form the core tracing benchmark and are compared against task-appropriate heuristic and learning baselines.

  • Transaction existence and Action assess foundational capabilities such as exact tool invocation, evidence grounding, and local action selection, with results reported in Appendix C.Source: Experiments · p. 5
  • Path and Subgraph are the core tracing tasks, measuring sequential investigation under cumulative error and seedless suspicious-region discovery, respectively.Source: Experiments · p. 6
  • For Path and Subgraph, ChainSleuth is compared with heuristic and learning-based baselines using matched candidate nodes, case splits, and inference budgets.Source: Experiments · p. 6

Experimental Settings

The experiments use case-disjoint splits and several LLMs, comparing ChainSleuth with rule-based, graph-based, and learning-based baselines. Metrics distinguish tracing quality, execution validity, and protocol compliance.

Dataset and Splits

  • The benchmark contains 22 publicly documented Ethereum incidents, with learning-based methods trained, validated, and tested on case-disjoint splits to prevent case-specific information leakage.Source: Dataset and Splits · p. 6

Agent Models

  • ChainSleuth is evaluated with Qwen3.6-27B, DeepSeek-V4-Flash, and DeepSeek-V4-Pro as its LLMs.Source: Agent Models · p. 6

Baselines

  • Baselines include adapted taint-propagation rules, graph-based tracing methods, classical classifiers, and blockchain-oriented GNNs adapted to edge-level prediction.The rule-based methods share initial taint budgets, candidate universes, and rollout limits; graph scores are mapped to edge rankings and decisions.Source: Baselines · p. 6

Evaluation Metrics

  • Path metrics assess trajectory reconstruction, local decisions, and over-expansion, while Subgraph metrics separate predicted-seed pipeline performance from propagation with oracle seeds.Structured-output validity and tool-protocol pass rate distinguish tracing performance from execution and protocol failures.Source: Evaluation Metrics · p. 6

Comparison Experiments

ChainSleuth outperforms specialized baselines on core multi-step tracing, while backend reliability and detector-tool use affect its results.

Comparison Experiments

  • Path complete-path rate is 0.665 and Path F1 is 0.734; Subgraph predicted-seed F1 is 0.375, while RF performs better with oracle seeds.On Path, over-expansion is 0.116 versus 0.158 for the strongest baseline. On Subgraph, RF has oracle-seed F1 of 0.588 but predicted-seed F1 of 0.309.Source: Comparison Experiments · pp. 6–7

Effect of LLM Backends

  • On Path, Qwen3.6-27B has structured-valid rate 1.000 and tool-protocol pass rate 0.979; DeepSeek-V4-Flash scores 0.318 and 0.099, respectively.On Subgraph, predicted-seed F1 rises from 0.094 for Flash to 0.213 for Pro, still below Qwen3.6-27B's 0.375; protocol reliability is associated with the backend performance gap.Source: Effect of LLM Backends · p. 7

Detector as Tool

  • Adding a detector raises Path completeness from 0.665 to 0.760, but lowers Path F1 from 0.734 to 0.628 and raises over-expansion from 0.116 to 0.221.The detector tool degrades all reported Subgraph metrics; the authors suggest detector scores may favor locally plausible but globally inconsistent expansions.Source: Detector as Tool · p. 7

Seed-Detector Sensitivity

The Subgraph seed-detector comparison uses shared case-disjoint evaluation and motivates reporting oracle-seed results separately from predicted-seed results.

Seed-Detector Sensitivity

  • RF and XGBoost have comparable seed-detector ranking performance and both outperform LR, supporting XGBoost for predicted-seed evaluation.All baselines share the same case-disjoint seed detector, public features, and target-case splits; oracle-seed results are reported separately to isolate propagation performance.Source: Seed-Detector Sensitivity · p. 7

Conclusion

The paper introduces a case-grounded Ethereum tracing benchmark and ChainSleuth, then reports stronger performance for ChainSleuth on sequential and seed-uncertain tracing than conventional methods.

Conclusion

  • CryptoFlowBench unifies transaction verification, local decisions, multi-step path completion, and seedless subgraph filtering under strict public–oracle separation.The benchmark is case-grounded and targets Ethereum fund-flow tracing.Source: Conclusion · p. 7
  • ChainSleuth retrieves auditable on-chain evidence and makes structured tracing decisions without requiring the full transaction graph in context.It models tracing as a sequential investigation in which agents can acquire evidence, revise decisions, and limit irrelevant expansion.Source: Conclusion · pp. 7–8
  • Conventional methods remain competitive on single-step decisions, whereas ChainSleuth performs better on sequential and seed-uncertain tracing.The conclusion presents evidence-grounded agents as having potential for practical and reproducible cryptocurrency investigations.Source: Conclusion · pp. 7–8

Related papers

Loading 2610.16449v1…