PaperlayerView exact revision on arXiv

Source-linked AI summary

OPBench: A Graph Benchmark to Combat the Opioid Crisis

Tianyi Ma, Yiyang Li, Yiyue Qian, Zheyuan Zhang, Zehong Wang, Chuxu Zhang, Yanfang Ye

arXiv:2602.14602v1cs.LGcs.AI

TL;DR

Graph learning lacks a comprehensive benchmark for comparing methods across real-world opioid-crisis scenarios. OPBench addresses this gap with five curated datasets, diverse graph structures, and standardized evaluation protocols. Experiments reveal method strengths and limitations, while the benchmark supports systematic research and practice within its stated scope.

  • Problem

    No comprehensive benchmark systematically evaluates graph learning methods across the opioid crisis’s diverse real-world scenarios.

  • Method

    OPBench provides five expert-curated datasets across three application domains, heterogeneous and higher-order graph structures, standardized protocols, and reproducible baselines.

  • Results

    Experiments reveal strengths and limitations of existing graph learning methods, including degraded performance from homogeneous projections and strong HAN performance on NHANES-Diet.

  • Takeaways & Limitations

    OPBench provides a standardized evaluation framework and rich real-world datasets to guide graph-learning research and method selection for opioid-crisis applications.

  • Takeaways & Limitations

    The PDMP data covers only Ohio in 2016, social-media detection omits encrypted and dark-web markets, and nutritional biomarkers are proxy indicators rather than standalone diagnostics.

Abstract

from arXiv · show

The opioid epidemic continues to ravage communities worldwide, straining healthcare systems, disrupting families, and demanding urgent computational solutions. To combat this lethal opioid crisis, graph learning methods have emerged as a promising paradigm for modeling complex drug-related phenomena. However, a significant gap remains: there is no comprehensive benchmark for systematically evaluating these methods across real-world opioid crisis scenarios. To bridge this gap, we introduce OPBench, the first comprehensive opioid benchmark comprising five datasets across three critical application domains: opioid overdose detection from healthcare claims, illicit drug trafficking detection from digital platforms, and drug misuse prediction from dietary patterns. Specifically, OPBench incorporates diverse graph structures, including heterogeneous graphs and hypergraphs, to preserve the rich and complex relational information among drug-related data. To address data scarcity, we collaborate with domain experts and authoritative institutions to curate and annotate datasets while adhering to privacy and ethical guidelines. Furthermore, we establish a unified evaluation framework with standardized protocols, predefined data splits, and reproducible baselines to facilitate fair and systematic comparison among graph learning methods. Through extensive experiments, we analyze the strengths and limitations of existing graph learning methods, thereby providing actionable insights for future research in combating the opioid crisis. Our source code and datasets are available at https://github.com/Tianyi-Billy-Ma/OPBench.

1 Introduction

Opbench addresses the absence of a comprehensive benchmark for evaluating graph learning methods across real-world opioid-crisis scenarios. It contributes five datasets, diverse graph structures, expert-curated data, and standardized evaluation resources.

  • The opioid crisis spans healthcare overdose risk, digital drug trafficking, and substance dependency, but no benchmark systematically compares graph learning methods across these scenarios.
  • Opbench introduces five benchmark datasets covering opioid overdose detection, online illicit drug trafficking detection, and drug misuse prediction.
  • The benchmark includes heterogeneous, multi-relation, and hypergraph structures to represent diverse entities, relations, and dependencies in opioid-crisis data.
  • Domain experts and authoritative institutions curate and annotate the datasets under privacy and ethical guidelines.
  • Standardized settings, systematic experiments, and an open-source library support fair comparison and reproducible evaluation of graph learning methods.

2 Backgrounds

This section introduces graphs, heterogeneous graphs, and hypergraphs as foundational structures for the benchmark. It also situates graph learning within prior opioid-crisis applications.

  • A graph consists of nodes, edges, and node or edge attribute features.
  • A heterogeneous graph distinguishes node types and relation types while associating type-specific attribute feature matrices with nodes.
  • A hypergraph represents higher-order interactions through hyperedges connecting sets of at least two nodes.
  • Graph learning has been applied to opioid-crisis problems by modeling relationships among patients, healthcare providers, and prescription records.

3 Overview of Opbench

Opbench is designed around real-world opioid-crisis complexity, spanning multiple domains and graph structures while supporting rigorous, reproducible evaluation. Its framework combines curated datasets, baseline methods, standardized metrics, and controlled protocols.

  • Opbench is organized around core principles aligning benchmark design with public-health challenges and rigorous graph-learning evaluation.
  • The benchmark covers healthcare claims, illicit drug-supply activity, and dietary patterns to evaluate models across distinct opioid-crisis data domains.
  • Hypergraphs model group-wise interactions, while heterogeneous graphs represent different entity and relation types with richer semantics than homogeneous graphs.
  • Opbench provides standard data splits, common metrics including Accuracy, F1, GMeans, and AUC, and compatibility with graph-learning libraries for reproducible benchmarking.
  • The benchmark implements feature-based methods, GNNs, heterogeneous GNNs, and multi-relation GNNs as baseline categories.
  • For fair comparison, baseline methods use two layers, validation-based hyperparameter tuning, and a two-layer MLP classifier for downstream tasks.
  • Experiments run on Nvidia A40 GPUs with PyTorch 2.9.1 and PyG 2.7.0 to maintain consistent performance measurements.

4 Opbench Datasets

Opbench comprises five real-world datasets spanning opioid overdose detection, illicit drug trafficking detection, and opioid misuse detection. Across these domains, the benchmark uses heterogeneous graphs and hypergraphs to represent complex relations and evaluates graph-learning baselines under standardized comparisons.

  • Dataset scope: Opbench contains five datasets across three opioid-crisis domains: overdose detection, illicit drug trafficking detection, and drug misuse detection.The datasets cover healthcare claims, digital-platform activity, and dietary patterns.
  • Opioid overdose detection: HetGNNs outperform homogeneous GNNs and MLPs across training ratios and metrics on Pdmp-OD-Det, with HAN best on most metrics and HGT leading in selected settings.The results support explicitly modeling heterogeneous healthcare relationships for overdose-risk prediction.
  • Illicit drug trafficking detection: X-HyDrug-Comm models higher-order user collaborations as hyperedges, and ED-HNN outperforms all baselines across training ratios and evaluation metrics.The task detects illicit drug-trafficking communities on social-media-derived networks.
  • Illicit drug trafficking detection: For X-HyDrug-Role, ED-HNN reaches 68.55%, 69.40%, and 70.66% accuracy at 10%, 20%, and 50% training data, respectively.HyGNNs outperform GNNs, while naive message passing and clique expansion can introduce redundant or noisy information.
  • Drug misuse and role detection: AD-GSMOTE performs best across settings and metrics for X-MRDrug-Role, while HAN leads NHANES-Diet with accuracies of 74.06%, 75.10%, and 75.90%.The findings emphasize imbalance handling for role detection and heterogeneous relation modeling for dietary-pattern misuse detection.

5 Observations from Opbench

Opbench shows that preserving heterogeneous and higher-order structure is crucial for opioid-crisis graph tasks, while class-imbalance handling yields task-dependent gains.

  • 39.09% F1-Macro for GCN on clique-expanded graphs versus 57.62% for ED-HNN in community detection.
  • Homogeneous projections lose critical relational information, whereas heterogeneous and hypergraph representations preserve interactions relevant to overdose risk and trafficking communities.Patient risk involves patients, prescribers, pharmacies, and medications; trafficking communities involve group-wise collaborations.

6 Conclusion

Opbench is a comprehensive benchmark for graph learning in opioid-crisis applications, combining five datasets across three domains with standardized evaluation and open-source release.

  • Opbench provides five curated datasets spanning opioid overdose detection, illicit drug trafficking detection, and opioid misuse prediction from dietary patterns.
  • The benchmark offers standardized evaluation and datasets that mimic real-world scenarios, supporting research and practice in opioid-crisis graph learning.
  • Opbench is released as an open-source library for future research on graph-based methods addressing the opioid epidemic.

A Baseline Implementation Details

The benchmark implements feature-based, homogeneous, heterogeneous, multi-relational, and hypergraph baselines, with graph methods adapted to structure-specific evaluation settings.

  • Feature-based baselines include MLP, Oversampling, and SMOTE, while GNN baselines include GCN, GAT, and GraphSAGE.
  • Oversampling and SMOTE are applied only to training data, preserving validation and test distributions for unbiased evaluation.
  • For heterogeneous data, homogeneous GNNs ignore node and edge types; for hypergraphs, clique expansion converts hyperedges into pairwise edges.
  • Heterogeneous graph baselines include HAN, HGT, and HGMAE, implemented with original-paper source code and meaningful metapaths where applicable.
  • Multi-relational baselines include R-GCN, GraphENS, and AD-GSMOTE, with GraphENS and AD-GSMOTE addressing class imbalance through graph-aware augmentation strategies.
  • Hypergraph baselines include HGNN, HNHN, and HCHA, which model higher-order interactions through hyperedges.

B.1 Data Source

The Pdmp-OD-Det dataset is constructed from Ohio prescription records by cleaning the data, encoding heterogeneous entities and relations, and labeling patients by opioid dosage risk.

  • The source contains 1,395,958 controlled-substance prescription records from Ohio’s 2016 PDMP data.
  • Records are cleaned by removing missing essential fields, imputing categorical values, filtering invalid quantities and days supplies, and standardizing dates.
  • Four heterogeneous node types are represented with 768-dimensional SentenceBERT feature vectors generated from textual attribute descriptions.
  • Five edge types connect patients, prescribers, pharmacies, and drugs to capture medication profiles, pharmacy use, prescriber visits, prescribing, and dispensing patterns.
  • Patients are classified into high- or low-risk groups according to whether daily MME exceeds or falls below 90 MME/day.
  • The resulting graph contains 54,318 nodes and 321,437 edges across five relation types.

C.1 Data Collection

The benchmark constructs drug-related graph datasets from social-media and other domain sources, using expert annotation and relationship-specific structures. Social-media users are assigned overlapping communities or exclusive trafficking roles based on behavioral evidence.

  • Social-Media Data: 275,884,694 posts from 40,780,721 users were collected through the official X API, yielding 266,975 drug-relevant posts from 54,680 users for annotation.
  • Social-Media Data: Six domain experts annotated filtered users independently for 62 days and resolved disagreements through discussion until consensus.
  • Community Annotation: X-HyDrug-Comm assigns users to one or more of six pharmacological communities using promotion, overdose or addiction evidence, and purchase evidence.
  • Community Annotation: Table 7 maps drug types to the six community categories used for overlapping community detection.
  • Role Annotation: X-HyDrug-Role and X-MRDrug-Role classify each user into exactly one trafficking role based on dominant behavior, forming a multi-class task.
  • Graph Construction: The hypergraph datasets use hyperedges for shared follows, conversations, hashtags, and emojis, preserving multi-user relational patterns.

D.1 Data Source

NHANES-Diet combines dietary, demographic, and drug-use records into a heterogeneous graph whose entities and relations support opioid misuse prediction. Expert-derived dietary habits and public-health labeling define the prediction task.

  • Data Source: NHANES-Diet covers 2003–2020 dietary intake records, demographic information, and drug-use questionnaires encoded with FNDDS food codes.
  • Graph Schema: The heterogeneous graph defines five node types with distinct feature representations, including users, foods, and ingredients.
  • Node Features: User features include 37 demographic attributes, while food features combine 46 nutrient values with 768-dimensional BERT food-description embeddings.
  • Dietary Habits: Four domain experts identify dietary-habit features and apply threshold-based annotation using the top and bottom 10% of respondents.
  • Dietary Habits: The process derives 54 dietary habits, assigning contrasting tags such as “drink lots of milk” and “drink little or no milk.”
  • Graph Schema: User–food, food–ingredient, and food–category relations encode consumption patterns, nutritional composition, and food classification.
  • Prediction Task: Opioid misuse is positive for past-year heroin use or prescription-opioid use exceeding 90 continuous days; the graph contains 13,742 nodes and 445,468 edges.

E Additional Experiments

Additional experiments examine inference-time scaling across hidden dimensions and datasets while documenting privacy protections and responsible-use guidance. Efficiency varies by graph type and method, with simpler models fastest on NHANES-Diet.

  • Efficiency Analysis: Inference time is evaluated across hidden dimensions 64, 128, 256, and 512 on two hypergraph datasets and NHANES-Diet.
  • Efficiency Analysis: On X-HyDrug-Role, methods show comparable scaling, while ED-HNN has the lowest inference times and HCHA remains efficient.
  • Efficiency Analysis: On NHANES-Diet, MLP scales linearly from 0.22 ms to 1.09 ms and GCN ranges from 1.35 to 4.66 ms across hidden dimensions.
  • Ethics and Privacy: PDMP data are de-identified under a data-use agreement, NHANES is publicly de-identified, and social-media identifiers are hashed.
  • Ethics and Privacy: Released graph formats contain no raw text or user-level data, and Sentence-BERT features cannot reconstruct the original text.
  • Responsible Use: The authors recommend using models as decision-support tools by qualified professionals to avoid algorithmic bias or stigmatization.

F.3 Limitations

Opbench’s limitations concern geographic and temporal coverage, incomplete representation of online trafficking, and the proxy nature of nutritional biomarkers.

  • Scope Boundaries: PDMP data come from Ohio and 2016, limiting coverage of evolving synthetic-opioid patterns in other places and periods.
  • Scope Boundaries: Social-media trafficking detection represents only part of the illicit market because traffickers may migrate to encrypted platforms or the dark web.
  • Scope Boundaries: Nutritional biomarkers are proxy indicators and should be combined with multimodal assessment rather than used as a standalone diagnostic tool.
Loading 2602.14602v1…