PaperlayerView exact revision on arXiv

Source-linked AI summary

In Medical Claims Data, Enhancing Predictive Performance for Major Adverse Cardiovascular Events Using Cross Attention

Yuhei Fujioka, Daitaro Misawa, Tatsuyoshi Ikenoue, Shingo Fukuma

arXiv:2609.09824v1cs.LG

TL;DR

Medical claims data contain clinical information but lack clear medical structure, limiting effective use of diagnosis-treatment links. This paper uses cross-attention to weight those relationships for MACE prediction, with the proposed model achieving the highest scores among the evaluated models: ROC-AUC 0.7720 and MCC 0.1525.

  • Problem

    Medical claims data lack clear medical structure, and few researchers effectively use diagnosis-treatment relationships to develop disease prediction models.

    Source: §1.1 · p. 1, §1.3 · p. 2

  • Method

    The model combines health checkup and medical claims data for MACE prediction, using cross-attention to learn many-to-many relationships between diagnoses and treatments.

    Source: §1.4 · p. 2, §1.3 · p. 2

  • Results

    ROC-AUC 0.7720 and MCC 0.1525 were the highest scores among the evaluated MACE prediction models.

    Source: §5.1 · p. 5

  • Takeaways & Limitations

    The proposed method has potential for identifying individuals at high risk of MACE to improve public health services in Japan.

    Source: §6 · p. 5

    The study excluded duplicated-code and claims time-series data, and its findings should not be extrapolated to populations with very high monthly medical-code counts.

    Source: §7 · p. 6

Abstract

from arXiv · show

Medical claims data comprise the financial details, including the expenses and billing information, as well as the clinical information, such as the diagnoses and treatments, of patients visiting medical facilities. Recently, it has been acknowledged that large databases can be constructed from medical claims data for medical research purposes. However, the clinical information within these datasets is often medically unstructured, limiting its application in comprehensive analyses. This study enhances predictive model performance for major adverse cardiovascular events (MACE), a leading cause of death worldwide. Models that predict MACE are crucial to clinical practice guidelines. We utilize a cross-attention mechanism to develop a method that effectively weights the relationships between diagnoses and treatments. Effectively repre- senting the clinical information contained in medical claims data, this approach generates more representative features for predicting MACE. The ROC-AUC score of our proposed cross-attention-based model was 0.7720, higher than other benchmark models including the conventional atherosclerotic cardiovascular disease model, the light gradient boosting machine, and a self-attention-based model. These results indicate that integrating the clinical structure of medical claims data using a cross-attention mechanism significantly enhances the performance of predictive models.

1 INTRODUCTION

The introduction frames MACE prediction as a high-stakes task whose medical claims data contain clinically useful but poorly structured diagnosis-treatment information, and presents cross-attention as a way to use it.

1.1 Background

  • Claims and EHR record diagnoses, procedures, and prescriptions, yet MACE models do not fully use claims clinical information despite guideline use; MACE is a leading global cause of death.Identifying high-risk individuals and managing risk factors are considered effective approaches to prevention.Source: §1.1 · pp. 1–2

1.2 Task Definition

  • The task predicts each patient's probability of MACE-related hospitalization or death from health-checkup and claims data; MACE cases are ICD-10-defined and rare, making classification challenging.MACE is defined as diseases listed in Table 1, identified using ICD-10 codes.Source: §1.2 · p. 2

1.3 Challenges

  • Because claims data are reimbursement-oriented and lack clear medical structure, linking diagnoses to treatments is difficult, especially for many-to-many relationships; the study uses cross-attention to weight and match them.This approach is intended to use clinical information in claims data more effectively for MACE prediction.Source: §1.3 · p. 2

1.4 Contributions

  • The study proposes a transformer MACE predictor combining health-checkup and claims data; cross-attention learns many-to-many diagnosis-treatment links, unlike self-attention models learning many-to-one links, and improves performance over health-checkup-only models.The authors report improvement from combining the two data types and the cross-attention mechanism compared with using health-checkup data alone.Source: §1.4 · p. 2

2 RELATED WORK

Related work spans guideline-based cardiovascular risk scores and disease prediction from checkup, claims, and EHR data; attention methods can improve prediction but face challenges in representing diagnosis-treatment relationships.

2.1 MACE Prediction Models

  • Pooled Cohort Equations estimate 10-year ASCVD risk from health checkup data in African American and white adults aged 40–79, while the Suita score estimates 10-year coronary heart disease risk in Japanese urban residents aged 30–79.Pooled Cohort Equations are adopted in American guidelines; the Suita score is included in Japan Atherosclerosis Society guidelines.Source: §2.1 · p. 2

2.2 Disease Prediction Utilizing Health Checkup Data and Clinical Information

  • Studies using health checkup and claims or EHR clinical information reported improved ROC-AUC, while selecting only outcome-related diagnoses and medications rather than all recorded codes.Source: §2.2 · p. 2

2.3 Self-Attention in Clinical Information

  • Self-attention EHR models show promise [13] [21], but long sequences limit training scale, while ExBEHRT links treatments to diagnoses yet misses their many-to-many relationships.Including diagnoses and treatments in a patient sequence improves prediction [21]; manually linking them is difficult and time-consuming because their correlations can be unclear, and the process may introduce subjectivity.Source: §2.3 · p. 2

2.4 Examples of the Application of Cross-Attention Mechanisms

  • Cross-attention relates elements across two inputs and weights their connections; protein-compound studies show superior predictive outcomes using it to model many-to-many relationships.The mechanism calculates relevance between elements in the two inputs and aggregates those relevance weights.Source: §2.4 · p. 3

3 METHOD

The method represents health-checkup and monthly claims data in a model that uses cross-attention to integrate diagnoses with related treatments. Claims codes are structured into medical sub-tokens, and benchmark models and training procedures are specified for comparison.

3.1.1 Our Proposed Model (Our CA).

  • The proposed model combines health-checkup features with 12 monthly claims representations and classifies their concatenated outputs.A feed-forward block transforms tabular data; monthly diagnoses and treatments are separately encoded, cross-attended, and combined before the classification head. The embedding block organizes diagnosis and medication sub-tokens into vectors.Source: §3.1.1 · p. 3, Figure 1 · p. 3

3.1.2 Benchmark Models. •

  • The benchmarks were ASCVD, LGBM, and a self-attention model; models requiring finer temporal diagnosis and treatment data were excluded.Claims were recorded monthly, without the detailed temporal information needed to train models in [13] [21].Source: §3.1.2 · pp. 3–4

3.1.1 Our Proposed Model (Our CA).

  • Cross-attention uses diagnoses as queries and treatments as keys and values, weighting treatments and integrating them into the diagnosis representation.The resulting representation passes through layer normalization and a feed-forward network, then its length dimension is reduced by summation.Source: §3.1.1 · p. 3, Figure 2 · p. 4

3.2 Input Feature

  • Inputs comprised health-checkup data and claims lists; LGBM instead used tabular health-checkup and tabular claims features.The claims lists contain diagnoses, procedures, and prescriptions organized by month.Source: §3.2 · p. 4

3.2.1 Tabular Health Checkup data.

  • Eight health-checkup features were extracted and used as inputs to the ASCVD model.The source passage refers to Table C.5 for the features but does not list them.Source: §3.2.1 · p. 4

3.2.2 Multi-List Claims data.

  • Monthly diagnosis, procedure, and prescription codes were deduplicated and converted into meaningful sub-tokens for the multi-list claims input.Diagnosis codes were expanded into five hierarchical ICD-10 categories, with <PAD> used for not applicable; the monthly lists were combined across 12 months for the attention-based models.Source: §3.2.2 · p. 4

3.2.3 Tabular Claims data.

  • Tabular claims data represented nearly 7,000 diagnosis, procedure, and prescription codes as binary presence-or-absence columns.Each code’s value indicated whether it appeared in the 12 months of claims data.Source: §3.2.3 · p. 4

3.3 Training

  • Models were trained with AdamW and class-balanced focal loss, using 9-fold cross-validation; Optuna tuned LGBM hyperparameters.Class-balanced focal loss addressed imbalance between positive and negative task labels.Source: §3.3 · p. 5

4 EXPERIMENTS

The experiments compare MACE prediction models using health-checkup and claims data, with a defined cohort, cross-validation, and ROC-AUC and MCC evaluation.

4.1 Experimental Setting

  • Nine-fold cross-validation compared Our (CA) with ASCVD, LGBM, and Our (SA), with Wilcoxon and DeLong tests used to assess performance differences.ASCVD uses health-checkup data alone; the other models use both health-checkup and claims data. Data were split 8:1:1 into training, validation, and test sets.Source: §4.1 · p. 5

4.2 Datasets

  • The study obtained health-checkup data for 166,030 people, claims data covering 714,710 people, and updated insurance qualification data for 1,484,255 people.The sources came from the Health Insurance Association for Architecture and Civil Engineering Companies in Japan.Source: §4.2.1 · p. 5

4.2.1 Data Source.

  • The final cohort included 51,367 subjects: 41,094 for training, 5,136 for validation, and 5,137 for testing.The prediction task used claims from April 2015–March 2022; input features came from health-checkup and claims data from April 2014–March 2015.Source: §4.2.2 · p. 5

4.2.2 Creation of the Dataset.

  • MACE was defined by hospitalization or death, and Table 2 covers incidence of MACE and its AMI, cerebrovascular disease, HF, and PAD components.AMI is acute myocardial infarction; HF is heart failure; PAD is peripheral arterial disease.Source: §4.2.2 · p. 5, Table 2 · p. 5

4.3 Metrics

  • Model performance was evaluated with ROC-AUC and Matthews correlation coefficient (MCC), calculated at a 0.5 threshold for MACE occurrence.The metrics are identified with citation.Source: §4.3 · p. 5

5 RESULTS

Models using both health checkup and medical claims data outperformed the health-checkup-only ASCVD model, and the cross-attention model achieved the highest scores. Its advantage over the other best-performing model was statistically significant on both ROC-AUC and MCC.

5.1 Experiment Results

  • The cross-attention model scored ROC-AUC 0.7720 and MCC 0.1525, the highest scores; models using both health checkup and medical claims data outperformed the health-checkup-only ASCVD model.Wilcoxon rank-sum tests on nine ROC-AUC and nine MCC values confirmed significance for the two best-performing models (pvalues = 0.0039, 0.0039). MCC was evaluated at the 0.5 threshold.Source: Table 3 · p. 5, §5.1 · p. 5
  • The trained ASCVD model was applied directly to test data, while the (SA)3 model omitted the proposed architecture’s cross-attention block.SA denotes self-attention and CA denotes cross-attention.Source: §5.1 · p. 5

6 DISCUSSION & CONCLUSION

Cross-attention enhanced MACE prediction from clinically unstructured medical claims by learning and weighting many-to-many relationships between diagnoses and treatments.

6 DISCUSSION & CONCLUSION

  • Cross-attention enhanced MACE prediction by weighting and learning many-to-many relationships between diagnoses and treatments in clinically unstructured medical claims.The method effectively uses large-scale claims data.Source: §6 · p. 5
  • Identifying people at high risk of MACE with the proposed model has the potential to improve public health services in Japan.Details are described in Appendix I.Source: §6 · p. 5
  • Future work will use cross-attention to pre-train on large volumes of medical claims data for models predicting various diseases in downstream tasks.Source: §6 · p. 6

7 LIMITATIONS · APPENDIX

The study excluded potentially informative claims data and cautions against extrapolating its findings to populations with very high monthly medical-code counts.

7 LIMITATIONS

  • The experimental dataset excluded duplicated medical codes and time-series information that might improve prediction performance.The paper directs readers to Appendix J for further limitations.Source: §7 · p. 6
  • The findings should not be extrapolated to populations with a very high number of medical codes recorded in a single month, which were excluded from the dataset.Source: §7 · p. 6

A MODEL ARCHITECTURE SUPPLEMENT · B HYPERPARAMETER SUPPLEMENT

The architecture supplement distinguishes the proposed cross-attention model from Our SA and describes how diagnoses and treatments are embedded; the hyperparameter supplement identifies a table for the proposed model.

A MODEL ARCHITECTURE SUPPLEMENT

  • The proposed model adopts a cross-attention block, whereas Our SA does not.Source: Figure A.3 · p. 8
  • Figure A.3 depicts blocks in the models and directs readers to Figure A.4 for embedding details and Figure 2 for cross-attention details.Source: Figure A.3 · p. 8
  • Diagnostic sub-tokens are transformed into vectors and aggregated into a single diagnosis vector.Source: Figure A.4 · p. 8
  • Medical-procedure and prescription sub-tokens are embedded and aggregated into separate vectors, which are concatenated to represent treatment information.Source: Figure A.4 · p. 8
  • For a month of claims, diagnostic and treatment information are represented as tensors with batch-size, length, and embedding-size dimensions; length typically spans several dozen.Source: Figure A.4 · p. 8

B HYPERPARAMETER SUPPLEMENT

  • Table B.4 is identified as the proposed model’s hyperparameter table.Source: Table B.4 · p. 9

C HEALTH CHECKUP DATA SUPPLEMENT … G SELECTION PROCESS SUPPLEMENT

The supplements document health-checkup inputs, medical-code conversion, datasource and dataset characteristics, and the selection process for experimental subjects.

C HEALTH CHECKUP DATA SUPPLEMENT

D CONVERSION METHOD FOR MEDICAL CODE SUPPLEMENT

  • Table D.6 describes the method for converting medical codes to sub-tokens.Source: Table D.6 · p. 10

E SUMMARY OF DATASOURCE SUPPLEMENT

F SUMMARY OF DATASET SUPPLEMENT

  • 7,031 unique medical tokens and 12,068 unique medical sub-tokens were extracted from medical claims data.The dataset notes that age was calculated as of 31 March 2015; binary-variable means are percentages, and standard deviations exclude binary variables.Source: F SUMMARY OF DATASET SUPPLEMENT · p. 11
  • The dataset summary specifies that binary-variable means are percentages and standard deviations exclude binary variables.Age was calculated as of 31 March 2015.Source: F SUMMARY OF DATASET SUPPLEMENT · p. 11

G SELECTION PROCESS SUPPLEMENT

  • Figure G.5 presents the selection process for experimental subjects.Source: Figure G.5 · p. 12

H ADDITIONAL EXPERIMENTAL RESULTS SUPPLEMENT · I USE CASES SUPPLEMENT

The ensemble results favored cross-attention over self-attention, and the proposed claims-data model is described as applicable to medical and community health interventions in Japan.

H ADDITIONAL EXPERIMENTAL RESULTS SUPPLEMENT

I USE CASES SUPPLEMENT

  • The proposed model for identifying individuals at high risk for MACE can be applied in medical and community health fields in Japan.Source: I USE CASES SUPPLEMENT · p. 13
  • Adjusting prediction-probability thresholds supports interventions for many people with moderate or higher risk, small high-risk groups, or highest-risk-first allocation based on available resources.Source: I USE CASES SUPPLEMENT · p. 13
  • Japan's national screening program has operated since 2008, but models relying solely on health checkup results have inadequately captured many risk factors.Source: I USE CASES SUPPLEMENT · p. 13
  • Using medical claims data with cross-attention enables more accurate identification of individuals at high risk for MACE.Source: I USE CASES SUPPLEMENT · p. 13

J LIMITATION SUPPLEMENT

The study’s approach omits repeated diagnosis and treatment information, excludes some patients with many medical codes, and does not use monthly time-series information; the authors identify all three as areas for future work.

J LIMITATION SUPPLEMENT

  • Because of computational limits, training removed duplicate diagnoses and treatments, although these repetitions may be informative; future work will seek to incorporate them efficiently.Duplicate removal followed an approach used in a previous study.Source: J LIMITATION SUPPLEMENT · p. 13
  • The experiments excluded individuals with numerous medical codes even after deduplication, and future work will address models applicable to these special cases.The study began with general cases because building a highly general model for such cases is challenging.Source: J LIMITATION SUPPLEMENT · p. 13
  • The proposed model omitted monthly information because positional encoding of monthly data did not improve performance, although effective use of time-series information may enhance it.Source: J LIMITATION SUPPLEMENT · p. 13

Related papers

Loading 2609.09824v1…