Source-linked AI summary
In Medical Claims Data, Enhancing Predictive Performance for Major Adverse Cardiovascular Events Using Cross Attention
Yuhei Fujioka, Daitaro Misawa, Tatsuyoshi Ikenoue, Shingo Fukuma
TL;DR
Medical claims data contain valuable clinical information but often lack a clear structure linking diagnoses and treatments, limiting their use for MACE prediction. The paper integrates health checkup and claims data with cross-attention to learn these relationships, achieving the highest reported benchmark performance with ROC-AUC 0.7720. The findings support using this approach for MACE prediction, while the study excludes some data and populations from its evaluation.
Problem
Medical claims data contain clinically valuable diagnoses and treatments, but their relationships are often medically unstructured and difficult to use comprehensively for MACE prediction.
Method
The proposed deep learning model integrates health checkup and medical claims data using cross-attention to learn many-to-many relationships between diagnoses and treatments.
Results
ROC-AUC 0.7720 was highest among the evaluated ASCVD, LGBM, self-attention, and cross-attention models.
Takeaways & Limitations
Applying cross-attention to clinically unstructured medical claims data enhanced MACE prediction performance and may support identifying individuals at high risk in Japanese public health services.
Takeaways & Limitations
The study excluded duplicated medical codes and claims time-series information, and its findings should not be extrapolated to populations with very many monthly medical codes.
Abstract
from arXiv · showhide
Medical claims data comprise the financial details, including the expenses and billing information, as well as the clinical information, such as the diagnoses and treatments, of patients visiting medical facilities. Recently, it has been acknowledged that large databases can be constructed from medical claims data for medical research purposes. However, the clinical information within these datasets is often medically unstructured, limiting its application in comprehensive analyses. This study enhances predictive model performance for major adverse cardiovascular events (MACE), a leading cause of death worldwide. Models that predict MACE are crucial to clinical practice guidelines. We utilize a cross-attention mechanism to develop a method that effectively weights the relationships between diagnoses and treatments. Effectively repre- senting the clinical information contained in medical claims data, this approach generates more representative features for predicting MACE. The ROC-AUC score of our proposed cross-attention-based model was 0.7720, higher than other benchmark models including the conventional atherosclerotic cardiovascular disease model, the light gradient boosting machine, and a self-attention-based model. These results indicate that integrating the clinical structure of medical claims data using a cross-attention mechanism significantly enhances the performance of predictive models.
1 INTRODUCTION
MACE prediction is important but medical claims data lack a clear structure linking diagnoses with treatments. This study proposes cross-attention to learn many-to-many clinical relationships while integrating claims and health checkup data.
- 1.1 Background: Medical claims data comprehensively record diagnoses, procedures, and prescriptions, but their clinical relationships are difficult to structure for prediction.Treatments are defined here as medical procedures and prescriptions.
- 1.2 Task Definition: MACE models support risk calculation and interventions, yet existing models do not fully use clinical information in medical claims data.MACE refers to major adverse cardiovascular events, including diseases listed using ICD-10 codes.
- 1.3 Challenges: Cross-attention is used to complementarily weight diagnosis–treatment relationships, including many-to-many relationships that are difficult to match manually.The approach addresses medical claims data whose reimbursement-oriented design makes diagnosis–treatment links unclear.
- 1.4 Contributions: The proposed model integrates health checkup and medical claims data with cross-attention to predict MACE.Its contribution is learning diagnosis–treatment relationships rather than relying only on self-attention or many-to-one links.
2 RELATED WORK
Prior work shows that claims and EHR information can improve disease prediction, but self-attention approaches face sequence-length and diagnosis–treatment-linking challenges. Cross-attention provides a way to model relationships between distinct clinical inputs.
- 2 RELATED WORK: Health checkup data combined with clinical information from claims and EHR have produced high prediction performance for disease onset and progression.Prior studies reported significant improvements in ROC-AUC while using outcome-related clinical codes.
- 2 RELATED WORK: Self-attention models benefit from combining diagnoses and treatments, but longer patient sequences increase computational requirements and limit applicable training populations.This creates a practical constraint for models using detailed clinical sequences.
- 2 RELATED WORK: ExBEHRT links each treatment to a diagnosis to improve prediction and reduce hardware requirements, but this overlooks many-to-many diagnosis–treatment relationships.Manual linking is also difficult because correlations in claims and EHR data are not always clear.
- 2 RELATED WORK: Cross-attention processes two distinct inputs and weights relationships by calculating relevance between elements from each input.This mechanism differs from self-attention by relating elements across separate inputs.
3 METHOD
The proposed architecture combines health checkup data with 12 monthly lists of diagnoses, procedures, and prescriptions, using separate encoders and cross-attention to represent their relationships.
- Model Architecture: The model uses health checkup tabular data and 12 monthly lists containing diagnoses, medical procedures, and prescriptions.
- Model Architecture: An embedding block integrates medical procedures and prescriptions into treatment data before diagnoses and treatments are separately transformed for each month.
- Cross-Attention Block: The cross-attention block creates monthly clinical vectors by integrating diagnoses and treatments according to their interrelations.
- Model Architecture: The monthly tensors are concatenated into a length-12 representation, then combined with transformed tabular data and passed to a classification head producing probabilities.
- Benchmark Models: Benchmarks include the ASCVD model, LGBM, and a self-attention-based model, while prior temporal models were excluded because the claims data lacked detailed temporal information.
4 EXPERIMENTS
The experiments compare the proposed cross-attention model with three benchmarks using health checkup and claims data from a Japanese insurance population, evaluating MACE prediction with ROC-AUC and MCC.
- Experimental Setting: The evaluation compares Our (CA) with ASCVD, LGBM, and Our (SA) using stratified data splitting and nine-fold cross-validation.
- Dataset: The data sources include health checkups, medical claims from May 2014–April 2022, and updated insurance qualification records.
- Dataset: The final experimental cohort contains 51,367 subjects selected for sufficient MACE observation and prediction using all evaluated models.
- Dataset: Table 2 reports the incidence of MACE and its components in the cohort of 51,367 subjects.
- Evaluation Metrics: Performance was evaluated with ROC-AUC and Matthews correlation coefficient, with MCC calculated at the 0.5 threshold.
5 RESULTS
Models using both health checkup and claims data outperformed the health-checkup-only ASCVD model, and the cross-attention model achieved the highest reported scores.
- Experiment Results: ROC-AUC 0.7720 and MCC 0.1525 were the highest scores achieved by the cross-attention model.
- Experiment Results: Models using both health checkup and medical claims data outperformed the ASCVD model, which used only health checkup data.
- Compared Models: Our (SA) excludes the cross-attention block used in the proposed architecture.
- Compared Models: CA denotes the cross-attention mechanism, while SA denotes the self-attention mechanism.
6 DISCUSSION & CONCLUSION
The proposed cross-attention approach improves MACE prediction from clinically unstructured medical claims data by weighting relationships between diagnoses and treatments. The authors propose applying this mechanism in future pre-training for broader disease-prediction tasks.
- Discussion & Conclusion: Cross-attention enhanced MACE prediction performance on clinically unstructured medical claims data.The method learns many-to-many relationships between diagnoses and treatments.
- Discussion & Conclusion: The proposed approach has potential to improve public health services in Japan by identifying individuals at high risk of MACE.
- Discussion & Conclusion: Future work will apply cross-attention to pre-training on large medical-claims datasets for prediction of various diseases.
7 LIMITATIONS
The study excludes duplicated medical codes and time-series claims information, and its findings should not be extrapolated to populations with very many monthly medical codes.
- Limitations: The study excluded duplicated medical codes and time-series information from medical claims data.These excluded data might improve prediction performance.
- Limitations: The findings should not be extrapolated to populations with a very high number of medical codes recorded in one month.Such populations were excluded from the experimental dataset.
- Limitations: Further study limitations are described in Appendix J.
A MODEL ARCHITECTURE SUPPLEMENT
The model architecture supplement distinguishes the proposed cross-attention model from the self-attention baseline and describes embedding-based representations of diagnoses and treatments.
- Model Architecture Supplement: The proposed model adopts a cross-attention block, whereas the self-attention model Our SA does not.
- Model Architecture Supplement: The supplement includes a figure describing model blocks and a table listing hyperparameters for the proposed model.
- Model Architecture Supplement: The embedding block converts diagnosis, procedure, and prescription sub-tokens into aggregated vectors, concatenating procedure and prescription vectors as treatment information.
C HEALTH CHECKUP DATA SUPPLEMENT
The supplied supplement materials identify health-checkup input definitions, medical-code tokenization, datasource and dataset summaries, and subject-selection materials.
- Health Checkup Data Supplement: The supplement includes a table defining input data from health checkup data.
- Health Checkup Data Supplement: The supplement includes a table describing conversion from medical codes to sub-tokens.
- Health Checkup Data Supplement: Additional supplement materials summarize the datasource, dataset, and experimental-subject selection process.
- Health Checkup Data Supplement: The reviewed medical claims data span May 2014 through April 2022, and the insured population numbered 398,239 on 31 March 2022.
- Health Checkup Data Supplement: The dataset contains 7,031 unique medical tokens and 12,068 unique medical sub-tokens extracted from medical claims data.
H ADDITIONAL EXPERIMENTAL RESULTS SUPPLEMENT
The ensemble experiments compare cross-attention and self-attention models for MACE prediction, with cross-attention achieving stronger performance and supporting application to Japanese medical and community health settings.
- Ensemble results: 0.7791 ROC-AUC and 0.1571 MCC were achieved by Our (CA), compared with 0.7653 ROC-AUC and 0.1368 MCC for Our (SA).Our (CA) outperformed Our (SA) in prediction performance.
- Ensemble results: The ROC-AUC difference between Our (CA) and Our (SA) was statistically significant by the DeLong test, with p-value 0.0159.
- Evaluation definitions: The table reports ensemble results for MACE prediction and defines CA as cross-attention, SA as self-attention, HC as health checkup data, and MC as medical claims data.MCC is evaluated at the 0.5 threshold.
- Potential application: The proposed model is intended for medical and community health use in Japan, where prediction thresholds can be adjusted for different intervention strategies and available resources.The stated applications include targeting many moderately high-risk people, a smaller high-risk group, or people in descending risk order.
J LIMITATION SUPPLEMENT
The study’s limitations concern computationally simplified histories, exclusion of patients with unusually numerous codes, and omission of monthly information despite the potential value of repeated and temporal data.
- Data representation: Duplicate diagnoses and treatments were removed from monthly histories because of computational resource limitations, although repeated information may be informative.Future work will seek to incorporate duplicates without compromising computational efficiency.
- Study population: Patients with numerous medical codes after duplicate removal were excluded, so the model was initially evaluated on general rather than special cases.The authors identify models for special cases as an important future research task.
- Temporal information: Monthly information was omitted because positional encoding experiments did not improve performance, although the authors consider effective temporal modeling potentially beneficial.