Source-linked AI summary
Towards Automated ICD Coding Using Deep Learning
Haoran Shi, Pengtao Xie, Zhiting Hu, Ming Zhang, Eric P. Xing
TL;DR
Manual ICD coding from physician-written diagnosis descriptions is time-consuming, error-prone, and expensive. The paper proposes a hierarchical deep learning model with character-aware encoders and soft attention, finding that both components improve coding performance and that probability thresholds can support doctor-assisted code selection.
Problem
Manual ICD coding must translate informal, ambiguous diagnosis descriptions into appropriate codes, making the process time-consuming, error-prone, and expensive.
Method
A hierarchical model uses separate character- and word-level LSTMs for diagnosis descriptions and ICD-code titles, with soft attention to handle multiple descriptions per code.
Results
Removing character-level encoding reduced F1 by 0.024 and AUC ROC by 0.018, while replacing attention reduced F1 by 0.061 and AUC ROC by 0.018.
Takeaways & Limitations
Probability thresholds allow the model to trade recall for precision and help doctors pre-select a small set of candidate codes, potentially reducing workload.
Takeaways & Limitations
The model treats ICD-code assignments as mutually independent, although codes can correlate or be mutually exclusive, and it does not model the ICD hierarchy.
Abstract
from arXiv · showhide
International Classification of Diseases(ICD) is an authoritative health care classification system of different diseases and conditions for clinical and management purposes. Considering the complicated and dedicated process to assign correct codes to each patient admission based on overall diagnosis, we propose a hierarchical deep learning model with attention mechanism which can automatically assign ICD diagnostic codes given written diagnosis. We utilize character-aware neural language models to generate hidden representations of written diagnosis descriptions and ICD codes, and design an attention mechanism to address the mismatch between the numbers of descriptions and corresponding codes. Our experimental results show the strong potential of automated ICD coding from diagnosis descriptions. Our best model achieves 0.53 and 0.90 of F1 score and area under curve of receiver operating characteristic respectively. The result outperforms those achieved using character-unaware encoding method or without attention mechanism. It indicates that our proposed deep learning model can code automatically in a reasonable way and provide a framework for computer-auxiliary ICD coding.
Introduction
ICD coding supports clinical and financial decisions but remains time-consuming, error-prone, and expensive. The paper targets automatic coding despite differences between physician-written diagnoses and formal ICD descriptions and mismatches between diagnoses and codes.
- ICD provides a hierarchical system of diagnostic codes used for reporting conditions, reimbursement decisions, and morbidity and mortality statistics.
- Manual coding requires reviewing physician-written diagnosis descriptions and other records before assigning appropriate ICD codes under coding guidelines.
- Automatic coding must address stylistic differences between informal, noisy diagnoses and formally worded ICD definitions, as well as non-one-to-one diagnosis-to-code mappings.
Contributions
The paper presents a hierarchical neural approach for automatic ICD coding from diagnosis descriptions, using representations of both diagnoses and ICD definitions with attention to handle unequal numbers of descriptions and codes.
- The model automatically assigns ICD codes from diagnosis descriptions using a hierarchical neural network that captures latent semantics in both text types.
- Attention addresses the mismatch between the number of diagnosis descriptions and the number of assigned ICD codes.
- The study trains on 8,066 admissions, tunes on 1,728 admissions, and evaluates on 1,729 held-out admissions.
Related work
Prior ICD coding systems used automatic or semi-automatic methods, but earlier studies often relied on smaller, domain-specific datasets or narrower coding targets. The paper positions its experiment as broader in clinical coverage and dataset scale.
- Earlier automatic and semi-automatic ICD classification studies commonly used small, domain-specific clinical datasets.
- A prior shared task assigned ICD-9 codes to 1,954 radiology records, while another system identified cancer types from death certificates.
- Compared with these experiments, the paper uses a larger dataset spanning various domains of clinical practice.
- Discharge-summary studies included candidate-label generation focused on principal diagnoses and methods assigning one diagnostic code per description.
Methods
The study preprocesses MIMIC-III discharge summaries into diagnosis descriptions, restricts prediction to 50 frequent ICD codes, and evaluates models on a split of 11,523 admissions. The dataset includes multiple descriptions and assigned codes per admission.
- Dataset and preprocessing: MIMIC-III contains de-identified electronic medical records from 58,976 patient visits, with diagnosis descriptions extracted mainly from discharge- and final-diagnosis sections.
- Dataset and preprocessing: The study restricts prediction to the 50 most frequent ICD codes because many of the dataset’s 6,984 codes have sparse supervision.
- Dataset and preprocessing: Preprocessing removes irrelevant discharge-summary sections and converts noisy, enumerated physician text into extracted diagnosis descriptions.
- Dataset and preprocessing: The resulting dataset contains 11,523 admissions and 59,302 diagnosis descriptions, split into 8,066 training, 1,728 validation, and 1,729 test admissions.
Model design
The model encodes diagnosis descriptions and ICD definitions with hierarchical LSTMs, matches them through code-specific attention, and predicts assignment probabilities. It compares hard selection with soft attention while learning code-wise confidence scores.
- The ICD coding model has four modules: encoding diagnosis descriptions, encoding ICD codes, matching descriptions with codes, and assigning codes.
- Hierarchical character-level and word-level LSTMs generate hidden representations for diagnosis descriptions and ICD definitions.The diagnosis-description encoder uses character embeddings within words and word representations within sentences; the ICD-code encoder applies the same two-level structure to long-title definitions.
- Attention addresses the mismatch between the numbers of diagnosis descriptions and assigned ICD codes by weighting descriptions for each code.For each ICD code, the attention score for a diagnosis description is based on cosine similarity between their hidden representations.
- Hard-selection chooses the highest-scoring diagnosis description, whereas soft attention combines all descriptions using normalized attention weights.The soft-attention mechanism forms a weighted average of diagnosis-description representations, allowing varying levels of attention across descriptions.
- A code-specific perceptron projects the attentional hidden vector to a confidence score, which sigmoid normalization converts into an assignment probability.The model uses binary cross entropy for each ICD code and learns parameters with stochastic gradient descent; training uses ADAM with an initial learning rate of 0.001 and mini-batches of 10.
Analysis and evaluation
The evaluation uses micro F1-score and ROC AUC because ICD code assignments are sparse and most codes are labeled false.
- Micro F1-score and AUC ROC are used to evaluate the model because ICD code assignment is generally sparse.Micro F1 is the harmonic mean of precision and recall, while AUC ROC is the area under the receiver operating characteristic curve.
Results
The model’s Soft-attention mechanism improves ICD coding performance over Hard-selection, while attention weights vary by target code and emphasize relevant diagnosis descriptions.
- Soft-attention increases F1 by 5.2 percent and AUC ROC by 2.3 percent compared to the Hard-selection model.
- For different ICD codes, the model automatically allocates different attention scores to diagnosis descriptions.
- For ‘Neonatal jaundice associated with preterm delivery’, attention emphasizes prematurity and hyperbilirubinemia descriptions over ‘Sepsis ruled out’.
Discussion
The discussion attributes performance gains to the model’s architecture, character-aware representations, word-level sequence encoding, and attention mechanism, while showing how attention focuses on relevant descriptions.
- The architecture combines diagnosis-description extraction, hidden representations, and ICD-title representations to support automated coding.
- Removing the character-level LSTM drops F1 by 0.024 and AUC ROC by 0.018, while pre-trained word embeddings remain below the character-level model.
- Character-level encoding generates similar representations for typos, morphological variants, and related disease or procedure terms.
- Replacing the word-level LSTM with word averaging drops F1 by 0.028 and AUC ROC by 0.014.
- Replacing attention with a linear classifier drops F1 by 0.061 and AUC ROC by 0.018.
Limitations
The study has three main limitations: noisy diagnosis extraction, restriction to 50 frequent ICD codes, and independent treatment of code assignments despite correlations among codes.
- Data quality: Noisy electrical discharge summaries and simple diagnosis extraction may leave room for improved performance through cleaner data and more elaborate preprocessing.The authors state that performance could improve with higher-quality diagnosis descriptions.
- Code coverage: The candidate code set is restricted to the 50 most frequent ICD codes.Rare codes provide insufficient evidence for valid neural models and worsen label imbalance.
- Code dependence: Separate Perceptrons assume ICD-code assignments are mutually independent, although codes can correlate or be mutually exclusive.The authors suggest structured methods could model these relationships and improve performance.
Conclusions
The proposed model uses attention to match diagnosis descriptions with ICD definitions and can assign codes automatically, including on noisy data. The authors also identify adaptation to newer ICD revisions as straightforward when definitions and labeled training data are available.
- Conclusions: Soft attention allocates greater weight to informative diagnosis descriptions and less to irrelevant ones when assigning ICD codes.This enables automatic and efficient coding from multiple descriptions.
- Conclusions: Table 4 compares hidden-vector neighbors from character-level LSTM and pretrained word-vector encoders, while its lower section shows diagnosis neighbors in the Soft-attention model.Italic text marks typos in diagnosis descriptions.
- Conclusions: The model shows potential for real-life applications because it performs highly even on noisy-formatted data.Its probability scores can also support doctor-assisted candidate-code selection.
- Conclusions: The approach can be adapted from ICD-9 to newer revisions such as ICD-10 when code definitions and gold-standard training labels are available.The paper describes this adaptation as straightforward under those conditions.
Author contributions statement
The authors report distinct responsibilities for study design, data processing, experiments, writing, and senior authorship, with all authors reviewing the manuscript.
- Author contributions: H.S. and P.X. conceived and designed the study.
- Author contributions: H.S. processed the data and performed the experiments.
- Author contributions: H.S., P.X., and Z.H. wrote the paper, while M.Z. and E.P.X. served as co-senior authors.
- Author contributions: All authors reviewed the manuscript.