Multimodal Machine Learning for Automated ICD Coding

1. Introduction

• Assigning ICD codes is time-consuming and error-prone because coders must extract complex diagnostic details from EMRs.Coding errors can also lead to billing mistakes, claim denials, and underpayment.Source: §1 · p. 1
• EMRs contain unstructured, semi-structured, and tabular data, making it non-trivial to leverage all information and provide evidence for predictions.The paper notes that black-box predictions are not convincing to physicians and insurance companies.Source: §1 · pp. 1–2
• The study integrates three modality-specific models in an ensemble to exploit complementary information and improve ICD-code prediction accuracy.The authors also report addressing data imbalance in ICD-code prediction.Source: §1 · p. 2
• The authors propose interpretation approaches for unstructured text and structured tables to make predictions more explainable and convincing to physicians.Source: §1 · p. 2

2. Related Work

• Earlier ICD-coding studies used single-label classifiers, multi-label ranking with engineered features, and SVM-plus-rule systems for selected diseases.Source: §2 · p. 2
• Unsupervised and semi-supervised approaches have also used medical concept extraction from unlabeled corpora to support ICD-code prediction.Source: §2 · p. 2
• Deep-learning approaches include recurrent and CNN models, attention mechanisms, LSTM encoders, and adversarial learning for code prediction from clinical text.Source: §2 · p. 2

3. Methods

• The ensemble combines Text-CNN classification for unstructured text, diagnosis-description similarity for semi-structured text, and a decision tree for binary-transformed tabular features.Retrieved evidence from raw data is presented to support explainability.Source: §3 · p. 3

3.1. Data Structure Description

• The study selected 44,659 MIMIC-III admissions, about 77% of the dataset, with at least one discharge summary and labels among 32 selected ICD codes.ICD-9 codes with multiple corresponding ICD-10 codes were excluded to avoid ambiguity.Source: §3.1 · p. 3
• Six MIMIC-III tables provide admission information, laboratory measurements, prescriptions, microbiology results, charted health data, and clinical notes.Source: §3.1 · p. 4

3.2. Classification Based on Unstructured Text

• Preprocessing tokenizes and normalizes note text, removes tokens appearing fewer than ten times, and maps test-time out-of-vocabulary words to UNK.Source: §3.2 · p. 4
• Text-CNN embeds note tokens and applies convolution to capture correlations among adjacent tokens for multi-label ICD-code prediction.The model minimizes average cross-entropy for each code class.Source: §3.2 · p. 4
• Guideline-derived keywords and phrases become TF-IDF vectors added to the Text-CNN fully connected layer in Text-TF-IDF-CNN.Figure 2 depicts the modified Text-CNN architecture.Source: §3.2 · p. 4, Figure 2 · p. 5
• Label Smoothing Regularization addresses imbalance across 32 code classes by replacing hard labels with a mixture of the truth label and a uniform prior.The smoothing weight is sampled from a beta distribution during training.Source: §3.2 · p. 5

3.3. Ranking Based on Semi-structured Text

• Diagnosis-based Ranking maps diagnosis descriptions and code descriptions into a shared vector space, then ranks ICD codes by distance from the diagnosis vector.Source: §3.3 · pp. 5–6
• Triplet loss trains diagnosis examples against positive and negative code descriptions by bringing positive pairs closer and separating negative pairs.Source: §3.3 · p. 6
• A neural network combines character- and word-level features, then uses a bidirectional LSTM and max pooling to encode diagnosis and code-description representations.Character features capture form similarities, while pre-trained word embeddings represent semantic relationships.Source: §3.3 · p. 6
• Because MIMIC-III lacks one-to-one code-description mappings, synonyms supply positive examples and edit-distance-selected similar n-grams supply negatives.At inference, distances are min-max normalized, and admissions with multiple diagnoses combine normalized results by their mean.Source: §3.3 · p. 6

3.4. Classification based on Tabular Data

• Tabular inputs cover LABEVENTS, PRESCRIPTIONS, MICROBIOLOGYEVENTS, and CHARTEVENTS.Source: §3.4 · p. 6
• Laboratory features mark tests abnormal or normal, measurement features encode whether BMI, heart rate, and blood pressure are in range, and microbiology features indicate positive or negative organism tests.Repeated lab results use majority vote, missing lab values count as normal, and measurement features are combined with LABEVENTS.Source: §3.4 · pp. 6–7
• Medication features indicate whether a drug was prescribed, treat missing medications as not prescribed, and exclude medications with frequency below 50.The resulting medication feature set contains 1,135 medications.Source: §3.4 · p. 7
• A decision tree classifies the binary features using one-versus-all multilabel prediction, with higher weights assigned to minority-class samples.Source: §3.4 · p. 7

3.5. Model Ensemble

• For each ICD class, the ensemble computes the final predicted probability as a weighted sum of probabilities from individual models.The model weights are tuned on a validation set.Source: §3.5 · p. 7

3.6. Methods for Interpretation

• The interpretation approach extracts influential textual and tabular features to explain final ICD-code predictions.Source: §3.6 · p. 7

3.6.1. Text Interpretability

• Text interpretation identifies influential phrases by summing path influence scores between words and an ICD code, then ranking consecutive nonzero-score words.Each path score multiplies hidden-unit values and edge weights along that path.Source: §3.6.1 · p. 7

3.6.2. Table Interpretability

• For tabular interpretation, LIME estimates instance-specific feature importance by locally approximating the model’s decision boundary with a simple model.Source: §3.6.2 · p. 8

4.1. Experimental Settings

• The 31,155/4,484/9,020-admission train/validation/test split kept each patient's admissions together; hyperparameters were tuned on validation data.Source: §4.1 · p. 8, §4.2 · p. 8

4.2. Hyperparameters

• Text-CNN used kernel sizes 2, 3, and 4 with 128 feature maps, 256-dimensional embeddings, dropout 0.1, and Adam learning rate 10^-3; diagnosis ranking used a one-layer bidirectional LSTM.Source: §4.2 · p. 8

4.3. Evaluation Metrics

• Classification used F1 and AUC, with micro-averages weighting classes by sample count and macro-averages treating classes equally; interpretability used JSC for evidence-annotation overlap.Source: §4.3 · p. 8

4.4. Evaluation of Code Classification

• The combined Text-TF-IDF-CNN + LS + DR + TD model led all models with 0.6867 Macro-F1, 0.7633 Micro-F1, 0.9337 Macro-AUC, and 0.9541 Micro-AUC.Table 1 covers model and baseline classification evaluations.Source: Table 1 · p. 9, §4.4 · pp. 9–10
• Adding tabular data to Text-CNN+LS improved all four metrics; Text-CNN+LS+TD led CNN models on macro-F1 (0.6137) and micro-F1 (0.6929), while adding DR raised AUC but slightly lowered F1.Source: §4.4 · p. 9

4.5.1. Text-oriented evaluation

• The model's text evidence averaged JSC 0.1806, below the 0.2784 average overlap between physicians; physicians also differed in whether they highlighted keywords or sentences.Source: §4.5.1 · p. 10

4.5.2. Table-oriented evaluation

• On tabular data, model-physician evidence JSC averaged 0.3105, compared with 0.5002 agreement among physicians; the model captured extra features, some informative for diagnoses.Source: §4.5.2 · p. 11

5.1. Code Classification

• Clinical-guideline keywords are added to the Text-CNN fully connected layer as TF-IDF features, building on the strong F1 performance of TF-IDF.The authors describe this as a way to leverage human domain knowledge in clinical machine learning.Source: §5.1 · p. 11
• Macro-averaged metrics improve more than micro-averaged metrics; tabular data particularly benefits small-sample classes, while larger code lists will make imbalance more important.The discussion attributes limited text-based inference partly to small training samples and notes that inpatient exams and lab tests are usually mandatory.Source: §5.1 · p. 11

5.2. Interpretability

• Text-TF-IDF-CNN surfaces disease-related phrases, while tabular evidence adds glucose and insulin for E11.9, absent from top-ranked text keywords.For N17.9, the text evidence also links reported drug use with acute kidney failure.Source: §5.2 · p. 12
• Interpretation quality varies with classification performance: evidence is more relevant for classes with AUC above 0.9 than for D64.9, around 0.7.The discussion points to Figure 3 for this comparison.Source: §5.2 · p. 12

5.3. Future Work

• The authors plan to expand beyond 32 representative hospital codes to cover more minority classes.Source: §5.3 · p. 12
• Tabular feature reduction and duplicate removal may improve both classification performance and interpretability.The current tabular feature space is described as very large, with some duplicated features.Source: §5.3 · p. 12
• The authors identify methods beyond keyword matching as a future direction for using existing human knowledge.They state that incorporating domain expertise improves prediction.Source: §5.3 · p. 12

• The ensemble of separate models for unstructured, semi-structured, and tabular EMR data outperformed all baseline methods in classification.Source: §6 · p. 12
• Incorporating human knowledge into machine learning models improved prediction performance.Source: §6 · p. 12
• The interpretability method extracts code-specific text phrases, important lab records, and prescriptions, which the authors say has potential for real clinical practice.Source: §6 · p. 12

• Table 3 presents 32 ICD-10 codes alongside their descriptions.Source: Table 3 · p. 17
• Table 4 presents prediction-interpretability examples from different modalities in descending order, with rankings shown in brackets.Source: Table 4 · p. 18
• Table 4 includes only the top three unstructured-text results, except for examples discussed in Section 5.2.Source: Table 4 · p. 18