Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding

1 Introduction

• Traditional ICD coding requires specialized coders and is described as expensive, time-consuming, and error-prone.The task assigns ICD codes to electronic medical records.Source: §1 · p. 1
• Recent methods frame coding as multi-label classification, encoding EMRs with RNNs or CNNs and predicting codes with a classifier.Label-attention methods use code representations as queries to extract code-related text snippets (Mullenbach et al., 2018).Source: §1 · p. 1
• Code expressions in EMRs can differ from ICD descriptions, motivating the use of UMLS synonyms alongside code hierarchies and descriptions.For code 244.9, the ICD description “Unspecified hypothyroidism” may appear in EMRs as “low t4” or “subthyroidism”; prior methods use code hierarchies and descriptions.Source: §1 · p. 1
• MSMN encodes EMR text and each synonym with a shared LSTM, uses synonym queries for multi-synonyms attention, then classifies with biaffine similarity.The attention mechanism is inspired by multi-head attention.Source: §1 · p. 1
• Experiments on MIMIC-III show that MSMN outperforms previous state-of-the-art methods in both full-code and top-50 settings.Source: §1 · p. 1

2 Approach

• Given EMR text, the task is to assign each code a binary label.Source: §2 · p. 2

2.1 Code Synonyms

• Code synonyms are collected by aligning codes to UMLS CUIs and selecting matching English terms, with additional hyphen- and “NOS”-removed variants.Each synonym is represented as a word sequence.Source: §2.1 · p. 2

2.2 Encoding

• A bidirectional LSTM encodes the text, while the same encoder and max-pooling produce a representation for each code synonym.Source: §2.2 · p. 2

2.3 Multi-synonyms Attention

• Multi-synonyms attention splits text hidden representations into heads and uses each synonym representation to query a head for code-related text.The resulting text snippets are aggregated into code-wise text representations; the architecture figure distinguishes code synonyms by color and shows the hidden-representation heads.Source: §2.3 · p. 2, Figure 1 · p. 2

2.4 Classification

• For classification, synonym representations are max-pooled into a code representation and compared with code-wise text using a biaffine transformation.Unlike code-dependent classification parameters that can suffer with rare codes, the proposed biaffine function uses code-independent parameters.Source: §2.4 · p. 2

2.5 Training

• The model is optimized with binary cross-entropy between predicted probabilities and labels.Source: §2.5 · p. 2

3.1 Dataset

• MIMIC-III evaluation uses full codes and the top 50 frequent codes, truncates discharge summaries at 4,000 words, and reports AUC, F1, and precision@k.Precision@k uses k = 5 for MIMIC-III 50 and k = 8 and 15 for MIMIC-III full.Source: §3.1 · p. 3

3.2 Implementation Details

• Training samples M = 4 synonyms per code for MIMIC-III full and M = 8 for MIMIC-III 50, with random sampling and repetition when a code lacks enough synonyms.The implementation uses pretrained word embeddings, R-Drop, dropout, an LSTM text encoder, and AdamW; the full setting uses eight GPUs while the top-50 setting uses one.Source: §3.2 · p. 3

3.3 Baselines

• The baselines include CNN, multi-scale attention with GCN, residual CNN, hyperbolic-code, and hierarchical joint-learning methods; LAAT and JointLAAT are the closest baseline.Source: §3.3 · p. 3

3.4 Main Results

• MSMN achieves state-of-the-art results on most full-code metrics and beats LAAT across all top-50-code metrics.The reported full-code values include macro-AUC 95.0 (+2.0), micro-AUC 99.2 (+0.0), macro-F1 10.3 (-0.4), micro-F1 58.4 (+0.9), P@8 75.2 (+1.4), and P@15 59.9 (+0.8).Source: §3.4 · pp. 3–4, Table 3 · p. 4, Table 4 · p. 4

3.5 Discussion

• Using more UMLS synonyms than descriptions alone consistently improves performance; M = 4 or 8 performs best across the reported AUC, F1, and P@5 comparisons.The median and mean UMLS synonym counts are 5.0 and 5.4, and the biaffine-based similarity performs best among the compared scoring functions.Source: §3.5 · p. 4, Table 5 · p. 4
• Synonym representations cluster for some codes, diverge for codes with different synonym expressions, and remain similar for a pair of sibling codes.The authors interpret these patterns as matching similar or different text snippets and implicitly capturing code hierarchy.Source: §3.5 · p. 4, Figure 2 · p. 5

3.6 Memory Complexity

• The memory complexities of the two dominant computations are linearly proportional to the number of synonyms M.The computations form intermediate tensors with shapes B × C × N × M and B × C × h × M.Source: §3.6 · pp. 4–5

• Recent ICD coding methods encode text with RNNs or CNNs and use label attention to extract relevant text for code classification.Some methods initialize label representations randomly, while others use word embeddings or code hierarchy.Source: §4 · p. 5
• Prior code-representation approaches use averaged word embeddings, hierarchical structures, or CNNs, whereas this paper represents codes with multiple synonyms.The authors argue synonyms provide more comprehensive code expressions than a single description.Source: §4 · p. 5
• Biomedical entity linking maps labeled text entities to larger concept dictionaries, unlike ICD coding, which maps unannotated text to ICD codes.BioSYN marginalizes probabilities across synonyms, whereas this work uses multi-synonym attention to extract different clinical-text parts.Source: §4 · p. 5

5 Conclusions

• MSMN achieves state-of-the-art results on MIMIC-III and outperforms previous methods with label attention.Source: §5 · p. 5
• MSMN leverages code synonyms from UMLS and uses multi-synonyms attention to extract code-wise text representations.Source: §5 · p. 5
• A biaffine transformation calculates similarities between texts and codes for classification.Source: §5 · p. 5
• Ablation studies show the effectiveness of multi-synonyms attention and biaffine-based similarity.Source: §5 · p. 5