Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes

• Prior CT models generally predicted one abnormality at a time, and multilabel classification of whole 3D chest CTs across diverse abnormalities had not been reported.Source: §1 · p. 2
• Large-scale multilabel CT classification requires sufficient data, structured labels for each volume, and a model capable of handling the task.Source: §1 · p. 2
• Manual labeling by radiologists across the thousands of cases needed for training is described as virtually impossible, motivating automatic extraction from reports.Source: §1 · p. 2
• Whole-report classifiers can be uninterpretable and require costly manual report labels that scale with both report count and abnormality count.Source: §1 · p. 2
• The proposed rule-based approach uses the rigid language of radiology reports to extract 83 abnormality labels for chest CT.Source: §1 · p. 3
• The model learns from whole CT volumes without pixel-, patch-, or slice-level labels, using transfer learning and craniocaudal feature aggregation for multiple abnormalities.Source: §1 · p. 3

2.1 Chest CT Data Set Preparation

• A retrospective Duke dataset comprised 36,316 chest CT volumes obtained without intravenous contrast, paired with reports, and collected from January 2012 through April 2017 under IRB approval and HIPAA compliance.Each volume was resampled to 0.8 × 0.8 × 0.8 mm to provide consistent physical distance per pixel across patients.Source: §2.1 · pp. 3, 5

2.2 Automated image labeling through analysis of radiology reports

• SARLE extracts 83 labels through either hybrid sentence classification plus medical-term search or fully rule-based normality and negation detection.Its report-level evaluation used 427 held-out reports and manually labeled only nine abnormalities because obtaining ground truth for all labels was costly.Source: §2.2 · pp. 6–7

2.3 Development and evaluation of a whole CT volume multi-organ, multi-disease classifier

• CT-Net applies ImageNet-pretrained ResNet-18 features to three-channel image stacks, aggregates features across whole-volume depth with 3D convolutions, and predicts 83 labels through fully connected layers.The authors note that a fully connected layer directly on ResNet output would require 1,116,114,944 parameters.Source: §2.3 · p. 8, Figure 2 · p. 9

2.4 Architecture Comparison and Ablation Study

• On a 2,000-training and 1,000-validation scan subset, the study compared CT-Net with BodyConv and 3DConv and ablated ImageNet pretraining and 3D convolutions.The ablations used random ResNet weights or replaced 3D convolutions with max pooling; all comparison models were trained on 83 labels.Source: §2.4 · p. 10

2.5 Performance and Statistical Analysis

• Performance was summarized with AUROC and average precision; CT-Net-83 versus CT-Net-9 AUROCs were compared using DeLong tests with Benjamini–Hochberg correction and DeLong confidence intervals.Average precision was reported alongside abnormality frequency because its baseline depends on positive-label frequency.Source: §2.5 · p. 10

3.1 Automatic Label Extraction from Free-Text Reports

• SARLE-Rules achieved an average F-score of 0.976 across nine labels, versus 0.930 for SARLE-Hybrid, and performed better on rare findings.Rules and Hybrid tied on common labels such as atelectasis; SARLE-Rules labels were used to train and evaluate the multilabel CNN.Source: §3.1 · p. 10
• For six overlapping labels, SARLE-Rules generally outperformed CheXpert; CheXpert scored better on pleural effusion, and both tied on atelectasis.On the 427-report test set, SARLE-Rules scored 0.998 versus 0.888 on opacity and 0.941 versus 0.727 on pneumothorax.Source: §3.1 · pp. 10–11
• Nodule was the most frequent training-set abnormality, with 19,567 examples (77% positive); only 139 scans, under 0.6%, were negative for all 83 labels.The nodule category included micronodules under 3 mm; nodules larger than 1 cm accounted for 12%.Source: §3.1 · p. 11

3.2 Multilabel CNN Predicts Abnormalities from CT Volumes

• CT-Net-83 outperformed CT-Net-9 on every abnormality, indicating the value of training with 74 additional labels.CT-Net-9 was trained on nine labels; CT-Net-83 was trained on those nine plus 74 additional labels.Source: §3.2 · p. 11
• 18 abnormalities achieved AUROC >0.9; six were below 0.6, while the other 59 fell between 0.6 and 0.9.The remaining groups were 17 abnormalities at 0.8–0.9, 24 at 0.7–0.8, and 18 at 0.6–0.7 AUROC.Source: §3.2 · p. 12
• On the 2,000-scan training and 1,000-scan validation subset, CT-Net-83 outperformed BodyConv, 3DConv, and its Rand and Pool ablations.The Rand variant used a randomly initialized ResNet feature extractor; the Pool variant replaced 3D convolutions with max pooling.Source: §3.2 · p. 12, Figure 3 · p. 14

4 Discussion

• RAD-ChestCT is the largest reported multiply annotated chest CT data set, with 36,316 volumes from 19,993 patients.Source: §4 · p. 14
• RAD-ChestCT includes diverse focal and diffuse abnormality annotations and supports multilabel classification, weakly supervised localization, and exploratory unsupervised research.Source: §4 · pp. 14–15
• SARLE extracts 83 labels and achieves an average F-score of 0.976 across 9 abnormalities; its rules distinguish normal from abnormal phrases before vocabulary lookup.Source: §4 · p. 15
• Unlike separate single-label models, the multilabel setup accommodates interactions among coexisting findings in unfiltered data, where fewer than 1% of scans are normal.Source: §4 · p. 16
• Nodule AUROC is 0.718, below methods in the upper 0.90s that use manually acquired bounding boxes; CT-Net uses whole-volume labels without boxes.Source: §4 · p. 16
• Over 10%: training on 83 labels instead of 9 raises average AUROC from 0.726 to 0.802; the 83-label model exceeds AUROC 0.90 for almost twenty abnormalities.Source: §4 · p. 16
• CT-Net outperforms two alternative models by leveraging transfer learning and 3D convolutions that combine abnormality features across the scan.Source: §4 · p. 17

A.1.1 Background & Downloading CT Reports

• The MRN uniquely identifies patients and keeps each patient’s scans in one train, validation, or test set; accession numbers match reports to their corresponding CT volumes.Source: §A.1.1 · p. 21
• Protocol descriptions record the imaged body part, contrast use, and other scan-acquisition details.Source: §A.1.1 · p. 21

A.1.2 Removing Duplicate CT Reports

• The report sources supplied 414,438 DEDUCE reports and 26,384 Epic reports; Table A1 outlines the process for removing duplicate reports.Source: §A.1.1 · p. 21, Table A1 · p. 21

A.1.3 Merging CT Report Datasets

• 330,710 DEDUCE reports were merged with 26,384 Epic reports.Source: §A.1.3 · p. 22
• 77% of Epic reports overlapped with DEDUCE, while 6,093 Epic reports were unique to Epic.Source: §A.1.3 · p. 22
• 336,800 unique reports remained in the merged data set after removing rows with missing values.Source: §A.1.3 · p. 22

A.1.4 Filtering by Protocol

• A total of 36,861 reports matched either of the specified unenhanced chest CT protocol descriptions, accounting for 11% of the total report dataset.Source: §A.1.4 · p. 23

A.1.5 Cleaning CT Reports

• Report cleaning extracted Findings and Impression, split text into sentences, lowercased it, standardized whitespace, retained decimal points, and replaced times, dates, and years with tokens.Source: §A.1.5 · p. 23

A.2.1 Downloading CT Volumes

• The volume download took 49 days and succeeded for 36,316 of the 36,861 specified accessions, a 99% success rate.Source: §A.2.1 · p. 23

A.2.2 The Data Interchange Standard for Biomedical Imaging (DICOM)

• DICOM images pair a pixel array with patient and scan metadata, and Table A3 identifies attributes important for building RAD-ChestCT.Source: §A.2.2 · p. 23, Table A3 · p. 24

A.2.3 Selecting Series

• For each accession number, the pipeline keeps ORIGINAL series and, when several qualify, selects the one with the most slices.A scanning event may contain multiple scans or reformats, each treated as a separate series.Source: §A.2.3 · p. 24

A.2.4 Ordering Slices

• Correct slice order is critical because shuffling slices destroys the volumetric representation.Source: §A.2.4 · p. 25
• InstanceNumber is unreliable because some scanners fill it incorrectly, so it is not used to order slices.Source: §A.2.4 · p. 25
• ImageOrientationPatient and ImagePositionPatient provide the reliable basis for ordering slices along the patient’s craniocaudal direction.The orientation identifies the z direction, while sorting by z-position reconstructs the volume; chest CTs typically use orientation “1,0,0,0,1,0,” but it can vary.Source: §A.2.4 · p. 25

A.2.5 Rescaling Pixel Values to Hounsfield Units

• RescaleSlope and RescaleIntercept convert raw DICOM pixel values to Hounsfield units, after which values are clipped to [-1000 HU,+1000 HU].For model training, values are dynamically clipped to [-1000 HU, +200 HU] to focus on heart and lung abnormalities.Source: §A.2.5 · pp. 25–26

• PixelSpacing provides x- and y-distances in millimeters, while z-spacing is inferred from adjacent slices’ ImagePositionPatient values.Source: §A.2.6 · p. 26
• 19,619 volumes had 0.625 mm z-spacing and 14,871 had 0.6 mm, together accounting for 95% of volumes.Other volumes had z-spacing below 0.6 mm (23 volumes) or above 0.625 mm (1,803); x- and y-spacing can differ from z-spacing.Source: §A.2.6 · p. 26
• Resampling every CT to 0.8 × 0.8 × 0.8 mm makes each pixel represent a consistent physical distance across the data set.The step also mildly reduces volume size, lowering storage and model memory requirements.Source: §A.2.6 · pp. 26–27

A.2.7 Resampling with Irregular Z-Spacing

• About 4% of scans contain at least one irregular z-spacing interval, so resampling uses the scan-wide modal z-spacing to prevent warping.Source: §A.2.7 · p. 27

A.2.8 Sanity Checks

• All 25 inspected CT scans passed checks of slice ordering, resampling artifacts, and grayscale appearance across organs.The sample included 9 severely abnormal scans, 9 with no abnormalities, and 7 random scans, viewed in axial, coronal, and sagittal planes.Source: §A.2.8 · p. 27

A.2.9 Efficient Storage of CT Volumes

• 2.8 terabytes of zipped preprocessed arrays replaced 9.2 terabytes of raw DICOMs, with final pixel values stored as 16-bit integers.The arrays use lossless zip compression.Source: §A.2.9 · p. 27

B.1 Sentence Analysis for Radiology Label Extraction: Term Search

• SARLE’s vocabulary supports automatic extraction of 83 abnormality labels from CT reports.The table documents terms and example matches used to assign labels.Source: Table B1 · p. 28
• A match in “Any” marks an abnormality present; alternatively, a Term 1 and Term 2 match does so unless an Exclude term is present.The search is applied only to abnormal phrases.Source: Table B1 · p. 28
• Measurement-sensitive rules identify nodules and lymphadenopathy by size, while non-lung terms are excluded to avoid matching liver fibrosis.The special rules use measurements in the sentence text; lymph nodes over 1 cm count as lymphadenopathy.Source: §B.1 · pp. 28, 30

B.2 Radiology Label Extraction Related Work

• SARLE uses an explainable, customizable vocabulary for unenhanced chest CT and sentence-text negation rules, rather than MetaMap tagging or fixed negation scopes.Its abnormality detection also recognizes normal-state terms such as “patent,” and special rules extract size measurements for nodules and lymphadenopathy.Source: §B.2 · p. 31
• 0.671 F-Score was reported for all labels in one prior report-classifier study, while another CT pulmonary-embolism classifier reported 0.938 F-Score on internal validation.The surveyed work spans report classifiers, rule-based systems, and hybrid methods.Source: Table B2 · p. 32, §B.2 · p. 33

Appendix C. Whole CT Volume Multilabel Classification

• 10 abnormalities was the median per CT scan (interquartile range 6) in the whole-volume multilabel classification data.Table C1 reports CT-Net-83 test-set AUROC and average precision across 83 abnormalities, but this caption provides no values.Source: Figure C1 · p. 35, Table C1 · p. 36