Development and Validation of Deep Learning Algorithms for Detection of Critical Findings in Head CT Scans

• Rapid interpretation of hemorrhage and other critical findings is important for stroke and trauma decisions, including whether urgent neurosurgical treatment is needed.Source: §1 · p. 2
• Automated initial screening and triage could streamline interpretation, reduce time to diagnosis, and support queue management or remote decision-making.Source: §1 · p. 2
• Deep learning has been applied to medical-image classification and segmentation, while accurate model development requires large, well-labeled datasets from diverse sources.Source: §1 · p. 2
• The algorithms detect five hemorrhage types, calvarial fractures, mass effect, and midline shift on non-contrast head CT scans.Source: §1 · p. 2

• The retrospective dataset comprised 313,318 head CT scans with clinical reports from multiple centers and varied scanner models.Source: §2.1 · p. 2, Table 1 · p. 3
• Scans from randomly selected patients formed the Qure25k validation set, which was not used for algorithm development and excluded postoperative scans and patients younger than seven.Source: §2.1 · p. 2
• CQ500 came from New Delhi centers with no overlap with development centers and included hospital and stand-alone outpatient radiology sites.Source: §2.1 · p. 2
• CQ500 batch B1 covered 30 days of scans, while B2 used report-based sampling to select scans with target findings.Source: §2.1 · p. 3
• CQ500 eligibility required a complete non-contrast axial series with soft reconstruction, no postoperative defect, and patient age of at least seven years.Source: §2.1 · p. 3

• Three independent senior radiologists, with 8, 12, and 20 years of cranial CT experience, read randomized CQ500 scans without access to clinical history.Source: §2.2 · p. 3
• Readers used a web-based viewer that allowed window-level adjustment and access to all scan series.Source: §2.2 · p. 3
• Readers recorded hemorrhage presence and type, fracture, midline shift, and mass effect for each scan.Source: §2.2 · pp. 3–4
• The study defined mass effect broadly, treated shift greater than 5 mm as positive, and counted a fracture extending into the skullcap as calvarial.Source: §2.2 · p. 4
• Majority vote determined CQ500 diagnoses when readers disagreed, while rule-based NLP inferred findings from clinical reports used as gold standards for development and Qure25k.Source: §2.2 · p. 4

2.3 Developing the algorithms

• The 3D nature of CT and the limited development of 3D image classification posed a challenge; the approach drew on video classification and slice- and pixel-level annotation.Source: §2.3 · pp. 4–5
• Separate models addressed intracranial hemorrhage, midline shift and mass effect, and calvarial fractures.Source: §2.3 · p. 5

2.3.1 Intracranial Hemorrhage

• For hemorrhage detection, modified ResNet18 models predicted slice-level confidences for each bleed type, which a random forest combined into scan-level predictions.Source: §2.3.1 · p. 5
• A UNet-based segmentation model localized IPH, SDH, and EDH using pixel annotations from a subset of slice-annotated images.Source: §2.3.1 · p. 5

2.3.2 Midline Shift and Mass Effect

• Modified ResNet18 predicted slice-wise mass-effect and midline-shift confidences, and a random forest combined them into scan-level predictions.Source: §2.3.2 · p. 5

2.3.3 Calvarial Fractures

• For fractures, DeepLab produced pixel-wise heatmaps, hard-negative mining addressed sparse annotations, and engineered heatmap features fed a random forest for scan-level confidence.Source: §2.3.3 · p. 5

2.3.4 Preprocessing

• Preprocessing resampled axial slices to about 5 mm and 224 × 224 pixels, then stacked brain, bone, and subdural windows as input channels.Source: §2.3.4 · p. 6

2.4 Evaluating the algorithms

• Each scan receives nine real-valued confidence scores from 0 to 1, covering ICH, its five types, midline shift, mass effect, and calvarial fracture.The gold standard is inferred from reports for Qure25k and obtained by majority vote for CQ500.Source: §2.4 · p. 6
• ROC curves plot sensitivity against 1 − specificity as the threshold varies, and AUC summarizes algorithm performance.Two operating points are selected for sensitivity ≈0.9 and specificity ≈0.9.Source: §2.4 · p. 6

2.5 Statistical Analysis

• For a finding with 1% prevalence, approximately 20,000 scans are needed to estimate AUC within a 95% confidence interval half-length of 0.05.Approximately 80 positive scans are needed to estimate sensitivity of 0.7 within a 95% confidence interval half-length of 0.10.Source: §2.5 · p. 6
• Qure25k was randomly sampled and exceeded 20,000 scans; CQ500 used selective sampling because radiologist time was constrained.Random selection based on clinical reports was used to mitigate the selection bias that manual scan curation could favor more significant positive cases.Source: §2.5 · p. 6
• Sensitivity and specificity confidence intervals used the exact Clopper–Pearson method, while AUC intervals used the distribution-based Hanley–McNeil approach.Reader concordance on CQ500 was assessed with percentage agreement, Cohen’s kappa for pairs, and Fleiss’ kappa across all three readers.Source: §2.5 · p. 6
• Sample sizes, sample statistics, and confidence intervals were calculated with scikit-learn and statsmodels.Source: §2.5 · p. 6

3 Results

• Qure25k contained 21,095 scans, including 2,494 reported positive for ICH and 992 for calvarial fracture; CQ500 comprised 491 scans across B1 (214) and B2 (277).B1 had 35 ICH-positive and 6 fracture-positive scans; B2 had 170 and 28, respectively.Source: §3 · pp. 6–7, Table 2 · p. 7
• The CQ500 selection flow analyzed 4,462 reports and yielded 491 scans after exclusions; the most frequent stated exclusion was unavailable images (113).The selection process is summarized in Figure 2.Source: §3 · p. 7, Figure 2 · p. 8

3.1 Qure25k dataset

• The Qure25k report-NLP reference was evaluated on 1,779 reports; subdural hemorrhage was the least sensitive finding (0.9318), while fracture inference had sensitivity and specificity of 1.Table 3a presents reliability estimates for the NLP algorithm.Source: §3.1 · p. 7, Table 3 · p. 9
• On Qure25k, AUC was highest for IVH (0.9559) and lowest for mass effect (0.8583); ICH, calvarial fracture, and midline shift AUCs were 0.9194, 0.9244, and 0.9276.At the high-sensitivity operating point, sensitivities for ICH, fracture, and midline shift were 0.9006, 0.9002, and 0.9114, respectively.Source: §3.1 · pp. 7–8, Table 4 · p. 10

3.2 CQ500 dataset

• On CQ500, the highest AUCs were for EDH (0.9731) and midline shift (0.9697), while mass effect had the lowest AUC (0.9216).At the high-sensitivity operating point, sensitivities for ICH, fracture, and midline shift were 0.9463, 0.9487, and 0.9385, respectively.Source: §3.2 · pp. 8, 11, Table 4 · p. 10
• AUCs were generally higher on CQ500 than Qure25k; the largest reported gap was for mass effect (0.9216 vs 0.8583), and the smallest was for intracranial hemorrhage (0.9419 vs 0.9149).Source: §3.2 · pp. 8, 11, Table 4 · p. 10

4 Discussion

• Earlier systems reported 0.8864 sensitivity and 0.8125 precision for hemorrhage on 77 scans [11], while another reported AUCs of 0.91 for HMH and 0.81 for SAI.Source: §4 · pp. 11–12
• The authors frame this as the first large separate-finding head-CT evaluation, clinically validated on CQ500 and releasing its reads for future benchmarking.Source: §4 · p. 12
• CQ500 AUCs were significantly better than Qure25k, while CQ500 scores were lower against a single reader than against majority vote.The authors suggest clinical-history access and differences in reference standards may help explain the dataset comparison.Source: §4 · p. 12, Table 5 · p. 12
• Qualitative review found stronger predictions for medium or large IPH, extraaxial bleeds, fractures, and shift, but errors persisted on small IPH, IVH, skull-base bleeds, and chronicity-related mimics.Acute and chronic hemorrhages were not separated; infarcts and prominent CSF spaces were occasionally predicted as ICH.Source: §4 · p. 12
• Only 13 CQ500 scans had extradural hemorrhage, making those results somewhat statistically insignificant.Source: §4 · p. 12
• Radiologist concordance was not high for all findings; fracture pairwise Cohen’s kappas were 0.5771, 0.3704, and 0.3637.Source: §4 · p. 12
• CQ500 validation did not exclude follow-up scans, and the authors could not quantify this limitation because patient identifiers were unavailable.They report no patient or scan overlap between CQ500 and the development dataset.Source: §4 · p. 13
• The algorithms can output hemorrhage masks except for SAH and fracture localizations, but output accuracy was not independently assessed and fracture detection was limited to the calvaria.The localization outputs are shown in Figure 4.Source: §4 · p. 13, Figure 4 · p. 13

5 Conclusion

• The algorithms detect critical findings on head CT with good accuracy and could provide a lower performance bound for radiologic interpretation.The findings include hemorrhages, fractures, mass effect, and midline shift.Source: §5 · p. 13
• Automating head CT triage with these algorithms may be feasible.The conclusion presents triage automation as a possibility, not a demonstrated clinical outcome.Source: §5 · p. 13
• Further research is needed to determine whether the algorithms improve radiologist efficiency or patient care and outcomes.Source: §5 · p. 13