Source-linked AI summary
Computer aided detection of tuberculosis on chest radiographs: An evaluation of the CAD4TB v6 system
Keelin Murphy, Shifa Salman Habib, Syed Mohammad Asad Zaidi, Saira Khowaja, Aamir Khan, Jaime Melendez, Ernst T. Scholten, Farhan Amad, Steven Schalekamp, Maurits Verhagen, Rick H. H. M. Philipsen, Annet Meijers, Bram van Ginneken
TL;DR
Automated chest-radiograph analysis could support tuberculosis prescreening where sputum testing is costly and expertise is limited. This study evaluates CAD4TB v6 on independent data against Xpert and expert-derived radiological standards, finding improved performance and cost efficiency over earlier versions while matching human-observer performance against Xpert.
Problem
The study addresses the need for practical, affordable, and efficient tuberculosis prescreening in high-burden settings where radiological expertise is limited.
Method
CAD4TB v6 was evaluated on independent chest-radiograph data against Xpert and an expert-derived radiological reference standard, with comparisons to earlier versions and five human observers.
Results
CAD4TB v6 was more cost-effective than earlier versions, achieving 76% specificity at 90% sensitivity against Xpert and performing within the range of human observers.
Takeaways & Limitations
At 90% sensitivity, CAD4TB v6 provides higher throughput and lower screening cost than version 3, supporting its use as a prescreening system in high-burden regions.
Takeaways & Limitations
The Xpert reference standard is imperfect, and the study retains previously estimated costs for Xpert testing and digital radiography.
Abstract
from arXiv · showhide
There is a growing interest in the automated analysis of chest X-Ray (CXR) as a sensitive and inexpensive means of screening susceptible populations for pulmonary tuberculosis. In this work we evaluate the latest version of CAD4TB, a commercial software platform designed for this purpose. Version 6 of CAD4TB was released in 2018 and is here tested on a fully independent dataset of 5565 CXR images with GeneXpert (Xpert) sputum test results available (854 Xpert positive subjects). A subset of 500 subjects (50% Xpert positive) was reviewed and annotated by 5 expert observers independently to obtain a radiological reference standard. The latest version of CAD4TB is found to outperform all previous versions in terms of area under receiver operating curve (ROC) with respect to both Xpert and radiological reference standards. Improvements with respect to Xpert are most apparent at high sensitivity levels with a specificity of 76% obtained at a fixed 90% sensitivity. When compared with the radiological reference standard, CAD4TB v6 also outperformed previous versions by a considerable margin and achieved 98% specificity at the 90% sensitivity setting. No substantial difference was found between the performance of CAD4TB v6 and any of the various expert observers against the Xpert reference standard. A cost and efficiency analysis on this dataset demonstrates that in a standard clinical situation, operating at 90% sensitivity, users of CAD4TB v6 can process 132 subjects per day at n average cost per screen of \$5.95 per subject, while users of version 3 process only 85 subjects per day at a cost of \$8.38 per subject. At all tested operating points version 6 is shown to be more efficient and cost effective than any other version.
Methods
CAD4TB v6 was evaluated on an independent chest-radiograph dataset using Xpert results and an expert-derived radiological reference standard, with comparisons across software versions and human observers.
- Observer analysis: 500 scans, balanced between 250 Xpert-positive and 250 Xpert-negative cases, were selected for human-observer comparison after matching CAD4TB v6 performance to the full dataset.Five observers independently scored every selected scan while blinded to clinical information and one another’s scores.
- Observer analysis: Observers classified each scan as No TB, Possible TB, or Likely TB, with optional free-text observations.The five observers included radiologists, a public-health TB doctor, and an X-ray technician operating CAD4TB in Pakistan.
- CAD4TB analysis: CAD4TB outputs a 0–100 score and an abnormality heatmap, with v6 using deep learning and four software versions compared.Version 6 uses a deep neural network for lung-field segmentation and a deep convolutional neural network for abnormality analysis.
- CAD4TB analysis: The pipeline normalizes radiographs, verifies postero-anterior chest-image validity, segments lung fields, and analyzes parenchymal texture with version-specific classifiers.The shared normalization step reduces appearance differences between radiographs acquired from different machines.
- Study design: 5565 chest radiographs were processed, with CAD4TB outputs analyzed against Xpert results and a radiological reference standard from 500 expert-reviewed scans.The radiological standard classified scans as positive when at least 3 of 5 observers assigned scores of 2 or 3.
- Cost analysis: Cost analysis modeled a point-of-care unit with one digital radiography system and three four-cartridge GeneXpert IV machines, evaluating four high-sensitivity settings.The modeled unit had capacity for 300 CXR screens and 45 Xpert tests per day; CXR screening cost was $1.49 and Xpert testing $13.06 per test.
Results
CAD4TB v6 improved performance over earlier versions against both Xpert and radiological references, matched expert-reader performance, and reduced screening cost while increasing throughput.
- 854 of 5565 individuals had positive Xpert results.
- CAD4TB v6 showed marked improvement over previous versions against Xpert, particularly at higher sensitivities.
- CAD4TB v6 achieved an Az of 0.987 and 98% specificity at the selected operating point against the radiological reference standard.
- CAD4TB v6 performed within the range of expert observers and improved on Observer 5 in both reference-standard analyses.
- At 90% sensitivity, CAD4TB v6 achieved 76% specificity versus 70% or less for earlier versions, saving 283 Xpert tests in this population.
- At 90% sensitivity, CAD4TB v6 had 1.6 times higher throughput and 1.4 times lower cost per screen than version 3.
- The study concludes that CAD4TB v6 enables larger-scale TB prescreening at a fraction of the cost while operating at the level of expert readers.
Competing interests
Three authors were employed by Thirona, the developer of CAD4TB, during manuscript preparation; additional disclosures are reported for one author.
- J.M., R.H.H.M.P., and A.M. were employed by Thirona, the developer of CAD4TB, when the manuscript was prepared.
Additional information
The paper provides correspondence information, reprints and permissions details, licensing terms, and a publisher’s neutrality statement.
- Correspondence and material requests should be addressed to K.M.
- Reprints and permissions information is available through Nature’s website.
- Springer Nature states neutrality regarding jurisdictional claims in published maps and institutional affiliations.
- The article is distributed under a Creative Commons Attribution 4.0 International License, subject to attribution and related conditions.