PaperlayerView exact revision on arXiv

Source-linked AI summary

COVID-MobileXpert: On-Device COVID-19 Patient Triage and Follow-up using Chest X-rays

Xin Li, Chengyin Li, Dongxiao Zhu

arXiv:2004.03042v3eess.IVcs.CVcs.LG

TL;DR

COVID-MobileXpert addresses the need for rapid point-of-care COVID-19 triage and follow-up using chest X-rays. It transfers lung-disease imaging knowledge through a three-player network and distills to a lightweight on-device model, with reported compact-model triage performance comparable to a large network and stronger trajectory prediction from longitudinal features than from a single CXR.

  • Problem

    Rapid point-of-care COVID-19 triage and follow-up are needed to support resource use and clinical workflow amid radiologist shortages and expected surges in suspected cases.

    Source: §I · p. 1

  • Method

    A three-player KTD framework transfers lung-disease CXR knowledge through AP and RF networks, then distills to a lightweight MS for on-device triage and follow-up.

    Source: §I · pp. 1–2

  • Results

    MobileNetV2 and SqueezeNet achieved AUROC values of 0.970 and 0.964, respectively, for COVID-19 discrimination against mixed pneumonia and normal cases, comparable to DenseNet-121; longitudinal CXR features outperformed single-image features for trajectory prediction.

    Source: D. Evaluation of COVID-19 Patient… · p. 5, E. Evaluation of COVID-19 Patient… · p. 6

  • Takeaways & Limitations

    The authors conclude that the three-player framework can train a compact on-device model with a smaller dataset while preserving performance.

    Source: §VI · pp. 7–8

Abstract

from arXiv · show

During the COVID-19 pandemic, there has been an emerging need for rapid, dedicated, and point-of-care COVID-19 patient disposition techniques to optimize resource utilization and clinical workflow. In view of this need, we present COVID-MobileXpert: a lightweight deep neural network (DNN) based mobile app that can use chest X-ray (CXR) for COVID-19 case screening and radiological trajectory prediction. We design and implement a novel three-player knowledge transfer and distillation (KTD) framework including a pre-trained attending physician (AP) network that extracts CXR imaging features from a large scale of lung disease CXR images, a fine-tuned resident fellow (RF) network that learns the essential CXR imaging features to discriminate COVID-19 from pneumonia and/or normal cases with a small amount of COVID-19 cases, and a trained lightweight medical student (MS) network to perform on-device COVID-19 patient triage and follow-up. To tackle the challenge of vastly similar and dominant fore- and background in medical images, we employ novel loss functions and training schemes for the MS network to learn the robust features. We demonstrate the significant potential of COVID-MobileXpert for rapid deployment via extensive experiments with diverse MS architecture and tuning parameter settings. The source codes for cloud and mobile based models are available from the following url: https://github.com/xinli0928/COVID-Xray.

I. INTRODUCTION

COVID-MobileXpert addresses the need for rapid, point-of-care CXR triage and follow-up with a lightweight mobile system. Its three-player KTD framework transfers lung-disease imaging knowledge into a compact model for COVID-19 screening and radiological trajectory prediction.

I. INTRODUCTION

  • A shortage of radiologists, reactive resource allocation, and anticipated surges in suspected cases motivate rapid, automated, point-of-care patient management.The stated goals are to optimize resource use and clinical workflow through triage and radiological trajectory prediction.Source: §I · p. 1
  • The mobile-AI challenge is to build lightweight networks that remain accurate and robust, while reducing latency and protecting privacy on device.The paper notes that medical-image mobile applications were largely limited to basic functions such as DICOM viewing.Source: §I · p. 1
  • The KTD framework pre-trains an Attending Physician on large-scale lung-disease CXRs, fine-tunes a Resident Fellow on COVID-19, pneumonia, and normal cases, then distills to a lightweight Medical Student.The MS is trained for on-device triage and follow-up.Source: §I · p. 1
  • Knowledge transfer from large-scale lung-disease images aims to make learned representations more expressive, while novel losses aim to make MS distillation more robust.The resulting MS is described as able to identify COVID-19 cases and predict radiological trajectory.Source: §I · pp. 1–2
  • COVID-MobileXpert uses portable X-rays to screen for COVID-19 at the ED and track possible deterioration from longitudinal images in the ICU or general ward.The system is intended to support both immediate triage and follow-up of admitted patients.Source: §I · p. 2

II. RELATED WORK

Prior COVID-19 CXR studies commonly follow a pipeline of preprocessing, feature extraction, and interpretation, addressing diagnosis, severity, or trajectory prediction. The paper distinguishes its use of longitudinal images to forecast radiological trajectory from approaches based on a single CXR.

A. COVID-19 CXR Interpretation

  • COVID-19 CXR interpretation studies generally use image preprocessing, feature extraction, and interpretation, with augmentation and lung segmentation among their preprocessing approaches.Reported augmentation ranges from transformations to GAN-generated samples, while segmentation methods preserve lung regions of interest.Source: A. COVID-19 CXR Interpretation · p. 2
  • Studies extract discriminative patterns such as GGO using CNNs, often adopting ResNet, InceptionV3, DenseNet, or VGG architectures.The cited approaches apply these architectures after preprocessing.Source: A. COVID-19 CXR Interpretation · p. 2
  • Prior interpretation tasks include diagnosis by classification or anomaly detection, and severity assessment by regression or combined classification and regression.Trajectory prediction has used severity-score changes between adjacent CXRs or feature-based classification from a single image.Source: A. COVID-19 CXR Interpretation · p. 2
  • The paper forecasts radiological trajectory from features extracted across one patient's longitudinal CXRs, since a single image may not capture enough information for prediction.The authors state that longitudinal features may reveal progressive-disease changes too subtle for the human eye.Source: A. COVID-19 CXR Interpretation · p. 2

B. On-device AI Model

On-device medical imaging must fit mobile hardware while addressing label scarcity and the difficulty of separating diseases against similar, dominant image backgrounds. The paper positions its three-player framework as a way to train a lightweight network for COVID-19 triage and follow-up.

B. On-device AI Model

  • Most existing COVID-19 interpretation models are full DNNs, while prior lightweight-network comparisons largely use small natural-image benchmarks rather than on-device medical imaging.The cited lightweight architectures have not shown one network superior across all reported natural-image tasks and resource measures.Source: B. On-device AI Model · p. 2
  • Fig. 2 depicts knowledge transfer from AP to RF and distillation from RF to MS, with separate training paths for patient triage and follow-up.The caption assigns blue arrows to triage and purple arrows to follow-up.Source: Figure 2 · p. 3
  • Compression can reduce model cost, but quantization and pruning may reduce prediction accuracy, while other reported approaches aim to reduce latency or energy without sacrificing accuracy.The passage also describes in-training compression and mixed-precision methods.Source: B. On-device AI Model · p. 3
  • Knowledge distillation trains a compact student to mimic a full teacher's outputs while also minimizing cross-entropy with true labels.Prior work cited in the passage reports compact students outperforming compact models trained from scratch.Source: B. On-device AI Model · p. 3
  • Medical-image label scarcity and similar, dominant foreground and background make discriminative learning difficult; the paper proposes a three-player framework for lightweight on-device COVID-19 triage and follow-up.These are identified as challenges distinct from natural-image applications.Source: B. On-device AI Model · p. 3

III. METHOD

The method uses DenseNet-121 for the AP and RF networks and MobileNetV2 or SqueezeNet as compact MS candidates. Its training scheme transfers lung-disease CXR knowledge through the AP and RF stages before using the RF to train the MS for triage and trajectory prediction.

A. Model Architecture

  • DenseNet-121 is the template for AP and RF, while MobileNetV2 and SqueezeNet are selected as candidate lightweight MS networks for screening and trajectory prediction.The paper characterizes these choices as, respectively, a well-applied and a particularly lightweight network.Source: A. Model Architecture · p. 3

B. The KTD Training Scheme

  • The AP is pre-trained on ChestX-ray8's 108,948 lung-disease cases, then general CXR and pneumonia weights initialize RF fine-tuning on COVID-19, normal, and pneumonia images.The COVID-19 and normal class weights are randomly initialized; the RF then trains the MS through distillation.Source: B. The KTD Training Scheme · pp. 3–4
  • For follow-up, the trained MS extracts disease-specific features from a patient's longitudinal CXRs, aggregates them by feature difference or chronological concatenation, and predicts trajectory with two fully connected layers.The trajectory predictor is trained with softmax loss.Source: B. The KTD Training Scheme · p. 4

C. Loss Functions

The section addresses the hard-sample problem in medical imaging and presents Probabilistically Compact (PC) loss as a way to improve robustness by widening probability margins among classes.

C. Loss Functions

  • Medical-image hard samples have subtle ROI differences against large shared foreground and background regions.PC loss is compared with ArcFace and classical softmax as a response to this challenge.Source: C. Loss Functions · p. 4
  • PC loss encourages a margin between the true class probability and the most probable competing classes.It penalizes non-target classes that violate a probability margin, encouraging probabilistic intra-class compactness without assumptions about feature space.Source: C. Loss Functions · p. 4

IV. EXPERIMENT AND RESULTS

The experiments assess compact medical-student networks for COVID-19 screening and radiological trajectory prediction, using separate CXR datasets for triage and follow-up.

IV. EXPERIMENT AND RESULTS

  • The experiments evaluate compact medical-student networks across loss functions and tuning parameters for screening, and feature aggregation schemes and classifiers for trajectory prediction.Source: §IV · p. 4

A. Datasets

  • The triage dataset contains 179 images each from normal, pneumonia, and COVID-19 classes, split 125/18/36 per class for training, validation, and testing.Images from the same patient were kept from appearing in both training and test sets.Source: A. Datasets · p. 4
  • The trajectory dataset has 159 sequences from 100 patients, labeled Worse, Stable, or Improved from opacity-score changes across up to four time points.It includes 80 Worse, 28 Stable, and 51 Improved samples, split 111/16/32 for training, validation, and testing.Source: A. Datasets · p. 5

B. Implementation Details

Training uses Adam and validation-selected parameters; the tuning discussion explains the roles of distillation strength and temperature.

B. Implementation Details

  • Models were trained with Adam for 50 epochs, using mini-batches of 32 for triage and 10 for follow-up.The best validation-set parameters were used for testing; trajectory prediction used a fixed pre-trained medical-student feature extractor and dropout of 0.5.Source: B. Implementation Details · p. 5

C. Tunning Parameters

  • The tuning discussion identifies ξ as a hyperparameter that encourages probabilistic intra-class compactness in the PC loss.Source: C. Tunning Parameters · p. 5
  • The distillation parameter α controls the relative contribution of distillation and classification losses, while temperature T softens the softmax probabilities.Higher α strengthens distillation from the resident-fellow model; T = 1 is standard softmax.Source: C. Tunning Parameters · p. 5

D. Evaluation of COVID-19 Patient Triage Performance

Compact student networks approach the cloud-based resident-fellow model on COVID-19 discrimination, while distillation and loss choice affect classification performance.

D. Evaluation of COVID-19 Patient Triage Performance

  • 0.935 accuracy was reported for the resident-fellow network after knowledge transfer from the pre-trained attending-physician network.Source: D. Evaluation of COVID-19 Patient… · p. 5
  • 0.843 (MobileNetV2) and 0.732 (SqueezeNet) were the baseline classification accuracies without knowledge distillation.The paper reports that most distilled-network accuracy values were above 0.850 and higher than these baselines.Source: D. Evaluation of COVID-19 Patient… · p. 5
  • Performance was sensitive to loss choice but not to distillation temperature or strength; PC loss performed best across the tested settings.Source: D. Evaluation of COVID-19 Patient… · p. 5
  • 0.970 and 0.964 AUROC were achieved by MobileNetV2 and SqueezeNet, respectively, for COVID-19 versus mixed pneumonia and normal cases.Both compact networks were reported comparable to DenseNet-121 across the discrimination tasks.Source: D. Evaluation of COVID-19 Patient… · p. 5

E. Evaluation of COVID-19 Patient Follow-up Performance

Longitudinal feature aggregation with an FC-classifier gives the strongest reported on-device trajectory accuracy, matching the DenseNet-121 upper bound. MobileNetV2 also shows close-to-RF performance for radiological trajectory discrimination by AUROC.

E. Evaluation of COVID-19 Patient Follow-up Performance

  • The evaluation selects classifier and feature-aggregation combinations using “Worse” versus “Improved” accuracy, then assesses discrimination with AUROC.Four classifiers were trained on MS-network features: logistic regression, gradient boosting, random forest, and the FC-classifier.Source: E. Evaluation of COVID-19 Patient… · pp. 5–6
  • 0.800 accuracy from MobileNetV2 with the FC-classifier and feature concatenation matches DenseNet-121’s upper bound; the reimplemented single-CXR model scored 0.600.The single-CXR result was measured on the paper’s more balanced dataset.Source: Table III · p. 6, E. Evaluation of COVID-19 Patient… · p. 6
  • Longitudinal feature concatenation significantly improves on using only the difference between the last two feature sets, and the FC-classifier outperforms conventional classifiers.The conventional classifiers were logistic regression, gradient boosting, and random forest.Source: E. Evaluation of COVID-19 Patient… · p. 6
  • MobileNetV2 reaches an AUROC of 0.883 for distinguishing “Worse” from “Improved” cases and performs close to the RF network.Fig. 4 also shows comparisons for “Worse” versus “Stable” and “Improved” plus “Stable” cases.Source: Figure 4 · p. 6, E. Evaluation of COVID-19 Patient… · p. 7

V. PERFORMANCE EVALUATION ON MOBILE DEVICES

The mobile-device evaluation finds a resource–accuracy trade-off between the two compact networks, with resource consumption also decreasing across newer device generations.

V. PERFORMANCE EVALUATION ON MOBILE DEVICES

  • The MS networks were deployed with PyTorch Mobile on six Android devices spanning low-, mid-, and high-end systems, with CPU, memory, and energy use compared.Source: §V · p. 7
  • MobileNetV2’s higher accuracy comes at higher resource consumption than SqueezeNet, making it more suitable for high-end devices and SqueezeNet more suitable for low-end devices.The authors report a downward resource-consumption trend across the device generations tested.Source: §V · p. 7
  • Resource consumption trends downward across the tested device generations, reflecting continuous improvement in mobile hardware.Source: §V · p. 7

VI. CONCLUSIONS

The conclusion presents three-player knowledge transfer and distillation as a way to train compact on-device models from smaller datasets while preserving performance. It also identifies potential application beyond COVID-19 to other medical-imaging tasks.

VI. CONCLUSIONS

  • Conventional two-player distillation faces a natural-image-to-medical-image domain gap, compounded by the small number of labeled COVID-19 training images.Source: §VI · p. 7
  • The three-player framework transfers knowledge from AP to RF and then to MS; the AP and RF share an architecture, while RF and MS share a training set.The authors describe AP-to-RF transfer as a more effective regularization and highlight novel loss functions for refining distilled knowledge.Source: §VI · p. 7
  • The framework is presented as a more effective way to train compact on-device models on smaller datasets while preserving performance.Source: §VI · p. 8
  • The authors describe broader applicability to on-device medical-imaging classification and segmentation for diseases including lung and musculoskeletal abnormalities.Source: §VI · p. 8

Related papers

Loading 2004.03042v3…