PaperlayerView exact revision on arXiv

Source-linked AI summary

Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model

Quande Liu, Lequan Yu, Luyang Luo, Qi Dou, Pheng Ann Heng

arXiv:2005.07377v1cs.CV

TL;DR

Medical image classification often lacks abundant expert labels, so the paper develops a semi-supervised framework that exploits unlabeled data through prediction and sample-relation consistency. Its self-ensembling SRC approach models semantic relations across samples and reports superior performance on two benchmark datasets spanning single-label and multi-label tasks.

  • Problem

    High-quality medical-image labels are laborious and expensive to obtain, while existing consistency methods neglect relations among samples that can provide useful semantic information.

  • Method

    The framework combines perturbation-based prediction consistency, an EMA self-ensembling teacher, and SRC, which regularizes semantic relation matrices across samples.

  • Results

    The framework consistently outperforms state-of-the-art semi-supervised methods on ISIC 2018 and ChestX-ray14 for single-label and multi-label classification.

  • Takeaways & Limitations

    SRC is presented as a general strategy that can be incorporated into other semi-supervised learning approaches.

Abstract

from arXiv · show

Training deep neural networks usually requires a large amount of labeled data to obtain good performance. However, in medical image analysis, obtaining high-quality labels for the data is laborious and expensive, as accurately annotating medical images demands expertise knowledge of the clinicians. In this paper, we present a novel relation-driven semi-supervised framework for medical image classification. It is a consistency-based method which exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations, and leverages a self-ensembling model to produce high-quality consistency targets for the unlabeled data. Considering that human diagnosis often refers to previous analogous cases to make reliable decisions, we introduce a novel sample relation consistency (SRC) paradigm to effectively exploit unlabeled data by modeling the relationship information among different samples. Superior to existing consistency-based methods which simply enforce consistency of individual predictions, our framework explicitly enforces the consistency of semantic relation among different samples under perturbations, encouraging the model to explore extra semantic information from unlabeled data. We have conducted extensive experiments to evaluate our method on two public benchmark medical image classification datasets, i.e.,skin lesion diagnosis with ISIC 2018 challenge and thorax disease classification with ChestX-ray14. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.

I. INTRODUCTION

The paper targets the high cost of expert medical-image labeling by using unlabeled data in a relation-driven semi-supervised framework. Its SRC paradigm adds consistency of semantic relations among samples to conventional prediction consistency, with strong results on two benchmark tasks.

  • Motivation: Expert annotation makes large, high-quality labeled medical-image datasets laborious to obtain, motivating effective use of unlabeled clinical data.The paper frames semi-supervised learning as a way to reduce large-scale annotation labor.
  • Prior consistency methods: Existing consistency-based methods regularize predictions for the same input under different perturbations, while Mean Teacher improves consistency targets through an ensembled teacher.The teacher model is updated from student weights using exponential moving averaging.
  • Motivation: The framework is motivated by clinical diagnosis practices that use analogous prior cases and by the additional semantic information encoded in sample relations.The paper identifies neglected inter-sample relations as a limitation of existing consistency methods.
  • Proposed approach: SRC explicitly enforces consistency of intrinsic relations among different samples, rather than only enforcing individual prediction consistency.The method models an n × n correlation matrix in high-level semantic feature space for each mini-batch.
  • Results: Experiments on ISIC 2018 and ChestX-ray14 show consistent superiority over state-of-the-art semi-supervised methods for both single-label and multi-label classification.These datasets represent skin-lesion diagnosis and thorax-disease classification, respectively.

II. RELATED WORK

Related work includes consistency, self-training, co-training, feature-alignment, reconstruction, adversarial, graph-based, and medical-image-specific semi-supervised methods. Consistency approaches are especially relevant because they exploit perturbed unlabeled inputs, often using improved targets.

  • Consistency-based methods: Consistency-based methods exploit unlabeled data by encouraging prediction agreement for perturbed versions of the same input.Perturbations can include Gaussian noise or network-weight changes.
  • Medical image analysis: Medical-image semi-supervised learning also includes self-training, co-training, feature-space alignment, and image-reconstruction approaches.These methods have been applied to tasks including segmentation and breast-cancer analysis.
  • Medical image analysis: Recent medical-image methods can be grouped into adversarial-learning-based, graph-based, and consistency-based categories.The cited examples include lung segmentation, breast-cancer analysis, and other medical imaging applications.

C. Skin Lesion Classification

The framework combines supervised classification with consistency training and an EMA teacher, while SRC models relations among medical-image samples. The cited passages describe its backbone and semantic-feature design choices.

  • Application context: The framework targets skin-lesion and thorax-disease classification, areas involving difficult medical-image variation and clinically significant diagnosis.The related application passages describe skin-lesion inter-class similarity and intra-class variation, plus chest-radiography disease classification.
  • Sample Relation Consistency: SRC estimates semantic relations among samples by calculating a relation matrix from high-level feature activations and regularizes that structure across teacher and student perturbations.The framework uses this stable relation structure to extract richer semantic information from unlabeled data.
  • Backbone framework: The objective combines supervised loss on labeled inputs with unsupervised consistency loss under different input perturbations.The supervised term evaluates labeled network outputs, while the unsupervised term measures consistency for the same inputs.
  • Backbone framework: An EMA teacher provides consistency targets while only the student model is optimized during network training.Teacher weights are updated from student weights using a smoothing coefficient.

B. Sample Relation Consistency (SRC) Paradigm

The SRC paradigm models pairwise sample relations from feature activations and enforces their stability across perturbations, extracting additional semantic information from unlabeled data.

  • Relation modeling: SRC models structured relations among mini-batch samples with a case-level Gram Matrix computed from reshaped activation maps.The Gram Matrix contains pairwise inner products between vectorized sample activations.
  • Relation modeling: The sample relation matrix is obtained by applying row-wise L2 normalization to the Gram Matrix.
  • Consistency objective: SRC requires sample relation matrices to remain stable when the same mini-batch undergoes different weights and perturbations.Its loss compares relation matrices computed under two weight–perturbation pairs.
  • Consistency objective: Minimizing Lsrc encourages robust, discriminative representations that preserve intrinsic sample relations and extract additional semantic information from unlabeled data.
  • Layer selection: Deeper features are preferred for relation estimation because they contain more high-level semantic information, while same-class samples form high-similarity blocks.The post-pooling relation matrix shows stronger diagonal-block responses than the pre-pooling matrix.

C. Overall Loss Function and Technical Details

The framework combines supervised learning with conventional prediction consistency and SRC losses, while ramping the unsupervised contribution to avoid unreliable early targets dominating training.

  • Objective: The total objective is L = Ls + λLu, where the unsupervised objective is Lu = Lc + βLsrc.
  • Objective: Ls is the supervised objective, while Lc and Lsrc respectively enforce prediction consistency and sample relation consistency.
  • Loss weighting: β balances conventional consistency and SRC losses and is generally set to 1, whereas λ weights the combined unsupervised loss against supervised learning.
  • Technical details: Weighted cross-entropy is used for Ls to mitigate class imbalance in both experimental tasks.
  • Technical details: λ ramps from 0 to 1 during the first T epochs and remains 1 thereafter, preventing unreliable early consistency targets from dominating training.

IV. EXPERIMENTS

The method is evaluated on two medical image classification settings: single-label skin lesion diagnosis and multi-label thorax disease diagnosis, with ablations and state-of-the-art comparisons.

  • Experimental scope: Experiments cover skin lesion classification from dermoscopy images and thorax disease diagnosis from chest X-ray images.
  • Experimental scope: The evaluation includes extensive ablation analysis and comparisons with state-of-the-art semi-supervised learning methods.

A. Datasets and Experimental Setup

The experiments use ISIC 2018 for seven-class skin lesion classification and ChestX-ray14 for 14-class thorax disease diagnosis, with task-specific metrics and semi-supervised training setups.

  • ISIC 2018: ISIC 2018 contains 10015 dermoscopy images labeled across 7 common skin lesion types for single-label classification.Images are resized to 224 × 224 and normalized with ImageNet statistics.
  • Evaluation: Table I compares the method with state-of-the-art semi-supervised learning methods on the ISIC 2018 dataset.
  • Implementation: The framework is trained with DenseNet121 pretrained on ImageNet as the network backbone.
  • ChestX-ray14: ChestX-ray14 contains 112120 frontal-view chest X-rays from 30805 unique patients, labeled with one or more of 14 thorax disease classes.Images are resized from 1024 × 1024 to 384 × 384 for training.
  • Evaluation: ISIC 2018 is evaluated with AUC, Accuracy, Sensitivity, and Specificity, while ChestX-ray14 is evaluated with AUC.

B. Comparison on Skin Lesion Classification Dataset

The framework is compared with several semi-supervised methods on skin lesion classification under a 20% labeled-data setting, using a common backbone and perturbations for consistency-based methods.

  • The comparison uses identical input perturbations for consistency-based methods and the same network backbone across comparison methods.These controls are used to support a fair comparison among the evaluated approaches.
  • Under 20% labeled data, the study compares self-training, SS-DCGAN, TCSE, TE, and Mean Teacher approaches on skin lesion classification.The fully supervised models trained with 100% and 20% labeled data serve as upper-bound and baseline references, respectively.
  • Consistency-based methods achieve larger improvements than SS-DCGAN, while TE slightly exceeds TCSE through more reliable ensembled consistency targets.SS-DCGAN improves AUC over the self-training method by 0.70%; TCSE and TE achieve larger improvements, with TE performing slightly better than TCSE.

C. Comparison on Thorax Disease Diagnosis Dataset

On thorax disease classification, the framework is evaluated against GraphXNET and Mean Teacher, showing strong performance across labeled-data percentages and relative robustness as supervision decreases.

  • Our method overwhelmingly outperforms GraphXNET across all labeled-data percentages and shows smaller performance decay when labeled data decreases.GraphXNET’s AUC drops by 10% from 20% to 15% labeled data and falls to 53% at 2%, whereas the proposed method is described as more robust.
  • The implemented fully supervised DenseNet obtains 81.75% average AUC on the official ChestX-ray14 split, close to a reported DenseNet result of 81.80%.A higher reported result of 83.02% relies on complicated architecture design and testing-model ensembling.
  • Mean Teacher reaches 78.83% average AUC with 20% labeled data, while the proposed SRC-enhanced method reaches 79.23%.Mean Teacher improves AUC on 13 of 14 disease types versus the fully supervised baseline, and SRC improves 12 of 14 diseases versus Mean Teacher.

2) The impact of different loss weight

The SRC loss weight β generally improves classification over Mean Teacher across a moderate range, while relation matrices become more aligned and stable as training progresses.

  • 2) The impact of different loss weight: β values from 0.1 to 5 generally improve classification performance over Mean Teacher, whereas β = 0.01 provides only marginal improvement.β = 0 corresponds to the original Mean Teacher framework; the loss is dominated by classification and conventional consistency losses when β is very small.
  • 2) The impact of different loss weight: At β = 10, SRC may impose overly strict consistency constraints, identifying a limitation of excessive loss weighting.
  • 3) Evolution of sample relation matrix: Early in training, student and teacher relation matrices are poorly structured and differ little across perturbations; later, the learned relations become meaningful.The matrices and their absolute differences are visualized at 1, 10, 20, and 50 epochs under 20% labeled data.
  • 3) Evolution of sample relation matrix: At convergence, SRC produces cleaner absolute-distance matrices than Mean Teacher under two perturbations, indicating more stable sample relationships.The visualization compares student-model relation distances for Mean Teacher and the proposed approach at the 50th epoch.
  • Training behavior: As β ramps up, SRC and conventional consistency losses gradually decrease while classification performance increases and cross-entropy loss decreases.The training curves track classification loss, conventional consistency loss, SRC loss, and β.

5) Performance on other consistency-based frameworks:

The SRC paradigm generalizes consistency-based semi-supervised learning by enforcing consistency among sample relations, and improves other frameworks while avoiding overly strict direct feature regularization. The method remains subject to backbone, similarity-estimation, and classification-performance limitations.

  • 5) Performance on other consistency-based frameworks:: SRC can be incorporated into Π model and Temporal Ensembling, where it consistently improves AUC and F1 under 20% labeled data.This supports SRC as a general strategy across different consistency-based semi-supervised frameworks.
  • V. DISCUSSIONS: SRC regularizes the sample relation matrix R_l rather than directly constraining latent features A_l under perturbations.The design aims to preserve intrinsic sample relations while exploiting additional semantic information from unlabeled data.
  • V. DISCUSSIONS: Direct feature consistency is inferior to Mean Teacher at β = 1, marginally better at β = 0.1, and offers limited advantage at β = 0.01.The authors attribute the β = 0.01 result to classification and conventional consistency losses dominating training.
  • V. DISCUSSIONS: The relation is modeled using the inner product of two vectors, while alternative similarity estimators and stronger perturbation schemes remain future directions.The paper specifically mentions cosine similarity, instance relation graphs, and automatic data transformations as possibilities.
  • V. DISCUSSIONS: The method’s classification performance still leaves room for improvement, with relatively limited gains in Specificity on skin lesion classification.The authors propose exploring class re-balancing to improve discrimination between positive and negative samples.
  • V. DISCUSSIONS: The experiments cover both single-label and multi-label medical image classification, and the framework is intended for integration with other network designs.The authors used a basic DenseNet backbone and identify more accurate classification with other architectures as future work.

VI. CONCLUSION

The paper presents a self-ensembling semi-supervised framework with sample relation consistency to reduce labeling effort in medical image classification. Experiments on two benchmark datasets support its effectiveness across single-label and multi-label tasks and its compatibility with other semi-supervised approaches.

  • VI. CONCLUSION: The framework introduces SRC to exploit intrinsic relations between samples and better use unlabeled medical images.The conclusion frames SRC as the paper’s central addition to a semi-supervised self-ensembling framework.
  • VI. CONCLUSION: Experiments on two large-scale public benchmark datasets demonstrate superiority over state-of-the-art semi-supervised methods for single-label and multi-label classification.The reported evaluation establishes effectiveness across both task types.
  • VI. CONCLUSION: SRC is feasible to incorporate into other semi-supervised learning approaches.The conclusion presents this compatibility as a general property of the proposed strategy.
Loading 2005.07377v1…