Source-linked AI summary
Fully Convolutional Architectures for Multi-Class Segmentation in Chest Radiographs
Alexey A. Novikov, Dimitrios Lenis, David Major, Jiri Hladůvka, Maria Wimmer, Katja Bühler
TL;DR
Accurate multi-class segmentation of anatomical structures in chest radiographs supports computer-aided diagnosis, while training on small datasets raises overfitting concerns. The paper evaluates fully convolutional architectures, including InvertedNet, using delayed subsampling, restrictive regularization, and high-resolution features; its models achieve strong overlap scores and compare favorably with state-of-the-art and human performance.
Problem
Accurate anatomical segmentation supports computer-aided diagnosis, but small chest-radiograph training sets can make overfitting probable because of large parameter spaces.
Method
The paper evaluates three fully convolutional architectures and introduces InvertedNet with delayed subsampling, restrictive regularization, reordered feature maps, and Dice-based training.
Results
InvertedNet achieves better segmentation results across all considered organs, with statistically significant improvements over the original U-Net and a 2.2% higher Jaccard overlap than the best state-of-the-art method.
Takeaways & Limitations
The proposed architectures match or surpass state-of-the-art methods across the considered tasks and outperform human observers for lung and heart segmentation.
Takeaways & Limitations
Experiments focus on 256 × 256 resolution for comparability, while training at 1024 × 1024 would require multiple GPUs or changing the architecture.
Abstract
from arXiv · showhide
The success of deep convolutional neural networks on image classification and recognition tasks has led to new applications in very diversified contexts, including the field of medical imaging. In this paper we investigate and propose neural network architectures for automated multi-class segmentation of anatomical organs in chest radiographs, namely for lungs, clavicles and heart. We address several open challenges including model overfitting, reducing number of parameters and handling of severely imbalanced data in CXR by fusing recent concepts in convolutional networks and adapting them to the segmentation problem task in CXR. We demonstrate that our architecture combining delayed subsampling, exponential linear units, highly restrictive regularization and a large number of high resolution low level abstract features outperforms state-of-the-art methods on all considered organs, as well as the human observer on lungs and heart. The models use a multi-class configuration with three target classes and are trained and tested on the publicly available JSRT database, consisting of 247 X-ray images the ground-truth masks for which are available in the SCR database. Our best performing model, trained with the loss function based on the Dice coefficient, reached mean Jaccard overlap scores of 95.0\% for lungs, 86.8\% for clavicles and 88.2\% for heart. This architecture outperformed the human observer results for lungs and heart.
I. INTRODUCTION
Chest radiographs are central to diagnosis but pose difficult segmentation problems because of ambiguous boundaries, anatomical variation, artifacts, and severe class imbalance. The paper motivates fully convolutional, multi-class approaches tailored to CXR anatomy and limited medical datasets.
- Chest radiographs are among medicine’s most common diagnostic images and help solve 70-80% of diagnostic questions.
- Accurate anatomical boundaries support computer-aided diagnosis, including measurements related to lung disease and cardiomegaly.Lung shape, size, and area provide diagnostic clues, while heart segmentation supports cardiothoracic-ratio measurement.
- CXR segmentation is difficult because anatomical variation, organ overlap, artifacts, and fuzzy or poorly visible boundaries complicate interpretation.
- 73.53% of pixels in the SCR database’s JSRT annotations belong to the lung class, severely outweighing the clavicle class.
- Large medical-data requirements make many standard semantic-segmentation approaches impractical, motivating finely tailored strategies.
- The paper adapts fully convolutional segmentation ideas and U-Net information-flow concepts to improve challenging clavicle and heart segmentation.
B. Contributions
The paper contributes a multi-class CXR segmentation approach and evaluates architectural and training choices for lungs, clavicles, and hearts. Its best architecture reduces parameters while maintaining or improving segmentation performance and statistical competitiveness.
- The proposed multi-class approach segments lung fields, clavicles, and hearts in chest radiographs.
- Dice-coefficient training produces higher overlap scores than cross-entropy training.
- ELUs instead of ReLUs can speed training and achieve higher overlap scores.
- Around ten times fewer parameters than U-Net preserves accuracy and improves segmentation across all considered organs.The largest improvements are reported for clavicles.
- Overlap scores are comparable to or surpass state-of-the-art techniques and human performance across the considered tasks.The paper specifically includes challenging heart and clavicle segmentation cases.
- The best architecture is reported as the fastest multi-class CXR segmentation approach to date.The authors associate this with efficient processing of large amounts of data.
II. METHODOLOGY
The methodology formulates CXR organ segmentation as multi-class pixel labeling with mutually exclusive semantic classes, trains fully convolutional networks on minibatches, and evaluates their multi-label outputs on held-out images.
- The dataset is represented as images with pixel intensities and corresponding binary ground-truth masks for semantic classes.
- A multi-class ground-truth matrix assigns each pixel a semantic label, assuming that each pixel belongs to exactly one class.
- Projection mappings recover the binary mask for a selected semantic class from the multi-class ground-truth matrix.
- The image dataset is split into three non-overlapping sets: training, validation, and test.
- During training, minibatches from the training set pass through the network to produce per-pixel semantic-class probabilities.
- A loss function measures deviation from ground truth and updates network parameters until stopping criteria are met.
- At testing time, unseen images produce multi-label outputs whose channels correspond to clavicles, heart, and lung fields.
B. Improvements to U-Net Model for Chest Radiographs
The paper modifies U-Net architectures to address overfitting, excessive parameters, uninformative features, and imbalanced organ representation in chest-radiograph segmentation.
- All-Dropout: All-Dropout places Gaussian dropout after every convolutional activation to impose restrictive regularization in deep networks trained on limited data.The drop probability is denoted by d.
- All-Convolutional: All-Convolutional replaces each pooling layer with a convolution using the pooling layer’s filter size.This adaptation follows the All-CNN-C design.
- InvertedNet: Delayed subsampling and reordered feature-map widths define InvertedNet, reducing expressivity while emphasizing high-resolution low-level features.Feature maps decrease in depth through contraction and increase after upsampling.
- Class imbalance: Class weighting in cross-entropy and Dice losses addresses severe pixel-count imbalance between organs such as clavicles and lung fields.Weights are based on class pixel counts in training batches.
- Training objectives: The study evaluates cross-entropy and Dice distance functions with softmax outputs for cross entropy and sigmoid outputs for Dice.The output formulation differs because the losses operate on different domains.
A. Training Details
Experiments use the JSRT chest-radiograph dataset and SCR reference masks, with two image resolutions and distinct ground-truth constructions for Dice and cross-entropy training.
- Dataset: The JSRT dataset contains 247 posterior-anterior chest radiographs at 2048 × 2048 resolution, with SCR organ boundaries provided at 1024 × 1024.Reference masks cover both lungs, the heart, and both clavicles.
- Experimental setup: Networks were trained at 128 × 128 and 256 × 256 resolutions to enable comparison with state-of-the-art methods.The same train, validation, and test subsets were used across evaluations.
- Ground truth: Dice training uses Gdice with three organ classes, whereas cross-entropy training uses Gentropy with four classes including background and lungs without clavicles.Gdice permits overlapping pixel labels; Gentropy was used only for training in this study.
- Preprocessing: Images were zero-centered using the training-set mean and normalized using the training-set standard deviation.
- Optimization: ADAM training used an initial learning rate of 10^-5 with β1 = 0.9 and β2 = 0.999.Larger learning rates produced less stable training and sometimes local minima or trivial solutions.
B. Performance Metrics
Performance is evaluated with Dice and Jaccard similarity coefficients, supplemented by symmetric mean absolute surface distance for the best architecture.
- Overlap metrics: Dice and Jaccard similarity coefficients measure segmentation overlap against ground-truth masks.
- Pixel sets: For an image, ground-truth mask, and feature channel, Pl(I) denotes pixels where the model is sufficiently certain that they are not background.The set is defined using a threshold ε on the predicted probability.
- Surface distance: Symmetric mean absolute surface distance Sd was computed for the best-performing architecture.The paper attributes the definition to Babalola et al.
C. Implementation Details
Implementation uses Keras with a Theano backend, zero-padding, convolutional output layers, and scikit-image mask downsampling.
- Software: Experiments were implemented in Keras with Theano as the backend and Python 3.5.The backend handled automatic differentiation and optimization during training.
- Network implementation: Zero-padding preserves output-channel dimensions, while convolutional final layers reduce parameters compared with dense layers.The number of output maps matches the considered classes, with one additional background map for cross entropy.
- Mask processing: Original SCR masks were downsampled using the scikit-image Python library.
IV. RESULTS AND DISCUSSION
The proposed architectures were compared with U-Net across resolutions, training behavior, feature representations, and ensemble configurations. InvertedNet generally achieved the strongest results, particularly for clavicle segmentation, with sharper borders and slower but ultimately superior convergence.
- Performance comparison: 8.5% higher Jaccard overlap than U-Net was achieved by InvertedNet for clavicles at 128 × 128 resolution.The lower resolution was especially challenging for U-Net and related architectures.
- Feature analysis: InvertedNet features favored sharper borders over shape separation, and its larger pre-classification feature-map count yielded higher overlap scores.Feature visualizations used the same test image for All-Dropout, All-Convolutional, and InvertedNet.
- Training behavior: InvertedNet started extremely slowly during training but ultimately achieved the best validation result.U-Net initially improved faster, then plateaued and oscillated, whereas better-regularized models reached similar or higher final scores.
- Performance comparison: InvertedNet had the most test samples in the highest Jaccard-score interval for clavicle segmentation.The distribution was computed using three-fold cross-validation scores on the testing set.
- Statistical analysis: Statistical tests supported selecting InvertedNet as the winning model, with significant differences versus other architectures for lungs and clavicles.All-Dropout also showed significant differences for lungs, clavicles, and heart, but the authors selected InvertedNet based on both evaluations.
B. Multi-class segmentation with loss function based on Dice coefficient
The Dice-based evaluation examined InvertedNet across training splits and weighted versus non-weighted losses. Class weighting remained important under severe imbalance, while Dice loss improved clavicle performance over cross entropy.
- Experimental design: InvertedNet performance was evaluated across several training and validation splits using weighted and non-weighted Dice-based losses.The same testing set was used across the training runs.
- Experimental design: The evaluation tables reported InvertedNet results for three data splits with a fixed testing subset containing 33% of the JSRT images.The split-specific comparisons were performed for weighted and non-weighted Dice losses.
- Class imbalance: Class weighting remained important for Dice loss in the presence of severe between-class pixel imbalance.The dominant class slightly benefited from using the non-weighted loss.
- Loss-function comparison: More than 4% improvement in clavicle final score was obtained with Dice loss compared with cross entropy.The comparison was made for the InvertedNet evaluation under severe class imbalance.
C. ReLU vs ELU activation functions
Replacing ReLUs with ELUs improved InvertedNet overlap scores across lungs, clavicles, and heart. The resulting model outperformed the human observer on lung and heart segmentation while remaining computationally expensive to train.
- Activation functions: ELU activation produced a clear increase in overlap scores for all three organs, with the largest improvements for clavicles and heart.The comparison used InvertedNet evaluations across three training and validation splits with Dice-based loss.
- State-of-the-art comparison: InvertedNet exceeded the human observer on lung segmentation and slightly surpassed the human observer on heart segmentation.Its clavicle score remained below human performance while significantly exceeding state-of-the-art results.
- State-of-the-art comparison: 2.2% higher Jaccard overlap than the best state-of-the-art method was achieved by InvertedNet for lung segmentation.It also slightly exceeded the human observer for heart segmentation but did not surpass human performance for clavicles.
- Computational cost: 33 hours were required to train InvertedNet, compared with 12.4–14.5 hours for the other listed architectures.The longer training time was attributed to its larger number of high-resolution features despite fewer parameters.
- Clinical relevance: The authors describe the method as the fastest multi-class CXR segmentation approach to date and note its potential benefit in large clinical environments.The stated benefit concerns settings screening hundreds or thousands of people daily.
F. Limitations
The evaluation is constrained to 256 × 256 JSRT images, while higher-resolution training remains computationally demanding. The approach also leaves shape-intrinsic plausibility insufficiently captured by Dice overlap alone, with additional shape-prior regularization deferred to future work.
- Resolution and computational scope: 256 × 256 resolution was used for comparability, although 1024 × 1024 training would require multiple GPUs or fewer feature maps.Using fewer feature maps would change the Original U-Net architecture and compromise comparison across resolutions.
- Architectural constraint: Pooling-based contraction can lose location-specific information and favor local image context over intrinsic shape.This trade-off motivates concerns about whether learned features preserve anatomically meaningful structure.
- Evaluation limitation: Dice overlap alone does not ensure plausible structures with sufficient shape-intrinsic information.The paper identifies stronger shape-prior regularization or alternative loss objectives as possible remedies, but leaves them for future work.
- Architectural constraint: The altered-image examples indicate that InvertedNet classifies clavicle pixels from neighborhood context rather than intrinsic shape information.This behavior is observed even when the images retain lung fields and surrounding padding.
- Future work: Future improvements are expected to require more training data and transfer-learned features from other chest-radiograph segmentation tasks.This scope boundary is stated alongside the paper’s conclusion about improving segmentation performance.