Source-linked AI summary
BACH: Grand Challenge on Breast Cancer Histology Images
Guilherme Aresta, Teresa Araújo, Scotty Kwok, Sai Saketh Chennamsetty, Mohammed Safwan, Varghese Alex, Bahram Marami, Marcel Prastawa, Monica Chan, Michael Donovan, Gerardo Fernandez, Jack Zeineh, Matthias Kohl, Christoph Walz, Florian Ludwig, Stefan Braunewell, Maximilian Baust, Quoc Dang Vu, Minh Nguyen Nhat To, Eal Kim, Jin Tae Kwak, Sameh Galal, Veronica Sanchez-Freire, Nadia Brancati, Maria Frucci, Daniel Riccio, Yaqi Wang, Lingling Sun, Kaiqiang Ma, Jiannan Fang, Ismael Kone, Lahsen Boulmane, Aurélio Campilho, Catarina Eloy, António Polónia, Paulo Aguiar
TL;DR
BACH addresses limited clinical readiness in automatic breast histology analysis by organizing a public challenge with microscopy and whole-slide data. It evaluates competing algorithms across four-class classification and segmentation, finding 0.87 accuracy for microscopy classification and 0.69 score for whole-slide labeling, while identifying generalization, subtle-class discrimination, and interpretability as remaining concerns.
Problem
Whole-slide analysis is complex, and prior automatic methods had mainly addressed binary classification while broader clinical classification remained insufficiently explored.
Method
BACH organized a public challenge using annotated microscopy and whole-slide datasets, with CNN-based methods and fine-tuning among the principal approaches.
Results
0.87 accuracy was achieved for four-class high-resolution microscopy classification, and 0.69 score for labeling entire whole-slide images.
Takeaways & Limitations
Large-context inputs and appropriately fine-tuned CNN designs were associated with strong performance, while the dataset supports further research in digital pathology.
Takeaways & Limitations
The regional, relatively small dataset and small test set may limit generalization and evaluation across tissue structures and staining deviations.
Abstract
from arXiv · showhide
Breast cancer is the most common invasive cancer in women, affecting more than 10% of women worldwide. Microscopic analysis of a biopsy remains one of the most important methods to diagnose the type of breast cancer. This requires specialized analysis by pathologists, in a task that i) is highly time- and cost-consuming and ii) often leads to nonconsensual results. The relevance and potential of automatic classification algorithms using hematoxylin-eosin stained histopathological images has already been demonstrated, but the reported results are still sub-optimal for clinical use. With the goal of advancing the state-of-the-art in automatic classification, the Grand Challenge on BreAst Cancer Histology images (BACH) was organized in conjunction with the 15th International Conference on Image Analysis and Recognition (ICIAR 2018). A large annotated dataset, composed of both microscopy and whole-slide images, was specifically compiled and made publicly available for the BACH challenge. Following a positive response from the scientific community, a total of 64 submissions, out of 677 registrations, effectively entered the competition. From the submitted algorithms it was possible to push forward the state-of-the-art in terms of accuracy (87%) in automatic classification of breast cancer with histopathological images. Convolutional neuronal networks were the most successful methodology in the BACH challenge. Detailed analysis of the collective results allowed the identification of remaining challenges in the field and recommendations for future developments. The BACH dataset remains publically available as to promote further improvements to the field of automatic classification in digital pathology.
1. Introduction
Breast histology assessment remains demanding, motivating computer-aided analysis beyond binary classification toward multi-class microscopy and whole-slide interpretation. BACH advances this direction by combining four-class microscopy classification with whole-slide segmentation.
- Computer-aided diagnosis can complement pathologists because whole-slide analysis is difficult due to data volume and task complexity.
- Prior systems achieved strong binary classification performance, while the standard clinical multi-class procedure had only recently begun to be explored.
- Earlier four-class histology approaches reported approximately 68% and 78% accuracy on a difficult 36-image test set.
- Whole-slide analysis is challenging because gigapixel images contain diverse local tissue behavior and require methods beyond small-region processing.
- BACH expanded the benchmark with H&E microscopy-image classification and breast-cancer whole-slide-image segmentation.
2. Challenge description
BACH structured a two-part challenge around four-class microscopy classification and pixel-wise whole-slide labeling, using labeled training data and hidden independent test sets. Its evaluation combined microscopy accuracy with a custom whole-slide score designed to reflect class-distance errors and pathological-region detection.
- Challenge structure: BACH Part A classified microscopy images into Normal, Benign, In situ carcinoma, and Invasive carcinoma, while Part B produced pixel-wise labels for whole-slide images.
- Datasets: The microscopy dataset contained 400 training and 100 test images with the four classes equally represented.
- Evaluation: The microscopy test set used independent patients, supporting a fairer evaluation of generalization.
- Datasets: Part B provided 30 training and 10 testing whole-slide images, including pixel-wise annotated regions and additional potentially pathological unannotated slides.
- Evaluation: Microscopy performance was measured by overall prediction accuracy on 100 hidden test images.
- Evaluation: The custom whole-slide score penalized predictions farther from the ground-truth class and could ignore dominant correct Normal predictions while penalizing wrong Normal predictions.
3. Competing solutions
The competing-solutions section summarizes participant methods that met minimum performance thresholds and directs readers to detailed descriptions by challenge part. Joint methods and part-specific methods are covered in separate subsections.
- Tables 2 and 3 summarize methods achieving accuracy ≥0.7 on Part A and score ≥0.5 on Part B.
- Detailed approaches are organized into joint Part A and B methods, Part A-only methods, and Part B-only methods.
3.1. Introduction to Convolutional Neural Networks
CNNs dominated BACH because they offer strong medical-image performance with limited handcrafted expertise, but their many parameters create overfitting risks when training data are scarce. The section reviews architectures and transfer-learning practices used to address these challenges.
- CNNs were the predominant approach among BACH participants, especially for Part B.
- Limited medical-image datasets make CNNs prone to overfitting, motivating fine-tuning from networks trained on natural or medical images.
- VGG uses sequential convolutional blocks, max pooling, and fully connected layers to produce classifications.
- Inception reduces parameter counts through locally sparse blocks, parallel multi-scale convolutions, and global average pooling.The described 159-layer network has 23M parameters and supports different input sizes.
- Batch Normalization in newer Inception models addresses vanishing gradients by normalizing intermediate activation maps using training-batch statistics.
- DenseNet uses dense blocks that connect each layer to all preceding layers, addressing feature redundancy and gradient-vanishing issues.
- Fine-tuning replaces a pretrained network’s classification head with a randomly initialized task-specific head before adapting the model.
3.2. Part A
Part A approaches primarily used transfer learning with pretrained CNN ensembles or hierarchical architectures to classify four microscopy-image classes. Designs varied in normalization, spatial reduction, model configuration, and voting or probability aggregation.
- An ensemble of one ResNet-101 and two DenseNet-161 models used different normalization schemes and majority voting for final classification.Pretrained initialization was used to reduce the difficulty of learning from limited labeled data.
- A three-model ResNet ensemble with configurations 34, 50, and 101 classified each test image using the highest class probability.The approach reduced image complexity through downsampling and central-patch selection.
- VGG16 was fine-tuned after resizing and normalization, with random 224 × 224 crops and Sample Pairing augmentation during training.
- A hierarchy of three ResNeXt50 models first separated carcinoma from non-carcinoma, then classified the two classes within each group.The parent network was fine-tuned first, and its learned filters initialized the child networks.
3.3. Part
The WSI-focused approach described here used Candy Cane, a fully convolutional DenseNet-based segmentation network designed to preserve low-level information while respecting GPU-memory limits.
- Candy Cane uses an auto-encoder with downsampling and upsampling paths plus skip connections between corresponding feature maps.Its downsampling path is substantially longer than the upsampling path to account for GPU-memory restrictions.
3.4. Part A and B
Part A and B approaches combined patch extraction, transfer learning, ensembles, and encoder-decoder designs to handle microscopy and whole-slide images. Methods adapted preprocessing, validation, and prediction aggregation to each task’s data constraints.
- Part A and B: A two-stage method used clustered image origins for data splitting, extracted 1495 × 1495-pixel patches, and fine-tuned Inception-ResNet-v2.
- Part A and B: An ensemble of four modified Inception-v3 networks used adaptive pooling and cross-validation to improve robustness to small-scale changes.
- Part A and B: Predictions were aggregated across overlapping regions for Part A and sliding-window outputs for WSIs, with a ResNet34 refining the resulting WSI map.
- Part A and B: A pretrained DenseNet approach fine-tuned its fully connected layers before training the entire network for Part A.For Part B, it classified filtered 330µm × 330µm patches extracted from annotated WSIs.
- Part A and B: An encoder-decoder model used a dense-skip, group-convolution, dilated-convolution, and self-attention encoder for Part A classification.
- Part A and B: For Part B, the same model used a U-Net-like decoder with skip connections and initialized its encoder from Part A weights.
4. Results
BACH evaluated automatic breast histology analysis through microscopy classification and whole-slide segmentation, comparing submitted methods with expert annotations and statistical baselines. The results identified strong performance alongside challenging classes, acquisition-related errors, and methodological limitations.
- Challenge participation: 677 registrations produced 64 submissions across Part A and Part B of the BACH challenge.Part A received 51 submissions and Part B received 13.
- Part A results: Benign and in situ images were the most challenging classes for the top-10 Part A methods, with some images misclassified despite 100% inter-observer agreement.At least 80% of the top-10 approaches misclassified the highlighted examples.
- Inter-observer analysis: 94%, 78%, and 73% were the accuracies of the three external pathologists, compared with 96% for the BACH pathologist.The BACH pathologist repeated the annotation after one month to reduce influence from prior patient-exam knowledge.
- Part B results: Part B methods were evaluated with sensitivity, specificity, BACH score, and quadratic-weighted kappa, with invasive-region identification more successful than benign or in situ-region identification.Tables 6 and 7 report team-wise performance, and Figure 9 shows pixel-wise predictions.
- Statistical analysis: Part A comparisons used an adapted McNemar test, rejecting equal classifier performance when χ2 > 3.841 at p = 0.05.The test compares discordant errors: n01 counts samples misclassified by B but not A, while n10 counts samples misclassified by A but not B.
5. Discussion
BACH results indicate that carefully designed deep-learning systems can reach human-level performance on breast histology classification, while revealing important generalization, class-imbalance, and challenge-design limitations.
- Performance in Part A: Large-image training generally outperformed local-patch approaches, suggesting that tissue architecture and integrated local-global context aid breast-cancer discrimination.This pattern aligns with pathologists’ emphasis on tissue architecture.
- Performance in Part A: Benign and In situ cases remained difficult, with Benign especially confusable with Normal because benign lesions preserve normal tissue architecture and vary morphologically.These difficulties affected both submitted methods and human observers for Normal-versus-Benign distinctions.
- Generalization: Acquisition-related cues and evaluation design weakened generalization: most methods differed by about 10% between internal estimates and the independent test set.The authors associate this gap with patient-wise overfitting, staining patterns, and inadequately designed data splits.
- Performance in Part A: 87% was the highest Part A accuracy, matching the average expert accuracy of 85±10% and approaching the expected human-performance ceiling.The dataset was annotated by two experts, and dubious diagnoses were discarded.
- Performance in Part B: Part B was substantially harder than Part A: Invasive sensitivity averaged 0.4, whereas In situ sensitivity averaged 0.06 and reached at most 0.18.In situ recognition depends on pathological-cell location and therefore requires difficult global and local context integration.
- Diversity in the Solutions: BACH submissions were dominated by deep learning, whose high performance was accompanied by limited methodological novelty and black-box behavior that complicates clinical use.The authors recommend challenge designs encouraging methodological diversity and automatic explanations.
- Limitations: The regional, relatively small training set and relatively small test set may have limited generalization and restricted evaluation across tissue variability.This is identified as a limitation of the challenge dataset and evaluation scope.
6. Conclusions
BACH advanced breast cancer histology classification through a public challenge and dataset, with best methods reaching strong microscopy performance but lower whole-slide labeling performance. The collective analysis also identified experimental-design practices, persistent whole-slide difficulties, and directions for future CAD development.
- BACH organized a public challenge to advance computer-aided systems for automatic breast cancer histology image analysis.
- 0.87 accuracy was achieved for four-class high-resolution microscopy classification, while whole-slide image labeling reached a 0.69 score.
- Proper fine-tuning of modern CNNs, robust handling of small H&E color variations, patient-aware splitting, and large-context inputs were associated with strong performance.
- Current deep learning solutions still struggle with large, high-resolution images, motivating further development of whole-slide image analysis methods.
- The authors hope BACH’s analysis will motivate medical-imaging challenges and new breast cancer CAD methods supporting earlier detection.