Deep Multi-Magnification Networks for Multi-Class Breast Cancer Image Segmentation

• Pathologists assess surgical margins by reviewing roughly 20–40 slides per case, a time-consuming process relevant to determining whether additional surgery is needed.Source: §1 · p. 2
• Single-magnification patch methods have a narrow field of view and do not use lower-magnification morphological features.Source: §1 · p. 4
• Higher-magnification patches capture cellular details, while lower-magnification patches show tissue distribution and architectural growth patterns.Source: §1 · p. 5
• The proposed DMMN consistently outperformed other architectures on two breast datasets for multi-subtype breast-tissue segmentation.Source: §1 · p. 6
• DMMN combines multiple encoders, decoders, and intermediate feature-map concatenations, and is trained using partial WSI annotations to reduce annotation burden.Source: §1 · p. 6

2. Proposed Method

• The pipeline partially annotates training slides, extracts patches, balances rare classes with elastic deformation, trains a segmentation CNN, and applies it to breast cancer images.Annotations cover C classes; elastic deformation multiplies patches from rare classes to balance annotated pixels.Source: §2 · p. 7

2.1. Partial Annotation

• Entire subtype components are annotated while unlabeled boundary regions are narrowed, retaining spatial transitions and growth patterns while reducing exhaustive labeling effort.An experienced pathologist can annotate one WSI in approximately 30 minutes.Source: §2.1 · p. 8

2.2. Training Patch Extraction

• Three 256 × 256 input patches at 20×, 10×, and 5× magnifications are extracted around each target region to provide the DMMN with broader context.Training inputs are extracted when more than 1% of target-patch pixels are annotated; target patches advance by 256 pixels to avoid overlap.Source: §2.2 · pp. 9–10

2.3. Class Balancing

• Elastic deformation multiplies minority-class patches to balance annotated pixels; mixed-class patches are assigned to the class with the higher class order.The per-class multiplication rate is a non-negative integer, and a rate of zero skips deformation for that class.Source: §2.3 · pp. 10–11

2.4. CNN Architectures

• DMMN-M3 uses multiple encoders and decoders, concatenating lower-magnification decoder features into the 20× decoder at intermediate layers.The compared alternatives include single-encoder/single-decoder and multi-encoder designs with either one decoder or final-layer-only concatenation.Source: §2.4 · pp. 12–13

2.5. CNN Training

• The CNNs are trained with weighted cross-entropy; unlabeled pixels are excluded, and the model with the highest validation-image mIOU is selected.Optimization uses SGD for 20 epochs, with random rotation, flips, brightness, contrast, and color jittering as augmentation.Source: §2.5 · pp. 13–14

2.6. Multi-Class Segmentation

• At inference, the trained CNN predicts each target patch from 20×, 10×, and 5× inputs, and the highest-scoring class becomes each pixel’s label.Patches cover the WSI with a 256-pixel stride; Otsu thresholding may optionally remove background regions before processing.Source: §2.6 · pp. 14–15

3. Experimental Results

• Evaluation used 32 Dataset-I slides for training and validation, then tested on 55 Dataset-I and 34 Dataset-II images.Source: §3 · pp. 15–16
• Only 5.34% of training-slide pixels were annotated; elastic deformation balanced class representation, and the models predicted six tissue classes.Source: §3 · pp. 15–16, Figure 5 · p. 17
• Dataset-II evaluation excluded necrotic, adipose, and background classes because necrotic regions were scarce and Otsu processing left most adipose and background unsegmented.Source: §3 · p. 18
• On Dataset-I, DMMN-M3 predicted all subtypes accurately and achieved the best mIOU, mRecall, and mPrecision.Source: §3 · pp. 21, 23, Table 3 · p. 25
• On held-out Dataset-II, DMMN-M3 produced smoother, clearer boundaries than alternatives, while DMMN-M2S falsely labeled large regions as benign epithelium.Source: §3 · p. 23
• On Dataset-II, DMMN-M3 mislabeled well-differentiated carcinomas absent from Dataset-I; excluding five such cases, it outperformed other methods on higher-grade cases across mIOU, mRecall, and mPrecision.Source: §3 · p. 25, Table 4 · p. 25

4. Conclusion

• Partial annotation reduced annotator time and effort while the model learned within-class spatial characteristics and relationships between classes.The passage describes the model as trained on partially annotated images.Source: §4 · p. 26
• Dataset-II confusion matrices exclude necrotic, adipose, and background pixels because few such pixels were evaluated.The comparison includes two single-magnification and four multi-magnification network architectures.Source: §4 · p. 27
• The authors report outstanding segmentation performance on breast datasets with their DMMN implementation.The architecture uses 20×, 10×, and 5× magnifications, transferring intermediate features from lower-magnification decoders to the 20× decoder.Source: §4 · p. 27
• Well-differentiated carcinomas were absent from training annotation, and background noise from other scanners may lead to background-region mis-segmentation.The authors identify both as challenges and plan to include varied cancer structures and background noise patterns in future training.Source: §4 · p. 27

6. Conflict of interest

• T.J.F. is Paige.AI’s Chief Scientific Officer, co-founder, and equity holder.Source: §6 · p. 28
• M.G.H. consults for Paige.AI and serves on Path-Presenter’s medical advisory board.Source: §6 · p. 28
• D.J.H. and T.J.F. have relevant intellectual-property interests, and MSK has financial interests in Paige.AI.Source: §6 · pp. 28–29