Utilizing Automated Breast Cancer Detection to Identify Spatial Distributions of Tumor Infiltrating Lymphocytes in Invasive Breast Cancer

Ethics Statement

• SEER images were de-identified before receipt, so the NIH Office of Human Subjects Research Protection excluded the study from NIH IRB review; Stony Brook IRB classified the dataset as non-human-subjects research.Each contributing SEER registry had obtained approval from its respective institution.Source: Ethics Statement · p. 2
• The study used high-resolution WSIs from SEER and publicly available, de-identified TCGA images to train and evaluate models and generate cancer region maps.Source: Ethics Statement · p. 2
• Breast cancer detection datasets used patches from 102 SEER WSIs for training, 7 for validation, and 89 for testing; images were scanned at 40x and manually segmented by an expert pathologist using a web-based application.Source: Ethics Statement · p. 2
• The detection models were additionally evaluated on 195 TCGA WSIs manually annotated by Cruz-Roa et al..Source: Ethics Statement · p. 2
• The trained detection models were applied to 1,015 diagnostic WSIs from TCGA invasive breast cancer cases.Source: Ethics Statement · p. 2
• TIL classification models used training and validation data generated by Saltz et al., comprising 86,154 and 653 image patches, respectively.Source: Ethics Statement · p. 2
• The same 1,015 TCGA WSIs were analyzed using TIL models, which were evaluated on 327 patches from TCGA invasive breast cancer WSIs.The TIL dataset statistics are referenced in Table 2.Source: Ethics Statement · pp. 2–3

• Patches of 350 x 350 pixels at 40x magnification, equivalent to 88µm x 88µm, gave the best classification performance and were labeled by intersection with manually segmented cancer regions.OpenSlide extracted patches inside and outside the annotated regions as cancer-positive and cancer-negative examples, respectively.Source: Patch Extraction for Breast Cancer… · p. 3
• The researchers tested cancer-negative to cancer-positive patch ratios on the same validation dataset, motivated by prior work linking more negative samples to generalization and fewer false positives,,,.The final training, validation, and test datasets are presented in Table 1, whose values are not included in the supplied passages.Source: Patch Extraction for Breast Cancer… · p. 3
• The study adapted VGG16, Resnet34, and Inception-v4 for patch classification, resizing 350x350 tumor patches at 40x and 100x100 lymphocyte patches at 20x to each network’s required input size.For Resnet34 and Inception-v4, the output layer was changed from 1,000 to two classes. For VGG16, intermediate classification features were reduced from 4,096 to 1,024 and only the first four classifier layers were retained, reducing trainable parameters from 138 million to 41 million.Source: Patch Extraction for Breast Cancer… · pp. 3–4
• CNNs were first trained with classifier-only updates, then fine-tuned for 20 total epochs with all weights updated using stochastic gradient descent and cross-entropy loss.Cancer models used 3 classifier-only epochs and batch size 256; lymphocyte models used 5 epochs and batch size 128. Both used learning rate 0.01, momentum 0.9, and weight decay 0.0001.Source: Patch Extraction for Breast Cancer… · p. 4
• Each patch’s RGB channels were normalized, and training augmentation applied rotations, flips, and brightness, contrast, and saturation perturbations; test patches received normalization only.The augmentation was intended to reduce effects of stain and acquisition variability; model predictions assigned each patch a positive-class probability between 0.0 and 1.0.Source: Patch Extraction for Breast Cancer… · p. 4

Experiments

• The evaluation used accuracy, F1-score, and AUC to assess classification performance, including performance across prediction thresholds.F1-score combines precision and recall; AUC evaluates the relationship between true-positive and false-positive rates at different thresholds.Source: Experiments · p. 4
• The best models generated cancer-region and TIL prediction probability maps for 1,015 TCGA invasive breast cancer whole-slide images.Each slide was partitioned into patches assigned probabilities from 0.0 to 1.0; cancer patches were labeled positive or negative, as were TIL patches.Source: Experiments · p. 4
• A web-based application displayed the prediction probability maps as heatmaps for visualization and interaction.The passage points to Figure 2 for the application.Source: Experiments · p. 4

Post-processing Step for Cancer Heatmaps

• Because invasive cancer regions tend to be spatially close, the probability that a patch is cancer-positive is correlated with its neighbors, unlike independent patch classifications [49],.Source: Post-processing Step for Cancer Heatmaps · p. 4
• The method converts patch probabilities into a map and aggregates each patch over neighbors within a specified distance, assigning patches in the window the same score.The input map is H and the aggregated output map is A.Source: Post-processing Step for Cancer Heatmaps · p. 5
• Max aggregation with a 4x4 window performed best on Tseer and was applied to post-process TCGA predictions.Average, Median, and Max aggregation functions were explored.Source: Post-processing Step for Cancer Heatmaps · p. 5

• The RGB image stores quantized lymphocyte probabilities in red, cancer probabilities in green, and tissue-versus-glass background status in blue.Source: Combined Tumor-TIL Maps · p. 5

Software Support for Analysis Workflow

• QuIP and caMicroscope support data management and visualization, converting probability maps into heatmaps for interactive review alongside the underlying images.Whole-slide images can be several gigabytes each, making storage, indexing, and interactive analysis challenging.Source: Software Support for Analysis Workflow · p. 5
• PathDB indexes image and heatmap metadata and links them for retrieval, while browser-based FeatureMap displays low-resolution previews and enables detailed high-resolution examination.FeatureMap is designed to be lightweight enough to run on mobile devices.Source: Software Support for Analysis Workflow · p. 5

RESULTS Evaluation of Cancer Detection Models

• C-VGG16, C-Resnet34, and C-Incepv4 were compared with the original ConvNet and a ConvNet-ours implementation trained on the study’s datasets.The original ConvNet had been trained on HUP and UHCMC/CWRU; the authors reimplemented it in PyTorch following the original network description.Source: RESULTS Evaluation of Cancer Detection… · p. 5
• Post-processing raised ConvNet-ours average F1-score from 0.75 to 0.77 and PPV from 0.69 to 0.73; ConvNet-ours slightly outperformed original ConvNet across all metrics.Average F1-score was computed across 195 test images using thresholds from 0.0 to 1.0 in 0.01 steps; other metrics included NPV, TPR, TNR, FPR, and FNR.Source: RESULTS Evaluation of Cancer Detection… · p. 5, Table 4 · p. 6
• Visual inspection showed that C-Resnet34 probability maps detected and segmented cancer regions well in representative TCGA whole-slide images.Map shades of red indicated the model’s predicted probability that a patch was cancer-positive.Source: RESULTS Evaluation of Cancer Detection… · p. 6

Evaluation of Lymphocyte Classification Models

• Training on 2,912 invasive-breast-cancer patches yielded more accurate classifiers than training on the original 86,154 patches from multiple cancer types.The three models were L-VGG16, L-Resnet34, and L-Incepv4.Source: Evaluation of Lymphocyte… · p. 6
• The best models were applied to 1,015 TCGA invasive breast cancer slides to generate Tumor, TIL, and combined Tumor-TIL maps for public availability.Figure 5 shows example combined maps overlaid on whole slide images.Source: Evaluation of Lymphocyte… · pp. 6–7
• Combined maps visualize lymphocytes and tumor regions, including TILs and tumor-associated lymphocytes surrounding cancer regions.The authors describe these representations as information for further analyses.Source: Evaluation of Lymphocyte… · p. 7
• The new lymphocyte models consistently outperformed Saltz et al.’s model across performance metrics, with evaluation reporting F1-score, accuracy, and AUC.The supplied passages do not provide the metric values.Source: Evaluation of Lymphocyte… · p. 6, Table 5 · p. 7

Assessment of Inter-rater and Machine Versus Human Scoring of TIL Patches

• Three pathologists rated TIL content as low, medium, or high in 500 super-patches, each comprising an 8×8 patch grid.Human–human ordinal concordance was assessed with the polychoric coefficient, and machine-count versus human-ordinal concordance with the polyserial coefficient.Source: Assessment of Inter-rater and Machine… · p. 7
• Human–machine concordance was somewhat consistently better than human–human concordance, and machine predictions showed lower variability by confidence-interval width.Concordance generally improved when machine predictions were compared with the median score across pathologists rather than with individual experts.Source: Assessment of Inter-rater and Machine… · p. 7, Table 6 · p. 8
• Median machine-derived TIL scores were distinct across the low, medium, and high pathologist-assessed bins.Machine scores were counts of TIL-positive patches on a 0–64 scale.Source: Assessment of Inter-rater and Machine… · p. 7, Figure 4 · p. 8

Discussion

• In an example, TILs occupied approximately 20–30% of the cancer region, with a sparse multifocal distribution; maps were generated for 1,015 TCGA breast cancer slides.The cancer region occupied approximately 50–60% of total tissue area in the example.Source: DISCUSSION · p. 8
• Although TIL evaluation may predict treatment response, routine adoption remains limited despite straightforward H&E-based guidance from the International Immuno-Oncology Biomarker Working Group.The authors argue that scalable computational insight could ease pathologists’ time constraints, preserve diagnostic tissue, and avoid the additional costs of IHC for identifying TIL subsets.Source: DISCUSSION · p. 8
• The tool automatically maps breast cancer and TILs in H&E whole-slide images to quantitatively characterize their spatial relationship.The authors describe this as a response to subjective, laborious assessment and the heterogeneity of the breast tumor microenvironment.Source: DISCUSSION · p. 10
• Potential clinical uses include mapping intra- and peritumoral TIL patterns, improving tumor-boundary reporting, and informing treatment selection alongside molecular subtyping.The authors also propose screening for immunotherapy candidates, noting that such treatment may not be useful without substantial TIL amount and distribution.Source: DISCUSSION · p. 10
• The publicly available TCGA tumor–TIL dataset and interactive interface enable tumor-area TIL quantification, rapid prediction checks, and exploration alongside H&E slides.The authors also describe plans to characterize immune heterogeneity across tumor centers, invasive margins, and metastases, and to combine maps with other digital pathology analyses.Source: DISCUSSION · p. 10
• The authors report that open-source tools, datasets, and methods are available, and plan to refine lesion differentiation and prediction speed for potential real-time clinical use.They envision tumor–TIL maps supporting correlative studies in clinical trials, epidemiology, and surveillance.Source: DISCUSSION · p. 10