PaperlayerView exact revision on arXiv

Source-linked AI summary

Merlin Plus: A Large-Scale, Multi-Cancer, Image-Mask-Report Dataset

Pedro R. A. S. Bassi, Wenxuan Li, Szymon Plotka, Ruby Honjol, Jakub Przado, Xinze Zhou, Kang Wang, Yang Yang, Malte Jensen, Akshay S. Chaudhari, Curtis P. Langlotz, Alan L. Yuille, Zongwei Zhou

arXiv:2609.36436v1cs.CV

TL;DR

Public CT tumor masks are scarce across organs, limiting multi-cancer segmentation. Merlin Plus adds radiologist-created masks and longitudinal metadata using report-based active learning, and its models outperform the original Merlin classifier and public models. The authors report a greater than 22% tumor-detection F1-Score improvement over the Merlin classifier, while noting low inter-reader agreement for the tumor types studied.

  • Problem

    Public CT datasets provide tumor masks for only a few organs, while cancers in other organs remain largely unaddressed by current AI research.

    Source: §1 · p. 2

  • Method

    Merlin Plus adds radiologist-reviewed tumor masks and longitudinal metadata, using report-based active learning to generate and prioritize masks for annotation.

    Source: §1 · p. 2

  • Results

    Merlin Plus segmentation models outperformed the Merlin classifier by over +22% in tumor detection F1-Score and surpassed state-of-the-art public multi-tumor segmentation models.

    Source: §5 · p. 9

  • Takeaways & Limitations

    AI-assisted analysis of routine CT could help identify suspicious tumors and prompt targeted examinations for confirmation.

    Source: §4 · p. 9

    Average inter-reader DSC was 53% for the studied tumor types, indicating difficulty identifying and delineating them on CT.

    Source: §4 · p. 7

Abstract

from arXiv · show

Multi-cancer segmentation in computed tomography (CT) is fundamentally limited by the scarcity of tumor masks across different organs. We present Merlin Plus, the first large-scale CT dataset with radiologist-created tumor masks across 9 organs. Merlin Plus extends the Merlin dataset by adding 1,153 per-voxel tumor masks and longitudinal metadata. To create these tumor masks, we developed a report-based active-learning framework in which radiology reports identify tumor cases for annotation and support training of a tumor segmentation model. The model generates initial masks, which radiologists review and correct to produce the final masks, reducing annotation burden while maintaining high-quality annotations. Besides tumor masks, the longitudinal metadata in Merlin Plus enables temporal modeling of cancer progression. By directly addressing the major bottleneck of limited multi-cancer segmentation masks, Merlin Plus supports scalable multi-organ cancer detection, segmentation, and longitudinal analysis in CT. Dataset is available at: https://github.com/MrGiovanni/MerlinPlus

1 Introduction

Merlin Plus addresses the scarcity of public CT tumor masks across organs by adding masks and longitudinal metadata, with report-based active learning supporting their creation. Models trained on Merlin Plus significantly surpassed public AI models and models trained on the original Merlin dataset.

1 Introduction

  • Public CT datasets mainly provide tumor masks for five organs, while cancers in other organs account for around half of global cancer mortality.The paper identifies this mismatch as a gap in current AI research.Source: §1 · p. 2
  • Merlin Plus adds 1,153 public tumor masks across nine organs, along with metadata that supports demographic and protocol audits and longitudinal analysis.The metadata includes age, sex, race, scanner types, de-identified patient IDs, and scan dates.Source: §1 · pp. 2–3
  • Report-based active learning uses a segmentation model to generate masks for radiologists to revise, then retrains the model with revised masks and reports.The workflow made mask creation 3 to 5× faster than manual annotation alone.Source: §1 · pp. 2–3
  • Merlin Plus segmentation models significantly surpassed public AI models and models trained on the original Merlin dataset.The dataset’s masks cover nine tumor types that are poorly represented in public CT datasets.Source: §1 · p. 3

2 Merlin Plus Dataset

Merlin Plus extends the Merlin CT-and-report collection with radiologist-verified tumor masks and metadata that identifies repeat scans for longitudinal analysis.

2 Merlin Plus Dataset

  • Merlin Plus adds 1,153 masks for tumors in nine organs to 25,494 CT scans and corresponding radiology reports.The masks include tumors reported as malignant in the nine organs, plus 325 benign tumors.Source: §2 · pp. 3–4
  • Patient IDs and scan dates identify repeat scans and intervals, supporting longitudinal analyses and development of longitudinal AI algorithms.Among 18,317 patients, 3,830 have at least two scans and 1,524 have at least three.Source: §2 · p. 3
  • Four radiologists manually verified and corrected the masks, which are three-dimensional and were created with report-based active-learning assistance.Source: §2 · p. 3, Figure 1 · p. 4

3 Report-based Active Learning

The report-based active-learning workflow initializes segmentation from reports, refines and scores generated masks against report descriptions, and directs radiologist review toward less reliable masks. Repeated cycles fine-tune the model using reports and corrected masks.

3 Report-based Active Learning

  • Report-supervised R-Super initialization addresses the cold-start problem by generating reasonable masks without requiring an existing tumor segmentation model.The team used three cycles, creating around 450 radiologist-revised masks in each cycle.Source: §3 · p. 4
  • In a 20-scan timing comparison, AI-assisted annotation averaged 3 versus 16 minutes for small tumors and 18 versus 52 minutes for larger tumors.The overall workflow accelerated annotation by 3 to 5×; assisted and unassisted order was alternated and reversed in a second round.Source: §3 · p. 4
  • Report refinement extracts tumor count, diameters, and locations, then restricts probabilities to reported organs and, when available, nearby reported slices.Organ masks are dilated by 2 cm to account for segmentation inaccuracies and tumors extending beyond organs.Source: §3 · p. 5
  • Sequential Ball Convolutions locate reported tumors by diameter, remove probabilities outside selected tumor regions, and enforce the reported tumor count.Previously selected regions are ignored to avoid locating the same tumor twice.Source: §3 · pp. 5–6
  • Report-refined masks are binarized to match reported tumor volume, using a 0.5 probability threshold when estimated volumes agree within 30%.Otherwise, the highest-probability voxels are selected until the reported volume is reached.Source: §3 · p. 6
  • The report-confidence score is the Dice coefficient between the soft mask and binary report-refined mask; low scores flag masks for review.A low score can flag a missed tumor that refinement placed at a likely location based on the report.Source: §3 · p. 6
  • Radiologists revise masks with the lowest report-confidence scores, and R-Super is fine-tuned using accumulated CT-report and corrected CT-mask pairs.Training with both reports and masks produced a more accurate segmentation than training with masks alone.Source: §3 · p. 6

4 Results

Merlin Plus-trained models substantially surpassed the original Merlin classifier and leading public tumor models in detection and segmentation. The reported inter-reader agreement also shows that the evaluated tumor types are difficult to delineate on CT.

4 Results

  • The evaluation compared two Merlin Plus segmentation models with a Merlin-trained classifier and public models, using internal and external validation.Detection metrics were sensitivity, specificity, and F1; segmentation was measured by DSC.Source: §4 · p. 7
  • Merlin Plus-trained models substantially surpassed the original Merlin classifier and public multi-tumor segmentation models in detection and segmentation.The comparisons include internal validation and external validation on a private dataset.Source: §4 · p. 7, Table 1 · p. 8
  • Average inter-reader DSC was 53%, compared with a reported median of 86% for pancreatic tumors, indicating difficulty identifying and delineating the studied tumors on CT.CT is not the primary detection modality for many of these tumor types.Source: §4 · p. 7
  • AI-assisted routine CT analysis could help identify suspicious tumors and prompt targeted examinations for confirmation.The paper notes that CT is performed at a far greater scale than modalities commonly used for some of these tumors.Source: §4 · p. 9

5 Conclusion

Training on multi-tumor masks yields better tumor segmentation and detection. Merlin Plus models surpass public multi-tumor segmentation models, and the dataset provides resources toward incidental cancer detection using CT scans.

  • Multi-tumor masks yield substantially better tumor segmentation and detection through precise per-voxel supervision that is more informative than classification labels.Source: §5 · p. 9
  • Over +22% in tumor detection F1-Score: a Merlin Plus segmentation model trained with reports and masks outperformed a Merlin-trained classification model.Source: §5 · p. 9
  • Merlin Plus models surpassed the state-of-the-art public multi-tumor segmentation models Voxtell, FLARE, and ULS.Source: §5 · p. 9
Loading 2609.36436v1…