PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Anomaly Detection for Image Dataset Quality Assurance in Multi-Center Breast MRI

Chiara Tappermann, Steffen Renisch, Lars Ole Schwen, Hans Meine, Horst K. Hahn, Eike Petersen

arXiv:2608.16725v1cs.CVcs.AI

TL;DR

Automated dataset-quality assurance for heterogeneous, multi-center medical imaging remains underdeveloped. This study benchmarks unsupervised anomaly and OOD detection on realistic breast MRI anomalies, finding that performance depends strongly on distribution shift and domain-specific adaptation, with reconstruction-based methods offering the best balance of detection and institutional transferability.

  • Problem

    Automated detection of data-level irregularities remains underdeveloped for heterogeneous, multi-center medical imaging datasets used in medical AI.

  • Method

    The study constructs a realistic anomaly benchmark and evaluates unsupervised anomaly and OOD detection methods for multi-center breast MRI dataset quality assurance.

  • Results

    Detection was mostly reliable for medium-far- and far-OOD samples, while near-OOD and external normal data exposed method-specific differences; reconstruction-based approaches best balanced performance and institutional transferability.

  • Takeaways & Limitations

    AD and OOD effectiveness depends on distribution shift, perceptual distance from in-distribution data, and the degree of domain-specific adaptation.

  • Takeaways & Limitations

    The 3D reconstruction-based approach required substantial architectural, objective, and input-dimensionality modifications to produce reliable anomaly scores for volumetric data.

Abstract

from arXiv · show

Corrupted, inconsistent, or anomalous data silently threatens the safety and reliability of medical AI. Despite growing regulatory recognition of dataset quality assurance (QA) for high-risk medical AI, scalable automated detection remains underdeveloped. We employ unsupervised anomaly detection (AD) and out-of-distribution (OOD) detection as an automated dataset QA mechanism for multi-center dynamic contrast-enhanced breast MRI. We build a controlled AD benchmark of 17 realistic QA-relevant anomaly types from six public datasets (protocol violations, processing errors, incorrect anatomical regions) and propose a taxonomy of radiological image anomalies based on human visual perception, enabling fine-grained analysis of AD failure modes. The benchmark includes near-, medium-far-, far-OOD samples, as well as in-distribution and external normal data. Four methods are evaluated: a projection-based method extended with a domain-specific feature extractor and a novel positional encoding, a reconstruction-based approach extended to full 3D volumes with an augmented training objective, and two unmodified hybrid OOD detection methods. Medium-far- and far-OOD samples are detected reliably, whereas near-OOD samples and external normal data from unseen institutions expose method-specific differences. The 3D reconstruction-based approach best balances detection performance (AUROC: 0.936) and generalization to unseen institutions. The projection-based method with positional encoding achieves the highest overall detection performance (AUROC: 0.954). Both hybrid methods exhibit critical failure modes, confirming that methods validated for one modality or anatomy may not generalize without domain-specific adaptation. Implants and mastectomies remain an open challenge for all methods. Our results establish a foundation and practical guidance on scalable unsupervised QA in medical AI pipelines.

1. Introduction

Corrupted or inconsistent data threaten the safety and reliability of medical AI, while scalable automated detection for dataset quality assurance remains underdeveloped. This study addresses the gap in multi-center breast MRI by benchmarking unsupervised anomaly and out-of-distribution detection, introducing a QA-focused anomaly taxonomy, and adapting methods to volumetric DCE MRI.

  • Corrupted, inconsistent, or anomalous data can propagate through medical AI pipelines and compromise model failures, patient safety, and clinical outcomes.
  • Regulatory quality criteria increasingly encompass datasets, but scalable methodological approaches for automatically detecting data irregularities remain underdeveloped.
  • Unsupervised anomaly and out-of-distribution detection learn normal in-distribution data to identify anomalous, corrupted, or distributionally inconsistent samples at scale.
  • The study constructs a controlled benchmark for dataset QA from six public datasets, covering clinically motivated anomaly types in multi-center breast imaging.
  • It proposes a four-dimensional QA anomaly taxonomy and adapts projection- and reconstruction-based methods for volumetric DCE breast MRI.The taxonomy covers protocol and modality, anatomical and structural alterations, orientation and field of view, and spatial extent; adaptations add domain-specific features, positional encoding, full volumetric processing, and an augmented training objective.

2. Literature Review

Existing medical-imaging QA methods are limited by poor scalability, reliance on labels, per-image or conceptual assessment, and insufficient sensitivity to semantic and distributional irregularities. Unsupervised AD/OOD methods offer a scalable alternative, but multi-center DCE breast MRI remains challenging because of substantial heterogeneity and limited QA-focused evaluation.

  • Metadata checks and histogram analysis detect limited low-level errors but miss semantic inconsistencies, subtle preprocessing errors, and image-content distribution shifts.
  • Manual review is sensitive to content irregularities but cannot scale to thousands of multi-center samples, while existing quality frameworks mainly assess per-scan acquisition quality or diagnostic usability.
  • Supervised quality-control pipelines automate assessment at scale but require expert labels and are generally restricted to a single modality and anatomical region.
  • METRIC provides a structured conceptual foundation for medical-AI data quality but does not supply algorithmic methods for detecting anomalous or inconsistent samples.
  • Unsupervised AD/OOD methods learn normal in-distribution data and detect deviations without annotated anomalies, yet existing applications focus mainly on diagnosis or industrial defects rather than multi-center QA.These methods support large-scale curation and deployment safeguards, but breast MRI studies have primarily framed lesions as diagnostic anomalies and have not addressed real-world data-level irregularities.

3. Material and Methods

The study designed an experimental pipeline to assess whether unsupervised anomaly- and out-of-distribution detection can provide scalable dataset quality assurance for heterogeneous, multi-center DCE breast MRI.

  • Study objective: The pipeline evaluates unsupervised AD and OOD detection as a potential scalable mechanism for dataset QA.The experimental design is presented in Figure 1.
  • Study setting: The target application is heterogeneous, multi-center dynamic contrast-enhanced breast MRI.
  • Experimental design: The experimental pipeline is explicitly designed to test the suitability of these detection methods for medical imaging dataset QA.Its structure is shown in Figure 1.

3.1. Anomaly Dataset

The anomaly dataset is designed for comprehensive, practical evaluation of unsupervised anomaly and OOD detection using realistic deviations from normal unilateral DCE breast MRI subtraction images. It separates homogeneous in-distribution normal data, external normal data representing unseen institutions, and diverse anomalies spanning modality, anatomy, protocol, structural, and preprocessing deviations.

  • Dataset design: The benchmark covers structurally diverse, realistic anomalies that may arise during medical-AI model training or inference.Its design supports comprehensive evaluation of AD and OOD detection failure modes.
  • Data definitions: Normal data comprise unilateral DCE breast MR subtraction images, excluding cases with implants or post-mastectomy anatomy.The exclusion creates a consistent, homogeneous definition of normality while treating these alterations as anomalous categories.
  • Data definitions: External normal data are acquired at institutions absent from training and validation, capturing domain shifts from protocols, scanners, and clinical practices.Method behavior on this set is evaluated separately from anomaly-detection performance to simulate unseen deployment institutions.
  • Anomaly construction: Anomalies include different imaging modalities, non-breast anatomical regions, incorrect imaging protocols, implants or mastectomies, and preprocessing artifacts or errors.These samples are defined as deviations from the normal data distribution along one or more dimensions.
  • Data sources: Six publicly available medical-imaging datasets provide the benchmark’s normal, external, and anomalous data sources.ODELIA supplies normal, external, and anomalous data; DUKE supplies external normal data; and TCGA-KIRP, TCGA-LIHC, PFMRIP, and QIN-BREAST supply anomalous data.
  • Preprocessing and splitting: All images were resampled to 256 × 256 × 32, while patient sides remained in the same split and diagnostic labels were represented in each split.External data included the first ten DUKE patients and first 15 ODELIA RSH patients after excluding mastectomy and implant cases.

3.2. Taxonomy of Anomalies

The paper introduces a modality- and task-independent taxonomy of radiological image anomalies grounded in human visual perception. It categorizes anomaly causes, perceptual impact, and OOD distance through four graded dimensions, with summed scores defining ID-to-far-OOD categories.

  • Framework: The taxonomy supports systematic evaluation of AD and OOD detection by categorizing error sources and perceptual distance from in-distribution data.It is designed to be independent of any specific imaging modality or clinical task.
  • Dimensions: Protocol and modality, anatomical and structural alterations, and orientation and field of view describe anomaly causes, while spatial extent describes their impact.The taxonomy therefore separates why an anomaly occurs from how extensively it affects the image.
  • Grading: Each dimension is graded from 0 to 2, spanning correct or expected data, protocol or structural deviations, and major modality, content, or geometric changes.Protocol grading distinguishes same-modality same-protocol data from same-modality different-protocol data and different modalities; anatomical grading distinguishes expected organ variation from altered or different content; orientation grading captures increasing misalignment or missing breast visibility.
  • OOD Categories: Summed grades classify anomalies as ID (0), near-OOD (1-2), medium-far-OOD (3-4), or far-OOD (≥5).The categorization is inspired by Graham et al. (2022).
  • Transferability: Applying the taxonomy to head CT anomalies provided an initial test of transferability across a different medical use case, anatomy, and modality.The comparison extended beyond the breast MRI setting.

3.3. Implementation and Experiments

The experiments evaluated four unsupervised anomaly- and OOD-detection approaches, extending projection- and reconstruction-based methods with medical, positional, training-objective, and volumetric adaptations. Two hybrid methods used VQ-GAN compression with transformer-based density estimation or DDPM denoising.

  • Approaches: Four approaches were selected: PatchCore as a projection-based method, reconstruction-based RA, and two hybrid OOD detectors originally developed for volumetric brain CT.The hybrid approaches combine discrete latent-space compression with either sequential density estimation or a Denoising Diffusion Probabilistic Model.
  • Projection-based approach: PatchCore was adapted with a medical foundation-model feature extractor and normalized positional encoding capturing consistent thorax and breast locations.The positional extension addresses PatchCore’s lack of global spatial context by incorporating anatomical priors from preprocessed axial images.
  • Reconstruction-based approach: RA was extended with L1, perceptual, and SSIM losses to produce sharper reconstructions for fine structures and sharp tissue boundaries.These additions guide the encoder and decoder without altering the underlying architecture.
  • Reconstruction-based approach: A 3D RA extension replaced two-dimensional operations with three-dimensional counterparts to incorporate volumetric context and detect anomalies beyond middle slices.The original 2D setup discarded information from other slices, preventing detection of anomalies located outside the middle slice.
  • Hybrid approaches: The hybrid detectors used VQ-GAN to compress three-dimensional volumes, followed by either transformer-based density estimation or DDPM-based latent denoising.The transformer models distributions over flattened latent sequences, whereas the DDPM learns iterative denoising across multiple noise levels.

3.4. Method Evaluation

The evaluation repeated all methods across five random seeds and used anomaly-group AUROC, aggregated AUROC metrics, and separate external-normal analysis. Threshold comparisons also distinguished fully unsupervised applicability from optimistic test-set estimates.

  • All methods were run five times with different random seeds on normal, external-institution normal, and diverse anomalous samples.
  • AUROC was computed separately for each anomaly group against normal data, alongside two aggregated AUROC metrics.The sample-weighted average AUROC measures overall performance across all anomalous samples against normal data.
  • External-normal AUROC was reported separately and excluded from both overall metrics because it assesses transferability rather than anomaly-detection performance.The target in this setting was AUROC 0.5, indicating indistinguishable score distributions between internal and external normal samples.
  • Percentile-based thresholding is fully unsupervised because it requires only normal validation data, unlike the ROC-based threshold.Both thresholds were calculated on the test set here, making their performance estimates optimistic rather than realistic.

4. Results

Detection was consistent for medium-far- and far-OOD anomalies, while near-OOD and external normal data revealed method-specific generalization differences. PC+PE achieved the highest overall performance, whereas reconstruction-based methods transferred best to external normal data, and implants and mastectomies remained challenging.

  • Near-Out-of-Distribution Anomalies: AUROCs above 0.9 were generally obtained for near-OOD orientation and field-of-view anomalies, while subtraction errors reached AUROCs ≥0.919 for three method families.The addition of PE substantially improved PC on near-OOD categories, whereas DDPM OOD performed substantially worse for subtraction errors.
  • Near-Out-of-Distribution Anomalies: Implant AUROCs ranged from 0.622 to 0.762 and mastectomy AUROCs from 0.281 to 0.835; PC+PE improved mastectomy detection from 0.679 to 0.835.The improvement was attributed to absent breast tissue producing larger distances to patch embeddings in the memory bank.
  • Medium-Far-Out-of-Distribution Anomalies: AUROCs consistently above 0.9 characterized medium-far-OOD detection across almost all methods, but DDPM OOD scored 0.157 for wrong subtraction order.DDPM OOD performed well on protocol- and modality-level anomalies but failed on this processing error.
  • Dimensions of Anomaly Taxonomy: All improved methods reliably detected globally affecting anomalies, but none detected same-organ anatomical alterations with grade = 1 combined with spatial extent = 1.The dataset contained no same-organ anatomical alterations with a global effect, so detectability of those cases could not be reported.
  • Overall Performance: 0.954 and 0.949 were the highest group-weighted and sample-weighted performances, achieved by PC+PE.Reconstruction-based approaches nevertheless demonstrated superior transferability to external normal data.

5. Discussion

The discussion emphasizes method-specific strengths and failure modes rather than a universally superior detector. Reconstruction-based methods offer stronger institutional transferability, while hybrid methods require domain-specific adaptation and remain vulnerable to important anomaly categories.

  • Method-Specific Strengths and Failure Modes: No single method is superior across all evaluation criteria; deployment context and tolerance for specific failure modes should guide method selection.Projection-based detection is straightforward to deploy and performs strongly on globally anomalous and stylistically distinct samples.
  • Method-Specific Strengths and Failure Modes: The 3D reconstruction-based approach best balances detection performance and institutional transferability by encoding global image structure rather than local patch statistics.This benefit required more careful adaptation than the projection-based approach.
  • Method-Specific Strengths and Failure Modes: Both unmodified hybrid methods underperformed the projection-based and reconstruction-based methods, showing that methods validated for one modality may not transfer without targeted adaptation.Their multi-stage pipelines also make failure diagnosis and correction more resource-intensive because errors propagate between stages.
  • Implant and Mastectomy Anomalies: Implant and mastectomy anomaly AUROCs remained consistently low across all methods, indicating insufficient sensitivity to their specific structural changes.Positional encoding partially improved mastectomy detection by exposing the spatial absence of breast tissue, but implants and mastectomies remain challenging.
  • Anomaly Taxonomy for Evaluation: The proposed anomaly taxonomy enables fine-grained interpretation of method differences by mapping anomaly types to perceptual distance from in-distribution data instead of relying only on aggregate AUROC.This structure is intended to support interpretation of observed failure modes and incorporation of new partner institutions.
  • Method-Specific Strengths and Failure Modes: External AUROC provides a practical diagnostic of how strongly an unseen institution deviates from the learned notion of normality for a selected detection method.Evaluating a small subset of normal samples from the new institution provides a direct, interpretable estimate of distributional separability.

6. Conclusion

The study establishes unsupervised anomaly and out-of-distribution detection as a dataset QA framework for multi-center breast MRI, supported by a benchmark of seventeen anomaly categories and a transferable visual-perception taxonomy. Results show reliable detection of medium-far- and far-OOD samples but method-specific limitations under near-OOD, institutional shifts, and clinically important implants and mastectomies.

  • Conclusion: The benchmark contains seventeen QA-related anomaly categories plus external and normal data from six public datasets.It provides a systematic foundation for evaluating anomaly and out-of-distribution detection as a QA mechanism for multi-center radiological imaging AI.
  • Conclusion: The four-dimensional grading-based taxonomy categorizes radiological image anomalies from human visual perception and supports identification of method-specific failure modes.Its transferability was demonstrated on an independent head CT anomaly dataset.
  • Conclusion: Medium-far- and far-OOD samples are mostly detected reliably, whereas near-OOD samples and external normal data produce substantial method-specific differences.The DDPM-based approach substantially underdetects most far-OOD samples, likely because it lacks domain-specific adaptation.
  • Conclusion: The proposed 3D reconstruction-based approach best combines detection performance with transferability to unseen institutions, while projection-based methods offer strong overall performance with minimal setup but greater domain-shift sensitivity.Both hybrid approaches perform substantially worse despite being designed for volumetric radiological image data.
  • Conclusion: Methods for image dataset QA must be tailored to each target modality’s normal distribution, and adaptation becomes more difficult as pipeline complexity increases.The two hybrid approaches with shared compression bottlenecks are particularly challenging to tune for new domains; implants and mastectomy cases remain an open challenge across all methods.

Ethics Statement

The study analyzed only publicly available, fully anonymized retrospective imaging data, with no new patient data collected and no additional ethical approval required.

  • Ethics Statement: The study used publicly available, fully anonymized retrospective imaging datasets and collected no new patient data.The datasets were ODELIA, DUKE, QIN-BREAST, TCGA-KIRP, TCGA-LIHC, and PFMRIP.
  • Ethics Statement: No additional ethical approval was required because the work used existing anonymized public data.Ethical approval and informed consent for original data collection were obtained by the respective contributing institutions.

CRediT authorship contribution statement

The CRediT statement assigns contributions across conceptualization, methodology, validation, writing, and project execution. Chiara Tappermann additionally contributed investigation, software, visualization, and the original draft.

  • Chiara Tappermann contributed conceptualization, investigation, methodology, software, validation, visualization, and writing the original draft.
  • Steffen Renisch contributed conceptualization, methodology, project administration, validation, and writing review and editing.
  • Lars Ole Schwen contributed conceptualization, methodology, validation, and writing review and editing, while Hans Meine contributed conceptualization and writing review and editing.
  • Horst K. Hahn contributed writing review and editing, and Eike Petersen is listed with conceptualization, methodology, and supervision.

A. Supplementary Material

The supplementary material documents test-dataset subgroups, illustrates normal, external, and anomalous data, and details the anomaly taxonomy and grading. It also reports anomaly-score distributions by anomaly and distribution category for the evaluated OOD methods.

  • Figure 5 provides a detailed overview of the different subgroups within the test dataset.
  • Figure 6 presents examples of normal, external, and anomalous data.
  • Table 3 reports the anomaly taxonomy and grading assigned to anomaly types defined by Graham et al. (2023) and Graham et al. (2022).
  • Figures 7–12 present anomaly-score distributions for the 3D RA, Transformer OOD, and DDPM OOD methods by anomaly and distribution category.
Loading 2608.16725v1…