PaperlayerView exact revision on arXiv

Source-linked AI summary

Pos4CHD: A Positive-Free Mask-Guided Counterfactual Synthesis Framework for Fetal Congenital Heart Defect Analysis

Wenjie Xuan, Yapeng Li, Hang Su, Chen Cheng, Juhua Liu, Sheng Zhao, Ziming Huang, Bo Du

arXiv:2610.31139v1cs.CV

TL;DR

Rare fetal CHD positive cases are difficult to collect, and existing synthesis approaches require positive samples. Pos4CHD generates pseudo-positive images from manipulated semantic masks using FE-Gen. Experiments report improved image fidelity and anatomical consistency, alongside consistent improvements across downstream diagnosis tasks.

  • Problem

    Rare fetal CHD positives are difficult to collect because of low incidence and privacy constraints, while existing synthesis methods require positive samples and offer limited fine-grained anatomical control.

    Source: §I · p. 2

  • Method

    Pos4CHD manipulates normal fetal heart masks using clinical priors and uses FE-Gen to render corresponding pseudo-positive images.

    Source: §I · p. 2, B. Pos4CHD: A Positive-free… · p. 4

  • Results

    Experiments report consistent improvements on zero-shot CHD classification, lesion detection, and fetal heart anatomy segmentation.

    Source: §I · p. 2, §V · p. 9

  • Takeaways & Limitations

    The synthesized image-mask pairs show potential for fetal CHD diagnosis and analysis across classification, detection, and segmentation tasks.

    Source: §V · p. 9

Abstract

from arXiv · show

The scarcity of positive cases remains a major bottleneck in developing automated diagnostic systems for fetal Congenital Heart Defects~(CHDs), particularly for rare subtypes. Although normal cases are accessible with effort, positive cases are hard to collect due to low incidence rates and privacy concerns, causing severe class imbalance or absence of positive training samples. Existing works resort to synthesizing positives with conditional generative models, but they are task-specific and require positive training samples, hindering application for fetal CHDs. To this end, we propose a positive-free mask-guided counterfactual generation framework for fetal echocardiography, namely Pos4CHD, which enables realistic, high-quality synthesis of pseudo-positive samples from semantic masks, with only normal samples at hand. Technically, we first develop FE-Gen, a mask-guided generative model tailored for fetal heart ultrasound synthesis that mitigates histogram drift and enhances image-mask spatial alignment. Building on FE-Gen, Pos4CHD employs a mask-to-image generation paradigm for CHD synthesis. It first manipulates semantic masks to simulate fetal-heart structural anomalies and then renders anatomically consistent images. Extensive experiments on in-house datasets, together with expert evaluations, demonstrate the visual fidelity~($-12.97$ FID) and anatomical consistency~($+3.96\%$ DSC) of synthesized images. Further validations across diverse real-world CHD diagnosis tasks exhibit promising results and consistent increments, including zero-shot classification~(achieving $93.25\%$ ACC on EbA and $72.92\%$ ACC on TAPVC), lesion detection~($+6.95\%$ mAP50 on VSD), and anatomy segmentation~($+10.49\%$ DSC on fetal heart) compared with baselines. Codes and models are publicly available on \href{https://github.com/DREAMXFAR/Pos4CHD}{GitHub}.

I. INTRODUCTION

Fetal CHD analysis faces scarce high-quality data, especially for rare positive cases, while existing synthesis methods rely on positive samples or offer limited anatomical control. Pos4CHD uses mask-guided counterfactual synthesis to generate pseudo-positive samples from normal cases.

A. Medical Image Generation

  • Mask guidance provides dense structural control and fine spatial alignment for counterfactual synthesis of CHD anomalies.Medical image generation research also explores anatomical fidelity, controllable synthesis, and privacy preservation.Source: Figure 1 · p. 1, A. Medical Image Generation · pp. 2–3

I. INTRODUCTION

  • Rare fetal CHD positives are difficult to collect because of low incidence and privacy constraints, leaving downstream models with severe class imbalance and potential generalization issues.High-quality fetal echocardiography data are also limited by expert-dependent view acquisition and labor-intensive anatomical annotation.Source: §I · p. 2
  • Existing fetal CHD synthesis methods require positive samples and provide insufficient fine-grained anatomical control for diverse structural subtypes.Labels and prompts offer limited spatial control, while bounding boxes do not depict morphology adequately.Source: §I · p. 2
  • Pos4CHD manipulates semantic masks using clinical priors, then uses mask-guided synthesis to produce pseudo-positive image-mask pairs from normal samples.The framework targets zero-shot classification, lesion detection, and anatomy segmentation.Source: §I · p. 2

B. Ultrasound Image Generation

Prior fetal ultrasound generators support synthesis and augmentation, but Pos4CHD targets positive-free counterfactual synthesis of heterogeneous CHD subtypes. It combines rule-based mask manipulation with mask-guided ultrasound generation.

B. Ultrasound Image Generation

  • FetalFlex synthesizes multi-plane ultrasound using anatomical layouts and global prompts, while Pos4CHD adopts fine-grained mask-to-image synthesis for CHD analysis.The paper identifies precise spatial control and histogram-shift mitigation as its distinctions from FetalFlex.Source: B. Ultrasound Image Generation · p. 3

C. Counterfactual Synthesis

  • Pos4CHD targets subtype-specific fetal CHD synthesis from normal samples by combining rule-based mask manipulation with mask-guided ultrasound synthesis.Its semantic masks provide verifiable references for heterogeneous structural anomalies.Source: C. Counterfactual Synthesis · p. 3

1) Diffusion Models:

  • Diffusion models synthesize images by reversing a learned denoising process that starts from random Gaussian noise.The forward process progressively corrupts training images with Gaussian noise.Source: 1) Diffusion Models · p. 3
  • Stable Diffusion moves the diffusion process into a VAE-constructed latent space to reduce computational cost.The encoder maps images to latents and the decoder reconstructs images from them.Source: 1) Diffusion Models · p. 3

2) Spatially Controllable Image Generation:

Pos4CHD uses spatial conditions for image synthesis and constructs pseudo-positive fetal CHD samples through a sequence of mask collection, clinical-prior mask manipulation, image generation, quality filtering, and downstream use.

2) Spatially Controllable Image Generation:

  • ControlNet injects spatial conditions into Stable Diffusion features, with conditioning scale λ adjusting control strength at inference.The output feature is modulated by the spatial condition.Source: 2) Spatially Controllable Image… · p. 3

B. Pos4CHD: A Positive-free Mask-to-image Counterfactual Generation Framework for Fetal CHD Analysis

  • The framework collects normal masks, manipulates them into counterfactual masks, synthesizes pseudo-positive images, filters low-quality pairs, and uses the retained data downstream.Downstream uses include classification, detection, and segmentation.Source: Figure 2 · p. 4, B. Pos4CHD: A Positive-free… · p. 4
  • Pos4CHD uses semantic masks as interpretable anatomical representations that can be manipulated to simulate CHDs with fine-grained spatial control.Counterfactual masks also provide references for checking anatomical correctness and pixel-level supervision without extra annotation.Source: B. Pos4CHD: A Positive-free… · p. 4
  • Clinical-prior morphological rules, including translations, editing, and free-form deformation, transform normal masks into subtype-specific counterfactual masks.The study focuses on EbA, TAPVC, and VSD.Source: B. Pos4CHD: A Positive-free… · p. 4
  • FE-Gen renders realistic echocardiographic images aligned to counterfactual masks, and the resulting image-mask pairs can be converted into labels or boxes for analysis tasks.Candidate pairs are filtered manually or with pretrained segmentation models because stochastic sampling can yield artifacts or unfaithful CHD samples.Source: B. Pos4CHD: A Positive-free… · p. 4

C. FE-Gen: A Mask-guided Controllable Generation Model for Fetal Echocardiography

FE-Gen is designed to generate fetal ultrasound with both realistic appearance and precise alignment to conditioning masks. Its training data pair a large unlabeled 4CH-view collection with a smaller expert-annotated set.

C. FE-Gen: A Mask-guided Controllable Generation Model for Fetal Echocardiography

  • FE-Gen requires both image fidelity and image-mask alignment, since poor alignment can introduce noisy labels that degrade downstream performance and data efficiency.The model is intended to produce realistic fetal heart ultrasound with precise alignment to conditioning masks.Source: C. FE-Gen: A Mask-guided Controllable… · p. 5

1) Data Collection:

  • WHU-4CH-8k contains 7,974 fetal 4CH views from 2,855 patients collected between 2017 and 2023 for generator training.The data mostly cover gestational weeks 21–23 and 28–30, after plane quality checks.Source: 1) Data Collection · p. 5
  • WHU-4CH-Seg517 comprises 517 high-quality 4CH samples annotated by clinical experts for 14 key anatomical structures.These semantic annotations provide anatomical and spatial guidance for simulating CHD subtypes associated with those structures.Source: 1) Data Collection · p. 5

2) FE-Gen Architecture:

FE-Gen addresses histogram drift and image-mask misalignment with learnable semantic queries and histogram regularization for fetal ultrasound synthesis.

2) FE-Gen Architecture:

  • Learnable semantic queries and histogram regularization target FE-Gen’s two identified challenges: image-mask alignment and ultrasound fidelity.The queries replace plain color embeddings, while the regularization addresses histogram drift.Source: 2) FE-Gen Architecture · p. 5
  • Each anatomical category receives a learned query that is assigned to pixels by the category-index mask to construct the conditioning mask.The queries are jointly learned during diffusion-model training.Source: 2) FE-Gen Architecture · p. 5
  • Histogram regularization uses a sorted L1 loss between clean and estimated clean latents to mitigate histogram drift.The estimate is obtained from the current noisy latent and predicted noise; sorted pixel values are compared in order.Source: 2) FE-Gen Architecture · p. 5

3) Multi-step Training:

FE-Gen adapts Stable Diffusion to fetal ultrasound through VAE adaptation, SD-UNet fine-tuning, and mask-conditioned training.

3) Multi-step Training:

  • The three-step strategy adapts the SD-VAE, fine-tunes the SD-UNet, then trains FE-Gen for mask-guided synthesis.FE-Gen training uses image-mask pairs and the adapted VAE and UNet initialization.Source: 3) Multi-step Training · pp. 5–6
  • Fine-tuning the VAE decoder on WHU-4CH-8k uses reconstruction, perceptual, and GAN losses, while the encoder stays frozen.The paper reports improved performance after this adaptation in Table V.Source: 3) Multi-step Training · p. 6
  • SD-UNet is fine-tuned with LoRA on WHU-4CH-8k using denoising loss and histogram regularization, with a fixed fetal cardiac four-chamber prompt.The fine-tuned SD is reported to better match the real-data distribution and serve as a basis for controllable generation.Source: 3) Multi-step Training · p. 6
  • FE-Gen jointly optimizes its embedding module and semantic queries with the adapted SD components, and adds no inference cost over baseline ControlNet.Jointly fine-tuning SD-UNet LoRA with FE-Gen empirically yields better performance.Source: 3) Multi-step Training · p. 6

IV. EXPERIMENTS

The experiments use in-house fetal ultrasound datasets and evaluate synthesis in three CHD-analysis settings, with specified metrics and implementation choices.

1) Datasets:

  • WHU-4CH-8k comprises 6,466 training and 1,508 test images for fine-tuning SD-VAE and SD-UNet; WHU-4CH-Seg517 has 14 anatomical annotations and 417 training/100 test images for training FE-Gen.Source: 1) Datasets · p. 6
  • Evaluation covers zero-shot EbA and TAPVC classification, imbalanced VSD lesion detection, and fetal cardiac anatomy segmentation.The zero-shot test sets contain normal and positive cases but have no real positives available for training; VSD uses 28 positive training cases, and segmentation avoids test-image or mask use in generative training.Source: 1) Datasets · p. 6

2) Metrics:

  • Synthesis evaluation measures reconstruction, image fidelity, and image-mask alignment; downstream tasks use standard classification, detection, and segmentation metrics.Experiments are repeated three times or use 5-fold cross-validation.Source: 2) Metrics · p. 6

3) Implementation:

  • Experiments use PyTorch, four NVIDIA 3090 GPUs, SD v1.5 with ControlNet, and 512×512 images unless otherwise specified.The passage also specifies training settings and downstream models: ConvNeXt, YOLO-v5s, and nnUNet.Source: 3) Implementation · p. 6

B. Experimental Results

FE-Gen produces fetal-heart ultrasound images with strong fidelity and image-mask alignment, and Pos4CHD-generated samples support gains across CHD diagnosis tasks. Expert ratings also suggest the synthesized images are difficult to distinguish from real samples.

1) Image Generation:

  • 3.96% higher DSC than CtrlNet, competitive FID, and the lowest LPIPS and KL divergence indicate improved image-mask alignment and image fidelity.Clinicians’ average real-or-synthesized classification accuracy was 46.4%, near the 50% chance level.Source: 1) Image Generation · p. 7, Table I · p. 7, Figure 5 · p. 7

2) Downstream Tasks:

  • 93.25% ACC on EbA and 72.92% ACC on TAPVC were achieved, outperforming CtrlNet in zero-shot classification.Models trained entirely on synthesized data generalized to real test images and outperformed models trained with real negatives.Source: 2) Downstream Tasks · p. 7, Table II · p. 8
  • 6.95% higher average mAP50 than real data and 3.41% higher than CtrlNet were achieved for VSD detection; segmentation DSC rose 10.49% when synthetic data augmented 20 real samples.FE-Gen retained 66.2% of generated VSD samples after quality control versus 26.4% for CtrlNet; for segmentation it retained 57.5% versus CtrlNet’s 35.4%.Source: 2) Downstream Tasks · p. 8, Table III · p. 8, Table IV · p. 8

C. Ablation Studies

Ablations indicate that histogram regularization improves image fidelity, while learnable semantic queries improve image-mask alignment.

1) Effects of regularization Lhist:

2) Effects of learnable semantic query:

3) Effects of training strategy:

The experiments report benefits from multi-step fine-tuning, human filtering, and mask augmentation, while parameter discussions identify trade-offs and diminishing returns in synthesis quality.

3) Effects of training strategy:

4) Effects of human filtering & mask augmentation:

1) Discussions on regularization weight ωh:

2) Discussions on training data scales for FE-Gen:

3) Discussions on key synthesis hyperparameters:

4) Discussions on augmentation data scale:

Pos4CHD’s augmentation performance improves with more synthesized pseudo-VSD samples before saturating near ×3 scale; synthesized samples vary across structures and appearance, while some pseudo-VSD failures remain. The conclusion reports consistent improvements across real-world diagnosis tasks.

4) Discussions on augmentation data scale:

  • Pseudo-VSD augmentation performance improves with more synthesized samples and gradually saturates at approximately ×3 scale.The scale experiment uses all 417 WHU-4CH-Seg517 training masks to synthesize ×1 samples, then repeats synthesis to construct larger scales.Source: 4) Discussions on augmentation data… · p. 9

5) Discussions on synthesized sample diversity:

6) Analysis of failure cases:

  • Some pseudo-VSD failures leave the interventricular septum visually continuous despite the defect in the counterfactual mask.The paper attributes these failures to diffusion-sampling stochasticity and difficulty controlling subtle localized structures; precise synthesis remains future work.Source: 6) Analysis of failure cases · p. 9, Figure 14 · p. 9

V. CONCLUSION

  • Pos4CHD yields consistent improvements on real-world zero-shot classification, lesion detection, and anatomy segmentation tasks.The paper presents these results as evidence of the potential of synthesized data for automated fetal CHD diagnosis and analysis.Source: §V · p. 9

Related papers

Loading 2610.31139v1…