PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated 2D and 3D Segmentation of AMD and DME Lesions in OCT

Lucia Sundberg, Zhihao Zhao, M. Ali Nasseri

arXiv:2608.27095v1cs.CV

TL;DR

AMD and DME OCT segmentation is rarely tested beyond its training dataset, leaving generalization across acquisition conditions unresolved. This work builds and ablates four 2D/3D pipelines, evaluates them in-domain and on ground-truth-free OLIVES using proxy metrics, and finds biomarker-tracking evidence that remains weaker than in-domain performance.

  • Problem

    Most OCT segmentation studies use a single dataset, leaving generalization to different acquisition conditions or patient populations unresolved despite AMD and DME causing vision loss.

  • Method

    The study systematically ablates four AMD and DME lesion-segmentation pipelines, then evaluates external generalization on OLIVES through biomarker AUROC, CST correlation, and longitudinal concordance proxies.

  • Results

    Dice scores reached 0.76–0.82 with strong volumetric and surface calibration, while OLIVES predictions tracked clinical biomarkers outside the training distribution but were generally attenuated.

  • Takeaways & Limitations

    Full-volume, calibration-aware evaluation identified ensemble composition as the most consistent improvement lever and supports cautious evidence for automated lesion-burden tracking.

  • Takeaways & Limitations

    OLIVES proxy metrics remain indirect substitutes because lesion-level ground truth and clinical outcome data were unavailable, so clinical validation remains unfinished.

Abstract

from arXiv · show

Age-related macular degeneration (AMD) and diabetic macular edema (DME) are leading causes of vision loss, and optical coherence tomography (OCT) is the standard modality for detecting and monitoring the subtle lesions that drive treatment decisions. Most deep-learning segmentation work for OCT is validated only in-domain, leaving generalization to clinical data collected under different acquisition protocols largely untested. This work develops and systematically ablates four lesion-segmentation pipelines -- 2D and 3D variants for AMD and DME -- reaching Dice scores of 0.76 to 0.82 with strong volumetric and surface calibration (r vol, r surf greater than or equal to 0.97 across all four pipelines) on an in-domain validation set. The ablation process establishes a full-volume, calibration-aware adoption standard that catches mechanisms an ordinary slice-level evaluation would keep, and identifies ensemble composition as the most consistent driver of improvement. To test generalization, the models are evaluated on OLIVES, an external clinical cohort with no lesion-level ground truth, using a proxy-metric framework built around biomarker AUROC, central subfield thickness (CST) correlation, and longitudinal concordance. Predictions track clinical biomarkers outside the training distribution, though less strongly than in-domain -- evidence for, not validation of, automated lesion-burden tracking as a clinical tool.

1 Background

AMD and DME cause major vision loss, while OCT enables rapid structural monitoring of their lesions. Automated segmentation is motivated by labor-intensive manual delineation and the uncertain generalization of models across acquisition settings.

  • AMD and DME are leading causes of vision loss, with untreated disease potentially progressing to irreversible impairment.AMD involves drusen and PED, whereas DME involves vascular leakage and IRF accumulation.
  • OCT is a high-resolution, non-invasive modality that produces cross-sectional images compilable into 3D volumes for lesion visualization.Accurate lesion segmentation supports disease-severity quantification and treatment-response monitoring.
  • Speckle noise, low tissue contrast, and hundreds of B-scans make manual lesion delineation time-consuming, variable, and impractical for routine use.These constraints motivate automated deep-learning segmentation approaches.
  • Most published OCT segmentation models are evaluated on single datasets, leaving their generalization to different acquisition conditions or patient populations uncertain.

2 Introduction

The paper compares 2D and 3D U-Net-based pipelines for AMD and DME lesion segmentation, emphasizing volumetric clinical relevance and external generalization. It evaluates unseen clinical data with proxy metrics because lesion-level ground truth is unavailable.

  • 3D segmentation is clinically relevant because disease severity and treatment response are typically assessed by lesion volume across the retina.Although 2D pipelines are more computationally efficient and more extensively validated, 3D models can use inter-slice context for subtle lesions.
  • The paper builds and compares 2D and 3D U-Net-based pipelines for AMD and DME, using extensive ablations to assess their tradeoffs.
  • External generalization is assessed on a clinical cohort using biomarker presence and clinical-label correlation as proxy metrics when lesion-level ground truth is unavailable.

3 Methods

The study develops four U-Net-based pipelines from AMD and DME OCT volumes, evaluates them at native resolution, and systematically searches training, architecture, inference, and ensemble choices. External volumes undergo preprocessing and full adopted-pipeline inference.

  • 3.1 Data: The dataset contains 62 AMD and 42 DME volumes split 80/20 into training and validation sets, with native-resolution processing and no held-out test set.The split yields 49/13 AMD and 33/9 DME training/validation volumes.
  • 3.2 Model Training: Both dimensionalities use U-Net-based encoder-decoder models trained with Adam and ReduceLROnPlateau scheduling.
  • 3.2 Model Training: The 2D pipelines use ResNet-34 ensembles, while the 3D pipelines use MONAI U-Net-family ensembles with sliding-window reconstruction for patch-based inference.
  • 3.3 Ablation Studies: Ensemble compositions and combination weights are selected through systematic searches over training and architectural variants.
  • 3.3 Ablation Studies: Ablations vary loss functions, sampling, z-axis receptive field, architecture, augmentation, regularization, post-processing, and ensemble composition.Candidate mechanisms are adopted only when they improve both Dice and calibration relative to baseline.
  • 3.4 External Evaluation: External OLIVES B-scans are cropped to the tissue band, resized to 512x512, grouped into near-complete visits, and processed through the adopted DME 3D ensemble.

4.1 Final Model Performance

The four final pipelines are evaluated on their own validation sets using slice- or volume-level Dice together with full-volume calibration measures. The reported examples span worst- to best-performing cases for both 2D and 3D settings.

  • Table 9 reports final segmentation and calibration performance for all four pipelines on their respective validation sets.Figures 2 and 3 show representative worst-, typical-, and best-performing 3D volumes and 2D slices.
  • Full-volume pooled Dice, rvol, and rsurf complement positive-slice 2D Dice and mean-over-volume 3D Dice in the final evaluation.
  • DME 3D reaches 82.47% mean Dice, exceeding Huang et al.'s 81.00% plain 3D U-Net baseline under a harder complete-volume evaluation protocol.AMD 3D reaches 76.23% and trails the cited 79.73% baseline; the comparisons are not directly equivalent.

4.2 Ablation Studies

The ablations evaluated mechanisms across four pipelines using raw full-volume Dice and calibration-aware adoption criteria. Ensemble composition, sampling, loss, architecture, and inference choices produced strongly pipeline-dependent outcomes.

  • Loss Functions: Loss sensitivity depended on disease and dimensionality: Dice-BCE helped 3D but regressed both 2D pipelines, while milder Tversky weighting helped AMD 3D and hurt DME 3D.Dice-BCE changed AMD 3D mean Dice from 0.4643 to 0.4793 and DME 3D from 0.3894 to 0.4959, but DME 2D fell from 0.7558 to 0.7379.
  • Sampling Strategies: Lesion-guaranteed-only sampling caused severe over-prediction, collapsing AMD 3D mean Dice to 0.12 and DME 3D to 0.10.The all-positive supervision interacted adversely with recall-heavy Tversky-BCE weighting.
  • Sampling Strategies: Elongation-aware oversampling improved AMD 3D mean Dice from 0.6477 to 0.6901 while improving DME 3D calibration rather than trading it off.The mechanism targeted failures associated with lesion z-extent.
  • Sampling Strategies: Small-lesion oversampling improved all four pipelines, raising AMD 3D mean Dice from 0.710 to 0.738 and improving or preserving calibration.It also increased AMD 2D full-volume Dice from 0.8214 to 0.8253 and DME 2D balanced-slice Dice from 0.7688 to 0.7730.
  • Architecture Variants: Attention gating was adopted only for DME 3D, increasing mean Dice from 0.8167 to 0.8242 and pooled Dice from 0.8349 to 0.8454 with calibration essentially unchanged.It was rejected or traded off against Dice in the other tested pipelines.
  • Architecture Variants: The cascade verifier was integrated into DME 3D after improving mean Dice from 0.8242 to 0.8260 and pooled Dice from 0.8454 to 0.8467 without changing calibration.Its adoption reflected general low-confidence component denoising on the in-domain validation set.

4.2.5 Augmentation

Augmentation, regularization, test-time augmentation, thresholding, post-processing, and ensemble design were judged by full-volume Dice and calibration. The final choices favored mechanisms that improved both axes, while ensemble averaging remained the dominant improvement source.

  • Augmentation: Removing the fuller augmentation recipe failed for AMD 2D, while Dice-only gains in DME 2D and AMD 3D failed secondary checks.The tested minimal recipe retained only horizontal flip and rotation.
  • Augmentation: The domain-shift augmentation recipe was rejected in all four pipelines because it failed the same in-domain Dice and calibration adoption bar.It widened brightness and contrast and added resolution degradation based on measured OLIVES differences.
  • Regularization: Weight decay was adopted for AMD 2D, increasing full-volume Dice from 0.8265 to 0.8282 with a clean win across tested axes and thresholds.It was also retained as a load-bearing DME 3D ensemble member but rejected in later AMD 3D and DME 2D comparisons.
  • Test-Time Augmentation: The z-flip TTA pass improved DME 3D mean Dice from 0.7981 to 0.8024 and pooled Dice from 0.8225 to 0.8285 with calibration essentially unchanged.It was rejected for AMD 3D because of calibration cost and for both 2D pipelines.
  • Thresholding: Decision-threshold tuning was adopted universally, with DME 3D mean Dice rising from 0.4999 to 0.7205 at t=0.95.The sweep corrected systematic volume over-prediction that morphological erosion alone did not resolve.
  • Post-Processing: The current DME 3D plausibility filter uses a size ceiling that exempts large components, improving in-domain performance while suppressing an OLIVES artifact pattern.The exemption was motivated by evidence that real large in-domain lesions rarely triggered the suppression criteria.
  • Ensembling: Ensemble probability averaging reduced large run-to-run variation and remained the project’s single largest improvement lever.Repeated identical AMD 3D configurations ranged from mean Dice 0.42 to 0.62, motivating variance reduction through averaging.

4.3 OLIVES validation

OLIVES evaluates generalization on an independent cohort without lesion-level ground truth, using biomarker and longitudinal proxies plus qualitative review. The proxies show signal, but artifacts and indirect evaluation limit validation strength.

  • External Cohort and Metrics: OLIVES was collected on different scanners and under different clinical protocols, making it an external test of generalization.Because lesion-level labels are unavailable, evaluation uses biomarker AUROC, CST/volume correlation, cross-visit concordance, and qualitative review.
  • Proxy Results: On TREX-DME, DME 3D reached IRF AUROC 0.7282 and cross-visit concordance 83.3% (30/36), while DME 2D reached AUROC 0.9072 and concordance 82.4% (14/17).DME 2D had weak, non-significant CST correlation: r(CST, pred vol) = 0.1883 (p = 0.45, n.s., n = 18).
  • Proxy Results: On disease-mismatched PRIME, IRF AUROC fell to 0.5583 for DME 3D and 0.8285 for DME 2D.These values were lower than the corresponding TREX-DME AUROCs of 0.7282 and 0.9072.
  • Guardrails and Artifacts: The envelope guardrail addressed anatomically implausible predictions that slice-level AUROC could not detect.Without it, predictions extended outside the retinal envelope despite biomarker-presence scoring.
  • Guardrails and Artifacts: A guardrail tracing bug erased predictions inside edema domes; an asymmetric-smoothness fix reduced total-erasure cases from 7.6% to 2.0%.The original smoothness penalty caused the traced ILM line to cut across steep dome elevations.
  • Artifact Investigation: Patients 203 and 204 exhibited a persistent thin-streak false-positive pattern that multiple geometry, image, and characterization tests failed to separate from real components.The investigation included filters, shadow removal, persistence checks, classification, vessel continuity, graded suppression, and ensemble disagreement.
  • Artifact Investigation: The final plausibility filter abandoned discrimination for large components, exempting those above a voxel-count ceiling and improving both in-domain Dice and OLIVES proxies.This choice followed evidence that large real in-domain lesions rarely met the suppression criteria.
  • Limitations: OLIVES remains an open validation effort because proxy metrics substitute for unavailable ground truth and can be muddied by domain-shift failure modes.The paper identifies a direct, mechanistically grounded relationship between predictions and clinical biomarkers as unfinished work.

4.4 PED Analysis

OLIVES PED analysis was constrained by a rare, confounded positive cohort and imperfect label alignment. AMD 3D failed at the standard threshold, while AMD 2D recovered coherent near-RPE signal only after lowering its threshold.

  • Data limitations: Only 10 serous-PED-positive slices from one patient were available, and all had substantial co-occurring pathology, preventing formal AUROC evaluation.The positives were also associated with preretinal tissue/hemorrhage and attached vitreous face; 6/10 had DME-related fluid.
  • Data limitations: OLIVES grades serous PED specifically, whereas training masks may include other PED subtypes, making correspondence strong but imperfect.Near-RPE activation cannot therefore be attributed uniquely to serous PED.
  • AMD 3D: AMD 3D detected almost none of the known-positive scans at the standard threshold, with mean predicted area of 0.5px and 9/10 scans exactly zero.A relative/local-threshold follow-up produced uniform areas and was diagnosed as a method artifact.
  • AMD 2D: AMD 2D produced nonzero predictions on 6/10 positive slices at the standard threshold, but areas ranged from 16-169px and 4/10 remained zero.Guardrail, representation-shape, and photometric-normalization fixes each yielded negative results.
  • AMD 2D: Lowering the threshold within the RPE-proximity guardrail recovered coherent signal in 8/10 positives, while 15 negative controls averaged only 1.7px.This supported anatomical specificity in a qualitative single-patient analysis.

5 Discussion

The four pipelines show strong in-domain segmentation and calibration, while external testing indicates weaker but clinically relevant biomarker tracking. The evidence supports automated lesion-burden tracking as a research direction, not clinical validation.

  • In-domain performance: Dice scores were 0.76–0.82 across pipelines, with strong volumetric and surface calibration; AMD 3D was modestly weaker than the other three.AMD 3D had mean Dice 0.7623, pooled Dice 0.7573, r vol 0.9723, and r surf 0.9927, versus pooled Dice 0.844–0.849 and r vol ≥0.988 for the other pipelines.
  • Ablation findings: Ensemble composition was the most consistent improvement driver, whereas several mechanisms that helped lesion-present slices degraded full-volume Dice or calibration.The adoption standard required gains in both segmentation accuracy and calibration, rather than relying on balanced or positive-slice evaluations alone.
  • External validation: DME 3D achieved biomarker AUROC 0.73 (IRF), Pearson correlation 0.57 with CST, and 83.3% longitudinal concordance in disease-matched TREX-DME data.External performance was more modest than the in-domain results.
  • PED analysis: AMD 2D showed anatomically specific serous-PED signal after lowering the threshold, but the finding relied on 10 positive slices from one patient and qualitative evidence.AMD 3D showed essentially no corresponding signal, and OLIVES provided no lesion-level ground truth.
  • External validation: DME 2D reached AUROC 0.91 (IRF) but had a much weaker, non-significant CST correlation on a smaller eligible sample, while PRIME performance was weaker.The external evaluation used biomarker and longitudinal proxy metrics because OLIVES lacks lesion-level ground truth.
  • Clinical interpretation: The study provides evidence for automated lesion-burden tracking, not validation, because proxy metrics replace direct lesion-level accuracy and clinical-outcome evaluation.The same limitation constrains interpretation of external and PED findings.

6 Conclusion

This work builds and ablates four 2D and 3D AMD and DME OCT lesion-segmentation pipelines, then evaluates external generalization without lesion-level ground truth. It contributes full-volume, calibration-aware model selection and proxy-metric external evaluation, while leaving direct clinical validation for future work.

  • Four AMD and DME pipelines achieved Dice scores of 0.76–0.82 with strong volumetric and surface calibration.
  • A full-volume, calibration-aware adoption standard rejected mechanisms that ordinary slice-level sweeps would have retained.
  • A proxy-metric framework enabled external evaluation where lesion-level ground truth was unavailable, but direct clinical validation remains unresolved.

Declarations

The declarations report funding, competing interests, ethics, consent, and data-availability information for this study.

  • The authors received no funding and declared no competing interests.
  • The study used publicly available, deidentified imaging datasets, with no human subjects collected as part of the work.
  • The training dataset was publicly available.
Loading 2608.27095v1…