PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning from Scarce Labels: Multi-View Echocardiography for Ejection Fraction Prediction

Zhiyuan Gao, Dominic Yurk, Yaser S. Abu-Mostafa

arXiv:2609.02969v1eess.IVcs.CVcs.LG

TL;DR

Public PLAX–EF supervision and reproducible benchmarks are scarce, despite PLAX being routinely acquired and useful when apical views are difficult. The paper constructs a public resource using view mining, note-derived labels, and A4C-based proxy supervision, achieving 6.86% MAE for PLAX and 6.37% MAE with A4C–PLAX late fusion.

  • Problem

    Public PLAX–EF supervision and reproducible benchmarks are scarce, despite PLAX being routinely acquired and useful when apical views are difficult.

  • Method

    The pipeline mines views, correlates echocardiographic studies with clinical notes, and uses A4C predictions as proxy labels for PLAX training.

  • Results

    6.86% MAE: the PLAX ensemble achieves this study-level error, while A4C–PLAX late fusion reaches 6.37% MAE on studies containing both views.

  • Takeaways & Limitations

    PLAX-only estimates are meaningful, and multi-view integration offers a practical way to improve upon A4C-only prediction.

  • Takeaways & Limitations

    Proxy supervision is affected by A4C teacher noise and domain shift, while view-classification and note-alignment errors can introduce additional label noise.

Abstract

from arXiv · show

We present, to the best of our knowledge, the first publicly available resource for predicting left ventricular ejection fraction (EF) from parasternal long-axis (PLAX) echocardiography. Because no PLAX-EF datasets previously existed, our work focuses on an innovative data generation strategy to overcome this scarcity. By leveraging a time-based correlation between clinical notes and echocardiographic videos, combined with fine-tuning view classifiers and proxy labeling, we created a labeled dataset of over 25,000 PLAX videos. This enables us to train the first reproducible PLAX EF model, achieving a mean absolute error (MAE) of 6.86%. Given that apical four-chamber (A4C) methods, the clinical standard, report MAE values of 6%-7%, our results demonstrate that EF estimation from PLAX views is both feasible and clinically relevant. This surpasses the performance of existing methods and provides a clinically relevant solution for situations where apical views may not be feasible. Going further, we demonstrate that combining PLAX and A4C predictions via simple unweighted late fusion improves both single-view baselines to a 6.37% MAE, underscoring the value of multi-view integration. To promote continued research, we release the dataset labels, trained models, and runnable demos on GitHub, Hugging Face, and Google Colab: https://github.com/Jeffrey4899/PLAX_EF_Labels_202509

1. Introduction

The paper addresses scarce public PLAX–EF supervision by constructing a reproducible resource and pipeline from publicly accessible data. It also evaluates multi-view fusion and releases artifacts for independent use.

  • Motivation: PLAX is routinely acquired and often easier to obtain, but public datasets provide limited large-scale supervision linking PLAX videos to EF.Apical acquisitions can be difficult because of poor acoustic windows, positioning constraints, and time-limited scans.
  • Approach: The pipeline mines A4C and PLAX views, correlates studies with clinical notes, and uses A4C predictions as proxy labels for PLAX training.This combines view classification, time-based note correlation, and proxy supervision.
  • Artifacts: The authors release label files, pretrained models, runnable demos, and usage documentation to support reproducible PLAX EF research.Artifacts span GitHub, Hugging Face, and Google Colab.
  • Results: 6.86% MAE: the proxy-supervised PLAX ensemble reaches this study-level error on a note-derived ground-truth cohort.The result is presented as a reproducible PLAX EF benchmark.
  • Results: 6.37% MAE: unweighted late fusion of A4C and PLAX predictions improves both single-view baselines on studies containing both views.The expanded evaluation uses a subset requiring both views within each study.

2. Background

The background defines EF, contrasts standard A4C measurement with PLAX-based assessment, and motivates video-based learning for scarce PLAX–EF supervision. It also establishes study-level evaluation as the clinical alignment for prediction and fusion.

  • EF measurement: EF is the fraction of blood ejected from the left ventricle during systole, based on end-diastolic and end-systolic volumes.Guideline-recommended volume estimates commonly use traced endocardial contours in A4C and A2C views.
  • PLAX-based estimation: PLAX formulas can be fast and interpretable but are sensitive to image quality, view alignment, and regional wall-motion abnormalities.These limitations motivate end-to-end models that map cine dynamics directly to EF.
  • Echocardiographic views: A4C supports guideline-recommended volumetric EF measurement, whereas PLAX shows the LV, left atrium, outflow tract, and mitral valve apparatus.PLAX is commonly available and useful for rapid qualitative LV-function assessment.
  • Label scarcity: Public PLAX datasets rarely provide EF labels despite frequent PLAX acquisition, creating a central supervision mismatch for PLAX EF modeling.The paper frames this scarcity as a key challenge for public-resource development.
  • Evaluation protocol: Study-level evaluation averages predictions across clips to better match clinical EF reporting and supports the study-level multi-view fusion strategy.Clinical EF is generally reported per study, while machine-learning datasets often label individual clips.

3. Related Work

Prior work establishes A4C EF prediction and multi-view learning, while PLAX methods remain constrained by scarce supervision and limited reproducibility. Proxy labeling and report mining provide relevant strategies for scaling supervision.

  • EF estimation from apical views: A4C is the dominant target for EF regression because apical views support standardized volumetric measurements and curated labels.Prior work shows spatiotemporal networks can achieve clinically competitive error on A4C cine videos.
  • PLAX and non-apical EF estimation: PLAX is routinely acquired and informative for identifying LV systolic dysfunction, but it is not commonly the primary view for EF reporting.This contributes to scarce PLAX–EF supervision.
  • PLAX methods: Existing PLAX approaches use indirect measurement pipelines or direct prediction, but public reproducible benchmarks with runnable artifacts remain scarce.Indirect methods trade interpretability for additional measurement-related error, while closed systems limit independent reproduction.
  • Multi-view learning: Multi-view learning combines complementary views through late fusion or learned fusion, reflecting different view-specific failure modes.Prior systems leverage available views including PLAX and A4C for EF-related tasks.
  • Scarce and noisy supervision: Proxy labels, teacher models, weak supervision, and clinical-report extraction are established responses to scarce or noisy clinical labels.The paper combines these ideas for echocardiographic EF supervision.

4. Dataset Generation and Model Training

The paper constructs PLAX EF training data by combining multi-dataset view classification, time-linked clinical-note labels, and A4C-derived proxy supervision. It validates and refines this pipeline through manual review, independent benchmarking, and classifier fine-tuning.

  • Dataset Generation: The pipeline identified A4C and PLAX views, then transferred A4C model EF predictions as proxy labels to PLAX videos from the same studies.A4C and PLAX selection was required because MIMIC-IV-Echo lacked view labels, while direct note-study pairs were insufficient for robust PLAX training.
  • View Classifier Evaluation: Manual review of 300 videos found reviewer agreement counts of 91/87 for predicted A4C, 78/72 for predicted PLAX, and 94/98 for OTHER.The evaluation addressed the absence of ground-truth view labels in MIMIC-IV-Echo.
  • View Classifier Training: The view-classification strategy combined EchoNet-Dynamic A4C videos, EchoNet-LVH PLAX videos, and MIMIC-derived OTHER videos.EchoNet-Dynamic contributed approximately 10,000 A4C videos, while EchoNet-LVH contributed approximately 12,000 PLAX videos.
  • Ground Truth Evaluation: 6.95% MAE on note-extracted labels supported the reliability of the A4C model and note-derived EF values in an independent 295-study test set.The test set contained 1,017 A4C videos and 1,320 PLAX videos and was used to evaluate PLAX predictions independently of proxy labels.
  • View Classifier Fine-Tuning: 6.83% to 5.14% MAE reduction followed X3D view-classifier fine-tuning using high-error A4C cases linked to incorrect or partial A4C views.The fine-tuning process improved A4C-versus-OTHER discrimination for downstream processing.
  • PLAX Dataset Generation: 25,532 videos formed the final PLAX training dataset across 4,822 studies.The split used 80% for training and 20% for validation.

5. Experimental results and analyses

The experiments establish PLAX EF prediction performance, evaluate study-level multi-view fusion, and analyze agreement, calibration, and limitations. Simple unweighted fusion improves over both single-view baselines, while proxy supervision and view-specific biases constrain interpretation.

  • 5.1 PLAX Model Training and Results: 6.86% MAE was obtained by averaging two PLAX model outputs for study-level EF estimation.Predictions were averaged across videos within each study before computing MAE.
  • 5.1 PLAX Model Training and Results: 0.670 Pearson correlation indicated positive agreement between PLAX predictions and ground-truth EF values.The Bland-Altman analysis reported -0.62% mean bias and limits of agreement from -18.10% to 16.87%.
  • 5.1 PLAX Model Training and Results: Proxy labels from an A4C model may compound errors and introduce domain shift because the A4C model was not trained on MIMIC datasets.The authors identify proxy supervision and dataset mismatch as potential explanations for variance and moderate correlation.
  • 5.2 Aggregating A4C and PLAX Views: 6.37% MAE and 0.709 Pearson correlation were achieved by unweighted study-level fusion of A4C and PLAX predictions on 284 common studies.The fusion used no additional training and combined the two view-specific predictions by averaging them.
  • 5.2 Aggregating A4C and PLAX Views: Fusion reduced bias and error spread relative to PLAX-only predictions, with -0.08% mean bias and limits of agreement from -16.08% to 15.93%.The fusion scatter plot showed closer alignment to the identity line and reduced dispersion.
  • 5.2 Aggregating A4C and PLAX Views: 6.77% versus 7.01% MAE favored PLAX numerically over A4C, but the difference was not statistically significant on the paired subset.The paired comparison reported Wilcoxon p = 0.66, paired t-test p = 0.48, and ΔMAE = 0.24% with 95% bootstrap CI [-0.40, 0.88].
  • 5.2 Aggregating A4C and PLAX Views: Inter-view disagreement was a weak reliability signal, with regression slope ≈0.08, Pearson r = 0.07, and no consistently monotonic quartile trend.The authors conclude that disagreement was insufficient as a standalone uncertainty measure on this dataset.
  • 5.2 Aggregating A4C and PLAX Views: A learned fusion weight did not improve over equal weighting, producing 6.38% cross-validated MAE versus 6.37% for the unweighted average.The selected weight was approximately 0.44, nearly recovering equal weighting.

6. Artifacts and Reproducibility

The authors release labels, model weights, prompts, and interactive inference resources to support reconstruction of the evaluation cohorts and reproduction of reported results. These artifacts also provide a foundation for future methodological and clinical extensions.

  • Artifacts and Reproducibility: Released label files and instructions support reconstruction of evaluation cohorts from MIMIC-IV resources under the data-use agreement.The repository also includes GPT prompts used for EF extraction from clinical notes.
  • Artifacts and Reproducibility: Interactive Hugging Face and Google Colab tools allow evaluation on sample videos or user uploads, with the notebook code available for modification.The browser-based demo is designated for research and education use only.
  • Artifacts and Reproducibility: Inference-ready A4C and PLAX EF models are available for direct download and integration into existing pipelines.The Hugging Face release includes models for both single-view predictions.
  • Artifacts and Reproducibility: Together, the released artifacts are intended to facilitate reproduction and support future methodological and clinical extensions.The resources combine cohort reconstruction materials with pretrained inference tools.

7. Conclusion

The study introduces a publicly available PLAX–EF resource and reproducible proxy-supervision pipeline, achieving 6.86% MAE for PLAX prediction. Late fusion of A4C and PLAX predictions reaches 6.37% MAE, while limitations include proxy-label noise, classification errors, and the need for external validation.

  • Conclusion: The study introduces the first publicly available PLAX–EF label resource and a reproducible pipeline using view mining, note-derived EF extraction, and proxy supervision.The resource supports EF estimation from PLAX cine echocardiography.
  • Conclusion: 6.86% MAE was achieved by the PLAX ensemble on an independent note-derived ground-truth cohort.
  • Conclusion: 6.37% MAE and 0.709 correlation were achieved by study-level late fusion of A4C and PLAX predictions on studies containing both views.
  • Conclusion: Released labels, pretrained weights, and runnable demos support reproduction and downstream research given authorized access to the underlying datasets.
  • Conclusion: PLAX-based EF estimation may extend automated assessment when standard apical windows are unavailable, incomplete, or suboptimal.
  • Conclusion: External validation is required before clinical deployment because proxy supervision, view-classification errors, and note-based ground-truth extraction can introduce noise.

Ethical Standards

The study used only publicly available, de-identified datasets under their respective data-use agreements, so institutional review board approval was not required.

  • Ethical Standards: The analysis used publicly available, de-identified datasets and involved no newly collected data or direct human-subject participation.
  • Ethical Standards: The study complied with data-provider ethical standards and applicable regulations.

Data availability

The study releases artifacts for reproducing its analyses, while access to the underlying MIMIC-IV videos remains governed by data-use agreements.

  • Data availability: All required reproduction artifacts are publicly available through GitHub, Hugging Face, and Google Colab resources.
  • Data availability: Raw MIMIC-IV-Echo videos are not redistributed because of data-use agreements.
  • Data availability: Released label files provide identifiers and instructions for locating corresponding samples under authorized access.
  • Data availability: An artifact usage guide documents implementation details and hyperparameters to support faithful reproduction.

A.1 Video View Classifier Training Details

The appendix describes training details for image- and video-based view classifiers and EF regression models, including their data sources, architectures, preprocessing, optimization, and compute settings.

  • A.1 Video View Classifier Training Details: The image classifier predicts five view categories after training on TMED-2 data with a ResNet-34 backbone adapted for echocardiography frames.Inputs are resized to 112×112 and grayscale images are replicated across three channels.
  • A.1 Video View Classifier Training Details: Label smoothing with ϵ = 0.05 is used to improve robustness to noisy labels and the heterogeneous combined class.
  • A.1 Video View Classifier Training Details: Training uses PyTorch with GPU acceleration, validation-based checkpointing, learning-rate schedules, and Weights & Biases logging for traceability.
  • A.1 Video View Classifier Training Details: The video classifier distinguishes A4C, PLAX, and OTHER using clips assembled from EchoNet-Dynamic, EchoNet-LVH, and MIMIC-IV-Echo.
  • A.1 Video View Classifier Training Details: The video model fine-tunes X3D-s with a three-logit classification head and selects its checkpoint using held-out validation performance.
  • A.1 Video View Classifier Training Details: The EF regression model uses R(2+1)D-18 with a single-neuron output and trains on 64-frame clips using MSE loss.

A.3 PLAX EF Model Training Details

The PLAX EF models use proxy-labeled study data, video backbones, and study-level aggregation, with released weights and demos supporting reproducibility.

  • Training data and proxy targets: Each PLAX video inherits a study-level proxy EF label by averaging A4C teacher predictions across available same-study clips.Study-level splitting prevents clips from the same study appearing across training and validation sets.
  • Architecture: The models use R(2+1)D-18 video backbones with a single-neuron linear head that outputs scalar EF predictions.Both models share the same architecture and differ only in batch size.
  • Clip sampling and preprocessing: Training uses 64-frame clips with temporal stride 2, random temporal sampling for long videos, and black-frame padding for short videos.Frame-wise preprocessing is applied before stacking the frames into a clip tensor.
  • Optimization and ensembling: The models are optimized with MSE loss using RAdam, learning-rate scheduling, validation-loss checkpoint selection, and two batch-size configurations.The final PLAX predictor averages the two study-level model predictions.
  • Reproducibility and released artifacts: Released checkpoints, label files, and inference workflows support reproduction through authorized data access, study-level metric aggregation, Hugging Face, and Colab.The main results use two PLAX checkpoints plus the A4C model, while lightweight demos may use one model per view.

B.5 Practical notes •

The practical guidance limits use to appropriate A4C or PLAX clips, interprets demo outputs as view-specific or averaged predictions, and emphasizes study-level evaluation and nonclinical use.

  • Input constraints: The online demo is intended for short MP4 clips, while very long clips may time out or increase inference latency.The practical recommendation is clips of approximately 10 seconds or less.
  • View requirements: Predictions may be unreliable for clips that are not true A4C or PLAX views or represent partial or atypical acquisitions.Rigorous evaluation should use view-filtered cohorts and study-level aggregation.
  • Meaning of demo outputs: EF (mean of available views) is the simple unweighted average of the available A4C and PLAX predictions.With only one view provided, this output equals that view’s estimate.
  • Study-level aggregation: Single-clip demo predictions are qualitative examples rather than substitutes for study-level reporting based on averaging multiple clips.The main paper reports study-level results by aggregating predictions within each study.
  • Privacy and intended use: The artifacts are for research and education only, not clinical diagnosis or treatment, and uploaded data must comply with privacy policies and data-use agreements.Running inference locally or in Colab on appropriately de-identified data is recommended when privacy is uncertain.
Loading 2609.02969v1…