Source-linked AI summary
Toward Vision Language Model-based Assessment of Clinical Quality and Usability of LGE-MR Images for Cardiac Ablation Planning
Bipasha Kundu, Abhishek Chaturvedi, Axel W. E. Wismueller, Richard Simon, Cristian A. Linte
TL;DR
Poor LGE-MRI quality can undermine cardiac ablation planning, while manual assessment is subjective and existing automated methods lack interpretable clinical reasoning. The paper develops a two-stage VLM-LLM framework that generates five-criterion quality reports and derives structured scores plus binary usability decisions. On 60 slices from 20 patients, InternVL2 had the highest criterion-level performance, while DeepSeek achieved perfect clinical usability agreement.
Problem
LGE-MRI quality assessment for ablation planning is subjective and difficult to scale, while existing automated approaches provide scalar outputs without interpretable clinical reasoning.
Method
A two-stage framework uses a fine-tuned VLM for five-criterion radiology-style reports and a GPT-based module for structured scores and binary usability decisions.
Results
InternVL achieves the highest criterion-level performance with ACC=0.65 and PLCC=0.79, while DeepSeek achieves perfect clinical usability agreement with Acc=1.00 and kappa=1.00.
Takeaways & Limitations
The results provide proof-of-concept evidence for VLM-based clinical image quality assessment of left atrial LGE-MR images under limited annotated data.
Takeaways & Limitations
Validation is limited by 60 image slices from 20 patients and single-radiologist annotation, motivating larger multi-center and multi-reader evaluation.
Abstract
from arXiv · showhide
LGE cardiac MRI is widely used for left atrial fibrosis assessment and ablation planning in atrial fibrillation patients as knowledge of fibrotic tissue regions identified from LGE-MRI is critical for catheter ablation. Often, poor quality images used during ablation planning can cause mis-localization of ablation targets, directly impacting procedure safety and outcome. The decision of whether a scan meets the minimum quality threshold for ablation planning is currently made informally by the reviewing radiologist and is not captured by any automated system, yet it is arguably the most safety-critical output of the image quality assessment (IQA) process. However, variations in image quality caused by noise, motion artifacts, and poor boundary definition significantly compromise the reliability of downstream segmentation and clinical decision-making tasks. Manual quality assessment by expert radiologists is subjective and difficult to scale, while existing automated methods produce scalar scores without interpretable clinical reasoning. In this work, we propose a two-stage vision language model (VLM) framework for clinically grounded image quality assessment of left atrial LGE-MRI. In the first stage, a fine-tuned VLM generates structured radiology-style quality reports predicting five radiologist-defined criteria: Noise, Motion Artifact, LA Boundary Accuracy, PV Region Accuracy, and Under-segmentation Severity. In the second stage, a GPT-based reasoning module maps the predicted quality and reports to a structured quality scores and binary clinical usability decision for ablation planning. We curate a dataset of 60 annotated image slice-text pairs from 20 patients and benchmark four state-of-the-art VLM architectures. InternVL2 achieves the highest criterion-level accuracy (Avg ACC=0.65, PLCC=0.79), while DeepSeek achieves perfect clinical usability agreement (Acc=1.00, kappa=1.00).
1 Introduction
VLMs extend language models to medical imaging, but their use for cardiac MRI quality assessment remains limited despite the clinical importance of reliable LGE-MRI for ablation planning. The paper proposes structured, clinically grounded reports and usability decisions to address subjective, non-scalable assessment and scalar metrics that lack interpretable reasoning.
- Clinical motivation: LGE-MRI quality directly affects anatomical delineation and ablation planning because it supports scar localization relative to left atrial and pulmonary vein anatomy.Poor image quality can produce unreliable anatomical delineation and potentially sub-optimal treatment planning.
- Assessment gap: Expert MRI quality assessment is subjective, time consuming, difficult to scale, and often lacks reference images for quantitative comparison.Existing objective measures such as PSNR, CNR, and RMSE are not consistent with subjective evaluation.
- Proposed direction: VLMs can generate structured, human-interpretable reports covering multiple clinically relevant quality criteria simultaneously.The described criteria include noise severity, motion artifact extent, boundary clarity, and pulmonary vein visibility.
- Proposed direction: A clinically useful system must combine criterion-level quality assessment with a binary usability judgment because no single quality criterion determines ablation-planning suitability.For example, acceptable noise does not compensate for poor boundary definition.
- Paper contribution: The proposed framework generates radiology-style reports for five criteria and maps them to structured scores and a binary clinical usability decision.The five criteria are Noise, Motion Artifact, LA Boundary Accuracy, PV Region Accuracy, and Under-segmentation Severity.
2 Dataset and Annotation
The study constructs a left atrial LGE-MRI quality-assessment dataset from clinically informative slices and expert annotation, while recognizing that its limited scale constrains validation. Annotation covers five criteria and an overall ablation-planning usability decision.
- Dataset construction: Image samples were selected from LAScarQS 2022 and converted into image slice-text pairs for expert-annotated MRI quality assessment.The dataset construction was designed to adapt subjectively annotated MRI data to the VLM-based IQA task.
- Dataset construction: Three middle slices per patient were selected around the left atrium to emphasize anatomical visibility and reduce non-informative basal or apical slices.This selection focuses the model on clinically informative regions.
- Scope boundary: The dataset scale is limited, so the study is intended as proof-of-concept evidence requiring validation on larger, multi-center expert-annotated cohorts.The authors note that the current scale is consistent with other expert-annotated cardiac MRI quality benchmarks.
- Annotation protocol: Table 1 defines the quality-assessment criteria used for radiologist annotation.The table provides the study-specific rubric underlying the five criterion scores.
- Annotation protocol: Each case is assessed across Noise, Motion Artifact, LA Boundary Accuracy, PV Region Accuracy, and Under-segmentation Severity, alongside an overall clinical usability decision.The criteria use an ordinal scale from 0 (Unusable) to 3 (Good).
3 Methodology
The methodology treats MRI quality assessment as a two-stage reasoning problem: a VLM produces structured reports and an LLM converts them into scores and a clinical usability decision. The design separates visual perception from structured multi-criteria reasoning.
- Framework design: The protocol decomposes assessment into Image-to-Report Generation followed by Report-to-Score Mapping and Clinical Usability Decision.This decomposition is intended to support clinically interpretable representations and reduce ambiguity in subjective assessment.
- Image-to-Report Generation: The first stage fine-tunes a VLM to generate radiology-style quality reports from image slices and expert-conditioned textual supervision.The added descriptions act as a semantic bottleneck encouraging high-level clinical attribute encoding rather than direct discrete-score optimization.
- Report-to-Score Mapping and Clinical Usability Decision: The second stage converts VLM-predicted quality and reports into structured ordinal scores and a binary usability decision using explicit multi-criteria reasoning.A constrained JSON schema reduces failures from free-text variation in field names, punctuation, and phrasing.
- Implementation: The pipeline uses LoRA-adapted fine-tuning for the VLM, followed by LLM processing of generated scores and textual reports.Figure 2 summarizes data curation, annotation, VLM fine-tuning, report generation, and LLM-based usability assessment.
- Evaluation: Performance is evaluated with ACC and PLCC for ordinal criteria, and ACC, F1, and Cohen’s kappa for binary clinical decisions.Tolerance accuracy (Acc±1) is also reported because human quality grading can vary by one ordinal level.
4 Result Analysis
The framework is evaluated on criterion-level quality-report generation and binary clinical usability decisions, with performance varying across VLM backbones and quality criteria.
- InternVL2 achieves the highest average criterion-level performance, with ACC=0.65 and PLCC=0.79 across the test set.
- Noise and Motion criteria perform consistently better than boundary accuracy, PV region accuracy, and under-segmentation across models.The latter criteria show agreement of 0.50–0.67 for stronger models and 0.42–0.58 for weaker models.
- InternVL, DeepSeek, and LLaVA achieve perfect tolerance accuracy of 1.00 across all criteria, while Qwen2.5 reaches 0.92.Tolerance accuracy allows predictions within one ordinal level of expert ratings.
- DeepSeek achieves perfect agreement with radiologist usability decisions, recording Acc=1.00, F1=1.00, and k=1.00.Task 2 uses VLM-predicted reports rather than ground-truth annotations, so criterion-level errors propagate into usability reasoning.
- InternVL records Acc=0.92, F1=0.90, and k=0.83 for clinical usability, followed by Qwen2.5 with Acc=0.83, F1=0.80, and k=0.67.
- The qualitative comparison presents VLM predictions, Chat-GPT-extracted scores, clinical usability decisions, and ground truth.
5 Discussion
The study provides preliminary evidence that the two-stage VLM-LLM framework is feasible for clinically grounded LGE-MRI quality assessment, while dataset imbalance and limited scale constrain evaluation of rare failures and generalizability.
- 60 image slices from 20 patients and single-radiologist annotation make the findings preliminary rather than definitive validation.Future work will use multi-reader protocols to quantify inter-rater agreement.
- Few unusable cases limit evaluation of model performance on rare failure cases.The authors report Cohen’s kappa for binary usability decisions and PLCC alongside accuracy to address class imbalance and small-sample variance.
- Acc±1=1.00 for all models except Qwen2.5 indicates no two-level ordinal errors associated with clinically harmful misclassifications.
6 Conclusion
The work frames LGE-MR image quality assessment as structured clinical reasoning and demonstrates proof-of-concept feasibility for translating quality attributes into ablation-planning usability judgments.
- InternVL achieves the highest criterion-level performance with perfect tolerance accuracy, while DeepSeek achieves perfect clinical usability agreement.The framework uses VLM-generated reports across five criteria and a GPT-based module for binary usability decisions.
- The report-to-decision pipeline translates structured quality attributes into reliable usability judgments that remain correct when criterion predictions deviate by one ordinal level.
- The results provide proof-of-concept evidence for VLM-based clinical image quality assessment with limited annotated data.Larger, multi-center cohorts are needed to assess generalizability and potential clinical deployability.