PaperlayerView exact revision on arXiv

Source-linked AI summary

Lessons learned from deploying imaging AI with the open PACS-AI platform

Samuel Kadoury, Julie G. Hussin, Pascal Thériault-Lauzier, Laurent Létourneau-Guillon, Rob Lewis, Adam McArthur, Gordon J. Harris, Houda Bahig, Pierre-Luc Déziel, Jay Kshirsagar, Jacob L. Jaremko, Julien Cohen-Adad, Jacques Delfrate, Robert Avram

arXiv:2609.26981v1cs.CVcs.CY

TL;DR

Clinical AI is often evaluated on retrospective, task-specific datasets without durable workflow integration or post-publication feedback and monitoring. We report PACS-AI, an open self-hosted platform deployed at six hospitals; at one center, 515 of 607 angiography jobs completed, with failures tied to absent target-modality series.

  • Problem

    Evidence remains limited on durable workflow integration and post-publication feedback or monitoring for task-specific models, while benchmark performance and clinical value can diverge [1, 2, 3].

    Source: Introduction · p. 4

  • Method

    We built and deployed an open-source, self-hosted imaging AI platform at six hospitals, with routine-care integration, structured clinician feedback, and versioned model governance.

    Source: Introduction · p. 4, Monitoring, governance and privacy · p. 7

  • Results

    515 of 607 study-by-model jobs completed in the Montreal angiography deployment; all 92 failures lacked a series matching the models’ target modality.

    Source: Two prospective deployment use-case… · p. 6

  • Takeaways & Limitations

    Across these deployments, infrastructure to deploy, monitor, and correct models after publication was rarely less limiting than model performance.

    Source: Key lessons from deployment · p. 10

    Model performance varies by site, protocol, and population, so estimates should not be assumed to transfer.

    Source: Key lessons from deployment · p. 9

Abstract

from arXiv · show

We describe deploying imaging AI at six hospitals through PACS-AI, an open self-hosted platform. The binding constraint is not model accuracy but infrastructure to route studies, display results, capture feedback, and audit what runs. At one center, angiography models completed 515 of 607 jobs (84.8%); failures reflected absent diagnostic views, and 78.1% of 638 clinician ratings were positive. Publishing honest readiness levels for every model is itself a governance practice.

6 University of Ottawa Heart Institute, Ottawa, ON, Canada · 7 Open Health Imaging Foundation, Boston, MA, USA · 17 HeartWise.ai Laboratory, Montreal Heart Institute, Montréal, QC, Canada

PACS-AI brings imaging AI into routine hospital workflows while making deployment, clinician feedback, and model readiness auditable. Across six hospitals, the paper finds that infrastructure, local validation, and governance—not model performance alone—constrain deployment.

17 HeartWise.ai Laboratory, Montreal Heart Institute, Montréal, QC, Canada

  • Prospective evaluations had institutional research ethics approval, aggregate operational data are available on reasonable request, and patient-level imaging data remain within hospitals.The complete PACS-AI framework and model-migration skill are available publicly; the paper also reports author contributions, competing interests, and funding acknowledgements.Source: §17 · pp. 2–3

Introduction

  • PACS-AI addresses the gap between benchmark performance and clinical value by integrating models into routine care with monitoring and feedback.It is open-source and self-hosted, installed at six hospitals, and supports model families across angiography, echocardiography, electrocardiography, radiography, and cross-sectional imaging.Source: Introduction · pp. 4–5

The PACS-AI platform

  • The platform routes eligible PACS studies to containerized models and returns results through the DICOM archive, with a versioned registry and structured-feedback interface.Its open-source stack adds rule-based routing, silent-mode evaluation, drift monitoring, and audit logging; a standardized manifest and agent skill support supervised model migration.Source: The PACS-AI platform · pp. 5–6

Two prospective deployment use-case studies evaluating impact in the field

  • 515 of 607 angiography jobs completed (84.8%); all 92 failures arose because retrieved studies lacked diagnostic coronary-angiography series.The 413 prospective studies were processed at Montreal Heart Institute from June 6 to July 22, 2026; mean inference time was 18.2 ± 6.1 s for DeepCORO-CLIP and 4.2 ± 2.1 s for CathEF and DeepRV.Source: Two prospective deployment use-case… · p. 6
  • Among 207 ACS patients, CathEF achieved AUROCs of 0.84 for LVEF ≤50% and 0.90 for LVEF ≤40%; DeepRV improved reader accuracy in 82 STEMI cases.DeepRV accuracy rose from 72.1% to 77.6% among cardiologists and from 43.5% to 64.0% among medical students; performance also depended on culprit lesion and echocardiogram timing for CathEF.Source: Two prospective deployment use-case… · pp. 6–7

Monitoring, governance and privacy

  • 498 of 638 clinician ratings (78.1%) were positive, while negative ratings most often cited missed findings (41) or measurement disagreement (33).Ratings covered 14.9% of returned reports over 13 weeks at Montreal Heart Institute; feedback captures agreement, corrections, uncertainty, image quality, and downstream action.Source: Monitoring, governance and privacy · p. 7

Beyond angiography

  • Models beyond angiography are integrated or retrospectively ready, but none has completed prospective platform validation; a registry audit found missing assets and other defects.The audit found echocardiography adapters unable to build cleanly, a Model Facts file with the wrong modality, and absent checkpoints; all must resolve defects and pass silent-mode evaluation before clinical display.Source: Beyond angiography · pp. 8–9, Key lessons from deployment · p. 9

Key lessons from deployment

  • Deployment feasibility and discrimination—not clinical outcomes—are reported; every site needs local PACS/network integration and GPU capacity, and adoption or performance may not generalize.Only CathEF and DeepRV have completed prospective deployment; clinicians review flagged or discordant cases more often, so agreement rates are not unbiased accuracy estimates. Standardized local retrospective and silent-mode evaluation, structured feedback, and open viewer/server infrastructure support safe deployment.Source: Key lessons from deployment · pp. 9–10

model performance, but also on reliable clinical integration, prospective validation,

PACS-AI provides a practical foundation for deploying and evaluating imaging AI across institutions, while effects on clinical outcomes remain unestablished. Its architecture routes studies through local model inference to viewer results, clinician feedback, and audit records.

provides a practical foundation for deploying and evaluating imaging AI across

Figures and Tables

  • PACS-AI defines four readiness levels: R1 integration demonstration, R2 internally validated, R3 externally validated, and R4 prospectively deployed.Table 1 presents PACS-AI model families and readiness levels.Source: Table 1 · p. 11, Figures and Tables · p. 13
  • The PACS-AI architecture routes studies from hospital PACS through scheduled ingestion and local storage to containerized models, returning results to the viewer and logging clinician feedback.Source: Figure 1 · p. 14
  • The operational dashboard streams study and model-job progress and records each run’s trigger, version, timing, outcome, and skip or failure reason.Clinicians can rate outputs during review; negative ratings open a fixed questionnaire, and ratings from models not in clinical use were collected in silent mode. The dashboard tracks approval against a configurable review threshold.Source: Figure 3 · p. 15, Figures and Tables · pp. 17–18

Supplementary Materials

Supplementary Note S1 sets out seven governance practices for deploying imaging AI and describes a PACS workflow in which predictions remain advisory and clinicians review results. Its platform comparison distinguishes broad imaging orchestration from constrained cardiovascular suites, while noting that open, customer-controlled model-retraining infrastructure is not documented.

Supplementary Note S1. Governance practices

  • The first four governance practices require declaring AI use in consent, hosting and authenticating locally, logging access, and setting retention and destruction schedules.These practices were applied at each installation.Source: Supplementary Note S1. Governance… · p. 22
  • The remaining practices require documented model-selection criteria, advisory rather than verdict alerts, local-law checks before go-live, and a site-specific privacy assessment where feasible.Source: Supplementary Note S1. Governance… · p. 22
  • CathEF and DeepRV predictions run on eligible angiography series, return to PACS, flag abnormal results, and prompt clinician review and feedback; all remain advisory pending review.Abnormal predictions route patients to confirmatory echocardiography and, if confirmed, guideline-directed therapy.Source: Figure S1 · p. 25, Supplementary Note S1. Governance… · p. 26
  • Existing platforms offer monitoring and governance, but an open institution-controlled outcome-linked retraining loop is not publicly documented; registries generate evidence, not customer-controlled retraining infrastructure.Source: Supplementary Note S1. Governance… · p. 27
  • The platform comparison distinguishes multi-vendor orchestration spanning clinical systems from cardiovascular analysis suites constrained by disease or modality.Source: Supplementary Note S1. Governance… · p. 27

coordination and signal-AI platforms

The section describes an open, self-hosted platform for local inference and integrated viewing across cardiovascular imaging and signals. It also highlights native clinician feedback and longitudinal prediction tracking, while noting that an open, institution-controlled model-updating loop across products is not publicly documented.

coordination and signal-AI platforms

  • The broader program includes CathEF, DeepRV, CathAI/DeepCORO, echocardiography, and ECG models; Anumana targets low ejection fraction, pulmonary hypertension, and amyloidosis, while Eko targets low ejection fraction, atrial fibrillation, and murmurs.Eko combines ECG and heart sounds.Source: coordination and signal-AI platforms · p. 28
  • An open, institution-controlled, cross-product model-updating loop is not publicly documented, although monitoring, feedback, and pathway tracking exist in parts of the group.Source: coordination and signal-AI platforms · p. 28
  • Native clinician feedback capture and longitudinal prediction tracking support continuous evaluation and governed periodic model updates.Source: coordination and signal-AI platforms · p. 28
  • The platform supports vendor-agnostic, containerized local inference, PACS integration, integrated viewing, point-of-care deployment, and prospective validation across cardiovascular imaging and signals.It has an open-source research core and supports self-hosted, on-premises deployment.Source: coordination and signal-AI platforms · p. 28
Loading 2609.26981v1…