Source-linked AI summary
The Collection of a Real-World, Large-Scale Dataset for Automated Skill Evaluation in Central Line Dressing Change
Ummey Tanin, Shannan K. Hamlin, Hsin-Mei Chen, Peizhu, Qian
TL;DR
CLDC competency assessment relies mainly on educator observation, and large clinically grounded datasets for automated evaluation are lacking. The paper collected and annotated real-world nurse videos and tested video-based skill models; the resulting dataset provides a basis for future assessment and training research.
Problem
CLDC assessment relies mainly on time-intensive educator observation, while large clinically grounded datasets for developing and evaluating automated nursing-skill assessment are lacking.
Source: §1 · p. 2Method
The authors collected real-world CLDC videos during Houston Methodist competency evaluations and paired them with temporal annotations and clinical performance assessments.
Source: §1 · p. 2, §4 · pp. 7–8Results
The collection retained 365 complete nurse recordings, and I3D + TSN achieved 95.45% overall training accuracy on JIGSAWS.
Source: §3.1 · p. 3, §4 · p. 6Takeaways & Limitations
The dataset provides a foundation for future automated procedural understanding, skill assessment, and feedback research in nursing education.
Source: §1 · p. 2, §5 · p. 8The dataset primarily captures nurses with at least some familiarity with CLDC rather than complete novices.
Source: §4 · p. 7
Abstract
from arXiv · showhide
Central line-associated bloodstream infections (CLABSIs) remain a major source of preventable morbidity and mortality in healthcare settings. Proper execution of the central line dressing change (CLDC) procedure is essential for infection prevention, yet current approaches to competency assessment rely primarily on manual ob- servation by clinical educators, making evaluation time-intensive, difficult to scale, and limited in consistency and granularity. These challenges highlight the need for AI-assisted skill evaluation methods that can support more objective, reproducible, and scalable assessment of clinical performance. However, progress in this area has been constrained by the lack of broad-scale, clinically grounded datasets for developing and evaluating such methods. In this work, we present the collection process of a large-scale video dataset for CLDC skill evaluation, comprising 365 recordings of registered nurses collected during the 2026 annual competency exam at Houston Methodist Hospital. Each video is paired with subtask-level temporal annotations and clinical performance assessments spanning 17 subtasks of the procedure. We describe the data collection and curation pipeline, including pilot testing of record- ing configurations, annotation protocol design, and post-collection processing. We further establish the feasibility of video-based skill assessment through benchmark experiments with spatiotemporal deep learning models. This dataset provides a foundation for future research on automated clinical skill assessment, feedback generation, and AI-assisted training in nursing education.
1 INTRODUCTION
CLDC requires careful sequencing and sterile technique, yet competency assessment depends largely on time-intensive educator observation. This work introduces a real-world video dataset with temporal annotations and clinical performance assessments to support automated evaluation.
1 INTRODUCTION
- CLABSIs carry substantial patient and healthcare burdens, and deviations during the complex, multi-step CLDC procedure can compromise sterility and increase patient risk.The cited estimates include 12%–25% mortality and approximately 28,000 annual U.S. ICU deaths.Source: §1 · p. 2
- Current CLDC competency evaluation relies mainly on educator observation, which is time-intensive, subjective, and difficult to scale; large, clinically grounded datasets are also lacking.Earlier efforts largely emphasized rule-based tracking rather than data-driven video evaluation of real clinical performance.Source: §1 · p. 2
- The paper introduces 365 real-world nurse recordings paired with structured temporal annotations and clinical performance assessments across CLDC subtasks.The dataset was collected during authentic competency evaluations at Houston Methodist Hospital.Source: §1 · p. 2
2 RELATED WORK
Prior work demonstrates video-based skill assessment and structured nursing evaluation, but the literature lacks a large-scale real-world video dataset for automated CLDC assessment. The paper positions its dataset as addressing that specific gap.
2.1 Healthcare Procedural Video Datasets
- Existing procedural video datasets support AI-driven healthcare analysis but are concentrated in surgical or instructional settings rather than real-world nursing competency evaluation.NurViD spans 51 procedures and 177 action steps, while SLAM includes 4,097 annotated laparoscopic clips.Source: §2.1 · p. 2
2.2 Video-Based Automated Skill Assessment
- Video-based skill assessment has been explored across laparoscopic, robotic, cardiac, open-surgery, and simulated tasks, but most studies use controlled operative or simulation settings.Reported approaches include action recognition, skill classification, motion-feature analysis, and video-based assessment.Source: §2.2 · p. 3
2.3 Clinical Nursing Skill Evaluation
- The authors identify no prior large-scale real-world video dataset specifically designed for automated CLDC skill evaluation and present their annotated dataset as addressing that gap.The dataset pairs real competency-evaluation video with structured temporal annotations and clinical performance labels.Source: §2.3 · p. 3
3 MATERIALS AND METHODS
The study combines benchmark experimentation, clinical task definition, and a real-world collection protocol to build a CLDC video dataset. Its methods include pilot-tested camera configuration, clinical collaboration, and concurrent temporal annotation and performance assessment.
3 MATERIALS AND METHODS
- The study was designed as a multi-phase effort spanning benchmark experimentation, task definition, data collection, pilot recording, and annotation workflow design.These phases connected computational validation with clinical data collection.Source: §3 · p. 3
3.2 Defining CLDC Subtasks
- The CLDC procedure was decomposed with clinical experts into 19 subtasks, including three activities retained for completeness although they were not performed during the competency exam.The taxonomy was refined using nursing-class observations and educator feedback.Source: §3.2 · p. 4, Table 2 · p. 4
3.3 Establishing Clinical Collaboration
- Houston Methodist provided the clinical setting and access to competency evaluations; the team targeted at least 300 recordings and agreed to exclude audio and facial imagery.The hospital’s annual assessment evaluates approximately 2,000 nurses across multiple stations over several weeks.Source: §3.3 · p. 4
3.5 Preparation for Recording at Houston Methodist
- Preparation included participant surveys, printed timestamp and assessment forms, ID cards, and staff training that rehearsed procedure recording and visible transition markers.Paper forms were chosen because pilot testing found them faster and more reliable for timestamps in a fast-paced setting.Source: §3.5 · pp. 5–6
3.6 Recording Workflow and On-Site Annotation
- During collection, up to four trained observers recorded timestamps for 17 activities and documented qualitative performance feedback, while participants completed a standardized video-and-survey sequence.The demographic survey remained digital because it was less time-critical than live annotation.Source: §3.6 · p. 6
4 DATA RECORDS
The dataset contains 365 complete CLDC recordings from nurses at Houston Methodist’s January 2026 competency evaluation, paired with subtask timestamps, clinical assessments, and feedback. Collection and curation included high-resolution video capture, structured annotation, timestamp verification, and documentation of teaching segments.
Dataset and participants
- 365 of 381 recordings were retained after 16 exclusions for technical, participant, educator, or logistical interruptions.The evaluation ran from January 6 to January 22, 2026, across three weeks.Source: §4 · p. 6
- Participants ranged from 22 to 70 years old (mean 38.6, SD 10.7); 83.7% were female, and Asian, White, and Black or African American nurses each formed substantial groups.Asian participants were 37.5%; White and Black or African American participants were each 26.8%.Source: §4 · p. 6
- Nurses averaged 11.4 years of experience (SD 9.1; median 9), with specialties including Critical Care, Medical/Surgical, and Acute Care, where central-line care is routine.Experience ranged from fewer than two years to over 20 years.Source: §4 · p. 6
- Nearly half of participants reported being somewhat confident in CLDC (47.8%), while 26.0% reported being extremely confident.Only 7.3% reported low or no confidence, so the dataset primarily reflects nurses with some procedure familiarity.Source: §4 · p. 7
Recording and annotation
- Recordings captured one nurse performing the procedure individually from start to finish, at two dedicated stations, in 4K resolution and 60 frames per second.The procedure was performed standing, consistent with standard bedside practice.Source: §4 · p. 7
- Educators and research staff annotated each subtask’s start and end times, Met/Not Met status, and clinical feedback; timestamps were checked against video and teaching segments documented for downstream handling.Teaching segments could be excluded from or flagged in downstream analysis.Source: §4 · p. 8
5 CONCLUSION AND FUTURE WORK
The paper reviews related resources and assessment methods, then presents a structured real-world CLDC data collection protocol and an initial foundation for automated evaluation. It also describes a possible future direction: integrating assessment with robotic tutoring for real-time practice feedback.
Contributions
- The work reviews CLDC-relevant data resources and skill-assessment methods and reports a structured collection protocol covering equipment selection and annotation and evaluation instruments.The review identifies an absence of an automated system directly addressing CLDC.Source: §5 · p. 8
- The authors describe the collection as the first large-scale video dataset of CLDC captured during authentic competency assessment, with approximately 365 recordings.The recordings were collected during the January 2026 nursing competency evaluation at Houston Methodist.Source: §5 · p. 8
Future Work
- A proposed future direction is a camera-observing robotic tutor that recognizes subtasks, evaluates execution, and provides spoken feedback or demonstrations during practice.The authors suggest this could provide immediate guidance when one-to-one educator supervision is unavailable.Source: §5 · p. 8