Source-linked AI summary
Evaluating Usability in Biomedical Visualization: Rethinking Heuristic Evaluation for Spatial Omics and Multidisciplinary Research Platforms
Yulia A. Levites Strekalova, Rachel Liu Galvin, Jessica M. Ray, Samuel P. Border, Mishal Khan, Samantha Hoffman, Christina D. Beharry, Katie Kloss, Philipp Haessner, David Manthey, Sanjay Jain, Michael T. Eadon, Laura Barisoni, Pinaki Sarder
TL;DR
High-dimensional CRI platforms create usability challenges that conventional heuristics may not fully capture. The paper combines two complementary studies of spatial omics visualization and finds that users need domain-specific support for complex data states, including transparency, contextual guidance, and phased feature exposure. The studies also report marginally acceptable usability for FUSION, with a SUS score of 54 below the cited benchmark of 68.
Problem
Conventional usability heuristics may not adequately capture the interaction challenges, visualization requirements, and data-state complexities of multidisciplinary CRI systems.
Method
The study combines think-aloud usability interviews, structured heuristic evaluation, deductive Nielsen-based coding, and inductive analysis across complementary participant studies.
Results
FUSION showed marginally acceptable usability with a SUS score of 54, below the commonly cited threshold of 68, while behavioral findings revealed specific interface-improvement opportunities.
Takeaways & Limitations
The findings support three CRI-specific heuristics: Active Parameter Transparency, Point-of-Use Guidance, and Phased Feature Disclosure.
Takeaways & Limitations
The remote asynchronous design captured predominantly behavioral data and could not correct task-system technical issues in real time.
Abstract
from arXiv · showhide
Introduction: Clinical research informatics (CRI) platforms support biomedical discovery by integrating advanced computational tools into research workflows. Emerging technologies such as spatial omics and AI-enabled imaging expand research capabilities but introduce complex interfaces that increase cognitive burden and alter established analytical processes. Traditional usability frameworks identify general usability issues but often miss challenges specific to high-dimensional biomedical data. Methods: We conducted two complementary studies involving 39 participants to evaluate conventional usability heuristics and identify CRI-specific criteria. Study 1 included 19 undergraduates completing interactive tasks, and Study 2 involved 20 clinical professionals completing an asynchronous hierarchical task framework. Observational and interview data were analyzed using deductive coding based on standard usability heuristics and emerging CRI-specific themes. Results: Simultaneous presentation of complex data overlays and analytical tools overwhelmed users, particularly those with limited spatial-omics experience. Participants relied on trial-and-error exploration and struggled with unlabeled tools in data-rich environments. Feedback indicated that users benefit from phased onboarding, contextual guidance, and progressive feature introduction rather than immediate access to all functionality. Discussion: High-dimensional research platforms require domain-specific usability criteria beyond traditional frameworks. We propose three specialized heuristics: Active Parameter Transparency, Point-of-Use Guidance, and Phased Feature Disclosure. These heuristics help developers manage complexity, provide contextual support, and improve accessibility for multidisciplinary research teams.
Introduction
CRI platforms expand biomedical research capabilities but introduce complex, high-dimensional interfaces that can disrupt users’ mental models and workflows. The study evaluates whether conventional heuristics adequately capture these demands and develops domain-specific criteria.
- Motivation: Spatial omics and AI-enabled imaging expand clinical research capabilities while introducing unfamiliar interfaces and interaction demands.These technologies integrate rich computational and imaging information into biomedical workflows.
- Motivation: CRI usability has historically received less attention than functionality and product development, risking misalignment with biomedical research workflows.The paper links improved usability with reduced cognitive workload and lower entry barriers across disciplines.
- Cognitive demands: High-dimensional spatial omics interfaces can overwhelm working memory when they expose many interacting features and data states simultaneously.Users must interpret spatial relationships, identify structures, maintain orientation, and receive continuous feedback during exploration.
- Heuristic gap: Standard heuristic evaluation is useful but limited because it lacks real users, task-based workflows, and domain-specific coverage for specialized technologies.The paper identifies interpretive scaffolding and progressive interface exposure as relevant design needs in multidisciplinary biomedical interfaces.
- Study objective: The study uses two complementary approaches to examine conventional heuristic adequacy and formulate operational criteria for complex CRI systems.Study 1 uses think-aloud interviews, while Study 2 applies structured heuristic evaluation with a multidisciplinary expert panel.
Technology
The paper examines FUSION, a cloud-based platform for interactive spatial omics and whole-slide-image analysis. Its usability data are analyzed through think-aloud sessions, remote study procedures, Nielsen-based coding, and inductive theme development.
- Technology: FUSION integrates whole-slide images, segmentation outputs, pathomic features, and spatial omics data across cell, tissue, and biopsy levels.The platform supports multidisciplinary analysis of high-resolution imaging and spatial data.
- Technology: FUSION uses a human-in-the-loop model that lets users iteratively refine segmentation, annotation, and feature selection during analysis.Users respond to data observations rather than only viewing preprocessed outputs.
- Study setting: The CIMAP internship brought undergraduate engineering and public health students together for interdisciplinary AI/ML biomedical research.The five-day virtual program formed the setting for Study 1’s usability activities.
- Study setting: Study 1 used remote paired tutorials, screen recording, think-aloud protocols, and peer interviews to capture participants’ actions and reasoning.Written instructions covered account setup, study materials, recording, and task procedures.
- Data Analysis: Rapid qualitative analysis coded sessions deductively against Nielsen’s ten heuristics and inductively captured usability issues beyond that framework.Analysts independently summarized participant pairs, resolved discrepancies by consensus, and used a case-by-domain matrix for comparison.
Results
Study 1 found that all ten Nielsen heuristics appeared in student feedback, with the strongest concerns involving system feedback and instructional support. Participants also valued automated cell identification and distinct visual coding.
- Participants: 19 students completed the think-aloud usability evaluation, while 18 completed the post-program survey.Twenty-one students participated in the broader CIMAP spring break experience.
- Heuristic findings: All 10 Nielsen usability heuristics appeared in student feedback, which concentrated on systemic and instructional issues.Visibility of System Status and Help and Documentation were the most discussed categories.
- Heuristic findings: Visibility of System Status was most frequently violated because of server lag, missing loading indicators, frozen screens, and overwritten text-box inputs.These problems made system state and participant input difficult to track.
- Heuristic findings: Participants requested contextual onboarding inside FUSION, including interactive tooltips, video crash courses, and pop-up guides.Initial PowerPoint materials helped some users but did not consistently provide integrated support during use.
- Heuristic findings: Users benefited from automatic cell segmentation and identification plus visually distinct color-coding for rapid analysis.These strengths supported Recognition Rather Than Recall and Aesthetic and Minimalist Design compared with legacy software.
RQ1: Adequacy of usability heuristics for CRI software
Standard heuristics identify CRI usability problems but do not fully explain the cognitive and interpretive demands created by high-dimensional, changing data states. Study 1 supports domain-specific criteria for transparency, visualization interpretation, and phased onboarding.
- High-dimensional overlays, heatmaps, and composition charts create cognitive load that general-purpose heuristics lack an operational vocabulary to describe.
- Users struggled to interpret visualizations and determine whether difficulties reflected system behavior, domain knowledge, or both.
- CRI usability criteria must extend standard heuristics with visualization interpretability, data-state transparency, and onboarding scaffolding.
- Users requested searchable help, step-by-step guidance, contextual feature explanations, and introductory walkthroughs to navigate the platform.
- The proposed criteria address active data-state visibility, point-of-use interpretive guidance, and progressive exposure to platform complexity.
Study 2: Pathologists, Biologists, and Students
Study 2 recruited potential pathologist, biologist, and student users through professional networks and evaluated them with demographic surveys, tutorials, hierarchical tasks, and usability analysis.
- Participants were recruited from July 2023 to July 2024 through renal pathology organizations, professional networks, grand rounds, and related outreach.
- Participants completed demographic and familiarity surveys before receiving individualized platform credentials and task-specific tutorial slides.
- Tasks used a three-level hierarchical framework co-developed by pathology, engineering, and human-factors experts for varied domain expertise.
- Descriptive statistics summarized demographics and SUS responses, with SUS benchmarks below 68 indicating below-average usability and 85–100 indicating exceptional usability.
- Behavioral video analysis used continuous timestamp coding to capture observable behaviors, exploratory actions, and reviewer observations across sessions.
- 20 participants completed usability sessions and submitted recordings, while 14 completed the post-study survey, yielding a 70% response rate.
FUSION system usability
FUSION received a marginally acceptable SUS score, while video analysis revealed recurring difficulties with system status, annotation recovery, and documentation. Users repeatedly relied on tutorials and requested embedded guidance.
- 54 was the mean SUS score for FUSION, based on 14 surveys, indicating marginal acceptability.Individual scores ranged from 30 to 80, with an overall SD of 1.06.
- Video analysis found friction across six of Nielsen’s ten heuristics, with Visibility of System Status the most pervasive challenge.
- Users struggled to clear manual annotations and access cell-state and cell-type data, prompting extensive exploration and complex recovery attempts.
- Participants repeatedly returned to tutorials because the interface lacked intuitive guidance, and requested embedded video tutorials and additional visualization training.
RQ1: Adequacy of usability heuristics for CRI software
Across undergraduate and professional users, standard heuristics flagged usability symptoms but did not capture the systemic complexity of translating layered CRI inputs into visual outputs.
- Professionals struggled with the same visualization features as undergraduates, including configuring spatial-omics and heatmap overlays.
- Participants used repeated tutorial review and blind trial-and-error toggling when the interface did not clarify how layered inputs produced visual outputs.
- Nielsen’s principles assume relatively stable, task-bounded interface states, whereas CRI platforms continuously alter data-state parameters across tissue, cell, and omics layers.
- Traditional heuristics can identify general navigation problems but lack specificity for the domain-intrinsic complexity generating them.
RQ2: Formulating domain-specific heuristic criteria for CRI tools
Study 2 findings supported three CRI-specific heuristics addressing visibility of analytical state, contextual interpretation, and progressive feature introduction.
- Active analytical layers should be visible at a glance, including clear confirmation of activated cell subtypes and annotation states.Users otherwise toggled heatmap inputs through trial and error and struggled to clear annotations without visual confirmation.
- Point-of-use guidance should provide contextual explanations within visualizations rather than relying on separate documentation.Separated tutorials interrupted workflows, while users requested descriptions of tool functions and sometimes searched the web to interpret heatmaps.
- Advanced functionality should be introduced progressively instead of presenting all analytical tools simultaneously.The proposed heuristic responds to workflow disruption and complexity in data-rich interfaces.
Summary of the present studies
The studies found marginal usability for FUSION, while behavioral observations identified interface and evaluation issues relevant to multidisciplinary users. Remote asynchronous testing broadened recruitment but constrained insight into participants’ cognitive processes.
- A SUS score of 54 fell below the commonly cited threshold of 68 for average usability.Future iterations target a score of 70 or above.
- Behavioral observations revealed confusion associated with spatial-omics novelty, task wording, and opportunities for interface improvement.The authors recommend involving users with varying expertise when developing future tasks.
- The remote asynchronous approach provided a scalable way to engage geographically distributed research-informatics users.Its recruitment scope extended across multiple countries and disciplines.
- Asynchronous testing primarily captured behavioral data and prevented real-time correction of technical problems unrelated to FUSION.The authors suggest hybrid studies combining asynchronous behavioral collection with synchronous debriefing.
Alignment with existing literature
The proposed CRI heuristics adapt established usability and cognitive theories to the density and layered structure of biomedical visualization platforms. Their intended application emphasizes visible tool states, embedded guidance, and staged complexity.
- Theoretical alignment: Data-state visibility adapts system-status principles and situation awareness theory to multi-layered omics viewports.Interfaces should indicate active analytical layers at a glance.
- Theoretical alignment: Point-of-use guidance extends documentation into embedded interpretive scaffolding within the visualization.The approach is described as lowering interpretative error rates in high-dimensional biomedical platforms.
- Theoretical alignment: Progressive interface exposure applies progressive disclosure to manage working-memory limits and software complexity.Advanced tools are phased in systematically rather than exposed all at once.
- Future research: The three heuristics are presented as a transferable framework for evaluation across other high-dimensional biomedical visualization platforms.The study examined one platform with a relatively small sample, and usability scores varied across disciplinary backgrounds and expertise.
- Implications for practice: Practical guidance recommends exposing foundational analytical layers by default and reserving advanced overlays for users completing structured onboarding.The recommendation specifically mentions cell-subtype heatmaps and multi-channel composition plots.
- Implications for practice: The framework responds to the risk that technically robust spatial-omics tools remain unusable and fail to achieve needed adoption.