PaperlayerView exact revision on arXiv

Source-linked AI summary

Making Sense of Animal-to-Human Drug Development Evidence: Stakeholder Practices, Challenges, and Requirements for AI Tools

Rosni Vasu, Simona E. Doneva, Benjamin V. Ineichen

arXiv:2609.39808v1cs.HC

TL;DR

Evidence-synthesis tools have received limited empirical study across the diverse stakeholder practices and decision contexts involved in animal-to-human evidence assessment. The authors conducted semi-structured interviews with 13 stakeholders about evidence practices and challenges, and gathered feedback on a curated evidence dashboard. Stakeholders’ practices and judgments varied by professional role and decision context rather than following one shared process.

  • Problem

    Evidence-synthesis tools have received limited empirical study across the diverse stakeholder practices and decision contexts involved in animal-to-human evidence assessment.

    Source: §2.1 · p. 2

  • Method

    The authors conducted semi-structured interviews with 13 stakeholders about evidence practices and challenges, and gathered feedback on a curated evidence dashboard.

    Source: §1 · p. 2

  • Results

    Stakeholders’ practices and judgments varied by professional role and decision context rather than following one shared process.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Develop interactive systems tailored to researchers’ and stakeholder groups’ differing needs that support retrieval, structuring, and initial comparison while accounting for evidence limitations and nuances.

    Source: §5 · p. 10

    The 13 participants were based in Switzerland, and transferability to other stakeholder groups and countries may be limited.

    Source: §6 · p. 12

Abstract

from arXiv · show

Animal models are widely used to study human biology and health interventions, yet translating findings from animal studies to humans remains challenging. Evidence across preclinical and clinical research informs experimental and translational decisions. Artificial Intelligence (AI) tools are increasingly reshaping how this evidence is searched, synthesized, and used, but it remains unclear how they should support the diverse stakeholders involved in assessing animal-to-human evidence. We conducted semi-structured interviews with 13 stakeholders to examine their evidence practices, challenges, and expectations for AI support. We found that stakeholders approach the same incomplete evidence base with different goals, expertise, and heuristics. Participants valued AI particularly for locating, screening, and extracting evidence, but were more cautious about automated interpretation and quality judgments. They emphasized transparency, source traceability, uncertainty communication, and human oversight. Based on these findings, we derive design implications for role-sensitive AI tools that support more systematic and transparent reasoning about animal-to-human translation.

1 Introduction

Animal-to-human translation depends on evidence from animal studies, but evidence synthesis is difficult and stakeholders’ needs for AI support vary. The study investigates these practices and challenges through stakeholder interviews and derives requirements for AI-assisted evidence tools.

1 Introduction

  • Evidence synthesis is challenging because studies are numerous, heterogeneous in design and reporting, and may include partly unpublished sources.Drug development is long, costly, and uncertain, and more than 90% of candidates entering clinical development ultimately fail.Source: §1 · pp. 1–2
  • The authors conducted semi-structured interviews with 13 stakeholders and elicited feedback on a curated evidence dashboard.Participants discussed their literature practices, challenges, and how AI tools might support their work.Source: §1 · p. 2
  • Stakeholders’ evidence practices and judgments vary by professional role and decision context rather than following one shared process.The study draws on stakeholders across basic and translational research, animal welfare and the 3Rs, and regulatory and ethics roles.Source: §1 · p. 2
  • The study identifies 15 evidence-work challenges and calls for role-sensitive, source-linked AI support with uncertainty communication and human oversight.Challenges span access, appraisal, study comparison, welfare assessment, cross-species synthesis, and AI use; quality assessment requires context-specific expert judgment.Source: §1 · p. 2
  • The findings inform interactive systems intended to support more efficient, transparent, and evidence-based research while supporting translation and reducing unnecessary animal experimentation.Source: §1 · p. 2

2 Related Work

Prior work has developed evidence-synthesis tools and AI literature support, but empirical understanding of diverse animal-to-human stakeholders’ practices and needs remains limited. This study extends that work across stakeholder roles and examines trust, verification, and design requirements.

2.1 Evidence Synthesis in Animal Research

  • Systematic reviews can inform animal-study robustness, experimental design, model selection, and the 3Rs, but remain resource-intensive; limited research examines broader stakeholder needs for evidence-support systems.Tools automate screening, extraction, and evidence organization, while SyRF has explored interactive platforms.Source: §2.1 · p. 2

2.2 AI-Assisted Scientific Literature Analysis

  • AI literature tools support retrieval, screening, extraction, and multi-step exploration, while effective systems also need to fit users’ goals, expertise, and practices [24].This work extends prior research to animal and translational research through a curated dashboard used as a design probe.Source: §2.2 · p. 3

2.3 Trust, Verification, and Human Oversight in AI-Assisted Evidence Work

  • Trust and appropriate reliance depend on inspectable, source-linked outputs and communication of system capabilities and limitations [3] [41] [58].These concerns remain underexplored where stakeholders judge heterogeneous evidence for quality, relevance, and translational value; this study examines related practices and oversight requirements.Source: §2.3 · p. 3

3 Methods

The study used semi-structured interviews with 13 stakeholders to examine evidence practices and AI-support requirements, then gathered feedback on a curated animal-and-clinical evidence dashboard. Interview transcripts were analyzed thematically, with collaborative theme development by the author team.

3.1 Participants

  • The study recruited 13 participants with relevant literature-use experience through professional networks and purposive sampling across four stakeholder categories.Participants were based in Switzerland; categories were basic research, translational research, animal welfare and the 3Rs, and regulatory and ethics roles.Source: §3.1 · p. 3

3.2 Procedure and Analysis

  • Interviews lasted around one hour and covered evidence workflows, information needs, and challenges before participants explored the dashboard and discussed its usefulness and trustworthiness.The two interview stages included literature practices and thinking aloud while using the dashboard.Source: §3.2 · p. 3
  • Researchers used reflexive thematic analysis, initially coding transcripts independently and then iteratively organizing and reviewing candidate themes as a team.They revisited excerpts and jointly developed interpretations for the findings.Source: §3.2 · pp. 3–4

3.3 Curated Evidence Dashboard

  • The dashboard linked preclinical neuroscience publications with clinical trials through drug–disease relationships and enabled filtering and inspection of studies and animal-study characteristics.Figure 2 shows evidence counts, animal species distributions and trends, and clinical-trial development phases.Source: §3.3 · p. 4, Figure 2 · p. 5

4 Results

The results describe how participants’ roles shaped evidence practices and judgments under incomplete, heterogeneous evidence, and what they valued in AI-supported evidence work. They also cover participants’ assessments of the dashboard and identified needs for role-specific detail, inspectable sources, human judgment, and workflow and animal-welfare extensions.

4.1.1 Evidence Use and Appraisal Depended on Role and Decision Context.

  • Evidence use, extraction, and appraisal followed stakeholders’ roles and decision contexts, so the same study could receive different judgments despite structured frameworks.Researchers focused on experimental planning and parameters, 3R stakeholders on welfare and refinement, and regulatory reviewers on compliance and risk of bias; RoB 2 did not eliminate disagreement or expert judgment.Source: §4.1.1 · pp. 5–6

4.1.2 Participants Relied on Selective Search and Personal Evidence-Management Strategies.

  • Because comprehensive searches were often impractical, participants combined selective search heuristics with individualized evidence-management systems.They relied on familiar databases, citation chains, networks, conferences, and journal monitoring, while using personal spreadsheets, reference taxonomies, or computational pipelines.Source: §4.1.2 · p. 6

4.1.3 Evidence Was Absent, Inaccessible, or Incompletely Reported.

  • Evidence was missing, inaccessible, or poorly reported, limiting study evaluation and comparison.Participants described underrepresented negative results, access barriers, missing experimental details such as species, dose, and sample size, and inconsistently reported welfare and refinement information that could obscure whether refinements were tested or simply unreported.Source: §4.1.3 · pp. 6–7

4.1.4 Cross-Study and Cross-Species Synthesis Required Interpretive Judgment.

  • Cross-study and cross-species synthesis required judgment about which similarities and differences mattered to the translational question.Variation in models, methods, outcomes, and analyses impeded comparison; cognitive and behavioral links across species were especially uncertain, and participants also lacked quantitative evidence such as effect sizes and dose–response relationships.Source: §4.1.4 · p. 7

4.1.5 AI Use Varied, but Reliability Concerns Limited Trust.

  • AI use ranged from minimal to workflow replacement, with common uses in search, screening, summarization, and document querying but reliability concerns shifting work toward verification.Participants reported fabricated sources, inaccurate summaries, and uncertainty about search completeness; source-level traceability was limited, and possible systematic overstatement of translational evidence required domain expertise to detect.Source: §4.1.5 · p. 7

4.2.1 Support Tools Should Adapt to the User’s Research Context.

  • Participants wanted tools tailored to users’ roles, research questions, and experimental contexts, with relevance explanations and interactive exploration.They also envisioned persistent topic workspaces and experiment-level records that separate multiple arms and expose relevant details without requiring users to read entire papers.Source: §4.2.1 · p. 8, §4.2.2 · p. 8

4.2.3 Support Tools Should Connect and Evaluate Evidence Across Research Domains.

  • Participants wanted tools to normalize terminology and connect animal models, mechanisms, and outcomes with clinical evidence, including contradictory findings and prior experiments.Such links could help trace clinical observations to animal studies and compare proposed experiments with existing work to assess justification, refinement, or replacement options.Source: §4.2.3 · p. 8

4.2.4 Trustworthy Evidence Support Requires Human Judgment and Inspectable Outputs.

  • Participants wanted AI outputs to remain inspectable and uncertainty-aware, with substantive claims traceable to sources and humans retaining responsibility for interpretation and quality judgments.They distinguished extracting how experiments were conducted from interpreting why choices were made, and cautioned that study quality depends on research questions and context.Source: §4.2.4 · p. 8
  • Participants saw AI evidence support as a way to assemble relevant literature quickly, while leaving contextual assessment and final conclusions to human experts.P09 wanted a system to assemble the literature within days of a stakeholder question, not provide a definitive answer.Source: §4.2.4 · pp. 8–9

4.3.1 Participants Perceived the Dashboard as Supporting Rapid Study Screening.

  • Participants found the dashboard’s overview, tables, filters, and visualizations useful for identifying study characteristics and deciding which publications merited closer inspection.They reported that it was clear and easy to navigate after a brief introduction, and that it helped them screen without first reading every publication in full.Source: §4.3.1 · p. 9

4.3.2 The Dashboard Required More Detailed and User-Specific Exploration.

  • Dashboard usefulness depended on role, prior knowledge, and decision context, with participants requesting more experiment-level detail, research-specific search and filters, and personalized displays.Basic researchers sought keyword search across titles, abstracts, or full texts, while pharmaceutical and regulatory participants also wanted greater detail.Source: §4.3.2 · p. 9

4.3.3 Further Analysis Should Be Supported In-Platform or Through Interoperability.

  • Participants wanted dashboard tools for organizing and analyzing evidence, retained collections and shared workspaces, or exports and integrations with existing workflows and evidence resources.Requested options included sorting, filtering, notes, custom ratings, spreadsheets, reference managers, machine-readable formats, APIs, and links to publications, registries, repositories, and databases.Source: §4.3.3 · p. 9

4.3.4 Trust Depends on Auditability, Coverage, and Independent Testing.

  • Participants expected to verify dashboard outputs against original publications and wanted extracted values and summaries linked to supporting passages so errors could be checked and corrected.They described checking a small sample of studies before relying on the system.Source: §4.3.4 · pp. 9–10
  • Because retrieval coverage was difficult to verify, participants expected to compare the dashboard with PubMed or other search systems and wanted transparency about its scope and updating.They requested information on databases, publication types, date ranges, and latest updates, along with date filters and continued updating.Source: §4.3.4 · p. 10

4.3.5 Potential 3R Value Depends on Concrete Decision Contexts.

  • Participants saw potential 3R value in better-informed animal study planning and review, with explicit welfare data and links to alternatives, disease models, and related human trials.Assessing whether expected benefits justify potential harms could help justify models, but formal review would require more detailed evidence.Source: §4.3.5 · p. 10

5 Discussion

Evidence work varies by stakeholder role, individual practice, and decision context, while the underlying evidence base is often incomplete and difficult to compare. AI may help organize and assess evidence, but systems should preserve traceability, uncertainty visibility, and expert interpretation.

5.1 Role- and Practice-Sensitive Tools over a Shared Evidence Layer

  • Evidence-support systems should tailor views and interaction to users’ responsibilities, while allowing priorities and criteria to remain visible and revisable within roles.Stakeholders within the same professional group also differed in tools, search strategies, and expertise-driven judgments.Source: §5.1 · p. 10
  • Planning, welfare review, licensing, and translational assessment are interdependent, supporting tools that adapt a partially shared evidence layer to each decision and access need.Evidence and rationale recorded during planning could inform later assessments; some regulatory, institutional, or commercial evidence may require differentiated access.Source: §5.1 · p. 10

5.2 Supporting Systematic Assessment under Evidence Limitations

  • Researchers relied on imperfect heuristics amid heterogeneous, incompletely reported evidence, while needing clearer accounts of what is known, uncertain, or conflicting before designing experiments.Structured search, selection, extraction, and comparison could make evidence and criteria more explicit; semantic representations may support comparison but do not establish direct comparability or translational relevance.Source: §5.2 · pp. 10–11
  • AI checks of protocols, preregistrations, and manuscripts may improve future reporting, but cannot recover unpublished findings or open restricted evidence.The authors call for coordinated efforts to strengthen open science and reporting transparency.Source: §5.2 · p. 11

5.3 Designing for Verifiable, Contextualized, and Collaborative AI Support

  • Participants envisioned AI as a first-pass synthesizer that gathers and organizes evidence while experts verify sources, assess coverage, and interpret findings in context.Validation needs vary by task; study-level tables and interactive evidence maps can provide structured overviews while retaining routes to original studies.Source: §5.3 · p. 11
  • Conversational exploration and human–AI collaboration may support progressive sensemaking, with AI aggregating evidence and experts retaining critical assessment and contextual interpretation.Future systems must balance shared evidence with different evaluative criteria, visible incompleteness, and automation with traceability and expert verification.Source: §5.3 · pp. 11–12

6 Limitations and Future Work

The study’s findings are bounded by its Swiss sample, self-reported workflows, brief dashboard evaluation, and a single prototype and literature collection. The authors identify broader sampling, observational and longer-term evaluation, and dashboard development as future work.

Sample scope and transferability

  • The 13 participants were based in Switzerland, and the sample excluded practising clinicians and FDA or EMA regulators, limiting transferability to other groups and contexts.Some participants fit multiple categories and were grouped by their main professional role.Source: §6 · p. 12

Self-reported workflows and first-impression evaluation

  • Workflow accounts relied on participant recall, and the dashboard received one brief think-aloud evaluation, so findings concern perceived usefulness and needs rather than sustained adoption or measured outcomes.The study did not assess efficiency gains, extraction accuracy, or 3R outcomes; future work could observe workflows and evaluate real tasks over longer periods.Source: §6 · p. 12

Scope and future development of the dashboard

  • Feedback concerned one dashboard version and literature collection, so it applies mainly to that prototype rather than AI-assisted evidence tools generally.Future versions could separate experiments, include quantitative outcomes and welfare data, support semantic search, link extractions to source sentences, and clarify collection scope and update status.Source: §6 · p. 12

7 Conclusion

Animal and translational evidence work comprises role- and decision-dependent practices constrained by incomplete, inaccessible, unpublished, and difficult-to-compare evidence. The interviews indicate that AI can help gather and organize evidence, while systems should preserve traceability, uncertainty visibility, and expert judgment.

7 Conclusion

  • Evidence work is shaped by roles and decisions rather than a single workflow, and the study examined practices and dashboard responses through interviews with 13 stakeholders.Participants described constraints including incomplete reporting, restricted access, unpublished findings, and difficult cross-study and cross-species comparison.Source: §7 · p. 12
  • AI-assisted tools can support evidence retrieval and structuring, but may shift effort toward verification and expert interpretation.Source: §7 · p. 12
  • Future systems should offer role- and practice-sensitive views over shared evidence, expose coverage and uncertainty, preserve source traceability, and retain human control over consequential judgments.Their aim is to reduce gathering and organization effort while supporting expert verification, contextual interpretation, and judgment.Source: §7 · p. 12

Ethics and Privacy Statement

The study received institutional data-protection review and did not require Cantonal Ethics Committee submission under the Swiss Human Research Act. The interview protocol introduced the study, allowed participants to stop or skip questions, and allocated time to background, problem exploration, and dashboard feedback.

Ethics and Privacy Statement

  • The institutional data-protection process determined that Cantonal Ethics Committee submission was not required; participants consented, could withdraw, and permitted recording.Identifiers were removed, pseudonyms and broad professional categories were used, and data were stored securely with research-team-only access.Source: Ethics and Privacy Statement · p. 12

Introduction

  • The interview opened by explaining its purpose and the interviewer’s use of follow-up “why” questions to clarify participants’ experiences.Participants were told there were no right or wrong answers and could decline questions or stop at any time.Source: Introduction · p. 15

A Interview Protocol

  • The approximately 60-minute session allocated 5 minutes to warm-up, 25 to problem exploration, and 30 to dashboard demonstration and feedback.Warm-up questions covered career level, role, animal-work experience, the participant’s understanding of translational research, and prior systematic-review experience.Source: Introduction · p. 15, Warm-Up · p. 15

Literature-Analysis Workflow.

The interview guide covered stakeholders’ literature practices, evidence challenges, animal-to-human connections, and desired tool support.

Literature-Analysis Workflow.

  • The workflow questions asked how participants search, explore, evaluate, and organize biomedical literature, including databases, other resources, and time spent.Source: Literature-Analysis Workflow · p. 15

Challenges and Information Needs.

Connecting Animal and Human Evidence.

The Ideal Tool.

  • Participants were asked what an ideal literature-analysis tool should ease, show, and achieve for translational research.Source: The Ideal Tool · p. 15

Topic 2: System Demonstration and Evaluation

The demonstration protocol gave participants time to explore the dashboard, then asked about their reactions, potential use, and desired additions.

Interviewer Instructions.

  • The interviewer was instructed to describe the available data and views, then allow about three minutes of think-aloud exploration.Source: Interviewer Instructions · p. 15

Initial Reactions.

  • Initial questions elicited what participants thought the tool did, what stood out, and whether its presentation was clear or suitable.Source: Initial Reactions · p. 15

Potential Use.

  • Participants were asked how and when they might use the output, whether it would complement existing work, and who else might benefit.Source: Potential Use · p. 15

Missing Information and Desired Functionality.

Trust, Risks, and Limitations.

The material covers questions about trust and 3Rs decision support alongside participants’ requested extraction, search, and system-design capabilities.

Trust, Risks, and Limitations.

Potential Contribution to the 3Rs.

  • A prompt asked where a tool could support 3Rs decisions, including finding existing evidence, examining translational assumptions, and considering alternatives or study justification.Source: Potential Contribution to the 3Rs · p. 16

Closing Questions

  • The closing questions invited participants to add topics, raise anything else, or ask questions.Source: Closing Questions · p. 16

B Desired Evidence-Extraction Characteristics

C Desired Search and Filtering Functions

D Design Criteria for AI-Assisted Evidence Systems

E Qualitative Analysis Materials

The appendix provides examples of transcript coding, codebook development, and collaborative theme development from the qualitative analysis.

E Qualitative Analysis Materials

Loading 2609.39808v1…