Source-linked AI summary
Advancing Clinical Genomic Analysis with a Skill-Centred Agent
Qiaoyu Zheng, Weike Zhao, Yanjie Fan, Ye Cao, Mengqi Jiang, Xiao Zhou, Xiaoman Zhang, Pengcheng Qiu, Yongguo Yu, Yanfeng Wang, Ya Zhang, Chaoyi Wu, Weidi Xie
TL;DR
Clinical genomic analysis is bottlenecked by fragmented evidence, expertise-intensive reasoning, and adaptive patient-specific workflows. ClinGeX organizes domain resources into executable skills that an LLM agent retrieves and composes for each task. It outperformed comparators in automated analysis and improved clinician concordance across three scenarios, while its real-world clinical impact remains to be established.
Problem
Genomic interpretation is constrained by fragmented evidence, expertise-intensive assessment, and dynamic workflows, leaving comprehensive analysis reliant on expert synthesis.
Source: §1 · p. 1Method
ClinGeX adapts general-purpose LLMs into clinically grounded genomic assistants by retrieving and composing curated executable skills for each analysis.
Source: §1 · pp. 2–3Results
ClinGeX outperformed comparators on automated analysis and improved clinician concordance across all three scenarios, reaching 71.7% concordance with expert consensus across 223 cases.
Source: §1 · p. 3Takeaways & Limitations
The findings support ClinGeX as a potential Co-Clinician that augments rather than replaces expert genomic decision-making.
Source: §1 · p. 3The prospective assessment focused on workflow outcomes rather than clinical endpoints, so downstream clinical impact requires evaluation in larger prospective studies.
Source: Limitations · p. 14
Abstract
from arXiv · showhide
Clinical genomic analysis requires experts to integrate evolving guidelines, databases, and computational evidence. As sequencing has become increasingly accessible, this complex and labor-intensive process has become a major bottleneck to the timely use of genomic information in clinical decision-making. To fill this gap, we developed ClinGeX, a large-language-model agent that composes 105 curated executable skills through task-specific retrieval and dependency-aware orchestration. On 223 temporally held-out benchmark cases, ClinGeX achieved 71.7\% macro-averaged task-level concordance with expert reference classifications, compared with 45.7\% for the strongest comparator. In a three-condition, counterbalanced human-AI collaboration study, 45 clinical genomic professionals assessed 135 clinically derived cases under routine (no AI assistance), DeepSeek-assisted and ClinGeX-assisted conditions. ClinGeX assistance increased raw case-level concordance across criterion-level variant analysis, genomic interpretation in rare disease and molecular oncology decision-making (51.1\%, 57.8\%, 64.4\% versus 15.6\%, 31.1\%, 20.0\% in comparator conditions) and reduced reported interpretation time in all scenarios. During an exploratory one-month real-world non-interventional assessment, 11 experienced professionals submitted feedback on 96 self-selected analysis episodes and reported frequent conclusion adoption and time savings. These findings support executable domain skills as a promising approach to augment specialist genomic interpretation.
1 INTRODUCTION
Clinical genomic interpretation is a bottleneck because evidence is fragmented, reasoning is expertise-intensive, and workflows vary by patient and question. ClinGeX combines curated executable skills with task-specific retrieval and was evaluated in automated, clinician-collaboration, and routine-use studies.
1 INTRODUCTION
- Genomic interpretation remains a bottleneck because evidence is fragmented, variant assessment requires contextual expertise, and patient-specific questions demand adaptive workflows.Conventional bioinformatics tools can automate individual steps, but comprehensive analysis still relies heavily on experts to synthesize actionable conclusions.Source: §1 · p. 1
- The skill-centred framework converts genomic resources into 105 specialized skills that an LLM agent retrieves and composes for different clinical questions.Skills are organized by operational role and prerequisites, with detailed instructions disclosed as needed.Source: §1 · pp. 2–3
- 71.7% concordance with expert consensus across 223 cases was 26.0 percentage points above the strongest comparator; 74.9% of ClinGeX reports were judged directly adoptable without substantial revision.The human-AI study also found higher case-level concordance with ClinGeX assistance in all three scenarios.Source: §1 · p. 3
- ClinGeX-assisted clinicians achieved 51.1%, 57.8%, and 64.4% concordance across three scenarios, versus 22.2%, 28.9%, and 31.1% with clinicians only and 15.6%, 31.1%, and 20.0% with DeepSeek.ClinGeX assistance also improved evidence identification and reduced median criterion-level variant-analysis time from 60 to 15 minutes.Source: §1 · p. 3
- In a one-month routine-use assessment, 11 professionals submitted feedback on 96 analyses; 94.8% of tasks had fully or partially adopted conclusions, and 77.3% of difficult tasks reported savings exceeding 15 minutes.This was a preliminary assessment of self-directed, naturally occurring tasks.Source: §1 · p. 3
2 Results
The Results section evaluates ClinGeX across benchmark, controlled collaboration, and exploratory real-world settings, covering genomic-analysis concordance, intermediate outcomes, reported clinical utility, and workflow support. It also presents two cases illustrating the resolution of interpretation ambiguities.
2.1 ClinGeX Overview
- ClinGeX accepts free-text requests and optional multimodal materials, then returns reports with concise workflow, evidence-source and intermediate-output summaries.Its skill-centred architecture comprises 105 atomic clinical-genomics skills selected and orchestrated for each task.Source: §2.1 · p. 3
2.2 System-level Analysis of ClinGeX on Automated Benchmarks
- Blinded expert review found the highest overall-acceptability ratings for ClinGeX reports in all three benchmark tasks.Ratings were 88.6% for SNV interpretation, 66.7% for gene–disease validity and 67.9% for clinical actionability; ClinGeX also rated higher across all assessed dimensions in the first two tasks.Source: §2.2 · p. 4
- ClinGeX achieved the highest final concordance in each benchmark task: 89.9% for SNV interpretation, 53.0% for gene–disease validity and 72.3% for clinical actionability.The corresponding best baseline results were 49.4% for DeepResearch, 35.5% for Biomni and 52.3% for TxAgent, respectively.Source: §2.2 · p. 4, Figure 2 · p. 5
2.3 Evaluation of ClinGeX in the MRMC Human-AI Collaboration Study
- In the MRMC study, ClinGeX improved evidence recall, evidence-strength concordance and phenotype interpretation, and increased molecular-oncology response accuracy versus both comparison conditions.Rare-disease phenotype-interpretation accuracy was 23.7 percentage points higher than human-only assessment, while its 8.8-point advantage over DeepSeek was not statistically significant; molecular-oncology free-response accuracy rose 38.3 points versus DeepSeek and 22.2 points versus human-only assessment.Source: §2.3 · pp. 7, 9
- ClinGeX assistance produced the highest case-level concordance in all three professional scenarios, with observed concordance of 51.1%, 57.8% and 64.4%, respectively.For criterion-level variant analysis, rare-disease interpretation and molecular oncology, corresponding human-only values were 22.2%, 28.9% and 31.1%, and DeepSeek-assisted values were 15.6%, 31.1% and 20.0%.Source: §2.3 · p. 7, Figure 4 · p. 8
- ClinGeX’s increased concordance was not accompanied by poorer confidence–correctness alignment across the assessed scenarios.It had the highest correlation and lowest Brier score in rare-disease interpretation and molecular oncology; for criterion-level variant analysis, 8 of 9 fully certain ClinGeX-assisted assessments were correct, with the small group limiting formal comparison.Source: §2.3 · p. 9
- ClinGeX shortened median time per question versus human-only assessment in variant analysis and rare-disease interpretation, but not significantly in molecular oncology.Times fell from 60 to 15 minutes and from 30 to 15 minutes, respectively (both P = 0.003); molecular-oncology time fell from 15 to 11 minutes (P = 0.078). ClinGeX also received higher mean user-experience ratings than DeepSeek on most dimensions, and 29 participants initiated 239 further conversations within two weeks.Source: §2.3 · p. 9
2.4 Prospective Assessment of ClinGeX on Real World Feedbacks
- In 96 exploratory real-world episodes, experts rated ClinGeX’s conclusion fully adoptable in 70 (72.9%), partially adoptable in 21 (21.9%) and not adoptable in 5 (5.2%).Experts reported saving more than 15 minutes in 40 episodes (41.7%) and 5–15 minutes in 50 (52.1%); savings exceeding 15 minutes were concentrated in difficult episodes, while most conclusions were fully adopted.Source: §2.4 · p. 11
- Across 96 episodes, mean helpfulness scores were 1.29, 1.35, 1.33 and 1.34, with some or substantial help reported in 93.8%, 94.8%, 92.7% and 95.8% of episodes across four workflow dimensions, respectively.The dimensions were information standardization, numerical analysis, literature analysis, and analytical or procedural guidance; 11 experienced professionals submitted feedback.Source: Figure 6 · p. 12, §2.4 · p. 13
- In the LRP6 case, ClinGeX detected that family evidence cited for p.Arg611Cys concerned p.Tyr418His, and the clinician reported full conclusion adoption and 5–15 min saved.The clinician adopted four of five recommended evidence criteria and said the analysis helped avoid incorrect use of family evidence.Source: §2.4 · p. 13, Figure 7 · p. 13
- For a low-allele-fraction TP53 finding in blood, ClinGeX distinguished possible variant origins and germline from somatic interpretation, prompting confirmation before hereditary risk assignment.The clinician reported full conclusion adoption and more than 15 min saved; the variant origin remained unconfirmed in the available record.Source: §2.4 · p. 13, Figure 7 · p. 13
3 DISCUSSION
The discussion presents ClinGeX’s skill-centred design as a potential way to support genomic interpretation and clinician workflow, while noting that the evaluations do not establish generalizability across genomic tasks or downstream clinical impact.
3 DISCUSSION
- Resource curation, graph-organized skills and progressive disclosure support task-specific retrieval; the discussion also reports workflow utility and local deployment flexibility.Source: §3 · p. 14
- ClinGeX raised case-level concordance from 22.2%, 28.9% and 31.1% to 51.1%, 57.8% and 64.4% across the three scenarios.Source: §3 · p. 14
- Median time per case fell from 60 to 15 min for criterion-level variant analysis and from 30 to 15 min for rare-disease interpretation.Source: §3 · p. 14
Limitations
- The MRMC study covered three scenarios selected around participating centres’ specialties and caseloads, so generalizability to other genomic areas, including polygenic risk assessment, is unestablished.Source: Limitations · p. 14
- The MRMC study was powered for assistance’s main effect, and concordance with expert consensus reflects interpretive quality rather than patient outcomes.Establishing effects on clinical management requires a prospective randomized trial with prespecified clinical endpoints.Source: Limitations · p. 14
- The prospective deployment assessed workflow outcomes rather than prespecified clinical endpoints; downstream clinical impact requires evaluation in larger prospective studies.Source: Limitations · p. 14
4 METHODS
The methods describe ClinGeX’s development as a skill-centred genomic-analysis agent and its evaluation through a counterbalanced human–AI study and an exploratory, non-interventional assessment of routine work. They cover resource curation, skill orchestration, clinical task scenarios and study procedures, including structured feedback on real-world episodes without collecting patient-level data.
4.1 Development of ClinGeX
- ClinGeX’s six-stage development scanned five resource categories and used LLM screening and expert consensus to select 105 resources for executable guide, search, and analysis skills.A builder–evaluator loop generated skill-specific tests and revised skills until failures were resolved or five revision cycles were reached.Source: §4.1 · pp. 15–16
- A graph of skill relatedness and prerequisite dependencies supported task-specific retrieval, workflow planning, skill execution by sub-agents, and multimodal file processing in the online system.The system converted PDFs to text and generated image descriptions before combining them with the user’s query.Source: §4.1 · pp. 16–17
4.2 Conduct of the MRMC Human-AI Collaboration Study
- Participants completed three scenarios—ACMG criterion-level variant classification, rare-disease variant–phenotype interpretation, and molecular-oncology action decisions—under routine, DeepSeek-assisted, or ClinGeX-assisted conditions.Only the assigned AI system was permitted in each assisted condition; otherwise, participants could use the resources available in routine practice.Source: §4.2.1 · pp. 17–18, §4.2.2 · p. 18
4.2.3 Data Preparation
- The study used 45 cases per scenario, 135 total, assembled from validated variants, curated rare-disease cases, and expert-reviewed molecular-oncology reports.The oncology cases were selected from 323 reports published after 2025; cases were de-identified and source-identifying details were redacted or reformulated.Source: §4.2.3 · p. 18
4.2.4 Participant Recruitment
- The study enrolled 45 readers and nine adjudicators; Latin-square-like allocation gave each reader nine non-overlapping cases across conditions, with each case assessed once per condition by different readers.Readers worked independently, spread their cases across two weeks, and received no case-specific guidance from expert panels.Source: §4.2.4 · p. 19, §4.2.5 · p. 19
4.2.6 Participant Post-study Survey
- After the study, readers completed an anonymous 13-item survey, including estimates of average time per case and five-point ratings of confidence, reuse willingness, satisfaction, system smoothness, and reliance.Estimated time was used for the main time comparison because platform timestamps could include pauses or unrelated clinical work.Source: §4.2.6 · pp. 19–20
4.3 Prospective, Non-interventional Feedback Collection of ClinGeX
- An exploratory prospective assessment collected structured feedback on self-selected routine clinical or research analysis tasks, without a fixed task set or control condition; outputs could not inform clinical decisions.The assessment was intended to characterize task-level utility in a workflow-relevant, non-interventional setting.Source: §4.3 · p. 20
4.3.1 Study Design and Participant Recruitment
- Eleven experienced clinical-genomics professionals used ClinGeX during routine work for one month and were asked to report objectively on episodes regardless of whether assistance helped.All had more than five years of variant-interpretation experience and prepared clinical reports for more than 500 cases annually; none had participated in the preceding MRMC study.Source: §4.3.1 · p. 20
4.3.2 Feedback Platform and Unit of Evaluation
- Each feedback submission evaluated one completed analysis episode as a whole, even when it involved multiple ClinGeX interactions; patient names, records, genomic data, and other patient-level information were not collected.Feedback coverage was at least 86.5% of platform conversations, providing some reassurance against selective reporting.Source: §4.3.2 · pp. 20–21
4.3.3 Feedback Measures
- Required categorical measures captured perceived difficulty, adoption of ClinGeX’s principal conclusion, and reported time saved or lost.For variant episodes, conclusion adoption reflected retention of ClinGeX’s proposed P/LP/VUS/LB/B classification in the participant’s final assessment.Source: §4.3.3 · p. 21
- Optional items recorded recommended and adopted evidence-criterion counts when applicable and a free-text description of ClinGeX’s contribution; categorical items were required.Submissions were stored with participant and selected-conversation identifiers.Source: §4.3.3 · p. 21
- The feedback form also assessed help with information standardization, numerical analysis, literature analysis, and analytical or procedural guidance.Each domain used categorical response options ranging from substantial help to little or no help.Source: §4.3.3 · p. 21
7 Extended Data
The Extended Data figures describe ClinGeX’s workflow and skill library, and outline evaluation designs for the system and human–AI collaboration. They also show how skills are compiled and selected for tasks.
Extended Data Figure 1
- The system-architecture evaluation covers three reasoning tiers, architecture scoring, component ablations, base-model metrics and evaluations across genomic resources.Responses to identical task inputs are graded by two independent LLM judges; the ablation compares a plain LLM, single-component variants and full ClinGeX.Source: Figure 1 · p. 26
Extended Data Figure 2
- The collaboration study assigns 45 professionals to three genomic scenarios and counterbalances routine, DeepSeek-assisted and ClinGeX-assisted case sets.Each clinician assesses nine cases, three per condition; structured responses use rule-based matching, while free-text and borderline items are reviewed by a blinded expert panel.Source: Figure 2 · p. 27
Extended Data Figure 3
- ClinGeX takes a clinical-genomics query and optional supporting files, selects a model, invokes relevant skills and tools, and returns an evidence-supported conclusion.The illustrated example is an SNV pathogenicity assessment under the ACMG/AMP framework.Source: Figure 3 · p. 28
Extended Data Figure 4
- The skill-compilation process iteratively tests and repairs expert-reviewed prototypes, adding only skills that pass all frozen tests to the 105-skill library.The suite includes positive, boundary, malformed-input, batch-processing and simulated external-failure cases; regression testing continues for up to five iterations.Source: Figure 4 · p. 29
Extended Data Figure 5
- Task-specific skills are selected by ranking embedding matches, expanding related skills through SkillNet and adding required dependencies.The final subset combines initial seed skills, graph-expanded skills and execution dependencies.Source: Figure 5 · p. 30
A.1 Main System Prompt of ClinGeX
The section reproduces the complete top-level ClinGeX/SKILL.md prompt. It identifies the prompt’s operating stance, routing order, progressive-disclosure policy and path-resolution rules as auditable elements.
A.1 Main System Prompt of ClinGeX
- The reproduced material is the complete top-level ClinGeX/SKILL.md router prompt.The text says its line wrapping, dash characters and directory-tree glyphs were normalized for typesetting.Source: §A.1 · p. 31
- The prompt’s operating stance and routing order are among the elements presented for audit.The passage identifies these topics but does not specify their contents.Source: §A.1 · p. 31
- The prompt’s progressive-disclosure policy and path-resolution rules are also identified as auditable elements.The passage identifies these topics but does not specify their contents.Source: §A.1 · p. 31
Front matter
ClinGeX is presented as an agentic chatbot for gene-related interpretation and analysis tasks, routing requests to specialized resources. Its router uses a restrained, evidence-weighing stance and progressive disclosure, with rules for reading content and resolving paths.
Front matter
- ClinGeX assists with literature-related and gene-related clinical interpretation, analysis and recommendation tasks by routing requests to specialized resources.The listed resources include authoritative guidelines, analysis tools and external knowledge databases.Source: Front matter · p. 31
- The top-level prompt acts as a router that directs concrete decisions to the relevant sub-skill category.The agent descends into that sub-skill’s own SKILL.md.Source: Front matter · p. 31
- The operating stance separates observed facts, authoritative rules, tool results, source assertions and inferences while weighing conflicts, evidence quality and context.It cautions against over-trusting any single database, publication, guideline summary, prediction score or prior model belief.Source: Front matter · p. 31
- Progressive disclosure reveals documents and routes stepwise, so newly surfaced information may require the agent to revise its plan.Nothing is preloaded, and each document reveals only its own scope and may point to deeper material.Source: Front matter · p. 31
- Sub-skill documentation is retrieved with get_documentation=True, while local bundle files are read using the described helper scripts and absolute paths.Documentation responses provide paths and base directories used by the path-resolution rules.Source: Front matter · p. 31
- Path resolution depends on the containing document: relative paths use its directory, while top-level bundle-root paths use the supplied BASE_DIR.Resolved paths are normalized to absolute paths and verified before use with a filesystem helper.Source: Front matter · p. 31
1. The three sub-skill categories
The bundle organizes sub-skills around common clinical gene-task needs in three categories: guideline lookup, external knowledge retrieval, and computation or prediction.
1. The three sub-skill categories
- guide/ covers authoritative procedural standards, including which guideline, section, and rule apply.These sub-skills address the need to consult a guideline.Source: §1 · p. 31
- search/ retrieves current, sourced facts from public databases and literature rather than relying on memory.These sub-skills address external knowledge lookup.Source: §1 · p. 31
- analysis/ computes scores, classifications, and structured evidence codes from a variant or gene.It covers computation and prediction needs.Source: §1 · p. 32
2. Routing protocol (follow this every time)
The routing protocol requires planning from relevant guideline documentation, executing documented sub-skills step by step, and revising the plan as evidence emerges. The final answer should state its conclusion, evidence sources, conflicts, and uncertainty.
2. Routing protocol (follow this every time)
- Identify potentially relevant guidelines, explain why each applies, explicitly state when none applies, and read the documentation for each candidate.Plan the next documents, sub-skills, and tools to use.Source: §2 · p. 32
- Execute step by step, revising the plan as new results, documents, or comparisons indicate further steps.Source: §2 · p. 32
- Decompose the task into knowledge points and operational steps, updating that decomposition after reading relevant guide documentation.Source: §2 · p. 32
- Before first use of any sub-skill, retrieve its SKILL.md documentation through ClinGeX; if unavailable, read that file directly using the prescribed helper and path convention.Use only documented actions and parameters; do not guess schemas.Source: §2 · p. 32
- After each substantive result, reassess whether new clues, conflicts, or limitations require another reference, tool, or narrower answer.Source: §2 · p. 32
- Synthesize the direct conclusion first, cite which skill produced each fact, weigh convergent and conflicting evidence, and state residual uncertainty, assumptions, and clinical risk.If no matching information surfaces, say so rather than guessing.Source: §2 · p. 32
3. Repository layout
The repository separates the router, generic OS-level helpers, reserved assets, and the guide, search, and analysis skill directories.
3. Repository layout
- The router is SKILL.md, while scripts/ contains OS-level helpers and assets/ is reserved and currently empty.Source: §3 · p. 32
- Guide skills are stored under skills/guide/<topic>/SKILL.md.Source: §3 · p. 32
- Search and analysis skills are stored under skills/search/<source>/SKILL.md and skills/analysis/<tool>/SKILL.md, respectively.Source: §3 · p. 32
4. OS-level helper scripts
The scripts/ directory provides generic, standard-library-only Python helpers for local file operations and code execution; these are not clinical tools.
4. OS-level helper scripts
- The four helpers read files, write files, list files, and run Python files or inline snippets.They are documented in scripts/README.md and use only the Python standard library.Source: §4 · p. 32
- read_file.py supports encoding fallback, size limits, line slicing, optional structured parsing, and JSON-envelope output.Source: §4 · p. 32
- write_file.py supports atomic writes, no-clobber, overwrite, and append modes, with optional directory creation, dry runs, and canonical JSON re-serialization.Source: §4 · p. 32
- list_files.py offers recursive listing with filters and controls for depth, symlinks, hidden files, and sorting.Source: §4 · p. 32
- run_python.py supports subprocess execution with argument forwarding, working-directory and environment settings, timeouts, and captured output.Each script accepts --help; --json produces a structured JSON envelope, and errors exit non-zero with a JSON error object on standard error.Source: §4 · p. 32
5. Authoring discipline
Sub-skill SKILL.md files under skills/ are the source of truth and must not be modified by users of this router.
- Sub-skill SKILL.md files under skills/ are the source of truth.Source: §5 · p. 32
- The source-of-truth files are located under skills/.Source: §5 · p. 32
- Users of this router must not modify those files.Source: §5 · p. 32
A.2.1 Skill Introduction
This section describes how ClinGeX’s skill-centred architecture was assembled and selected, including skill curation, retrieval, benchmark construction and evaluation. It also details the human–AI collaboration study and prospective deployment assessment, covering platforms, reference standards, outcomes, task construction and feedback coverage.
A.2.1 Skill Introduction
- A ClinGeX skill packages a recurrent clinical-genomics operation with workflow and evidence instructions, while optional scripts, references and assets support executable actions and other resources.Its required SKILL.md provides discovery metadata and instructions; full content and optional resources load only after selection. Construction action types and deployment classes are separate taxonomies, with deployed directory counts of 27 guide, 40 search and 38 analysis skills.Source: §A.2.1 · p. 33
A.2.2 Resource Discovery across the Five Capability Categories
- Candidate discovery distinguished raw records, normalized candidates and retained prototypes; category boundaries followed the primary operation, and conservative linkage favored residual duplicates over false merges.The predictor landscape was missense-enriched rather than complete across variant classes.Source: §A.2.2 · p. 33
A.2.3 Category-Specific LLM Triage and Expert Selection
- LLMs structured and prioritized candidates, but eight experts from seven institutions made final selections using clinical utility, standards compatibility, unmet need, legitimacy, stability, reliability and redundancy.Candidate triage used category-specific criteria and deterministic weighted scores; maintenance or adoption not verifiable from static metadata was flagged for live verification.Source: §A.2.3 · pp. 33–34
- 105 skill prototypes were frozen after expert review: 27 guideline-following, 13 entity-mapping, 30 data-source-lookup, 12 numerical-calculation and 23 model-based-prediction skills.Experts could retain lower-tier candidates that filled an unmet operation, remove redundant or non-invocable candidates, or consolidate candidates. The frozen landscape survey was not a preregistered systematic review, and coverage and triage repeatability were limited.Source: §A.2.3 · p. 34
A.2.4 Details of Skill Initialization and Iterative Optimization
- A builder agent converted each retained operation into a self-contained skill, and a separate test agent ran 5–20 predefined cases, prompting revisions and full-suite reruns for failures.Iteration stopped when no actionable failure remained or after five cycles; skills could comprise procedural instructions alone or include scripts, indexed references and static assets.Source: §A.2.4 · p. 34, Prompt used to initialize a ClinGeX… · pp. 35–36
A.3.1 Semantic Representations
- Skill retrieval used compact Scope and Use-when text encoded into 1,024-dimensional vectors, while dependency edges were scored separately from semantic similarity.All 5,460 unordered skill pairs received five 0–4 scores and a dependency label; the completed graph had mean relationship score 5.45 (s.d. 2.99), 28 directed dependencies and no mutual dependencies.Source: §A.3.1 · p. 36, §A.3.2 · p. 37
A.3.3 Per-Message Seed Retrieval and Graph Expansion
- For each message, ClinGeX decomposed needs, retrieved the top 30 matching skills, added up to 15 one-hop related skills and forced literature-search skills, then recursively included hard prerequisites.Related-edge expansion required a score of at least 8; prerequisites were included regardless of similarity rank, edge threshold or expansion cap.Source: §A.3.3 · pp. 37–38
A.4.1 Benchmark Composition and Case Construction
- The benchmark used 223 temporally held-out cases with frozen references and supporting public evidence; its SNV subset included 79 cases sampled across five classification tiers.The SNV reference retained expert-panel classifications and criterion-level annotations; gene–disease cases used ClinGen evidence categories, while clinical-actionability cases retained source-native CIViC, OncoKB and PharmGKB ontologies.Source: §A.4.1 · p. 38, Representative complete case: SNV… · p. 39, Representative complete case:… · p. 40
Common benchmark instruction and user-prompt template
- Common benchmark prompts supplied task-specific instructions, case inputs, literature and required outputs, while free-text answers were converted into structured fields and missing or unrecognized fields retained as null.System-specific execution controls were archived, and classification and evidence metrics used frozen single-output records, separate from the repetition-and-timing workflow.Source: §A.4.2 · p. 41, Common benchmark instruction and… · pp. 41–42
A.4.3 Metric Implementation and Denominators
- Benchmark outputs were evaluated by deterministic code against frozen references, not by an LLM judge, using task-specific denominators and rules for classification, evidence and severity.SNV concordance required an exact five-tier match; sensitivity and specificity excluded VUS cases, while significant errors were direction reversals across positive and negative groups.Source: §A.4.3 · p. 42, §A.4.4 · p. 42
A.4.5 Gene–Disease Validity-Specific Evaluation Metrics
- Gene–disease validity metrics used fixed seven-category ranks, while evidence coverage measured reporting a usable score rather than item-level recall.Actionability concordance used 65 applicable-tier cases, whereas complete-answer accuracy used all 78; missing or unmappable labels remained in the severity-analysis denominator, and OncoKB resistance-category ranks followed a prespecified analysis convention.Source: §A.4.5 · p. 43, §A.4.6 · p. 43
A.4.7 Blinded Clinician Assessment of Report Quality
- Nine clinicians rated blinded reports for all 223 cases, with reviewers assigned by task and report identities independently permuted for each case.All 669 reports received eight ordered three-category ratings, yielding 5,352 ratings with no missing values or imputation; ratings did not alter references or enter automated metrics.Source: §A.4.7 · p. 44
A.5.1 Study Objective and Controlled Comparison
- The controlled comparison tested the complete ClinGeX architecture against an unenhanced DeepSeek-V4 baseline and five single-strategy enhancements on 426 frozen evidence-grounded items.Items were generated from 80 anchors across five databases, organized into three evidence-operation tiers, and scored against item-specific rubrics by GPT-5.5 and Claude-4.8-Opus; the models, prompts and hidden-reference controls were held fixed across conditions.Source: §A.5.1 · p. 44, §A.5.2 · p. 45, §A.5.3 · p. 45
A.5.6 Framework-Selection Result
- 83.2% was ClinGeX’s overall rubric-normalised score, versus 72.2% for unenhanced DeepSeek-V4 and 74.0–78.4% for the five single-strategy conditions; it ranked highest overall and in every reasoning tier.ClinGeX scored 70.1% on variant-interpretation items, 56.7% on gene–disease-validity items and 84.9% on clinical-actionability items—improvements of 15.2, 11.1 and 15.1 percentage points, respectively, over the unenhanced base model.Source: §A.5.6 · p. 48
A.6.1 Purpose and Experimental Design
- The agent-core experiment held ClinGeX’s architecture, skills, tools, instructions and 3,662-item input set fixed while comparing six frontier models across three clinical-genomics domains.Items were linked to publication-level evidence, and adversarial QA generation required interpretive or synthesis questions with quotations matching the current full text; four answerers and a source-masked arbiter checked items before inclusion.Source: §A.6.1 · pp. 48–49, §A.6.2 · p. 49, §A.6.3 · pp. 49–50
A.6.5 Five Evaluation Dimensions
- Each model received the same questions and retrieval entry points, with reference answers and construction annotations withheld; every model–item pair ran independently three times.Evaluation combined claim-coverage accuracy, five binary reasoning indicators, judge-free execution-trace statistics, evidence faithfulness and three-run robustness; an external judge was not one of the candidate cores.Source: §A.6.4 · p. 50, §A.6.5 · p. 50
System message
- The evaluator checked atomic-claim support, reasoning criteria and factual claims against retrieved evidence, while scoring code applied deterministic accuracy thresholds and quotation matching.The judge was not given the candidate model name.Source: System message · pp. 51–52
A.7.1 Dedicated Online Evaluation Platform
- The password-protected platform assigned nine task–condition combinations, withheld reference materials and alternate responses, and required readers to enter their own conclusions after any assigned AI consultation.Completion time was self-reported, and oncology readers submitted open responses before viewing candidate statements.Source: §A.7.1 · p. 52
A.7.2 Reference Standards and Expert Adjudication
- Each scenario panel froze a task-matched reference record, and primary concordance was calculated separately by scenario and condition across 45 assessments.Enumerated fields used prespecified mappings; free-text and borderline cases were resolved by source-masked panel consensus based on clinical equivalence.Source: §A.7.2 · p. 52, Table 19 · p. 53
A.7.3 Secondary Performance Outcomes
- Secondary outcomes assessed criterion-level evidence and classification, rare-disease pathogenicity and phenotype interpretation, and oncology free-response and statement-judgement accuracy.Eligible instances were micro-averaged within condition; task-specific rules included accepted adjacent levels for rare-disease phenotype accuracy and strict supported–unsupported boundary handling for oncology judgements.Source: §A.7.3 · p. 53, Table 20 · p. 54
A.7.4 Confidence–Correctness Alignment, Time, Survey and Continued-Use Outcomes
- Confidence–correctness alignment used ordinal confidence measures, while the main time comparison used readers’ post-study estimates of mean minutes per case and paired, Holm-adjusted tests.Continued use counted new ClinGeX conversations in the 14 days after completion, excluding assigned study sessions, and was analysed descriptively.Source: §A.7.4 · pp. 53–54
A.7.5 Statistical Analysis of Proportions and Effect Differences
- Condition effects were estimated with binomial logistic GLMMs using crossed reader and case random intercepts, with standardized probabilities, absolute differences and two-sided 95% Wald intervals.Pairwise Wald contrasts were Holm-adjusted within each outcome and scenario.Source: §A.7.5 · pp. 54–55
A.7.6 Representative Complete Reader-Facing Tasks; A.7.7 Construction and Quality Control of the Molecular Oncology Tasks
- Representative reader-facing tasks covered criterion-level variant assessment, rare-disease candidate pathogenicity and phenotype interpretation, and oncology treatment recommendations.Oncology tasks were generated from case reports, independently answered and arbitrated, then reviewed by a three-member panel; 45 tasks were retained from 62 panel-reviewed candidates.Source: Representative MRMC task 1:… · p. 55, Representative MRMC task 2: genomic… · p. 55, Representative MRMC task 3: molecular… · p. 56
A.8 Feedback Coverage in the Prospective Deployment
- 96 feedback submissions for 111 conversations (86.5%) provided a conservative lower-bound proxy for episode-level feedback coverage during deployment.Participant-level ratios ranged from 66.7% to 100%.Source: Table 21 · p. 57, §A.8 · p. 57