Source-linked AI summary
From Research Question to Scientific Workflow: Leveraging Agentic AI for Science Automation
Bartosz Balis, Michal Orzechowski, Piotr Kica, Michal Dygas, Michal Kuszewski
TL;DR
Scientific workflow systems automate execution but leave translation from research questions to reproducible workflow specifications manual and expertise-intensive. The paper introduces semantic, deterministic, and knowledge layers, using expert-authored Skills to ground intent extraction and optimization. Skills raised Claude Opus full-match accuracy from 44% to 83%, while deferred generation reduced data transfer by 92%.
Problem
Scientists must manually translate research questions into workflow DAGs using both domain knowledge and infrastructure expertise, creating error and reproducibility risks.
Method
A three-layer architecture uses an LLM for structured intent extraction, deterministic generators for DAG creation, and markdown Skills for domain knowledge.
Results
Skills raised Claude Opus full-match intent accuracy from 44% to 83%, and deferred generation reduced data transfer by up to 92%.
Takeaways & Limitations
Separating intent extraction from workflow generation makes identical intents produce identical DAGs while Skills support accurate translation and execution-time optimization.
Takeaways & Limitations
The architecture was demonstrated in one domain, and implicit disease-to-gene-coordinate reasoning remained unreliable even with Skills.
Abstract
from arXiv · showhide
Scientific workflow systems automate execution -- scheduling, fault tolerance, resource management -- but not the semantic translation that precedes it. Scientists still manually convert research questions into workflow specifications, a task requiring both domain knowledge and infrastructure expertise. We propose an agentic architecture that closes this gap through three layers: an LLM interprets natural language into structured intents (semantic layer); validated generators produce reproducible workflow DAGs (deterministic layer); and domain experts author ``Skills'': markdown documents encoding vocabulary mappings, parameter constraints, and optimization strategies (knowledge layer). This decomposition confines LLM non-determinism to intent extraction: identical intents always yield identical workflows. We implement and evaluate the architecture on the 1000 Genomes population genetics workflow and Hyperflow WMS running on Kubernetes. In an ablation study on 150 queries, Skills raise full-match intent accuracy from 44% to 83%; skill-driven deferred workflow generation reduces data transfer by 92\%; and the end-to-end pipeline completes queries on Kubernetes with LLM overhead below 15 seconds and cost under $0.001 per query.
1 Introduction
Scientific workflow systems execute specified DAGs reliably, but translating research questions into those specifications remains manual, expertise-intensive, and error-prone. The paper proposes three layers and expert-authored Skills to automate translation while preserving reproducibility.
- Scientific workflow systems automate scheduling, fault tolerance, data staging, and resource management after a DAG is specified.
- Translating research questions into executable DAGs requires domain and infrastructure expertise, including vocabulary mappings, data selection, parallelism, and configuration.
- The semantic gap creates barriers to entry, propagates vocabulary errors, and undermines reproducibility when translation decisions remain undocumented.
- Direct LLM workflow generation is non-deterministic, so identical prompts may produce different DAGs and weaken reproducibility.
- The proposed architecture separates LLM intent extraction, deterministic DAG generation, and domain-expert Skills encoding vocabulary, constraints, and optimization strategies.
- The architecture is demonstrated on the 1000 Genomes population-genetics workflow using Hyperflow on Kubernetes.
2 Related Work
Prior approaches improve workflow generation through prompting, schemas, tool interfaces, or retrieval, but leave gaps in persistent domain semantics and deterministic execution. This paper positions expert-authored Skills as an auditable, dual-purpose knowledge layer.
- Prior studies find that LLMs produce plausible workflows but struggle with domain vocabulary, platform conventions, and reliable cross-platform translation.
- Few-shot prompting improves configuration accuracy but remains ephemeral, because its knowledge is tied to individual prompts rather than persistent artifacts.
- Schema-gated orchestration separates conversational and execution authority, whereas this paper delegates workflow generation to deterministic code after intent extraction.
- MCP standardizes agent-tool interaction but does not encode the semantic translation from research questions to structured intents.
- Skills are domain-scientist-authored markdown, deterministically routed, and designed to encode vocabulary mappings, parameter constraints, and optimization strategies without ML expertise.
- The same Skill supports both semantic interpretation and operational optimization, a dual-purpose design with no direct counterpart in the surveyed literature.
3 Architecture
The architecture routes a natural-language query through cooperating agents that extract a structured intent, obtain human validation and infrastructure measurements, then generate and execute a deterministic workflow.
- The system translates natural-language research questions into executable Kubernetes workflows through cooperating agents, with human judgment able to stop the process.
- Components: The Conductor classifies queries, selects a Workflow Composer, manages clarification and correction turns, and enforces human validation before provisioning and execution.
- Components: The Workflow Composer uses Skills and an LLM to extract a structured ResearchIntent, then later generates workflow.json from measured data sizes, vCPUs, and resolved resources.
- Components: The Deployment Service provisions Kubernetes resources and measures infrastructure conditions, grounding deferred DAG generation in actual rather than estimated data and compute requirements.
- Pipeline interaction: The Execution Sentinel asynchronously monitors pods, logs, anomalies, progress, and completion after workflow submission.
- Separating semantic interpretation from deterministic generation confines LLM non-determinism to intent extraction and makes identical intents produce identical DAGs with composition provenance.
- Pipeline interaction: The six-phase pipeline covers routing, planning, validation, infrastructure provisioning, deferred generation, and execution, with asynchronous monitoring omitted from the sequence diagram.
4 Skills
Skills are expert-authored markdown documents that make domain semantics and operational choices explicit, auditable, and reusable. Their dual role improves translation while selecting data-extraction strategies that reduce transfer and storage.
- Skills encode vocabulary mappings, parameter constraints, data locations, and optimization strategies in markdown documents authored by domain experts.
- Five 1000 Genomes Skills cover populations, genomic regions, research contexts, data sources, and Workflow Composer interpretation guidance.
- Population and region Skills map natural-language terms and disease contexts to valid population codes and GRCh37 coordinates, while context Skills connect topics to analyses.
- Skills support correct translation by resolving terms such as European and African populations, HLA coordinates, and population-comparison analysis types.
- Skills also drive optimization: tabix extraction transfers 50 MB for HLA instead of 943 MB for the full chromosome 6 VCF.
- Plain markdown provides transparency, version control, and domain ownership without requiring embeddings, retraining, or ML-engineer intervention.
5 Evaluation
On 150 stratified queries, Skills substantially improved intent extraction, while deferred generation reduced transfer and calibrated parallelism; the complete Kubernetes pipeline remained inexpensive relative to execution.
- Intent extraction accuracy and Skills ablation (C1, C2): Skills improved full-match intent extraction, with vocabulary Skills providing most of the gain and all Skills performing best except for GPT-4.1-mini.Vocabulary Skills raised Opus accuracy by 36 percentage points; strategy Skills alone improved accuracy by 9–13 percentage points but left region extraction unreliable.
- Intent extraction accuracy and Skills ablation (C1, C2): 100% accuracy was achieved on explicit-code and common-synonym tiers with all Skills, whereas no-Skills performance on implicit domain inference was only 0–10%.On T3, Opus reached 86.7% with all Skills, while GPT-5.4 and GPT-4.1-mini scored 63–70%; exact GRCh37 coordinates were the main challenge.
- Deferred Generation Impact (C4): Measured data volumes also reduced parallelism from 66 to 1 for HBB and APOE, avoiding unnecessary tasks for small regions.For HLA, parallelism decreased from 100 to 51, matching its 166K rows.
- Deferred Generation Impact (C4): 92% less data was transferred across six regions after deferred generation used measured data volumes instead of advisory estimates.Downloads fell from 21.6 GB to 1.69 GB; savings exceeded 99.9% for small gene regions, while whole-chromosome analyses still downloaded complete VCF files.
- End-to-End Demonstration (C3): Across three Kubernetes queries, intent extraction was correct on all fields, LLM overhead remained 11–14 seconds, and cost stayed below $0.001 per query.Execution accounted for 82–97% of total time; runtimes ranged from 10 minutes for small regions to 145 minutes for HLA.
- End-to-End Demonstration (C3): The Conductor completed the manually specified five-step process in 106 seconds, compared with an expert estimate of 30–50 minutes.The manual path required genomics, bioinformatics, workflow, and DevOps expertise.
6 Conclusion
The layered architecture automates translation from research questions to executable workflows while keeping generated DAGs deterministic. Skills support accurate intent extraction and execution-time optimization, but the demonstrated scope remains one domain and implicit domain reasoning remains difficult.
- 6 Conclusion: Separating intent extraction from deterministic workflow generation ensures that identical intents produce identical DAGs.Skills provide domain-expert-authored vocabulary and optimization knowledge in auditable markdown documents.
- 6 Conclusion: Vocabulary Skills raised full-match accuracy from 44% to 83% for Claude Opus and from 39% to 80% for GPT-5.4.Deferred generation reduced data transfer by up to 92% and eliminated overparallelization for small genomic regions.
- 6 Conclusion: The end-to-end pipeline completed three Kubernetes queries with correct intent extraction, LLM overhead below 15 seconds per query, and cost under $0.001 per query.
- 6 Conclusion: The architecture was demonstrated only on 1000 Genomes, and each new domain requires its own Skills and deterministic generator.Implicit disease-to-gene-coordinate reasoning remained unreliable even with Skills.