Source-linked AI summary
SciDER: Scientific Data-centric End-to-end Researcher
Ke Lin, Owais Aijaz, Yilin Lu, Yiyang Luo, Xuehang Guo, Preslav Nakov
TL;DR
Existing scientific agents have limited adaptability, domain generalization, and multimodal scalability when processing raw, domain-specific experimental data. SciDER combines data-centric analysis with four multimodal sub-agents to automate research, and evaluations report competitive or leading performance alongside open-source resources for autonomous discovery.
Problem
Existing agents struggle to autonomously process diverse raw experimental data across specialized domains and multimodal workflows.
Method
SciDER uses four multimodal sub-agents for ideation, data analysis, experimentation, and criticism, grounding code synthesis in structured analysis of dataset characteristics.
Results
Across six benchmarks, SciDER and OpenSciDER obtain competitive or leading results, with strong gains in data-centric analysis, end-to-end execution, and multimodal scientific visualization.
Takeaways & Limitations
SciDER bridges abstract scientific reasoning and reproducible experimentation synthesis while providing an accessible modular package, dataset, and fine-tuned model.
Abstract
from arXiv · showhide
While large language models accelerate scientific discovery, existing agents face severe limitations in adaptability, domain generalization, and multimodal scalability, often struggling to autonomously process raw, domain-specific experimental data. To overcome these barriers, we introduce SciDER, a multi-agent system designed to flexibly automate the entire research lifecycle. This framework employs a novel data-centric approach and integrates a dynamic multimodal skill system across four specialized sub-agents. Specifically, an ideation agent generates novel hypotheses via Evolutionary Idea Search, a data analysis agent systematically structures raw data, an experimentation agent synthesizes executable code grounded in dataset characteristics, and a critic agent drives iterative self-refinement. To democratize open-source scientific discovery, we release OpenSciDER-SFT-8K, a high-quality execution trajectory dataset, alongside the OpenSciDER-27B fine-tuned model. Across six benchmarks, SciDER and OpenSciDER obtain competitive or leading results, with especially strong gains on data-centric analysis, end-to-end research execution, and multimodal scientific visualization. By integrating data analysis with experimental execution, SciDER bridges the gap between abstract scientific reasoning and reproducible experimentation synthesis.
I. INTRODUCTION
Existing LLM-based agents struggle with diverse experimental data, specialized domains, and multimodal workflows because research automation often separates abstract ideation from data-grounded analysis. SciDER addresses this disconnect with four multimodal sub-agents, a data-centric workflow, and open-source research resources.
- Current agents often target public machine-learning datasets and struggle to independently process diverse real-world experimental data.
- Specialized domains expose a gap between abstract ideas and precise, sometimes proprietary, experimental data formats.
- Unimodal workflows limit scalability for multimodal tasks and the use of evolving AI skill plugins.
- SciDER integrates ideation, data analysis, experimentation, and criticism into a modular workflow with multimodal skills for interdisciplinary tasks.Its agents generate hypotheses, analyze and report on data, write and execute code, and evaluate outputs for improvement.
- SciDER grounds experiment-code generation in autonomous analysis of raw data, linking visual and structural characteristics directly to code synthesis.
- The work releases the OpenSciDER-SFT-8K trajectory dataset and OpenSciDER-27B model, while empirical results report stronger performance on complex scientific reasoning and coding tasks.
II. RELATED WORK
SciDER is presented as an end-to-end research system that combines ideation, data analysis, experimentation, and iterative critique. Its data-centric workflow structures raw inputs before guiding executable experiments and refinement.
- II. RELATED WORK: Prior research-agent frameworks primarily support idea generation and experiment execution rather than the full scientific pipeline.The comparison covers functionality, capability, deployment, and open-source contribution.
- II. RELATED WORK: SciDER integrates ideation, data analysis, experimentation, and iterative improvement while accepting text, raw data, code, and prior research assets.A lightweight web interface supports uploading data and research topics for a closed-loop research cycle.
- II. RELATED WORK: The data-analysis stage transforms a labelled file tree into a report covering structure, quality, semantics, and dependencies.The report guides data loading, preprocessing, and field selection during experimentation.
- II. RELATED WORK: Experimentation generates executable code from the proposal and data report, validates patches, executes the code, and feeds runtime failures back for revision.The critic evaluates intermediate outputs for accuracy, completeness, and neutrality to drive self-refinement.
B. Multimodal Skill System
SciDER’s multimodal skill system dynamically loads role-restricted procedural capabilities, keeping prompt costs largely independent of the full skill catalogue until skills are activated.
- Skills are procedural recipes that dynamically load capabilities into agent context rather than operating as isolated tool calls.
- Each skill is restricted to selected agents, optionally preloaded, and exposes its full body on demand.
- Agents initially load short skill descriptors, while keyword matches or explicit invocation materialize full skill bodies.
- Adding skills incurs negligible additional tokens until activation because per-call costs are decoupled from catalogue size.
- The system supports multimodal file processing and domain-specific skills across scientific and social-science disciplines.
C. OpenSciDER
OpenSciDER provides an open execution-trajectory dataset and a fine-tuned open-weights model intended to support autonomous scientific discovery.
- OpenSciDER-SFT-8K contains more than 8K high-quality execution trajectories for general research.
- The dataset combines 2,678 trajectories curated from scientific benchmarks with 5,854 additional successful benchmark trajectories.
- OpenSciDER-27B is a Qwen-3.6-27B model fine-tuned on OpenSciDER-SFT-8K and released with the dataset under Apache-2.0.
A. Evaluation Setup
SciDER is evaluated across ideation, data analysis, experimentation, multimodal performance, human feedback, and case studies using existing benchmark implementations.
- The evaluation spans ideation, data analysis, experimentation, and multimodal performance, supplemented by human feedback and case studies.
- The study includes ablations of the ideation and data analysis modules.
- Results for comparison methods and models are taken from their benchmarks and leaderboards where available.
B. Idea Generation
SciDER’s idea-generation evaluation uses AI-Idea-Bench 2025, while the reported results show OpenSciDER outperforming comparison systems on novelty and feasibility.
- Idea Generation: AI-Idea-Bench 2025 evaluates generated research ideas using 3,495 top-tier AI papers and quantitative alignment and reference-based judgments.
- Idea Generation: 63.22 (+12.98%) motivation and 56.11 (+6.41%) experiment novelty scores are reported for OpenSciDER against InternAgent-1.5.
- Idea Generation: 36.2 Feasibility Score represents a 17.15% gain over InternAgent-1.5 for OpenSciDER with EIS.
- Idea Generation: OpenSciDER surpasses standard SciDER and AI-Scientist-v2 across all reported AI-Idea-Bench metrics.
D. Experimentation
SciDER is evaluated on autonomous machine-learning and scientific coding tasks, including end-to-end research workflows. The reported results show gains over established agentic baselines, including for the open-weights OpenSciDER-27B variant.
- MLE-Bench: 63.64% for any medal and 40.90% for gold medals establish SciDER’s state-of-the-art performance on the MLE-Bench Lite split.Powered by Gemini-3-Pro, SciDER exceeds AIRA by absolute margins of 6.75% and 3.74%, respectively.
- MLE-Bench: 54.54% overall medal rate makes OpenSciDER-27B competitive with other agentic systems on MLE-Bench Lite.The open-weights model exceeds ML-Master’s 48.50% and AIDE’s 16.90%.
- SciCode: 11.44% absolute improvement in SciCode sub-problem success separates SciDER from AIRA, while SciDER also exceeds daVinci on main-problem success.SciDER achieves 16.46% main-problem success versus daVinci’s 15.40%, and its sub-problem gain represents a 31.97% relative improvement.
- SciCode: 38.07% SciCode success for OpenSciDER-27B exceeds AIRA’s 35.78% success rate.The comparison concerns the lightweight open-weights model and AIRA with Gemini-3-Flash.
- AIRS-Bench: 0.780 on AIRS-Bench exceeds AIRA’s 0.550 by 0.23, a 41.8% relative improvement across five benchmark iterations.AIRS-Bench evaluates the complete research lifecycle across approximately 20 complex machine-learning tasks without baseline code.
- AIRS-Bench: 0.519 for OpenSciDER-27B exceeds ReAct at 0.402 and AIRA at 0.424, while closely rivaling GPT-5 on AIRS-Bench.The result is reported for the lightweight open-weights variant.
F. Visual Understanding
The paper examines multimodal scientific visualization and reports an ablation of components supporting ideation and data-centric analysis. OpenSciDER performs strongly on specialized astronomical visualization, while EIS and data analysis improve evaluated tasks.
- AstroVisBench: OpenSciDER achieves a higher rate of correct visual outputs than both multimodal ReAct and the closed-source-backed SciDER framework on AstroVisBench.AstroVisBench tests long-tail astronomical APIs through specialized code synthesis and iterative visual refinement.
- Ablation Study: 23.8% relative improvement in Novelty raises the score from 45.79 to 56.67 after applying Evolutionary Idea Search.The ablation also reports an 11.5% Quality gain, from 3.64 to 4.06, while enhancing Feasibility.
- AstroVisBench: Correct, Failure, Minor Error, and Major Error rates are the reported AstroVisBench outcome categories.The figure organizes visualization results by these four outcome types.
- Ablation Study: 0.238 and 0.153 score improvements result from including data analysis on DiscoveryBench ML Req. Eng. and Meta-Regression, respectively.The comparison evaluates SciDER with and without the data analysis module.
H. Qualitative Feedback
The paper combines expert feedback with qualitative research and visualization case studies to assess SciDER’s practical utility. Experts rate the system highly, while the cases illustrate end-to-end execution and multimodal visual reasoning.
- Expert Feedback: 4.846/5.000 mean helpfulness, with a 0.376 standard deviation, is reported by 13 domain experts evaluating SciDER’s workflows.The questionnaire covers workload reduction from data parsing through experimental execution using a five-point rubric.
- End-to-end Research: SciDER autonomously executes a complete educational knowledge-tracing study from dataset analysis through method design, implementation, evaluation, and manuscript generation.The case proposes student-conditioned evidential meta-learning with hyper-conditioned priors and evaluates EvidentialHyperMAML against baselines using metrics such as AUC-ROC.
- Visualization: SciDER uses an iterative multimodal feedback loop to generate visualization code, inspect rendered plots, and refine anomalies in astronomical outputs.The case targets out-of-distribution visualization requiring coordinate transformations and specialized APIs.
- Conclusion: SciDER grounds high-level hypothesis generation in rigorous data analysis to connect research reasoning with reproducible code synthesis.The conclusion describes this connection as central to the four-agent, data-centric system.
APPENDIX
The appendix provides per-task score figures for SciDER on DiscoveryBench and AIRS-Bench.
- DiscoveryBench: Per-task scores of SciDER are presented for DiscoveryBench.The figure reports task-level performance rather than an aggregate result.
- AIRS-Bench: Per-task scores of SciDER are presented for AIRS-Bench.The figure reports task-level performance rather than an aggregate result.