Source-linked AI summary
Idea2Story: An Automated Pipeline for Transforming Research Concepts into Complete Scientific Narratives
Tengyue Xu, Zhuoyang Qian, Gaoge Liu, Li Ling, Zhentao Zhang, Biao Wu, Shuo Zhang, Ke Lu, Wei Shi, Ziqi Wang, Zheng Feng, Yan Luo, Shu Xu, Yongjin Chen, Zhibo Feng, Zhuo Chen, Bruce Yuan, Harry Wang, Kris Chen
TL;DR
Existing autonomous research agents often process scientific literature online at runtime, creating computational, context-window, and reliability challenges. Idea2Story shifts literature understanding offline by extracting methodological units into a reusable knowledge graph, then retrieves patterns for underspecified research intents. Qualitative analyses and preliminary empirical studies report coherent, grounded, novel research patterns and several high-quality end-to-end demonstrations.
Problem
Runtime-centric research agents repeatedly process large volumes of scientific literature online, incurring computational overhead while stressing context windows and increasing hallucination and reasoning risks.
Method
Idea2Story collects peer-reviewed papers and review feedback, extracts and normalizes methodological units, and organizes reusable composition patterns in a continuously updated knowledge graph.
Results
Qualitative analyses and preliminary empirical studies show that Idea2Story produces coherent, methodologically grounded, and novel research patterns, including several high-quality research demonstrations.
Takeaways & Limitations
Offline knowledge construction provides a practical and scalable foundation for autonomous scientific discovery grounded in reusable research patterns.
Takeaways & Limitations
The current framework does not yet provide a fully closed-loop research generation pipeline that experimentally instantiates, validates, and iteratively refines generated patterns.
Abstract
from arXiv · showhide
Autonomous scientific discovery with large language model (LLM)-based agents has recently made substantial progress, demonstrating the ability to automate end-to-end research workflows. However, existing systems largely rely on runtime-centric execution paradigms, repeatedly reading, summarizing, and reasoning over large volumes of scientific literature online. This on-the-spot computation strategy incurs high computational cost, suffers from context window limitations, and often leads to brittle reasoning and hallucination. We propose Idea2Story, a pre-computation-driven framework for autonomous scientific discovery that shifts literature understanding from online reasoning to offline knowledge construction. Idea2Story continuously collects peer-reviewed papers together with their review feedback, extracts core methodological units, composes reusable research patterns, and organizes them into a structured methodological knowledge graph. At runtime, underspecified user research intents are aligned to established research paradigms, enabling efficient retrieval and reuse of high-quality research patterns instead of open-ended generation and trial-and-error. By grounding research planning and execution in a pre-built knowledge graph, Idea2Story alleviates the context window bottleneck of LLMs and substantially reduces repeated runtime reasoning over literature. We conduct qualitative analyses and preliminary empirical studies demonstrating that Idea2Story can generate coherent, methodologically grounded, and novel research patterns, and can produce several high-quality research demonstrations in an end-to-end setting. These results suggest that offline knowledge construction provides a practical and scalable foundation for reliable autonomous scientific discovery.
1 Introduction
Existing autonomous research agents can automate end-to-end workflows, but runtime-centric literature processing repeatedly incurs computational and reasoning costs. Idea2Story addresses this by constructing methodological knowledge offline and retrieving reusable patterns online.
- LLM-based agents increasingly automate research workflows spanning literature review, code generation, experimentation, and manuscript drafting.
- Runtime-centric agents repeatedly retrieve, summarize, and reason over large, overlapping literature collections for each research attempt.This design increases computational overhead and can exacerbate hallucination and reasoning errors.
- At runtime, knowledge-grounded planning retrieves and composes research patterns rather than relying solely on open-ended generation and trial-and-error.The overview describes this process as grounding underspecified user intent into concrete research directions.
- Idea2Story separates offline knowledge construction from online research generation to reduce repeated literature reasoning and alleviate LLM context-window bottlenecks.
- The offline stage extracts methodological units and organizes composition relations into a continuously updated structured knowledge graph.The framework uses this graph as a compact, reusable representation of established scientific methods.
2 Related Work
Related work shows that autonomous research agents can automate parts of the research lifecycle, while remaining limited in exploration, feasibility, environmental interaction, and scientific evaluation. These limitations motivate more structured and reliable approaches to autonomous discovery.
- Early end-to-end systems demonstrate research automation but rely heavily on manually crafted code templates and largely linear exploration workflows.
- LLM-generated research ideas may appear novel initially yet underperform after implementation, while increasing generation can reduce meaningful diversity.
- Agents struggle with realistic codebases because of insufficient reasoning over hierarchical dependencies, structural constraints, and finite context windows.
- Autonomous LLM evaluations can hallucinate, overstate confidence, and substantially overestimate paper quality compared with human reviewers.
3 General Idea Generation
Idea2Story separates offline construction of a reusable methodological knowledge graph from online research generation. It extracts and clusters methodological structure from peer-reviewed papers, then retrieves and composes research patterns for underspecified ideas using multiple semantic views and review-guided refinement.
- Online Research Generation: Online generation aligns informal user ideas with research patterns through idea-, domain-, and paper-level retrieval, combining view-specific scores with fixed weighting coefficients.Retrieved patterns are composed into research directions and iteratively revised, retaining only revisions that improve reviewer scores.
- Offline Knowledge Construction: Offline construction curates accepted papers and review artifacts into a de-identified corpus that preserves technical content and evaluation signals.The corpus draws on accepted machine learning papers from NeurIPS and ICLR collected over the most recent three-year window.
- Offline Knowledge Construction: Method units abstract self-contained problem formulations or solutions while excluding implementation details unless they change the problem, model, or learning objective.Units are extracted from complementary signals in introductions, methods, and experiments, then normalized into structured methodological attributes.
- Offline Knowledge Construction: Paper embeddings are reduced and clustered with DBSCAN to produce coherent research patterns that capture recurring methodological structures across the literature.The resulting clusters provide higher-level abstractions over individual papers for later retrieval and composition.
- Knowledge Graph Construction: The knowledge graph connects canonicalized method units and meta-methods through composition and similarity relations, encoding reusable elements alongside compatibility observed in accepted papers.This structure supports method discovery and composition while remaining grounded in observed research practice.
4 Experiments and Analysis
Experiments analyze Idea2Story’s reusable methodological representations, knowledge-graph structure, and generated research patterns. Qualitative comparisons favor outputs with clearer problem framing, more specific methods, and stronger novelty signals.
- Method Unit Extraction: The extraction agent separates each paper into a base problem, solution pattern, story, and application while abstracting away implementation details.The representation supports reusable methodological structure rather than a collection of experimental components.
- Method Unit Extraction: Method units isolate reusable methodological structure, enabling comparison and composition of ideas across papers.
- Knowledge Graph Analysis: The knowledge graph exhibits a hub-and-spoke structure in which high-frequency domains connect to many papers and research patterns.This reflects uneven research activity and identifies domains that function as central hubs for methodological reuse.
- Knowledge Graph Analysis: The graph separates instance-level research artifacts from reusable methodological abstractions, supporting higher-level retrieval and composition.
- Qualitative Comparison of Generated Research Patterns: Idea2Story transforms underspecified research intents into structured proposals, while direct LLM generation remains closer to conventional task formulations.The comparison evaluates problem formulation, methodological structure, and innovation claims from the same user input.
- Qualitative Comparison of Generated Research Patterns: Across evaluated cases, an independent Gemini 3 Pro assessment consistently favors Idea2Story outputs for novelty, methodological substance, and overall research quality.Idea2Story patterns show clearer problem framing and more specific methodological structures, whereas direct generations rely more on standard techniques.
5 Future Work
Idea2Story currently grounds vague research intent in structured research patterns, while future work aims to close the loop through empirical validation and complete paper generation.
- 5 Future Work: Future work will integrate experiment-driven agents to instantiate, validate, and iteratively refine generated research patterns through empirical feedback.Planned capabilities include automated experimental design, dataset selection, and preliminary execution.
- 5 Future Work: Experimental outcomes could provide additional signals for refining instantiated research stories through a feedback loop between method design and empirical validation.
- 5 Future Work: The framework may also translate refined research patterns into complete paper drafts covering methods, results, and discussion.The proposed extension would ground paper generation in empirically validated research patterns.
6 Conclusion
Idea2Story shifts literature understanding from runtime reasoning to offline knowledge structuring, enabling agents to reason over stable research patterns. The authors report clearer problem reformulation, stronger methodological structure, and higher conceptual novelty than direct LLM generation, while identifying experimental integration as a path toward validated discovery.
- 6 Conclusion: Idea2Story shifts literature understanding from runtime reasoning to offline knowledge structuring.
- 6 Conclusion: By extracting reusable method units into a continuously updated knowledge graph, the framework lets agents reason over stable research patterns instead of repeatedly processing raw papers.
- 6 Conclusion: Qualitative analyses and comparative studies report clearer problem reformulation, stronger methodological structure, and higher conceptual novelty than direct LLM generation.
- 6 Conclusion: Integrating Idea2Story with experimental agents is presented as a direction toward fully autonomous and trustworthy scientific discovery.