Source-linked AI summary
OpenResearcher: Unleashing AI for Accelerated Scientific Research
Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, Pengfei Liu
TL;DR
Rapid growth in scientific literature makes it difficult for researchers to stay current, while existing assistants may not actively clarify queries. OpenResearcher combines retrieval-augmented LLM responses with query and workflow tools; the authors report that it outperforms industry applications in human and GPT-4o judgments.
Problem
Scientific publications are growing rapidly, making it demanding for researchers to review literature and stay current; existing assistants are also described as passive.
Source: §1 · p. 1Method
OpenResearcher combines LLMs with external information and flexibly uses tools to clarify queries and tailor a workflow to each inquiry.
Source: §3 · p. 3, §3.1 · p. 3Results
The authors report that OpenResearcher’s answers outperform those from industry applications, as judged by humans and GPT-4o.
Source: §6 · pp. 6–7Takeaways & Limitations
Its multi-turn question answering supports continuous and deeper discussions within OpenResearcher.
Source: §4 · p. 5LLM-based features may still hallucinate despite grounding in retrieved publications, so users are advised to verify crucial information.
Source: Ethical Considerations · p. 7
Abstract
from arXiv · showhide
The rapid growth of scientific literature imposes significant challenges for researchers endeavoring to stay updated with the latest advancements in their fields and delve into new areas. We introduce OpenResearcher, an innovative platform that leverages Artificial Intelligence (AI) techniques to accelerate the research process by answering diverse questions from researchers. OpenResearcher is built based on Retrieval-Augmented Generation (RAG) to integrate Large Language Models (LLMs) with up-to-date, domain-specific knowledge. Moreover, we develop various tools for OpenResearcher to understand researchers' queries, search from the scientific literature, filter retrieved information, provide accurate and comprehensive answers, and self-refine these answers. OpenResearcher can flexibly use these tools to balance efficiency and effectiveness. As a result, OpenResearcher enables researchers to save time and increase their potential to discover new insights and drive scientific breakthroughs. Demo, video, and code are available at: https://github.com/GAIR-NLP/OpenResearcher.
1 Introduction
Growing scientific literature makes staying current time-intensive, while existing academic tools often focus on single tasks and industry assistants are passive. OpenResearcher presents an open-source assistant for diverse research questions, combining active clarification, retrieval, flexible tool use, and conversation.
1 Introduction
- Annual growth of about 4%-5% in scientific publications adds to researchers’ time-intensive work of reviewing literature to stay current.The passage attributes the growth figure to Pinedo et al. (2024).Source: §1 · p. 1
- OpenResearcher brings diverse research inquiries into one open-source assistant rather than a single-task academic application.Its contributions include scientific text summarization and paper recommendation.Source: §1 · pp. 1–2
- OpenResearcher actively clarifies researchers’ questions, addressing the passive interaction model described for academic and industry applications.It can ask heuristic questions to help users clarify their intent.Source: §1 · pp. 1–2
- It retrieves up-to-date, domain-specific knowledge and flexibly calls tools, including refinement, to avoid unnecessary tool-use costs and support conversational follow-ups.The system also supports deep discussions through follow-up questions.Source: §1 · p. 2
2 Related Work
Academic research applications cover question answering, summarization, and recommendation, but typically specialize in one function. Industry assistants handle broader research inquiries in dialogue, while their closed-source nature limits development and academic research.
2.1 Academic Works
- Academic question-answering, summarization, and recommendation systems span varied approaches, but typically provide no unified solution for diverse inquiries.Examples include QASA, ACLSum (Takeshita et al., 2024), and ArZiGo.Source: §2.1 · pp. 2–3
2.2 Industry Research Applications
- Industry assistants handle broad research inquiries in dialogue and provide citations, but their closed-source nature limits development and academic research.The passage names Perplexity AI, iAsk, You.com, phind, and SearchGPT.Source: §2.2 · p. 3
3 OpenResearcher
OpenResearcher combines LLMs’ internal knowledge with external information and routes each query through a customizable set of tools. Its tools clarify and decompose queries, retrieve and post-process evidence, generate responses with citations, and refine those responses.
3 OpenResearcher
- OpenResearcher combines LLM knowledge with external information and flexibly customizes a tool workflow for each query.Its tool groups cover query handling, retrieval, post-processing, generation, and refinement.Source: §3 · p. 3
3.1 Query Tools
- Active querying adds context, rewriting improves query clarity, and decomposition splits complex questions into sub-queries for separate retrieval and answer generation.Active querying can ask users to specify their interest area or discipline.Source: §3.1 · p. 3
3.2 Retrieval Tools
- Internet, hybrid sparse-and-dense vector, and BM25 retrieval tools gather relevant information from online sources and the arXiv corpus.Hybrid retrieval uses vector representations of queries and documents to capture semantic similarities.Source: §3.2 · pp. 3–4
3.3 Data Routing Strategy
- Data routing partitions arXiv papers into time- and domain-aligned databases so retrieval scans only those relevant to a query.The strategy targets the processing cost of comparing a query with all paper chunks and is intended to speed search and improve accuracy.Source: §3.3 · p. 4
3.4 Post-Processing Tools
- Post-processing reranks, fuses, and filters retrieved chunks to prioritize relevant results, combine same-source content, and remove noise and redundancy.Source: §3.4 · p. 4
3.5 Generation Tools
- Generation prompts LLMs to answer using retrieved information, while a BM25-based citation tool links retrieved information to response sentences.Source: §3.5 · p. 4
3.6 Refinement Tools
- Reflection checks generated responses for accuracy, completeness, and language flaws; polishing then revises them using that feedback.Source: §3.6 · p. 4
4 Demonstration
OpenResearcher adapts its workflow to query complexity, can ask users to clarify their questions, and supports multi-turn dialogue. Its demonstration also shows retrieved Internet and arXiv evidence linked to generated text for verification.
4 Demonstration
- The implementation uses arXiv publications from January 2023 to June 2024, hybrid retrieval, BM25, and Bing search, then reranks results to 10 chunks.The paper reports that hybrid retrieval extracts 30 chunks per selected database, BM25 up to 80, and Bing 10 outcomes.Source: §4 · p. 4
- Simple questions can be answered from LLM knowledge, while complex queries use multiple tools; the paper says this flexibility can speed responses and reduce computational costs.For a complex PPO query, the interface displays rewritten and decomposed queries, retrieved outcomes, a generated answer, and citations.Source: §4 · p. 5
- OpenResearcher can ask users to specify question details rather than only answering, a clarification feature the paper identifies as useful for junior students.Source: §4 · p. 5
- Conversational question answering supports multi-turn dialogue for continuous, deeper discussions.Source: §4 · p. 5
- Retrieved Internet and arXiv evidence can be linked to generated text so researchers can verify sources and read the original papers.Source: §4 · p. 5
5 Experiment
The experiments compare OpenResearcher with industry systems and Naive RAG using pairwise human and GPT-4o preferences. Both evaluations favor OpenResearcher, though GPT-4o assesses relevance and richness rather than factual accuracy.
5.1 Evaluation Data
- The evaluation set contains 109 questions from more than 20 graduate students: 38 on recommendation, 38 on summarization, and 33 on other topics.Because annotating ground-truth answers is costly, the authors use pairwise comparisons instead.Source: §5.1 · p. 5
5.2 Evaluation Applications
- Comparators include four industry applications and Naive RAG; OpenResearcher is also tested with its Active Query tool removed.OpenResearcher uses its tools flexibly rather than following one fixed workflow.Source: §5.2 · p. 5
5.3 Evaluation Metric
- Outcomes are compared against Perplexity AI, with “Win” marking a candidate that outperforms it; answers are assessed for correctness, richness, and relevance.Richness measures depth and scope, while relevance concerns whether the answer addresses the question.Source: §5.3 · pp. 5–6
5.4 Human Preference
- OpenResearcher outperforms Perplexity AI with more “Win” than “Lose” and beats Naive RAG on all metrics; human annotator agreement was 90.67%.The human evaluation also reports superior information correctness, relevance, and richness compared with all other applications.Source: Table 1 · p. 6, §5.4 · p. 6
- Twelve students with good research experience evaluated 30 randomly selected questions, equally covering scientific question answering, text summarization, and paper recommendation; two annotators annotated each instance, with a third available to resolve disagreements.Source: §5.4 · p. 6
5.5 LLM Preference
- GPT-4o preference results rank OpenResearcher best for information relevance and richness among applications, and above Naive RAG on both metrics.GPT-4o evaluation omits factuality because the model struggles to verify it, even with citation papers.Source: Table 2 · p. 6, §5.5 · p. 6
6 Conclusion
OpenResearcher is an active AI assistant for diverse research questions, using RAG and flexible tools to build answers from scientific literature. The paper reports that its answers outperform industry applications in human and GPT-4o evaluations.
6 Conclusion
- OpenResearcher combines LLMs with up-to-date, verified, domain-specific knowledge through RAG and interacts with users to clarify their queries.Its tools support query understanding, literature search, filtering, answer generation, and refinement.Source: §6 · p. 6
- OpenResearcher flexibly combines its tools into query-specific pipelines for a broad range of researchers’ inquiries.The conclusion names query understanding, literature search, filtering, generation, and refinement as tool functions.Source: §6 · p. 6
- The paper reports that OpenResearcher answers outperform those from industry applications, as judged by human evaluators and GPT-4o.The conclusion frames the system as an active assistant intended to accelerate research.Source: §6 · pp. 6–7
Ethical Considerations
OpenResearcher grounds generated responses in retrieved scientific publications, but its LLM-based features may still hallucinate.
Ethical Considerations
- Despite grounding responses in retrieved publications, OpenResearcher may produce hallucinations, so users are advised to verify crucial information from its LLM-based features.The passage presents verification as advice, not as a guarantee that checking will prevent errors.Source: Ethical Considerations · p. 7
A Completed Case
Figure 3 is a screenshot showing the completed case in Figure 2.
A Completed Case
- Figure 3 shows the completed case in Figure 2.The passage identifies it as a screenshot.Source: Figure 3 · p. 10