Source-linked AI summary
Towards Recursive Self-Evolving Agentic Literature Retrieval
Yuwen Du, Tian Jin, Jing Kang, Xianghe Pang, Jingyi Chai, Tingjia Miao, Fenyi Liu, WenHao Wang, Sikai Yao, Yuzhi Zhang, Siheng Chen
TL;DR
Scientific literature retrieval must reconcile complex intent understanding with authentic sources. PaSaMaster addresses this gap through recursive evidence-guided retrieval and ranking over verified papers, with planning separated from retrieval. Across 38 disciplines, it improves retrieval over established baselines, maintains zero source hallucination, and operates at much lower cost.
Problem
Existing literature retrieval systems struggle to jointly understand complex search intents and preserve source authenticity.
Method
PaSaMaster recursively refines intent and retrieval from ranked evidence, ranks verified papers using evidence-grounded relevance scores, and separates planning from retrieval.
Results
Across PaSaMaster-Bench, PaSaMaster improves target-paper recovery, maintains zero source hallucination, and substantially reduces computational cost across complex search intents.
Takeaways & Limitations
Evidence-grounded ranking over verified papers narrows the gap between authentic but limited retrieval and richer but less trustworthy generative literature search.
Takeaways & Limitations
The benchmark is expert-curated, and the evaluation focuses on top-ranked paper retrieval rather than downstream synthesis, hypothesis generation, or manuscript writing.
Abstract
from arXiv · showhide
Scientific literature retrieval must understand complex search intents while preserving source authenticity. Traditional keyword and embedding-based systems return authentic sources but miss nuanced intents, whereas large language models capture richer intents but may fabricate citations. We introduce PaSaMaster, a Recursive Self-Evolving agentic literature retrieval system that iteratively analyzes intent, retrieves verified papers and ranks them with evidence-grounded relevance scores. PaSaMaster combines self-evolving retrieval that refines search intent from ranked evidence over time, hallucination-free ranking over verified papers rather than generated citations, and cost-efficient planning--retrieval separation that reserves frontier LLMs for intent understanding while delegating retrieval and scoring to lightweight models and customized corpora. Across 38 disciplines in PaSaMaster-Bench, PaSaMaster achieves a 16.5$\times$ higher F1-score than Google Scholar and a 37.8\% higher F1-score than GPT-5.2 at about 1\% of the cost, while reducing source hallucination from 32.66\% in generative LLMs to zero: https://github.com/sjtu-sai-agents/PaSaMaster
1 Results
PaSaMaster evaluates complex literature-search intents by making constraints explicit, retrieving and verifying papers through coordinated layers, and refining search with ranked evidence. On PaSaMaster-Bench, it achieves stronger retrieval quality, zero source hallucination, and substantially lower cost than the compared systems.
- System: PaSaMaster converts natural-language requests into explicit retrieval strategies, verification checklists, scored candidate papers, recommendation rationales, and evidence-linked constraint judgments.Its Navigator plans and reflects, while the Librarian Swarm performs multi-channel retrieval, evidence verification, scoring, and reranking over verified corpora and tools.
- Retrieval quality: 16.5× higher F1-score than Google Scholar: PaSaMaster improves F1-score@20 from 1.39 to 23.00 and reaches the best main retrieval metrics.Its reported metrics are NDCG@20 39.52, Recall@20 33.24, Precision@20 23.46, and F1-score@20 23.00.
- Source reliability: 0% source hallucination: PaSaMaster ranks only papers retrieved from verified corpora and grounds relevance judgments in original-paper evidence.Generative LLM baselines show substantial hallucination, including 32.66% for MiniMax-M2.7 and 5.65% for GPT-5.2.
- Cost efficiency: 37.8% higher F1-score than GPT-5.2 at about 1% of its computational cost, while PaSaMaster costs $0.05 per query versus GPT-5.2 at $6.06.The comparison supports separating high-level planning from large-scale retrieval.
- Robustness: Across disciplines, PaSaMaster shows higher F1-score distributions, a higher retrieval floor on difficult cases, and Ranker gains across most subject areas.The reported cross-disciplinary gains include improved NDCG@20, Recall@20, and Precision@20 after Ranker training.
2 Discussion
PaSaMaster frames literature discovery as recursive, evidence-grounded ranking over verified papers, addressing the tradeoff between complex intent understanding and source reliability. Its evidence feedback, source-constrained ranking, and planning–retrieval separation support broad retrieval gains while keeping recommendations auditable and practical.
- Core framing: PaSaMaster reframes literature discovery as Recursive Self-Evolving, evidence-grounded ranking rather than keyword matching or citation generation.The output space remains restricted to verified papers while the search process evolves from accumulated evidence.
- Cross-disciplinary results: PaSaMaster reports leading F1-score distributions across disciplines, with higher retrieval floors on difficult cases and higher upper ranges on easier cases.The benchmark spans 38 scientific disciplines, and the figure also summarizes source-error patterns and Ranker gains.
- Recursive retrieval: Ranked evidence continually updates the system’s interpretation of complex queries, enabling later searches to explore complementary directions beyond the initial query.This mechanism is especially relevant when relevance depends on combinations of topic, method, dataset, application context, and exclusion criteria.
- Evidence-grounded ranking: Verified-corpus retrieval and paper-level evidence make relevance scores traceable while ensuring recommended sources are real and auditable.PaSaMaster ranks papers rather than generating citations, reducing the risk of fabricated or misreported sources.
- Practical deployment: Planning–retrieval separation reserves high-level planning for frontier models while delegating large-scale retrieval and lightweight relevance scoring to make repeated multidisciplinary discovery more practical.The system is positioned as an assistive layer that expands and organizes candidate evidence rather than replacing expert judgment.
- Limitations: PaSaMaster-Bench is expert-curated, and the evaluation focuses on top-ranked paper retrieval rather than downstream synthesis, hypothesis generation, or manuscript writing.The authors call for human-in-the-loop evaluation of coverage, novelty, usefulness, and trustworthiness.
3.1 Overview of PaSaMaster
PaSaMaster maps complex natural-language intents to ranked, evidence-grounded paper sets through staged planning, retrieval, verification, and reranking. Its self-evolving loop updates search strategies and checklists from ranked evidence, while separating frontier reasoning from scalable retrieval and scoring.
- Overview: PaSaMaster retrieves real papers from customized scientific corpora, verifies relevance using original evidence, and returns a ranked paper set rather than generating citations from memory.Each recommendation is intended to remain traceable to paper-level evidence.
- Overview: The Navigator converts query q into a retrieval strategy and checklist whose items encode concrete requirements for relevant papers.Parallel Librarians then execute retrieval, verification, and reranking stages.
- Recursive Self-Evolving Retrieval: Ranked evidence from each round updates the Navigator’s strategy and checklist to address missing coverage, ambiguous constraints, or under-explored directions.This transforms retrieval from static query interpretation into an iterative self-evolving process.
- Hallucination-Free Ranking: Evidence Chunk Locator selects passages most semantically aligned with each checklist requirement, binding candidate judgments to explicit textual evidence.The Scorer assigns checklist satisfaction scores from 1 to 5 and provides evidence-grounded rationales.
- Hallucination-Free Ranking: The final paper-level score combines average checklist satisfaction with calibrated holistic confidence before listwise reranking produces the recommendation list.The denominator 6 normalizes the maximum possible sum because checklist satisfaction is at most 5 and confidence is at most 1.
- Cost-Efficient Planning–Retrieval Separation: Frontier LLMs handle complex-intent understanding and refinement, while customized corpora and lightweight parallel Librarians perform large-scale retrieval and relevance scoring.The toolset separates retrieval tools that construct candidate pools from reading tools that inspect metadata, abstracts, and evidence chunks.
3.2 PaSaMaster-Bench
PaSaMaster-Bench evaluates literature retrieval under realistic, compositional research intents rather than simple topical matching. It combines multidisciplinary, real-web-oriented task construction with expert checklist verification and metrics for retrieval quality, authenticity, and cost.
- Task Design: Each task combines dimensions such as scientific scope, methods, application setting, metadata restrictions, and exclusions, requiring systems to recover the complete intent.The benchmark includes direct-intent, problem-driven, metadata-constrained, and exclusion-oriented search scenarios.
- Benchmark Scope: PaSaMaster-Bench contains 244 independent tasks across 38 scientific disciplines, pairing natural-language search intents with expert-annotated target paper sets.The benchmark is designed around realistic paper-discovery requirements that existing benchmarks often cover only partially.
- Task Design: Domain experts decompose natural-language intents into objective checklist items and admit a paper into the target set only when it satisfies every required criterion.Candidates are gathered through multiple retrieval channels, then verified, deduplicated, and annotated item by item.
- Evaluation Protocol: The evaluation compares each returned ranked list with the expert-annotated target set, making correctness stricter than topical relevance.A useful returned paper must satisfy the complete intent rather than merely share the same field or topic.
- Evaluation Protocol: At K = 20, Recall@K, Precision@K, F1@K, and NDCG@K jointly measure target coverage, checklist satisfaction, balance, and ranking position.The benchmark also evaluates source hallucination through scholarly-record matching and cost through token usage and per-query expense.
- Evaluation Protocol: The benchmark targets four capabilities: intent comprehension, constraint-satisfying retrieval, ranking quality, and source authenticity.These capabilities correspond to PaSaMaster’s self-evolving retrieval, evidence-grounded ranking, and planning–retrieval separation.