Source-linked AI summary
GRAFT: Graph-Distilled Generative Retrieval for Facet-Aware Scientific Literature Exploration
Italo Luis da Silva, Hanqi Yan, Yujing Wang, Jiangnan Ye, Lin Gui, Yulan He
TL;DR
Scientific related-paper retrieval must expose multidimensional relationships rather than collapse them into one similarity score. GRAFT builds a typed graph over 11,359 NLP papers and distils it into a facet-specific generative retriever with coverage-aware training and graph-weighted fusion. It recovers 91% of the teacher’s Recall@20, outperforms the teacher on out-of-corpus queries, and reproduces facet attribution at 0.922 precision.
Problem
Existing scientific retrieval collapses relationships across problems, methods, results, and contributions into a single score, obscuring why papers are related.
Method
GRAFT distils a typed graph built from four facet types, facet items, and citation signals into a generative retriever using coverage-aware distillation and graph-weighted reciprocal rank fusion.
Results
91% of the dense-graph teacher’s Recall@20 is recovered, GRAFT leads the teacher on out-of-corpus queries, and facet attribution reaches 0.922 precision.
Takeaways & Limitations
GRAFT provides exploratory generative retrieval while preserving graph-supported facet provenance for each returned paper.
Takeaways & Limitations
The corpus covers only *ACL venues from 2019 to 2026, favours well-connected papers, and GRAFT requires retraining before it can return newly added papers.
Abstract
from arXiv · showhide
Scientific papers may relate by problem, method, result, or contribution, but document-level retrievers collapse these into a single similarity score without saying why they are related. Citation- and similarity-based retrieval alone also confines search to the neighbourhood of what is already known, whereas generative retrieval generates document identifiers directly, enabling the exploratory retrieval that scientific discovery depends on. We connect papers in a graph whose edges are typed by these four facets, derived from facet items and citation signals, and distil it into a generative retriever whose identifiers are the papers' own facet text. Two graph properties do not survive naive distillation. First, because every training pair is an edge, naive enumeration indexes just 84% of the corpus. Coverage-aware distillation makes every paper learnable through a reverse-neighbour fallback, a minimum-coverage threshold, and edge-importance weighting. Second, constrained decoding guarantees that every generated identifier is a valid paper, but not that the graph connects it to the query. Graph-weighted reciprocal rank fusion scales each candidate's rank term by its query-candidate edge weight, dropping unsupported ones. On LitWeave, our constructed corpus of 11,359 NLP papers, Graft recovers 91% of its graph teacher's Recall@20 with no nearest-neighbour index or encoder at inference, and outperforms the graph teacher on query papers outside the corpus. It reproduces the graph's own facet labels at 0.922 precision, so every returned paper arrives labelled with the facet that surfaced it rather than an opaque score.
1 Introduction
Scientific related-paper retrieval must represent multidimensional relationships and index them efficiently. The paper introduces LITWEAVE’s typed graph and GRAFT’s generative distillation, achieving strong recall and facet attribution.
- Motivation: Existing retrieval collapses heterogeneous paper relationships into one similarity score, obscuring whether papers share problems, methods, results, or contributions.Scientific papers can be related through different facets even when their topics or methods differ.
- Approach: GRAFT distils the typed graph into a generative retriever that preserves facet-aware retrieval behaviour while making the model itself the index.Naive distillation leaves coverage and structural-grounding gaps that the framework addresses with dedicated components.
- Results: 91% of the dense-graph teacher’s Recall@20 is recovered, while GRAFT outperforms baselines that do not consult the graph.The evaluation compares lexical, dense, graph-based, and generative systems.
- Results: On out-of-corpus query papers, GRAFT overtakes the teacher and other baselines, supporting exploratory retrieval beyond the indexed corpus.The out-of-corpus evaluation uses query-paper bibliographies.
- Results: Graph-RRF reproduces the graph’s facet attribution at 0.922 precision, labelling returned papers by the facet that surfaced them.The final ranking retains provenance rather than presenting only an opaque relevance score.
- Representation: LITWEAVE represents 11,359 NLP papers as a typed graph whose edges encode facet-specific relationships and strengths.The corpus combines semantic facet extraction with citation signals.
2 Related Work
Related work spans lexical, dense, citation-based, and generative retrieval. Generative systems differ chiefly in how they represent document identifiers and train targets.
- Scientific related-paper retrieval: Lexical methods preserve exact terminology, while dense and scientific-document methods use embeddings or citation structure for related-paper retrieval.These approaches provide established baselines for scientific retrieval.
- Generative retrieval: Generative retrieval generates relevant document identifiers directly, turning the model into the index instead of searching an external index.Prior work explores neural corpus indexing, semantic identifiers, and lexical identifiers.
- Generative retrieval: GRAFT differs from self-target generative retrieval by using graph-edge pairs, while related multi-view methods provide the closest comparison.MINDER represents documents through several views, whereas DSI-QG trains each document against pseudo-queries from its own text.
3 Task and Dataset
The paper defines facet-attributed scientific related-paper retrieval and builds LITWEAVE as a typed graph combining semantic facets with citation signals. The graph supports both retrieval and teacher supervision.
- Task: Scientific related-paper retrieval returns papers related to a query paper and records which facets retrieved each result.Retrieval runs separately over problems, methods, results, and contributions.
- Facet extraction: LITWEAVE represents papers with four LLM-extracted facets: Problems, Methods, Results, and Contributions.Each facet is a short list of bullet items, with every paper receiving at least one item per facet.
- Facet extraction: 96.5% of sampled facet items received groundedness ratings of at least 4, with Gwet’s AC1 of 0.91 on a shared core.The evaluation covered 280 ratings over 200 items and a unanimous-groundedness check on 40 items.
- Citation signals: Citation signals use bibliographic coupling and co-citation, log-compressed and averaged into one citation score.These signals complement semantic facet similarity.
- Multi-facet graph: The resulting typed graph carries facet types and importance weights, allowing papers to connect through several facets while preserving provenance.Facet similarity and citation score are combined, and only the top-20 edges per paper are retained for each facet.
- Graph retrieval: Graph retrieval performs parallel two-hop walks over the four facets and fuses the resulting provenance-bearing lists with reciprocal rank fusion.The graph serves both as a retrieval baseline and as the teacher for GRAFT.
4 Graph-Distilled Generative Retrieval
GRAFT distils a typed paper graph into facet-specific generative retrieval, then uses graph evidence to improve coverage and ground final rankings. Its identifiers are natural-language facet items rather than opaque codes.
- Retriever design: GRAFT uses the multi-facet paper graph as a teacher and decodes separate identifiers for Problems, Methods, Results, and Contributions.The retriever operates on facet items and produces per-facet candidate lists.
- Natural-language facet identifiers: Each facet item acts as a pseudo-query, while its normalised text becomes a target DocID, giving papers multiple retrieval paths.A prefix trie constrains decoding to valid identifiers, and duplicate outputs are collapsed into unique papers.
- Coverage-aware distillation: Coverage-aware distillation combines edge-importance sampling, a minimum target-coverage floor, and reverse-neighbour fallback to make every paper learnable.The fallback addresses papers lacking incoming edges after graph pruning.
- Coverage-aware distillation: 84% of the corpus initially had an incoming edge in some facet; reverse-neighbour fallback recovers the remaining papers for training.The fallback flips outgoing relations and samples the strongest edges for those papers.
- Coverage-aware distillation: The resulting training set contains 294,089 examples with Kmin = 3 and N = 10, compared with 213,447 under uniform sampling.Training uses standard cross-entropy on facet-item source–target examples.
- Graph-RRF: Graph-RRF combines per-facet candidate lists so graph-corroborated papers receive larger scores while retaining facet provenance.The method generalises to other scores expressing how well a candidate is supported by a facet.
- Graph-RRF: Graph-RRF scales each candidate’s reciprocal-rank contribution by its query–candidate edge weight and sets unsupported facet contributions to zero.The method reads weights from sparse top-20 facet adjacency matrices and filters candidates lacking graph support.
5 Experiments
Experiments show that GRAFT approaches its graph teacher in-corpus, transfers better to unseen queries, and preserves graph-supported facet attribution. Coverage-aware supervision, natural-language identifiers, and graph-weighted fusion each address distinct retrieval constraints.
- 5.2 Baseline Comparison: GRAFT reaches R@20 0.326, recovering 91% of the dense-graph retriever’s 0.357 and outperforming every non-graph baseline.
- 5.2 Baseline Comparison: At R@50, GRAFT reaches 0.453 versus dense-graph’s 0.560, indicating that deeper graph neighbourhoods are only partly recovered.Reverse-neighbour fallback raises R@50 from 0.447 to 0.453 by adding decodable tail papers at lower ranks.
- 5.3 Out-of-Corpus Queries: On unseen queries, GRAFT leads at every cutoff, improving R@20 from 0.501 to 0.548 (+9.4%) despite being simpler than the graph retriever.The evaluation uses 500 papers outside the corpus and estimates query–candidate edge weights for GRAFT.
- 5.3 Out-of-Corpus Queries: Only 33.4% of GRAFT’s retrievals are obvious similarity neighbours, while 47.4% of its correct retrievals are non-obvious.GRAFT’s non-obvious retrievals have 4.2× the precision of a most-cited non-obvious baseline.
- 5.4 Coverage: Coverage mechanisms raise learnable DocID coverage from 84% to all 11,359 papers, contributing roughly two thirds of the gain over uniform sampling.The coverage floor and reverse-neighbour fallback directly address DocID starvation.
- 5.4 Coverage: Three views per DocID peak at R@20 0.322, compared with 0.299 at one view and 0.294 at four views.Too few views leave identifiers unlearned, while too many add synthesised pairs without additional graph evidence.
- 5.5 Fusion and Attribution: Graph-RRF raises fused R@20 from 0.284 with plain RRF to 0.322 (+0.038, +13.38%), while retaining facet attribution.MINDER’s language-model-based combination reaches 0.310 at R@20.
- 5.5 Fusion and Attribution: GRAFT reproduces graph facet attribution at 0.922 precision, with 89.9% of credited facet sets contained in the graph-supported reference sets.Precision is 0.933 for results, 0.931 for problems, 0.915 for contributions, and 0.903 for methods.
6 Conclusion
The paper represents heterogeneous scientific-paper relationships with a typed, weighted facet graph and indexes that structure through a generative retriever. On 11,359 NLP papers, GRAFT approaches its graph teacher, preserves facet attribution, and outperforms the teacher on queries outside the corpus.
- The paper asks how to represent heterogeneous paper relationships and index that structure efficiently for retrieval.
- LITWEAVE uses typed graph edges carrying facet and strength, while GRAFT distils the graph into a generative retriever.
- Coverage-aware distillation and graph-weighted RRF address the coverage and structural-grounding properties lost under naive distillation.
- On 11,359 NLP papers, GRAFT approaches its graph teacher’s recall, reproduces facet attribution, and outperforms the teacher on outside-corpus queries.
- Future work will expand the corpus beyond NLP to test graph-construction cost and memorisation capacity on a larger, sparser graph.
Limitations
The corpus and deployment setting impose scope and scalability boundaries. GRAFT is limited to a narrow, connectivity-biased corpus, while graph construction and model capacity may constrain growth and updates.
- The corpus covers only *ACL venues from 2019 to 2026, and k-core sampling favours well-connected papers over less prominent work.
- Graph construction is quadratic in the number of nodes, and the required model size as the graph grows is unknown.
- GRAFT must be retrained before it can return newly added papers, although new papers can be queried without retraining.
Ethics Statement
The paper constructs and audits an open-access NLP corpus by extracting four facet types from full text, filtering and sampling papers for connectivity, and evaluating facet quality through automated checks.
- LITWEAVE is built from open-access S2ORC papers, with no human subjects and facet-groundedness annotation conducted by three authors.
- Papers are filtered to selected *ACL and affiliated NLP venues from 2019–2026 and must contain parsed full text for facet extraction.
- The corpus is downsampled to 10,000 papers using a 3-core, year-proportional selection strategy that preserves citation density.
- The resulting graph contains 112,644 citation edges at mean degree 22.5, producing a dense and connected corpus.
- Each paper receives problem, method, result, and contribution facet items extracted from its full paper text and separately encoded as embeddings.
- The extraction audit checks structural validity, grounding, redundancy, and specificity, while failed schema responses are discarded without retrying.
A.4 Facet Groundedness Study
The groundedness study combines a cosine-similarity proxy with manual ratings by three annotators. Manual ratings were predominantly high, while agreement depends on the chance-corrected measure used.
- The cosine-based grounding check is only a proxy: similarity to a paragraph does not guarantee that an extracted item’s claim is supported.
- Three annotators rated 200 facet items from 50 papers on a five-point groundedness scale using source-paper context.
- 96.5% of pooled items were rated ≥4, with a mean score of 4.78 and 95% CI [93.0, 99.0].
- 87.5% of shared-core items were unanimously rated grounded, while exact inter-annotator agreement ranged from 70–82.5%.
- Gwet’s AC1 was 0.91 with 95% CI [0.81, 0.98], whereas ordinal Krippendorff’s α was 0.30 with 95% bootstrap CI [0.05, 0.53].
B.1 Edge Construction and Weight Selection
The paper constructs typed paper-graph edges from facet similarity and citation signals, then selects a combined weighting for retrieval. It uses top-K graph pruning, two-hop traversal, facet-specific generative identifiers, and constrained decoding.
- Edge construction: Facet scores compare papers through embedded facet items, while citation scores combine bibliographic coupling and co-citation counts after rescaling.Edge weights combine these two signals linearly.
- Weight selection: α = 0.75 is selected as the best facet/citation weighting on the development split.The sweep evaluates the dense-graph retriever across candidate α values with β = 1 − α.
- Weight selection: Any mixture of facet and citation scores beats either signal alone, with citation similarity acting as a floor for weak facet matches.The citation-only setting reaches R@20 0.323, whereas facet-only reaches R@20 0.273.
- Coverage and traversal: At K = 20, 84.0% of papers have an incoming edge in some facet, while the reverse-neighbour fallback supplies full coverage from existing graph edges.Increasing K to 50 raises incoming-edge coverage to 98.5% but also adds weaker relations and graph cost.
- Generative indexing: GRAFT normalizes each facet item into a DocID, truncates it to twelve tokens, decodes valid identifiers with facet-specific tries, and fuses candidates across facets.Constrained beam search discards sequences that do not terminate at populated identifier leaves.
- Generative indexing: Natural-language identifiers produce 184,288 distinct identifiers, with only 41 shared by more than one paper.The mean number of papers per identifier is 1.0003, and trie-constrained decoding restricts outputs to valid identifiers.
- Inference cost: The full-width decoding mask costs O(BV) per step because every 128,256-way output distribution is evaluated.Trie lookups are negligible compared with transformer forward passes.
C.4 MINDER Baseline
The MINDER baseline uses untyped title, text-substring, and pseudo-query identifiers with language-model-score aggregation. GRAFT’s facet-item identifiers improve recall when the aggregation is held fixed.
- Baseline design: The generative baseline holds the base model, training edges, and constrained decoding fixed while replacing GRAFT’s identifiers and fusion.This isolates identifier and aggregation choices from the rest of the setup.
- Baseline design: MINDER represents each paper with titles, contiguous text substrings, and synthetic pseudo-queries, without typed graph edges.Its candidates are aggregated by generated-identifier language-model probability.
- Results: GRAFT’s facet-item lists reach 0.310 R@20 and 0.464 R@50 under MINDER aggregation, versus 0.284 and 0.392 for full MINDER.Facet-item identifiers contribute +0.025 R@20 and +0.071 R@50 over MINDER’s default views.
D.1 Out-of-Corpus Evaluation
The out-of-corpus evaluation tests retrieval for papers never seen during training, using citation-linked papers as gold targets. GRAFT retrieves many non-obvious papers and preserves facet-specific attribution, while supervision alternatives and facet ablations reveal additional design effects.
- Evaluation setup: The out-of-corpus queries are 500 unused papers, with 2,095 citation-linked query–paper gold pairs and a mean of 4.19 gold papers per query.The evaluation uses direct citation as a link type not used in the other experiments.
- Out-of-corpus fusion: Graph-RRF cannot directly score out-of-corpus queries because graph adjacency matrices contain only indexed papers.The method estimates support from mutual corroboration among retrieved corpus candidates instead of adding query-time dense indexing.
- Results: 66.6% of GRAFT’s top-20 results are non-obvious, compared with 19.0% for the dense-graph retriever.Non-obvious papers fall outside the graph’s 50 nearest facet-similarity neighbours.
- Results: GRAFT’s non-obvious retrievals have 4.2× the precision of a most-cited-paper baseline.This comparison tests whether citation frequency alone explains the non-obvious retrieval lift.
- Facet ablations: Removing facets reduces R@20 most for problems (-0.020) and least for methods (-0.002).The ablation reports that every facet contributes to performance.
- Facet attribution: Facet attribution surfaces method-related neighbours even when they lack the query’s topic, as in the LLM-as-a-judge paper returned for CEval.The per-facet graph-RRF terms provide the provenance for why each paper was recommended.
- Supervision alternatives: Enumerating every source–target bullet pair reduces R@20 to 0.242, while hard negatives reach 0.211 with a top-50 pool and 0.201 with top-100.The paper attributes these declines to redundant combinations and confusingly similar negative examples.
E Model Capacity
Model capacity is a central constraint for GRAFT’s memorization-based generative retrieval. Full fine-tuning outperforms LoRA, and smaller base models fail to memorize the corpus.
- Adaptation strategy: LoRA rank 128 reaches R@20 0.230, while rank 256 reaches 0.237 and remains below full fine-tuning.The marginal gain from rank 128 to 256 is only +0.007.
- Base-model capacity: Recall degrades sharply as base-model capacity falls, and the two smallest models fail to memorize the corpus.The comparison holds the retrieval method fixed while varying the base model.
- Training configuration: The retriever is fully fine-tuned with Llama-3.2-1B-Instruct rather than adapted through a parameter-efficient method.Training uses two epochs, effective batch size 16, learning rate 5e−5, and maximum sequence length 512.
- System configuration: Graph construction uses all-MiniLM-L12-v2 embeddings, α = 0.75 facet weighting, β = 0.25 citation weighting, top-20 edges, and two-hop traversal.Inference fusion uses RRF with k = 60.