PaperlayerView exact revision on arXiv

Source-linked AI summary

PaperTrail: A Claim-Evidence Interface for Grounding Provenance in LLM-based Scholarly Q&A

Anna Martin-Boyle, Cara A. C. Leckey, Martha C. Brown, Harmanpreet Kaur

arXiv:2602.21045v1cs.HCcs.CL

TL;DR

Scholarly QA systems use LLMs to synthesize literature, but errors and coarse provenance make rigorous verification difficult. PaperTrail maps claims and evidence between answers and source documents; in a study it lowered trust versus citations, without changing editing behavior. The findings highlight the challenge of presenting provenance in ways that fit time- and cognition-constrained scholarly work.

  • Problem

    LLM scholarly QA can produce hard-to-detect errors, while source citations may not provide sufficiently granular, reliable evidence for rigorous verification.

    Source: §1 · p. 2

  • Method

    PaperTrail decomposes source documents and LLM answers into claims and evidence, then matches them to expose unsupported answer claims and omitted paper claims.

    Source: §1 · p. 2

  • Results

    In a within-subjects study with 26 researchers, PaperTrail significantly reduced trust versus citation-based provenance but did not change editing behavior.

    Source: §9 · p. 18

  • Takeaways & Limitations

    Time pressure, system latency, and interface complexity constrained meaningful engagement with provenance, motivating designs that reduce cognitive load.

    Source: §7 · p. 16

    The sample came from one organization, limiting coverage of disciplines, verification norms, time pressures, and students’ experiences.

    Source: §8 · p. 18

Abstract

from arXiv · show

Large language models (LLMs) are increasingly used in scholarly question-answering (QA) systems to help researchers synthesize vast amounts of literature. However, these systems often produce subtle errors (e.g., unsupported claims, errors of omission), and current provenance mechanisms like source citations are not granular enough for the rigorous verification that scholarly domain requires. To address this, we introduce PaperTrail, a novel interface that decomposes both LLM answers and source documents into discrete claims and evidence, mapping them to reveal supported assertions, unsupported claims, and information omitted from the source texts. We evaluated PaperTrail in a within-subjects study with 26 researchers who performed two scholarly editing tasks using PaperTrail and a baseline interface. Our results show that PaperTrail significantly lowered participants' trust compared to the baseline. However, this increased caution did not translate to behavioral changes, as people continued to rely on LLM-generated scholarly edits to avoid a cognitively burdensome task. We discuss the value of claim-evidence matching for understanding LLM trustworthiness in scholarly settings, and present design implications for cognition-friendly communication of provenance information.

1 Introduction

Scholarly LLM tools promise to ease literature synthesis, but errors and coarse provenance make their outputs difficult to verify. PaperTrail exposes claim-evidence links; a study found lower trust than with citation-based provenance, but no change in reliance behavior.

1 Introduction

  • Researchers face growing cognitive demands from scientific literature, while LLM tools promise to automate synthesis and other scholarly tasks.Examples include accelerating systematic reviews, citation analysis, and identifying research patterns and gaps.Source: §1 · p. 2
  • Fluent LLM outputs can contain hard-to-detect hallucinations and misinformation, posing particular risks for literature reviews, peer assessment, and research design.These errors can persist in retrieval-augmented systems [70] [88].Source: §1 · p. 2
  • Citations can be inaccurate, while explanations and uncertainty cues may increase cognitive load without giving researchers actionable ways to verify evidence.Time pressure and cognitive load may also prevent researchers from acting on skepticism even when they recognize problems.Source: §1 · p. 2
  • PaperTrail decomposes source documents and LLM answers into claims and evidence, exposing unsupported answer claims and omitted paper claims.The design draws on argumentation structures in scholarly work and aims to support verification through users’ existing mental models of scholarly arguments.Source: §1 · p. 2
  • In a study of 26 domain experts completing two scholarly editing tasks, PaperTrail lowered trust versus citation-based provenance but did not change confidence or reliance behavior.The evaluation measured trust, confidence in revised outputs, and reliance via normalized Levenshtein edit distance.Source: §1 · p. 2
  • The paper contributes a provenance system and empirical evidence that reduced trust alone may not change reliance behaviors under constraints of time, usability, and cognitive resources.Participants valued the detailed breakdown, but feedback also indicated clutter and reduced usability in the study setup.Source: §1 · pp. 2–3

2 Related Work

Prior work studies provenance, explanations, argument structures, and scholarly QA, but often treats attribution as a backend problem or uses coarse evidence links. PaperTrail builds on these lines of work to study claim-evidence provenance in an expert-facing interface.

2.1 Evidence-Based Text Generation, Attribution, and Provenance

  • Most sourcing systems attach provenance at document, paragraph, or sentence granularity, whereas PaperTrail aligns discrete answer claims with paper claims and evidence snippets and exposes omissions and mismatches.Prior attribution work formalizes mapping answer statements to cited passages [72]; the paper characterizes PaperTrail as posterior external-data sourcing [93].Source: §2.1 · p. 3

2.2 LLMs and Explanations

  • Studies find that uncertainty cues can reduce overreliance, while explanations can increase reliance on correct and incorrect answers; verifiable sources may help users calibrate reliance.Persistent disclaimers can reduce overreliance, but token-level cues and removing direct answers can impose time costs without reliably improving appropriate reliance [15].Source: §2.2 · p. 3

2.3 Argument Structures in Textual Contexts

  • Argument structures organize claims and their support, and prior work reports benefits from explicitly organized arguments in medical QA and educational psychology.The paper proposes claims and evidence as the right unit to expose for scholarly sense-making.Source: §2.3 · p. 3

2.4 Scholarly Question Answering

  • Scholarly QA requires reasoning over long technical texts, and reported system problems include fabricated evidence, inconsistent accuracy, and errors in synthesis, formatting, interpretation, and completeness.These limitations motivate interfaces that make evidence intelligible; this work studies how a claim-evidence interface affects trust and reliance during scholarly writing and critique.Source: §2.4 · pp. 3–4

3 PaperTrail: a Scholarly Source Provenance System

PaperTrail is a system for investigating how argument-grounded provenance affects trust and reliance in LLM-based scholarly QA.

3 PaperTrail: a Scholarly Source Provenance System

  • PaperTrail generates and displays claim-evidence structures from source documents and real-time LLM-generated answers.The system is intended to investigate how argument-grounded provenance affects user trust and reliance.Source: §3 · p. 4
  • The system’s backend determines source provenance through argument extraction and matching.The paper describes the backend architecture before presenting the design goals and frontend interface.Source: §3 · p. 4
  • PaperTrail combines source-document processing with real-time analysis of LLM answers in scholarly QA.Its interface presents the resulting provenance annotations to users.Source: §3 · p. 4

3.1 Backend: Argument Extraction and Matching for Grounding Provenance

PaperTrail combines extraction methods with different cost and semantic-capability tradeoffs across offline source processing and real-time answer analysis. Its pipeline pre-extracts paper claims and evidence, decomposes answers, then matches their claims and evidence to produce provenance indicators.

3.1 Backend: Argument Extraction and Matching for Grounding Provenance

  • LLM-based, similarity-based, and RAG extraction trade off semantic capability and computational cost across offline paper processing, real-time answer extraction, and real-time claim-evidence matching.Stage-specific choices depend on query availability, timing, and text volume.Source: §3.1 · p. 4

3.1.1 Stage 1: Paper-Level Claim-Evidence Extraction.

  • Offline extraction uses LLM prompts to identify atomic, faithful, decontextualized, verifiable, declarative claims, then similarity search to find candidate supporting evidence.The one-time preprocessing shifts computation out of user-facing interactions and creates a consistent, reproducible corpus; evidence sentences above a 0.75 similarity threshold are candidates, with neighboring sentences added as context.Source: §3.1.1 · pp. 4–5

3.1.2 Stage 2: Answer-Level Claim-Evidence Extraction.

  • A separate extraction LLM decomposes each generated answer into claims and evidence in a single pass, then maps the extracted items to text spans for interface highlighting.The answerer receives the query, history, task context, and— in the study configuration—source documents; the extraction process is model-agnostic.Source: §3.1.2 · p. 5

3.1.3 Stage 3: Claim-Evidence Matching.

  • The matching stage produces global explanations of claim coverage and local explanations of individual claim provenance.Figure 2 encodes extraction-method computational cost by color: LLM red is highest, RAG gold is medium, and similarity teal is lowest.Source: §3.1.3 · p. 5, Figure 2 · p. 6
  • Matching retrieves relevant paper claims and evidence using similarity filtering and LLM selection, then aligns answer claims with semantically equivalent source claims.Answer evidence is flagged as potentially unsupported below a cosine similarity threshold of 0.55, chosen through iterative testing on five held-out examples to balance divergence detection and alert fatigue.Source: §3.1.3 · pp. 5, 7

3.1.4 Implementation Details.

  • The implementation uses Gemini 2.5 Pro for LLM operations and SPECTER for similarity operations, with average end-to-end response latency around 90 seconds per query.Sequential LLM calls in Stages 2 and 3 account primarily for the latency.Source: §3.1.4 · p. 7

3.2 Frontend Interface

PaperTrail’s three-panel interface organizes scholarly QA around linked claim-evidence provenance. Four design goals guide how users access provenance, navigate verification, and manage interaction overhead.

3.2.1 Design Goals.

  • Graduated engagement moves users from an answer overview to claim annotations and then source text on demand.The design also aims to minimize interaction overhead through coordinated views, automatic highlighting, and selective information hiding.Source: §3.2.1 · p. 7
  • Flexible verification supports bidirectional navigation from answer claims to evidence and from paper claims to omissions in the answer.The interface avoids enforcing a single verification pathway, accommodating both top-down checking and bottom-up discovery.Source: §3.2.1 · p. 7
  • The interface represents scholarly discourse through claims and evidence to align with researchers’ existing domain mental models.This goal reflects the structural and causal features experts use in their domain knowledge.Source: §3.2.1 · pp. 7–8

3.2 Frontend Interface

  • The three-panel web application separates the task workspace, scholarly QA chat, and provenance while synchronizing panels for coordinated navigation.Selections auto-scroll linked panels, and users can resize panels and scroll panes independently.Source: §3.2 · p. 7, §3.2.2 · p. 8

3.2.2 Interface Layout and Features.

  • The left panel keeps task instructions, source references, and the editable LLM-generated draft together, while the middle panel supports questions and answer-claim inspection.Users can select pre-written questions or ask their own, then click answer claims to inspect corresponding provenance.Source: §3.2.2 · p. 8
  • The right panel distinguishes matched paper claims from claims omitted in the answer and provides an overall Claim Coverage indicator.Clicking an answer claim highlights its corresponding paper claim; provenance cards remain organized by chatbot message and can be expanded or removed.Source: §3.2.2 · p. 8

4 Offline Evaluation for Backend Approach

The offline evaluation adapts two datasets because no gold-standard benchmark covers exhaustive atomic claim extraction from scientific texts. Results show different precision-recall profiles across the datasets, with the authors cautioning that reference-set granularity affects interpretation.

4 Offline Evaluation for Backend Approach

  • The extraction evaluation adapts SciClaimHunt and BioClaimDetect because no gold-standard benchmark measures exhaustive atomic claim extraction from scientific texts.Existing datasets address nonexhaustive sentence-level claim classification or claim verification.Source: §4 · p. 8

4.1.1 SciClaimHunt.

  • The evaluation uses 50 SciClaimHunt paragraphs with at least three associated claims and 50 BioClaimDetect abstracts sampled from its test set.SciClaimHunt claims were mapped to paragraphs using exact sentence matches, then SPECTER similarity for remaining claims; BioClaimDetect references were whole sentences labeled as claims.Source: §4.1.1 · p. 8, §4.1.2 · p. 8

4.2 Evaluation Procedure

  • Reference and extracted claims are embedded with SPECTER and matched when pairwise cosine similarity exceeds τ=0.9.Recall measures the share of reference claims matched, precision the share of extracted claims matched, and F1 their harmonic mean.Source: §4.2 · p. 10
  • Because neither dataset exhaustively annotates atomic claims, the authors expect precision to be underestimated and consider recall the more realistic comparison measure.Fine-grained valid extractions may not match references produced at lower granularity.Source: §4.2 · p. 10

4.3 Validation Results

  • On BioClaimDetect, precision was 0.63, recall was 0.88, and F1 was 0.73; on SciClaimHunt, precision was 0.69, recall was 0.62, and F1 was 0.65.The authors report that unmatched BioClaimDetect extractions were often valid fine-grained claims absent from sentence-level annotations, while SciClaimHunt references were generated by Llama-2-13B.Source: §4.3 · p. 10

5 User Study Design

The within-subjects study compares PaperTrail with a source-citation baseline across two scholarly editing tasks. The task materials and question bank were designed to elicit literature synthesis and manuscript critique.

5 User Study Design

  • The study compares PaperTrail with a source-citation baseline in a within-subjects design with 26 participants completing two scholarly tasks.Interface order was randomized and counterbalanced, while task order was fixed to control for task progression, fatigue, and learning effects.Source: §5 · p. 10

5.1 Baseline Interface

  • The baseline surfaces verbatim source text when users click sentence-level source highlights, unlike PaperTrail’s claim-to-evidence matching.This comparison tests whether more granular provenance can make scholarly LLM use more deliberate and tempered.Source: §5.1 · p. 10

5.2.1 Scholarly Writing Tasks.

  • Participants edit an approximately 300-word multi-paper synthesis and a 300-word devil’s-advocate defense generated by an LLM.The first task concerns integrating methods and findings across four papers; the second asks participants to verify and edit a draft defense of a paper’s novelty and soundness.Source: §5.2.1 · p. 10
  • The tasks reflect literature review and critical manuscript feedback, activities reported in prior work on researchers’ LLM use.The cited survey found 81% of 816 researchers already incorporated LLMs into research workflows, with literature review among frequent uses [75].Source: §5.2.1 · p. 10

5.2.2 Source Document Corpus.

  • The source corpus comprises four peer-reviewed Mars exploration papers selected for interdisciplinary accessibility, thematic coherence, and methodological diversity.The papers address optimization, Mars-base architecture, settlement design, and crewed-mission scale and specialization.Source: §5.2.2 · pp. 10–11

5.2.3 Question Bank.

  • A pre-written question bank and pre-generated answers reduce task burden and avoid response lag during provenance interactions.Questions target testing-category synthesis for Task 1 and deep reasoning about causal relations, rationales, and procedures for Task 2; surface questions were omitted.Source: §5.2.3 · p. 11

5.3 Procedure and Flow

Participants completed two editing tasks through a remote, browser-based study flow, with post-task surveys and interaction measures. The study assessed trust, reliance, confidence, workload, usability, and interaction behavior.

5.3 Procedure and Flow

  • Participants completed a pre-study survey, tutorials, two editing tasks, post-task surveys after each task, and a final preference and feedback form.Sessions ran remotely and asynchronously in a web browser.Source: §5.3 · pp. 11–12

5.4 Participants

  • The reported sample included 26 participants after excluding 12 completed studies for invalid responses or inadequate engagement.Excluded cases included tasks under 10 minutes without system interaction, gibberish responses, and reported latency that prevented engagement.Source: §5.4 · p. 12

5.5.1 Primary Measures.

  • Primary measures covered trust in the LLM, reliance on its placeholder text, and confidence in the edits and final output.Trust used the TXAI scale; confidence averaged two 7-point items, while reliance was based on changes to the draft.Source: §5.5.1 · p. 12
  • Reliance is measured as one minus token-level edit distance between the original draft and final text, normalized by the longer sequence.The method treats word-level additions, deletions, and substitutions equally, capturing editing behavior rather than semantic quality.Source: §5.5.1 · p. 12

5.5.2 Secondary Measures.

  • Secondary measures include workload, usability, task time, chat questions, provenance-feature clicks, interface preference, and open-ended feedback.Workload uses the raw NASA-TLX mean of five subscales, omitting Physical Demand; usability uses the SUPR-Q average.Source: §5.5.2 · p. 12

5.6 Analyses

The analyses combine paired tests for primary outcomes, descriptive comparisons for exploratory experience measures, qualitative coding, and correlations across participant and outcome variables.

5.6 Analyses

  • Paired baseline–PaperTrail comparisons used t-tests for normally distributed data and Wilcoxon signed-rank tests otherwise, with normality assessed by Shapiro–Wilk.All tests were two-tailed with alpha 0.05.Source: §5.6 · p. 13
  • Exploratory cognitive-load, usability, click, and message comparisons were descriptive, while interface preference was reported as a frequency count.Source: §5.6 · p. 13
  • Qualitative feedback was inductively coded, and Spearman correlations explored relationships among demographics, experiential measures, and primary outcomes across conditions.Source: §5.6 · p. 13

6 Results

PaperTrail lowered reported trust but did not significantly change reliance or confidence, while usability ratings were lower. Qualitative findings describe time pressure and interface complexity alongside participants’ interest in provenance for verification.

6.1 Primary Measures: Trust, Reliance, and Confidence

  • Trust was significantly lower with PaperTrail than baseline (t(25) = 2.61, p = 0.015; Cohen’s d = 0.44), but reliance and confidence did not differ significantly.Reliance: W = 146, p = 0.313; confidence: t(25) = 0.64, p = .525.Source: §6.1 · p. 13

6.2 Secondary Experiential Measures

  • PaperTrail was rated significantly less usable than baseline (W = 71.5, p = 0.026; r = 0.52), with the authors suggesting task difficulty and the interface learning curve as possible explanations.The authors note that usability perceptions might differ after users develop expertise in a real-world deployment.Source: §6.2 · p. 13
  • W = 97.50, p = 0.137: participants clicked more with PaperTrail (M = 12.47, SD = 14.96) than baseline (M = 8.26, SD = 6.02), but click counts could reflect either exploration or overhead.Cognitive load and messages sent to the LLM were similar across interfaces; the authors caution that click patterns cannot establish productive verification.Source: §6.2 · pp. 13–14

6.3 Correlation Analysis

  • In baseline, reliance correlated positively with trust (r = 0.57, p < 0.01) and usability (r = 0.60, p < 0.05); these associations were absent with PaperTrail.In PaperTrail, trust, confidence, and usability remained strongly related, but none was significantly correlated with editing reliance; education also correlated negatively with confidence (r = −0.41, p < .05).Source: §6.3 · p. 14

6.4.2 The Information Complexity–Usability Tradeoff.

  • Participants valued detailed provenance but found PaperTrail cluttered and its claim-evidence cards restrictive; the question bank helped exploration but did not resolve the usability tradeoff.Nested windows limited reading on small screens, while rigid cards constrained verification strategies despite the question bank’s entry point for exploration.Source: §6.4 · p. 14, §6.4.2 · pp. 14–15

6.4.1 Factors Affecting Reliance Behaviors.

  • Time constraints, compounded by slow system responses, limited participants’ engagement with provenance and claim verification.Over half explicitly mentioned time constraints, and one participant reported relying more on AI and spending less time revising and checking.Source: §6.4.1 · p. 14
  • Participants’ differing views of task authenticity shaped how much they familiarized themselves with papers and how closely they edited LLM output.Some wanted to read the papers first; others accepted the study’s artificial context and judged the output close enough for the task.Source: §6.4.1 · p. 14

6.4.3 The Ethos of Grounding LLM Outputs in Provenance Informa-

  • Participants recognized verification as important and valued tracing claims to original context, though usability issues limited positive experiences.Feedback linked LLM errors to stripped context and the need to locate claims in source papers for verification.Source: §6.4.3 · pp. 15–16

7 Discussion

PaperTrail lowered trust in LLMs, but did not change reliance; the discussion connects this trust-behavior gap to verification costs and outlines ways to improve provenance and its evaluation.

7 Discussion

  • PaperTrail lowered trust without changing reliance; translating skepticism into action requires addressing time, usability, and ingrained tool-use patterns.The authors say the implementation’s usability costs likely contributed to this trust-behavior gap.Source: §7 · p. 16

7.1 Trust-Behavior Gap: Why Didn’t Lower Trust Change Reliance?

  • System 1 defaults, limited feedback for learning reliable error-detection heuristics, and costly verification help explain why caution did not change reliance.The authors also describe a design trade-off: excessive friction hurts usability, while too little may not prompt analytical verification.Source: §7.1 · p. 16

7.2 Theoretical Implication: From Trust to Trustworthiness

  • Rather than always lowering trust, the authors frame the goal as making LLMs trustworthy without faithful model interpretability.They caution that excessive skepticism could lead people to under-use a system.Source: §7.2 · p. 17

7.2.1 Argument-Grounded Provenance as a Metric of LLM Trustwor-

  • Claim-level matching plus global claim coverage can measure faithfulness and completeness, including omissions, as an application-grounded evaluation for scholarly QA.The authors propose this approach as a complement to existing evaluation methods.Source: §7.2.1 · p. 17

7.3 Implications for Design

  • Future provenance designs should balance information completeness with interaction fluidity.The authors identify this as a competing demand for future systems.Source: §7.3 · p. 17

7.3.1 Managing the Cost of Information Granularity with Adaptive

  • Adaptive provenance could show simpler cues for straightforward queries and reveal fuller claim-evidence detail for complex or controversial ones.Progressive disclosure is one proposed way to manage information granularity.Source: §7.3.1 · p. 17

7.3.2 Scaffolding Critical Thinking Through Progressive Engagement.

  • Build verification skills progressively, reward genuine exploration rather than point-scoring, and account for scholarly QA’s open-ended, context-dependent judgments.The authors note that appropriate reliance differs conceptually between open-ended scholarly QA and decision-making tasks.Source: §7.3.2 · p. 17

7.4 Computational Considerations

  • Real-time deep semantic analysis is computationally expensive and caused latency that hurt the study’s user experience.The current pipeline retains real-time LLM processing for answer generation and claim matching, creating sequential bottlenecks.Source: §7.4 · p. 17

8 Limitations and Future Work

The evaluation’s artificial, time-constrained tasks, reliance and trust measures, and single-organization sample limit what the findings establish. The authors also identify design and source-structure assumptions and call for field and longitudinal work.

Task Realism

  • Participants used unfamiliar papers under artificial time constraints, with 20–30 minutes for tasks rather than authentic literature searches or familiar research materials.The standardized setup may have altered engagement patterns; the authors call for field studies using researchers’ actual work, familiar literature, and self-directed questions.Source: §8 · pp. 17–18

Metrics and Measurement

  • Edit distance may conflate passive acceptance with strategic delegation, and the study did not assess the quality of participants’ edited texts.The authors also note that participants may not have distinguished trust in the interface from trust in the LLM.Source: §8 · p. 18

Design Generalizability

  • Findings address one three-panel interface and assume source documents have clear claim-evidence structures; unsupported flags may instead reflect source coverage gaps.The authors note that users cannot easily distinguish a genuine LLM error from insufficient coverage in the source corpus.Source: §8 · p. 18

Participant Pool

  • Researchers from one organization are a narrow sample; disciplinary differences, students, and other external-validity questions remain unexamined.The authors leave evaluation of external validity to future work.Source: §8 · p. 18

Long-term Adaptation

  • A single-session study captured initial reactions, leaving open how trust and reliance may change as users develop mental models of system capabilities and limitations.The authors identify longitudinal deployment as a way to examine whether the trust-behavior gap narrows or selective engagement stabilizes.Source: §8 · p. 18

9 Conclusion

PaperTrail provides argument-grounded provenance for scholarly QA; in a within-subjects study with 26 researchers, it lowered trust but did not change editing behavior.

9 Conclusion

  • PaperTrail significantly reduced trust in LLM outputs compared with standard citation-based provenance, but participants edited generated text similarly across conditions.Time pressure, system latency, and interface complexity prevented meaningful engagement with provenance features.Source: §9 · p. 18
  • Participants recognized the value of claim-level verification, yet time pressures and cognitive constraints left the gap between verification needs and actions unresolved.The conclusion identifies granular provenance alone as insufficient to change behavior in typical research settings.Source: §9 · p. 18
  • PaperTrail’s claim-evidence extraction backend shows promise as an evaluation framework for the trustworthiness of scholarly LLM outputs.Source: §9 · p. 18

A Prompts

The appendix documents PaperTrail’s prompts and schemas across offline paper processing and real-time answer processing and claim matching.

A Prompts

  • All LLM-based operations use Gemini 2.5 Pro with structured JSON output; similarity-based retrieval and verification do not require prompts.Source: A Prompts · p. 22

A.1 Paper-Level Claim Extraction

  • Stage 1 extracts paragraph-level claims as atomic, verifiable, faithful, standalone assertions to build a structured claim database.Source: §A.1 · pp. 22–23
  • Stage 1 evidence retrieval uses cosine similarity with a threshold of 0.75 and does not require an LLM prompt.Source: §A.1 · p. 23

A.2 Answer Generation

  • Stage 2 generates answers using source documents as context and uses sentence-level citation tags for baseline attribution before removing them from display.Source: §A.2 · p. 23

A.3 Answer-Level Claim and Evidence Extraction

  • Stage 2 extracts answer claims and supporting evidence in one pass, retaining exact source spans for precise interface highlighting.Source: §A.3 · p. 23

A.4 Relevant Claims Selection

  • Stage 3 first retrieves candidate paper claims using SPECTER embedding similarity, then uses LLM reasoning to select claims relevant to the question.Source: §A.4 · p. 24

A.5 Relevant Evidence Selection

  • For each relevant claim, Stage 3 similarity-filters its supporting evidence before the LLM selects the most pertinent passages.Source: §A.5 · p. 24

A.6 Claim-to-Claim Matching

  • Stage 3 matches semantically equivalent answer and paper claims, while cosine similarity below 0.55 flags potentially unsupported evidence.Source: §A.6 · pp. 24–25

Related papers

Loading 2602.21045v1…