PaperlayerView exact revision on arXiv

Source-linked AI summary

From Papers to Mechanisms: An Evidence-Grounded Knowledge Substrate for Scientific Language Models

Qiuhui Chen, Yibo Liu, Tao Dai, Jiafan Lu, Zhenglei Zhou, Weimin Zhong

arXiv:2610.06248v1cs.AI

TL;DR

Scientific language models often access literature through untyped chunks, which poorly match mechanism questions requiring functional links across distributed evidence. The paper builds a provenance-linked, role-typed mechanism substrate and instantiates it as MS3 for conductive-fiber flexible sensors. It reports improved scientific correctness against Web search and a source-resolved Raw-PDF oracle, with a further advantage for hybrid retrieval under coverage shift.

  • Problem

    Scientific literature is often represented as untyped chunks, although mechanism reasoning requires connecting functional roles and evidence distributed across sources and formats.

    Source: Introduction · p. 1

  • Method

    The paper organizes source-grounded evidence into role-typed entities, directed relations, and complete or partial mechanism paths, with provenance links and retrieval across a 13,689-paper MS3 substrate.

    Source: Introduction · pp. 1–2

  • Results

    Verified MS3 improves macro-averaged scientific correctness by 26.9 percentage points over Web search and 5.8 points over a source-resolved Raw-PDF oracle; under coverage shift, Hybrid retrieval improves over the stronger single-source condition by 3.7 points on average.

    Source: Conclusion · p. 7

  • Takeaways & Limitations

    The findings support knowledge representation and evidence routing as material factors in grounded scientific question answering.

    Source: Conclusion · p. 7

    The current substrate is restricted to conductive-fiber flexible sensors, schema transfer remains unevaluated, and experiments assess single-pass retrieval rather than the complete source-repair workflow.

    Source: Mechanism-Aware Access for Language… · p. 4, Limitations · p. 7

Abstract

from arXiv · show

Scientific language models often access literature through untyped text chunks, which fragment the functional and evidential structure required for mechanism-rich questions. We introduce an evidence-grounded mechanism knowledge substrate that organizes scientific literature into provenance-linked evidence units, role-typed entities, and directed mechanism paths. We instantiate it as MS$^3$, a Material-Sensor-Signal-System schema for conductive-fiber flexible sensors, over 13,689 papers, 131,083 evidence items, and 26,648 mechanism objects. On in-domain and coverage-shift question-answering benchmarks, we compare closed-book generation, Web search, Raw-PDF RAG, and MS$^3$ retrieval across ten language models. MS$^3$ improves macro-averaged scientific correctness. It also improves citation entailment and answer completeness. These results support mechanism substrates as a reliable representation layer for scientific language models and motivate a source-repair workflow in which insufficient MS$^3$ evidence triggers targeted retrieval from its linked papers rather than assuming that a user has already supplied the correct PDFs.

Introduction

Scientific language models often retrieve literature as untyped chunks, although mechanism questions require functional links supported by evidence distributed across text, figures, captions, and tables. The paper introduces a provenance-linked, role-typed mechanism substrate and instantiates it for conductive-fiber flexible sensors.

Introduction

  • Mechanism questions can require linking material composition or fabrication to a device, measurable response, and downstream application using evidence spread across text, figures, captions, and tables.Existing systems may abstract away source evidence or retain untyped chunks, leaving the model to reconstruct functional structure and assess support at inference time.Source: Introduction · p. 1
  • The proposed substrate organizes literature into provenance-linked evidence units, role-typed entities, directed relations, and complete or partial mechanism paths.It is schema-configurable, while extraction, provenance preservation, mechanism construction, retrieval, and evidence packing remain shared; unsupported nodes and relations are left absent.Source: Introduction · pp. 1–2
  • MS3 instantiates the framework for conductive-fiber flexible sensors as a Material–Sensor–Signal–System path, with System distinguishing demonstrated use from proposed application.The schema also records validation setting and deployment status separately, and retains only complete or partial paths supported by the paper.Source: Introduction · p. 2
  • The MS3 substrate covers 13,689 papers and 26,648 mechanism objects, and improves scientific correctness for nine of ten language models in blinded cross-family evaluation.The substrate contains 131,083 source-grounded evidence items.Source: Introduction · p. 2
  • The paper's contributions include provenance-aware retrieval and evidence selection, alongside the representation, large-scale sensor-domain instantiation, and evaluation across ten language models.The evaluation uses paired uncertainty estimates and blinded cross-family judging.Source: Introduction · p. 2

Related Work

Prior work advances scientific retrieval, information extraction, knowledge graphs, and evidence verification, but these approaches generally do not organize source evidence into reusable, role-typed mechanism paths. This paper positions its substrate as a complementary representation layer for those systems.

Scientific literature retrieval and synthesis

  • Scientific RAG systems retrieve passages or citation-linked neighborhoods, but language models must still infer evidence roles and whether pieces form a coherent mechanism.PaperQA retrieves and evaluates full-text passages (Lála et al. 2024), PaperQA2 extends literature search and synthesis (Skarlinski et al. 2024), OpenScholar performs multi-paper synthesis, and SciRAG adds adaptive retrieval (Ding et al. 2026).Source: Related Work · p. 2

Scientific information extraction and knowledge graphs

  • Scientific extraction, schema-driven methods, and knowledge graphs represent entities and relations, but do not generally combine source-level evidence, functional roles, directed mechanisms, partial-path uncertainty, and a common model interface.Examples include SciERC (Luan et al. 2018), SciREX (Jain et al. 2020), SPIRES (Caufield et al. 2024), and materials-focused MatKG.Source: Related Work · p. 2

Evidence-grounded scientific question answering and verification

Related Work

  • The work is positioned at the intersection of scientific RAG, knowledge-graph construction, and evidence-grounded verification, with organized scientific evidence as its primary object of study.The substrate is presented as complementary to retrieval and agent-orchestration advances.Source: Related Work · pp. 2–3

Methodology

The method defines a schema-configurable substrate that links source-grounded evidence to typed mechanism paths, then retrieves and packs relevant mechanisms and evidence for language-model answers. Its evaluated access system uses deterministic lexical retrieval; the full source-repair loop is not evaluated in the experiments described here.

Problem Formulation

  • The framework maps a scientific corpus and domain-specific schema into evidence units, typed nodes and relations, mechanism paths, and links from retained elements to supporting evidence.For flexible sensors, the schema follows Material → Sensor → Signal → System.Source: Problem Formulation · p. 3

Constructing the Mechanism Knowledge Substrate

  • Evidence extraction preserves source locations and accepts only text aligned to its originating block, while the staged models normalize terms and build evidence-supported complete or partial paths.Unsupported layers remain absent rather than being completed from model priors.Source: Constructing the Mechanism Knowledge… · p. 3

Mechanism-Aware Access for Language Models

  • Deterministic query parsing searches separate mechanism and evidence indices, ranks candidate papers by equally weighted BM25 scores, and prioritizes evidence linked to selected mechanisms.The retrieval procedure expands linked evidence identifiers rather than traversing graph edges.Source: Mechanism-Aware Access for Language… · p. 3
  • Retrieved records are serialized as mechanism cards under fixed top-k limits, and the answer model is prompted to cite evidence identifiers or report insufficient evidence.The implementation does not use an explicit input-token budget or redundancy-aware packing objective.Source: Mechanism-Aware Access for Language… · p. 4
  • The proposed source-repair loop checks claim-level support and retrieves targeted passages from linked papers when cards are insufficient, but the reported experiments evaluate only single-pass MS3 retrieval.The full source-repair loop requires a separate controlled evaluation.Source: Mechanism-Aware Access for Language… · p. 4

Experiments

MS3 retrieval improves in-domain scientific correctness over closed-book generation and Web Search, while Hybrid retrieval also improves correctness under coverage shift. Ablations favor the complete verified construction pipeline, and the experiments remain limited to single-pass retrieval in the conductive-fiber sensor setting.

Experiments

  • The 300-question in-domain benchmark covers four question types across 362 target papers and is disjoint from the development benchmark at the paper level.It compares Closed-book Direct, Web Search, Verified MS3, and Raw-PDF RAG; the latter receives the known target paper as an oracle input.Source: Experiments · pp. 4–5
  • MS3 gains 54.4 points over Closed-book Direct and 26.9 points over Web Search in macro-averaged scientific correctness, outperforming Web Search for 9 of 10 generators.Qwen3-30B-A3B is the exception: Web exceeds MS3 by 5.0 points.Source: Experiments · p. 5, Table 1 · p. 6
  • Compared with Web Search, MS3 improves citation entailment by 28.2 points and completeness by 36.4 points on the in-domain evaluation.The authors note that these results do not establish the contribution of each structural component individually.Source: Experiments · pp. 5, 7, Table 2 · p. 7

Ablation Studies

  • The complete verified E/T/M pipeline achieves 74% semantic full-path accuracy and 72% effective quality, versus 43% full-path accuracy for decomposed extraction without verification.Adding verification recovers 31 points in semantic full-path accuracy and 22 points in effective quality.Source: Figure 5 · p. 7, Ablation Studies · p. 7

Experiments

  • Hybrid retrieval reaches 60.1% macro-average scientific correctness under coverage shift, exceeding the strongest single-source condition by 3.7 points on average.It improves over the better of Web Search and Verified-MS3 for nine of ten generators; Kimi-K3 is the exception at −7.5 points.Source: Experiments · p. 7

Limitations

  • The evaluation covers conductive-fiber flexible sensors, does not evaluate schema transfer to other domains, and tests single-pass MS3 retrieval rather than the complete source-repair workflow.The benchmark primarily measures evidence use after candidate sources can be identified, not unrestricted literature discovery.Source: Limitations · p. 7

Conclusion

The paper presents an evidence-grounded mechanism substrate and instantiates it as MS3 for conductive-fiber flexible sensors. MS3 improves scientific correctness over Web Search and a source-resolved Raw-PDF oracle in-domain, while Hybrid retrieval improves over the strongest single-source condition under coverage shift.

Conclusion

  • The paper introduces a substrate that organizes scientific literature into provenance-linked, role-typed mechanism paths.MS3 supports grounded mechanism reconstruction, quantitative lookup, cross-paper comparison, and evidence-boundary reasoning.Source: Conclusion · p. 7
  • Verified MS3 improves in-domain macro-averaged scientific correctness by 26.9 percentage points over Web Search and 5.8 points over a source-resolved Raw-PDF oracle.Under coverage shift, Hybrid retrieval improves over the stronger single-source condition by 3.7 points on average.Source: Conclusion · p. 7
  • The findings support knowledge representation and evidence routing as material factors in grounded scientific question answering.Source: Conclusion · p. 7

Related papers

Loading 2610.06248v1…