Do Agents Need a New Kind of Paper? SPRA: Structured Paper Representation for Agents

1 Introduction

• Research questions, comparisons, and proposals require connecting a paper’s problems, contributions, evaluations, findings, methods, and conditions.Conventional articles distribute this structure across narrative text, tables, and figures.Source: §1 · p. 2
• Passage retrieval and task-specific synthesis can leave research roles and dependencies to reconstruct, while repeated full-paper inputs incur token costs and use context.The introduction motivates evaluating retained scientific utility jointly with token use.Source: §1 · p. 2
• SPRA represents paper-level research objects with typed relations, persistent identifiers, selected citation dependencies, and source bindings for querying and reuse.Its objects connect contributions to evaluations and reported findings; source locations support checking evidence.Source: §1 · p. 2
• A linked evaluation question selects the relevant Measure and Finding, while a method comparison selects corresponding records from another paper.Agents can follow the links to evaluation conditions and reopen source evidence to check a reported advantage.Source: §1 · p. 2
• The paper contributes a paper-level representation and extraction workflow, a million-paper resource with aligned inputs and frozen outputs, and paired evaluations across three research tasks.The evaluated tasks are scientific QA, cross-paper reasoning, and idea generation.Source: §1 · p. 2

2 Related Work

• Scholarly document tools extract metadata, text, and layout, while JATS, PROV-O, PROV-CONSTRAINTS, and RO-Crate address markup, provenance, validity, and research-object packaging.The cited works include GROBID, S2ORC, and MinerU (Lo et al., 2020; Wang et al., 2024).Source: §2 · p. 2
• SPRA assembles research roles and extracted content into persistent paper-owned objects with typed connections and executable operations.Prior work includes role labeling, entity and relation extraction, document-level relations, contribution triples, and scientific events [Jain et al. marker not present].Source: §2 · p. 2
• SPRA’s task-oriented paper objects complement ORKG aggregation, with property alignment connecting the two approaches.ORKG supports paper-linked contributions, templates, comparisons, APIs, and ASK retrieval (Jaradeh et al., 2019; Open Research Knowledge Graph, n.d.; Oelen et al., 2024).Source: §2 · p. 3
• SciCore evaluates extracted science cores alongside full-manuscript review, whereas SPRA targets broader research operations through paper-owned units, typed relations, and source bindings.SciCore assesses stability under content-preserving rewrites and discrimination across papers (Wang et al., 2026).Source: §2 · p. 3
• Literature retrieval, cited synthesis, graph-based retrieval, and iterative idea development are existing agent workflows that SPRA’s persistent paper-level representation can support and compose.The passage cites PaperQA/- PaperQA2, OpenScholar, PaSa, ResearchAgent, GraphRAG, KG2RAG, and Intern-Atlas (Lála et al., 2023; Skarlinski et al., 2024; Asai et al., 2026; He et al., 2025; Baek et al., 2025; Edge et al., 2024; Zhu et al., 2025; Wu et al., 2026).Source: §2 · p. 3

3.1 Representation and Profiles

• SPRA stores query-independent research objects in structured JSON under a versioned profile, with six roles, typed relations, citation dependencies, and source bindings.Problems motivate Contributions; Components and ExperimentSetups describe what is built and tested; Measures and Findings capture evaluation targets and observations, including conditions.Source: §3.1 · pp. 3–4

3.2 Corpus and Community Access

• The September 2026 corpus contains 1,059,112 structured paper records spanning 1965–2026, and SciVerse hosts the collection with web and programmatic access.The evaluation sample has 9,849 eligible source-aligned papers, with 1,407 papers in each of seven AI subfields.Source: §3.2 · p. 4

3.3 Public Interfaces and Data Access

• SciVerse exposes paper-level research objects through the Paper Schema API and a Paper Schema Agent Skill for paper discovery, object retrieval, and evidence assembly.The online corpus and frozen evaluation materials serve complementary purposes; later service responses may differ from the September 2026 snapshot.Source: §3.3 · p. 5

3.4 Construction Implementation

• Construction starts from section-aware Markdown with paragraph markers; five core LLM calls establish units, predict relations, and fill Problem, Method, and Evidence content.Two citation passes join citation roles and verbatim signals to bibliographic metadata; assembly resolves source markers, captures tables, reconciles identifiers, and checks types, provenance, and census coverage.Source: §3.4 · p. 5

3.5 Operations on Research Objects

• Typed lookup, relation traversal, and faceted filtering assemble goal-specific objects, relations, and evidence for tasks including surveys, benchmark analysis, method comparison, and reproduction.Related-paper discovery extends queries to literature sets, and source opening resolves recorded locators for verification.Source: §3.5 · p. 5

4.1 Representation Quality

• Structural checks measure source-pointer addressability and relation-endpoint connectivity in sampled object registries.These checks assess structure; RQ1 separately evaluates content through scientific QA and structured operations.Source: §4.1 · p. 5

4.2 Functional Adequacy of Extracted Content

• RQ1 assesses source-conditioned QA across research topics and tests lookup, internal-relation recovery, and qualification filtering.Its dimensions include problems and contributions, methods and components, experiments and results, and limitations and assumptions.Source: §4.2 · p. 5
• Figure 2 orders SPRA construction from a shared unit census through relation planning and role-specific filling to assembly and validation.Assembly joins source-linked records and stores external citations separately; validation checks profile conformance, relation endpoints, and source-link integrity.Source: Figure 2 · p. 6

4.3 Tasks and Research Questions

• The evaluation covers 9,849 eligible papers across seven AI subfields for RQ1, 3,070 paired cross-paper tasks for RQ2, and 1,001 idea-generation variants for RQ3.RQ2 scores directed-relation and Top-5 synthesis; RQ3 assesses evidence-grounded proposals across seven AI domains.Source: §4.3 · p. 6

4.4 Controlled Representation Comparison

• The three conditions share questions, paper identities and order, answer format, rubric, and Reader, while varying the input between complete text, retrieved passages, and SPRA objects.Task evidence is fixed before inference; RQ2 retrieval selects one primary item per paper for each of four shared facets.Source: §4.4 · p. 6

4.5 Reader and Evaluation Protocol

• Llama 4 Scout is the primary Reader; DeepSeek-v4-pro generates RQ1/RQ2 references, and MiniMax-M3 scores Reader outputs without condition labels.For RQ3, DeepSeek-v4-pro assesses implementation, while MiniMax-M2.5, Qwen3.5-27B, and GLM-5.1 independently review proposals.Source: §4.5 · p. 6, Figure 3 · p. 7
• Frozen outputs and scripts support offline replay of RQ1–RQ3 statistics.Appendices document structural checks and the token-accounting scope.Source: §4.5 · p. 6

4.6 Metrics and Statistical Analysis

• RQ1 and RQ2 quality scores use a 0–5 scale, and paired bootstrap resampling estimates 95% confidence intervals.The RQ3 Overall contrasts receive Holm correction; Table 4 reports RQ1 means for semantic answer quality, structured operations, and Problem + Contribution.Source: §4.6 · p. 7, Table 4 · p. 7

5.1 Structural and Semantic Assessment

• SPRA points to 95.0% of core units and connects both endpoints for 97.1% of recorded relations in the sampled records.The rates measure addressability and connectivity, rather than content quality.Source: §5.1 · p. 7
• GPT-4.1 assigns SPRA a mean contribution-level semantic-alignment score of 84.49/100 against public NCG annotations.Information-unit alignment reaches 4.436/5.Source: §5.1 · p. 7

5.2 Extracted Structure for Source-Conditioned QA

• 0.381/5 is SPRA’s G-U1 gain over passage retrieval, while its score is 0.078/5 below Full Compact.G-U1 combines correctness, coverage, and qualification against complete-article references.Source: §5.2 · pp. 7–8

5.3 Cross-Paper Reasoning

• Across 3,070 paired RQ2 tasks, SPRA uses 28.5% fewer input+output tokens than Text-RAG and 95.3% fewer than Full Compact.Figure 4b covers token use across paired RQ2 tasks, including Top-5 synthesis.Source: §5.3 · p. 8, Figure 4 · p. 8
• 0.184/5 is Schema-RAG’s qualification gain over Text-RAG on directed judgments, the largest of its three measured gains.Correctness improves by 0.026/5 and grounding by 0.045/5; Table 5 reports paired RQ2 scores and token use.Source: §5.3 · p. 8, Table 5 · p. 8
• Top-5 synthesis improves correctness over Text-RAG by 0.156/5, but grounding and qualification decrease by 0.145/5 and 0.325/5.Full Compact has the highest means on all three criteria.Source: §5.3 · pp. 8–9

5.4 Idea Generation and Citation Expansion

• Across 1,001 idea-generation variants, SPRA’s Overall score is 0.067 points below Full Compact, alongside 93.7% fewer input+output tokens.The comparison uses shared paper selections.Source: §5.4 · p. 9
• Adding up to two resolved citation neighbors improves cross-paper integration by 0.253/5 under a shared 10,000-token ceiling.Both conditions start from the same seed objects.Source: §5.4 · p. 9, Table 6 · p. 9

• Findings remain connected to contributions, evaluations, and conditions, while selected citation dependencies link the focal paper to prior work.These connections support queries and relation traversal.Source: §6 · p. 9
• Citation expansion demonstrates using represented dependencies to assemble additional evidence under a shared context ceiling.The evaluations connect representation quality to scientific use across QA, cross-paper reasoning, and idea generation.Source: §6 · p. 9

• SPRA’s empirical-AI profile connects problems, contributions, methods, evaluations, and findings through source bindings and selected citation dependencies.SciVerse provides website, Paper Schema API, and Agent Skill access to the representation at million-paper scale.Source: §7 · p. 9
• RQ1 gains 0.381/5 over passage retrieval while scoring 0.078/5 below Full Compact; RQ2 uses 28.5% fewer input+output tokens than passage retrieval.Mixed Top-5 results and lower RQ3 Overall score than Full Compact show that compact structure must be assessed against each workflow’s information needs.Source: §7 · p. 9

• Evaluations cover the empirical-AI profile and specified workflows; transfer to other domains, successive-task reuse, and rewrite stability remain future directions.Comparisons do not isolate individual role or predicate label contributions.Source: Limitations · p. 9
• Token savings count downstream model inputs and outputs; SciVerse storage, retrieval, and maintenance are outside that accounting.Hosted-service updates can change behavior in new runs, so results characterize recorded service invocations.Source: Limitations · p. 10

Ethical Considerations

• Users should check consequential claims against source articles because extraction can omit or misstate qualifications, and corpus coverage is limited by venues, years, and source availability.Missing or underrepresented work should not be treated as evidence that a topic or finding is absent from the literature.Source: Ethical Considerations · p. 10

A.1 Resource Scope and Sampling

• The corpus contains 1,059,112 representations with 19,949,289 core units and 24,073,381 internal relations; the evaluation cohort covers 9,849 eligible 2025 papers across seven fields.The cohort follows the primary Reader’s documented August 2024 knowledge cutoff.Source: §A.1 · p. 13

A.2 Sources, Acquisition, and Resource Access

• The aligned RQ1 archive supports offline replay, while SciVerse exposes typed records, relations, citations, and source context under account permissions and quotas.Project software uses Apache-2.0 and project-authored benchmark materials use CC BY 4.0; third-party articles retain their source terms.Source: §A.2 · pp. 13–14

A.3 Construction Procedure

• A paper-owned registry supports staged role-specific extraction, typed internal relations, and separate citation layers, followed by deterministic assembly and schema validation.Unresolved validation issues exclude a paper from task use, while metadata or citation errors may remain with warnings when core extraction is usable.Source: §A.3 · p. 14
• The optional numeric-presence check flags Measure values missing from captured table cells but does not verify their units, comparison bases, or conditions.Numeric presence alone does not verify whether those fields are correctly associated.Source: §A.3 · p. 14

A.4 Structural Validation

• 95.0% of sampled units have provenance, and 97.1% of complete-record relations join both endpoints.These checks characterize source-pointer availability and relation connectivity for agent access.Source: §A.4 · pp. 14–15

A.5 Contribution-Level Semantic Alignment

• Gold contribution sentences are matched to extracted strings by token F1, then judged for semantic correspondence across five dimensions and a separate Overall score.The rubric accepts paraphrase and canonicalization while penalizing unsupported additions, incorrect attribution, and omitted contribution facts.Source: §A.5 · p. 15
• 84.49 mean Overall and 85 median accompany 4.436/5 information-unit alignment versus 3.890/5 contribution-fact coverage.The pattern suggests that SPRA captures contribution organization more consistently than fine-grained factual detail.Source: Table 8 · p. 15, §A.5 · p. 15
• Run-specific judge inputs and individual judgments are unavailable, so reproducing the reported aggregate scores requires additional NCG records.The supplement retains the implementations, rubric, aggregate scores, domain summaries, and paper-level call metadata.Source: §A.5 · p. 15

A.6 Structural Comparison of Scholarly Representations

• SPRA’s six-role profile makes the research argument explicit and integrates constrained relations, source-opening operations, and a common object registry; type counts describe granularity, not quality.The profile connects Problems, Contributions, Components, ExperimentSetups, Measures, and Findings through typed relations.Source: §A.6 · p. 16

A.7 ORKG Graph Sample and Interoperability Scope

• The interoperability analysis maps ORKG’s AI sample—200 papers from a 2,918-paper frame—to SPRA while preserving scientific identities, reference-paper links, and relation directions.The sample was selected by deterministic ordering before graph-property inspection.Source: §A.7 · pp. 16–17

B.1 Protocol and Analysis Population

• The analysis included 9,849 RQ1 items across seven AI subfields, 3,070 complete paired RQ2 tasks, and 1,001 RQ3 variants.RQ1 excluded 411 items without complete three-condition results; G-U3 is a subset of G-U1.Source: §B.1 · p. 17

B.2 Controlled Representation Comparison

• Task evidence and representation-specific input packs were fixed before inference, with shared task slots and distinct evidence forms across Text-RAG and Schema-RAG.Full Compact supplies available parsed article text, including retained table contents, formulas, and captions, but not figure pixels or page images.Source: §B.1 · p. 17, §B.2 · p. 17
• −0.078 points was the retained G-U1 SPRA–Full Compact contrast on the 0–5 reporting scale.Its 95% CI was [−0.110, −0.046].Source: §B.2 · p. 18

B.3 RQ2 Token Accounting

• Using shared offline BERT WordPiece counts of input plus output, Schema-RAG had 28.5% fewer tokens than Text-RAG and 95.3% fewer than Full Compact, calculated from unrounded aggregate totals.The aggregate sums were 9,511,821, 13,296,573, and 203,262,042 tokens for Schema-RAG, Text-RAG, and Full Compact, respectively (reported as 9.51M, 13.30M, and 203.26M).Source: §B.3 · p. 18

B.4 RQ3 Idea-Generation Quality and Token Use

• RQ3 compared conditions on 1,001 idea-generation variants, using independently judged Overall scores and family-balanced paired tests.Its token accounting covers digestion, generation, and binding; shared selection, reference generation, auditing, and review are excluded.Source: §B.4 · pp. 18–19

B.5 Statistical Inference

• RQ1, RQ2, and RQ3 inference preserved paired differences using bootstrap procedures, with RQ3 also applying exact family sign flips and Holm correction.RQ3 averages task-level Overall reviews and equally weights benchmark-cluster means.Source: §B.5 · p. 19

B.6 Citation-Guided Research Integration

• +0.254/5 on integration for the 973-question remaining cohort, while the four-dimension composite was +0.100 and opened neighbors appeared in 932/1,001 expanded proposals.The integration contrast was positive in all 14 benchmark families; the composite was positive in 13.Source: §B.6 · p. 20

C Research Objects in Use

• The examples and interface connect research problems and contributions to evaluations, findings, and source context for inspection and reuse.SciVerse presents research units and relationships around a question, with links to source passages and tables.Source: C Research Objects in Use · p. 21, §C.2 · p. 21, §C.1 · p. 21

D.1 Evaluation Materials and Reanalysis

• The data archive includes aligned inputs for the complete 9,849-paper RQ1 cohort, candidate and exclusion records, and saved outputs for offline inspection and numerical reanalysis.Numerical reanalysis uses saved records without hosted models; fresh extraction or answer generation requires source articles and hosted services.Source: §D.1 · pp. 22–23

D.2 Executable Implementation and Tests

• The released extraction implementation includes prompts, JSON schemas, assembly validation, regression tests, and documented offline commands.Fresh model runs require source articles and hosted services.Source: §D.2 · p. 23

D.3 Model Configurations and Request Settings

• The supplementary settings record model aliases, role-level output ceilings, backend request options, and temperature-0 primary evaluation calls.Configured output limits are distinct from actual consumption.Source: §D.3 · p. 23

D.4 Task-Based Assessment of Extracted Content

• RQ1 assesses extracted-representation adequacy through source-conditioned QA and structured operations scored against shared Full Compact-generated references.G-U1 measures correctness, coverage, and qualification; G-U3 focuses on Problem + Contribution questions.Source: §D.4 · p. 23

D.5 Token-Accounting Scope

• Token totals measure complete inputs and outputs for retained task calls, not service bills, storage, or retrieval costs.SciVerse supplies storage, retrieval, and maintenance.Source: §D.5 · p. 23