PaperlayerView exact revision on arXiv

Source-linked AI summary

Realize What Matters: Principled Context Representation for Large-Scale Reasoning

Michael Theologitis, Dean Light, Shuyue Stella Li, Benjamin Newman, Yulia Tsvetkov, Dan Suciu

arXiv:2609.27173v1cs.CL

TL;DR

Complex reasoning often depends on information scattered across large corpora, while principled guidance for constructing useful representations remains lacking. This paper derives design principles from relevance realization and implements them in R3CON. R3CON outperforms the strongest baseline on both evaluated benchmarks, and the authors identify context-window demands as a boundary for scaling to unbounded corpora.

  • Problem

    Large-corpus reasoning needs representations of scattered, interdependent information, but there is no principled account of how to construct them despite their influence on downstream reasoning.

    Source: §1 · pp. 1–2

  • Method

    R3CON operationalizes cognitively grounded principles by extracting task-specific relevant context, structuring it flexibly, and combining both for reasoning.

    Source: §1 · p. 2, §3.2 · pp. 3–4

  • Results

    R3CON outperforms the strongest baseline on both benchmarks, by +19.88 percentage points on CorpusQA.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Explicit context representation reduces reliance on model scale, with smaller R3CON models outperforming much larger evaluated systems.

    Source: §8 · p. 9

    Relevance snippets across documents must fit the model’s context window; scaling to unbounded corpora will require more selective attention allocation.

    Source: A LIMITATIONS · p. 16

Abstract

from arXiv · show

Solving complex tasks in domains such as science, medicine, law, and finance often requires assembling interdependent information scattered across vast, heterogeneous sources far beyond model context limits. Existing approaches tackle this challenge by organizing information into more manageable representations over which models can reason, such as graphs, textual memories, and retrieval collections. These representations dictate what downstream reasoning is possible and, ultimately, whether it succeeds; yet their design and construction remain largely ad hoc. In this work, drawing on the cognitive theory of relevance realization, we propose concrete principles for designing AI systems that construct effective representations of very large contexts. We analyze existing approaches and show how their successes and failures map onto their alignment with these principles, and introduce R3Con, a harness designed to operationalize the principles more systematically. We evaluate R3Con against nine state-of-the-art baselines on two recent benchmarks of reasoning over large document corpora. On these benchmarks, R3Con substantially outperforms the strongest baseline, by $20$ and $8.4$ percentage points. It also enables smaller models to outperform much larger ones: R3Con with 4B and 9B models outperforms all evaluated 35B baselines, while R3Con with a 35B-A3B model outperforms Claude Code with Claude-Sonnet-5 at $3.7\times$ lower cost. Our results show that context representations following our principled approach can reduce reliance on model scale, pointing toward a future of AI systems with frontier-level performance powered by smaller models. Our code is available at https://github.com/michaeltheologitis/r3con

1 INTRODUCTION

Complex tasks require reasoning across large corpora, but there is no principled account of how to construct representations that make solutions accessible. The paper proposes cognitively grounded design principles, instantiates them in R3CON, and evaluates it on large-corpus reasoning benchmarks.

1 INTRODUCTION

  • The paper identifies the lack of a principled account for constructing representations, despite their importance to which downstream reasoning is possible.Existing approaches make varied decisions about structure and construction, including fixed or dynamic structures and offline or task-evolving construction.Source: §1 · pp. 1–2
  • R3CON extracts task-relevant context, organizes it using structures chosen to fit that context and task, and combines both into a representation for reasoning.It uses recursive task-focused summarization and a coding agent to explore and reason over the representation.Source: §1 · p. 2
  • +19.88 percentage points on CorpusQA and +8.42 points on Loong over the strongest baseline; R3CON also outperforms larger-model baselines and Claude Code at lower cost.The evaluation covers two benchmarks and nine baselines; R3CON with Qwen3.5-4B on Loong and Qwen3.5-9B on CorpusQA beats evaluated Qwen3.5-35B-A3B baselines.Source: §1 · p. 2
  • The paper contributes cognitively grounded design principles, an analysis of existing approaches against them, and R3CON as an instantiation advancing information-integration performance.Source: §1 · p. 2

2 MOTIVATING EXAMPLE

The Dracula example illustrates how answering questions about events requires joining incomplete accounts distributed across a corpus. In the Demeter episode, records of the crew’s disappearances and the captain’s death help connect the deaths to Dracula.

2 MOTIVATING EXAMPLE

  • The novel’s letters, journals, and newspaper articles record partial knowledge, so solving the victims task requires piecing together scattered accounts.The task asks how many people Dracula killed, directly or indirectly, and who they were.Source: §2 · p. 2
  • The Demeter log records nine people setting sail and reports crew members disappearing one by one, but does not initially explain the deaths.The captain’s last entry says he planned to tie his hands to the wheel until his strength failed.Source: §2 · pp. 2–3
  • The newspaper’s unidentified corpse at the helm matches the captain’s final log entry, establishing that the captain was the last crew member to die.Source: §2 · pp. 2–3
  • All nine evaluated baselines fail to attribute the Demeter deaths, while R3CON identifies all 13 victims across the story and holds Dracula accountable.The crew never explicitly identifies Dracula, and the newspaper report of a corpse lashed to the helm must be connected with the captain’s final log entry.Source: §2 · p. 3

3 RELEVANCE REALIZATION

Relevance realization explains how task-specific focus and interaction with the environment help form useful problem representations. The paper translates this account into computational principles for large-context systems.

3.1 COGNITIVE THEORY

  • Relevance realization makes the search over possible representations tractable by focusing on information and structure relevant to the task.The theory describes relevance as task-specific and shaped through ongoing interaction with the environment, as understanding evolves.Source: §3.1 · p. 3

3.2 COMPUTATIONAL DESIGN PRINCIPLES OPERATIONALIZING THE THEORY

  • The computational principles extract relevant context in a task-specific, enactive way, then flexibly structure it according to that context and task.The relevant context and resulting structure together form a representation that the system can explore and reason over.Source: §3.2 · pp. 3–4

3.1 COGNITIVE THEORY

  • Enactive cognition emerges through reciprocal interaction with the environment, which changes understanding and what information receives attention.Source: §3.2 · p. 3, §3.1 · p. 3

4 R3CON

R3CON constructs a task-specific representation in stages: it extracts relevant context, uses that context to structure information, and then reasons over the result with a coding agent.

4 R3CON

  • R3CON treats the task as answering from information available in a corpus of documents and introduces a method instantiating relevance-realization principles.Source: §4 · p. 4

4.1 EXTRACTING RELEVANT CONTEXT

  • It extracts relevance iteratively, revisiting each document over N rounds with the task, its original text, and other documents’ previous-round snippets.Each document is summarized for its task contribution; its own previous snippet is excluded so it can be reinterpreted from scratch using current corpus-wide understanding.Source: §4.1 · p. 5

4.2 STRUCTURING

  • The model uses the task and extracted relevant context to propose a schema, then populates it by processing original documents independently.The schema specifies represented information, attributes, relationships, nesting, and data types; relevant context helps resolve ambiguities during parsing.Source: §4.2 · p. 5

4.3 ANSWER SYNTHESIS

  • A coding loop explores and manipulates the structured data in a Python REPL, while the relevant context is provided directly in the prompt.This lets the model reason programmatically without fitting the entire structure into its prompt.Source: §4.3 · p. 5

5 EXPERIMENTAL SETUP

The experiments test reasoning across large, interdependent document collections using two complementary benchmarks and a range of representation-based and agentic baselines. They use a shared default model setup, with additional smaller-model and implementation settings for comparison.

5.1 BENCHMARKS

  • CorpusQA tests integration across hundreds of interconnected documents, while Loong spans locating, comparing, clustering, and chaining evidence; both report accuracy using an LLM judge.The judge is gpt-5.4-mini, using the benchmarks’ original prompts.Source: §5.1 · p. 6

5.2 BASELINES

  • The nine baselines include graph, retrieval, and memory representation methods, dynamically structured representations, coding agents, recursive language models, and specialized retrieval tools.Representation baselines include HippoRAG2, LinearRAG, StructRAG, RAPTOR, ReadAgent, and MemAgent; agentic baselines include Claude Code, CodeAgent, and RLMs, alongside A-RAG.Source: §5.2 · p. 6

5.3 MODELS

  • Qwen3.5-35B-A3B is the default model for R3CON and baselines, with Qwen3.5-4B and 9B also tested for R3CON and the strongest baselines.Claude Code experiments use Claude-Sonnet-4.6 or Claude-Sonnet-5; MemAgent uses RL-MemoryAgent-14B.Source: §5.3 · p. 6

5.4 IMPLEMENTATION DETAILS

  • R3CON uses two relevance-extraction rounds by default, and reported USD costs include embedding costs priced at the lowest OpenRouter rates available when written.All baseline embeddings use text-embedding-3-small.Source: §5.4 · p. 6, §5.3 · p. 6

6 RESULTS

R3CON outperforms the strongest baseline on both benchmarks and across evaluated domains, with gains across Loong reasoning facets and lower-cost smaller-model runs. Ablations and the Dracula example connect its performance to relevance extraction, flexible structuring, and their support for preserving and linking task-relevant information.

6.1 OVERALL PERFORMANCE AND COST

  • R3CON leads the strongest baseline by +19.88 pp on CorpusQA and +8.42 pp on Loong overall, and has the highest accuracy across evaluated domains.Accuracy is micro-averaged; the table covers real-world domains requiring reasoning over large document corpora.Source: §6.1 · p. 6, Table 1 · p. 7
  • On Loong, R3CON with Qwen3.5-4B reaches 30.23% accuracy and beats 35B baselines at 6.58× lower cost; on CorpusQA, Qwen3.5-9B reaches 54.88% accuracy at 2.52× lower cost.Figure 6 places R3CON on the overall accuracy-cost Pareto frontier across model sizes, including 4B and 9B runs.Source: §6.1 · p. 7, Figure 6 · p. 7

6.2 FINE-GRAINED ANALYSIS OF REASONING FACETS

  • R3CON has the highest accuracy in every Loong reasoning category, with its largest gains over the strongest baseline in reasoning chains (+8.89 points) and clustering (+8.03 points).These gains exceed the +2.45-point gain on spotlight locating and +4.74-point gain on comparison; chain of reasoning and clustering are the most difficult facets across methods.Source: §6.2 · p. 7, Table 2 · p. 8

6.3 CAN’T CLAUDE CODE DO IT?

  • R3CON with Qwen3.5-35B-A3B reaches 71% accuracy, versus 67% for Claude Code with Claude-Sonnet-4.6 and 70% with Claude-Sonnet-5.R3CON costs $439, compared with $1,625 for Claude Code with Sonnet-5 and $1,714 with Sonnet-4.6; the reported cost ratios are 3.7× and 3.9× lower, respectively.Source: §6.3 · p. 8

6.4 ABLATION STUDIES

  • Removing either relevance extraction or structuring degrades performance, with removing relevance extraction having the largest effect and yielding performance comparable to StructRAG.The ablation concerns the two representation-construction stages.Source: §6.4 · p. 8, Table 3 · p. 8

6.5 UNDERSTANDING WHY R3CON SUCCEEDS WHILE OTHERS FAIL

  • Only R3CON satisfies all three design principles; the authors attribute its correct answer on the Dracula task to the interplay of its two stages.The taxonomy compares baselines by which representation-construction principles they satisfy; N/A means not applicable.Source: §6.5 · p. 8, Table 4 · p. 8
  • RAPTOR’s task-agnostic summaries discard crew identities and the initial crew count, making the solution inaccessible; StructRAG also misses the captain’s death because it cannot reinterpret earlier documents with later information.StructRAG discards the captain’s plan to tie his hands to the wheel before a newspaper reports a dead helmsman at the wheel.Source: §6.5 · p. 8
  • MemAgent’s memory loses the ship-log deaths before later context permits attribution to Dracula, while coding agents miss required multi-hop links, including the captain’s two-hop connection.The best-performing coding agent, RLMs, fails to make that two-hop connection.Source: §6.5 · p. 9

7 RELATED WORK

Prior work either constructs explicit context representations or leaves models free to develop agentic reasoning strategies. The paper frames its principles as guidance across these approaches, with R3CON constructing a representation before using a coding agent.

Explicit Representation-Construction

  • Explicit representation methods build or update textual memories, offline summaries, graphs, or dynamically selected structures, but their choices are described as largely ad hoc.Examples include textual memories and summaries, GraphRAG approaches, and StructRAG, Structure-R1, and LogicRAG.Source: §7 · p. 9

Agentic Reasoning

  • The paper presents its principles as general guidance for representation decisions and places representation construction before agentic reasoning.This contrasts with relying entirely on the model to discover an effective strategy inside the coding loop.Source: §7 · p. 9
  • Coding and tool-calling agents can structure reasoning at test time, but must discover effective strategies themselves, and evidence suggests difficulty in knowledge-intensive tasks.R3CON instead uses a coding agent after constructing a representation; the authors also suggest the principles could inform training trajectories for test-time context organization.Source: §7 · p. 9

8 CONCLUSIONS

The conclusion presents cognitively grounded principles and R3CON as an approach to organizing large heterogeneous contexts for accurate downstream reasoning. It highlights explicit, controllable representation construction and reduced reliance on model scale as potential benefits.

8 CONCLUSIONS

  • The paper proposes relevance-realization-grounded principles for building rich, task-specific, structured context representations and introduces R3CON to operationalize them.Across nine baselines on long-context benchmarks, R3CON outperforms prior state-of-the-art methods by up to 20 percentage points.Source: §8 · p. 9
  • Explicit representation construction keeps relevance and organization decisions inspectable, adaptable, and controllable by data owners rather than wholly internal to the model.The authors say R3CON enables models an order of magnitude smaller to outperform much larger frontier systems, and suggest smaller local models could retain control over sensitive knowledge.Source: §8 · p. 9

AI USE STATEMENT

  • The authors report using generative AI to polish the manuscript and state that they reviewed AI-assisted work for correctness and consistency.They take responsibility for the final content, including AI-assisted text, claims, or artifacts.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper provides method prompts, baseline implementation details, benchmark details, additional evaluation metrics, and evaluation code for reproducibility.The evaluation code is available at the paper’s stated repository.Source: REPRODUCIBILITY STATEMENT · p. 10

A LIMITATIONS

The limitations concern the cost of rebuilding task-specific representations and the context-window assumption in iterative relevance extraction. The Dracula example illustrates why answering over a large corpus requires connecting incomplete accounts across documents.

A LIMITATIONS

  • Task-specific representations must be recomputed for each new query, making R3CON more expensive than reusable task-agnostic representations that risk being subpar.The authors suggest storing and reusing relevance snippets and structures for similar questions as future work.Source: A LIMITATIONS · p. 16
  • Each relevance-extraction iteration feeds all documents’ snippets back to the model, assuming its context window can accommodate them.The combined snippets are roughly 1% of the original input in tested examples, but unbounded corpora such as the open web may require more selective attention.Source: A LIMITATIONS · p. 16

B.1 BRAM STOKER’S DRACULA (1897)

  • The novel comprises 46 reconstructed documents, including recurring journals and diaries assembled from entries and standalone artifacts treated separately.Its story requires connecting incomplete accounts written by different people at different times and for different purposes.Source: §B.1 · pp. 16–17
  • The answer to the Dracula query is 13 victims, but no individual document states it explicitly and each death requires non-trivial cross-document reasoning.The victims listed are the nine Demeter crew members, Mr. Swales, Mrs. Westenra, Lucy Westenra, and Renfield.Source: §B.1 · p. 17

B.2 EVIDENCE ATTRIBUTION

The evidence attribution traces deaths through clues distributed across the novel’s documents, including some deaths that must be excluded from the answer. The section also specifies its scoring criteria and tests whether a model can answer the Dracula question from parametric knowledge or the full novel in context.

B.2 EVIDENCE ATTRIBUTION

  • The Demeter’s logs trace nine crew deaths, while a newspaper account identifies the corpse lashed to the helm as the captain.Other accounts identify the mysterious tall, thin figure aboard the ship as Dracula.Source: §B.2 · pp. 17–18
  • Mina’s observations place Mr. Swales’s death at the same churchyard seat where she later sees a mysterious figure over Lucy.The account says it can be safely assumed the dog from the ship was Dracula in a guise and caused Swales’s death.Source: §B.2 · p. 18
  • A newspaper account links Dracula to the escaped wolf that frightens Mrs. Westenra, while diary entries trace Lucy’s persistent wounds, blood loss, and death.The connection to Mrs. Westenra relies on the reported appearance of the man, the wolf’s escape, and its return injured after the attack.Source: §B.2 · p. 18
  • Renfield recounts Dracula’s assault after he tries to stop him attacking Mina; his broken back and skull fracture are followed by confirmation of his death.The attribution also excludes deaths outside the query’s time window, deaths without evidence linking them to Dracula, and apparent victims who survive.Source: §B.2 · pp. 18–19

B.3 SCORING

  • Answers count as correct only when they give both the correct victim number and individual names; excluding the more inferential Mr. Swales does not change any system’s scores.The table concerns answers using parametric knowledge alone and the full novel in context.Source: §B.3 · p. 19, Table 6 · p. 19

B.4 PARAMETRIC KNOWLEDGE

  • Qwen3.5-35B-A3B fails to answer correctly both from parametric knowledge alone and with the entire novel in context.The ground-truth answer is 13 and also requires naming the correct people.Source: §B.4 · p. 19, Table 6 · p. 19

C EXPERIMENTAL DETAILS

This section describes the benchmarks and baselines used to evaluate R3CON, then examines smaller models, representation size, downstream reasoning, and relevance-extraction rounds. R3CON with Qwen3.5-4B outperforms the evaluated 35B-A3B baselines on both benchmarks.

C.1.1 CORPUSQA

  • CorpusQA tests aggregation across a dispersed document collection, while Loong covers three domains, four reasoning categories, and context bundles of 50k–250k tokens.The evaluation uses CorpusQA’s one-million-token split; larger splits were excluded because of computational budget.Source: §C.1.1 · p. 19, Parametric knowledge · p. 20, §C.1.2 · p. 21

C.2.1 RAPTOR

  • RAPTOR recursively clusters and summarizes passages into a fixed hierarchy without task knowledge; ReadAgent independently summarizes pages into a fixed gist list, also without task knowledge.RAPTOR’s summaries evolve bottom-up, whereas ReadAgent’s page summaries are made independently.Source: §C.2.1 · p. 21, §C.2.2 · pp. 21–22

C.2.3 MEMAGENT

  • MemAgent uses task-focused sequential memory but cannot revisit earlier chunks; HippoRAG2 and LinearRAG integrate corpus information into fixed graphs without broader reinterpretation.MemAgent’s memory is bounded; HippoRAG2 uses Personalized PageRank, while LinearRAG expands query entities into reasoning chains.Source: §C.2.3 · p. 22, §C.2.4 · p. 22, §C.2.5 · p. 23

C.2.6 STRUCTRAG

  • StructRAG dynamically selects a task-specific structure, then transforms documents independently, without further interaction across the corpus during construction.Its candidate structures include tables, graphs, and catalogs.Source: §C.2.6 · p. 23

C.2.7 CODEAGENT

  • CodeAgent, RLMs, and Claude Code reason through coding tools without constructing an intermediate corpus representation; A-RAG instead uses a fixed retrieval index.RLMs can recursively call themselves, while A-RAG offers keyword, semantic, and full-chunk search.Source: §C.2.7 · p. 23, §C.2.8 · p. 24, §C.2.10 · p. 24

D.1 EVALUATING BASELINES WITH SMALLER MODELS

  • 49.24% on CorpusQA and 30.23% on Loong with Qwen3.5-4B outperform StructRAG and CodeAgent using Qwen3.5-35B-A3B.Table 8 reports results for R3CON and the two strongest baselines across model sizes.Source: Table 8 · p. 25, §D.1 · p. 25

D.2.1 SIZE OF RELEVANCE AND STRUCTURING

  • Final relevant context and structured data each average around 1% of the original context on both benchmarks.The relevance snippets evolve across rounds, and the illustrated schemas include hierarchical organization for CorpusQA.Source: §D.2.1 · p. 25, §D.2.3 · p. 26, §D.2.2 · p. 26

D.2.4 DOWNSTREAM REASONING

  • Neither downstream reasoning paradigm dominates: coding improves CorpusQA accuracy from 65.43% to 70.64%, while direct reasoning improves Loong from 37.43% to 40.17%; N = 3 costs more without consistent accuracy gains.R3CON uses N = 2 by default; that setting costs $439 on CorpusQA and $281 on Loong.Source: §D.2.4 · p. 26, Table 9 · p. 26, §D.2.5 · p. 27

D.3 QUALITATIVE RESULTS

The qualitative examples trace how representation construction can preserve or lose evidence needed for cross-document reasoning. The results table reports R3CON’s accuracy advantage across domains.

D.3 QUALITATIVE RESULTS

  • RAPTOR’s task-agnostic summaries omit minor Demeter details, including crew deaths, which disappear further up its summary tree and cannot be recovered downstream.The example shows the deaths split across clusters and summarized separately.Source: Figure 14 · p. 29
  • StructRAG’s per-document table omits the captain’s plan to tie himself to the wheel, blocking its connection to a separate newspaper report of a dead seaman at the helm.The account requires inferring that the unidentified dead seaman is the captain.Source: Figure 15 · p. 30
  • R3CON achieves the highest accuracy across domains, exceeding the strongest baseline by +19.88 pp on CorpusQA and +8.42 pp on Loong overall.The table reports micro-averaged overall accuracy and standard errors.Source: Table 10 · p. 31

D.4 STANDARD ERRORS FOR MAIN RESULTS

The reported benchmark improvements are substantially larger than their standard errors. The appendix also provides prompts for extracting relevance, proposing and populating schemas, and downstream reasoning.

D.4 STANDARD ERRORS FOR MAIN RESULTS

  • 19.88 percentage points on CorpusQA and 8.42 points on Loong exceed the reported standard errors by a substantial margin.R3CON scores 70.64 ± 2.52 versus 50.76 ± 2.76 on CorpusQA, and 37.43 ± 1.35 versus 29.01 ± 1.28 on Loong.Source: §D.4 · p. 31

E PROMPTS

  • Figures 18–21 provide prompts for task-focused relevance extraction, schema proposal, per-document schema population, and the coding agent’s final reasoning step.The prompts use double-curly-brace text for template variables populated at runtime.Source: E PROMPTS · p. 31, Figure 18 · p. 34, Figure 19 · p. 35
Loading 2609.27173v1…