PaperlayerView exact revision on arXiv

Source-linked AI summary

Better Reasoning, Not Just Bigger Models: A Structure-Grounded Framework for Efficient and Reliable Inference

Amirhossein Aghaei, Liangyan Li, Shiyi Yang, Shaohua Wang, Yan Wang

arXiv:2610.30611v1cs.AIcs.IR

TL;DR

Reliable multi-turn inference remains challenging when agents rely on fluent generation without structurally grounded hypothesis refinement, especially under incomplete observations. SGUR couples prior-weighted partial graph alignment with information-gain-driven evidence acquisition. In sequential diagnosis, it improves Hit@3 over the strongest GraphRAG baseline while reducing token overhead, and smaller backbones can match or exceed substantially larger models.

  • Problem

    Many agentic systems rely on token-level generation and linguistic similarity, leaving multi-turn inference weakly grounded when relational structure does not jointly constrain semantic evidence.

    Source: §1 · p. 2

  • Method

    SGUR is a training-free framework that aligns incomplete observations with structured hypotheses and uses Expected Information Gain to select the next query node.

    Source: §1 · p. 3

  • Results

    SGUR improves Hit@3 by 9.4 percentage points over the strongest GraphRAG baseline while reducing token overhead by approximately 5×, and enables smaller backbones to match or exceed substantially larger LLMs.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results suggest reliable agent performance depends not only on backbone scale but also on how the reasoning process is organized.

    Source: §5 · p. 10

    SGUR is a research framework, not a deployed clinical decision system, and its use for real-world medical decisions requires expert validation, clinician oversight, and regulatory review.

    Source: §5 · p. 10

Abstract

from arXiv · show

Scaling large language models has substantially improved linguistic fluency, yet reliable autonomous agents require more than fluent generation. In multi-turn inference, current agentic systems often produce semantically plausible but structurally ungrounded reasoning, especially under incomplete observations. We propose Structure-Grounded Uncertainty Reduction (SGUR), a training-free framework that complements token-level generation with active uncertainty reduction over structured hypothesis spaces. SGUR represents incomplete observations as a query graph and aligns them with candidate hypothesis graphs through a prior-weighted partial graph matching mechanism, where database-derived empirical priors assign greater importance to decision-relevant relations rather than treating all graph edges uniformly. The resulting inference state is iteratively refined by an active information-gain strategy that selects the next most informative node to query, systematically pruning the candidate space. We instantiate SGUR on a challenging PrimeKG-derived sequential medical diagnosis benchmark, demonstrating that structural grounding can partially substitute for model scale: starting from only two observed symptoms, SGUR achieves 92.5% Hit@3, outperforming the strongest Graph-RAG baseline by 9.4 percentage points while using approximately 5x fewer tokens. Notably, SGUR enables smaller backbone models to match or exceed the performance of models up to 10x larger, suggesting that structural grounding provides a more robust and efficient paradigm for scaling agent performance than relying exclusively on increasing model size.

1 Introduction

The introduction argues that reliable multi-turn inference needs structured hypothesis refinement and goal-directed evidence acquisition, not fluent generation or graph retrieval alone. SGUR couples prior-weighted partial alignment with information-gain-driven querying; on sequential diagnosis, it improves Hit@3 over the strongest GraphRAG baseline while reducing token overhead.

1 Introduction

  • Existing systems can leave multi-turn inference structurally underconstrained because graph evidence is often retrieved and flattened into text rather than used as an active hypothesis space.Sequential diagnosis makes this consequential because intermediate inferences shape later evidence acquisition and hypothesis revision.Source: §1 · pp. 2–3
  • SGUR reformulates multi-step reasoning as uncertainty reduction over structured hypothesis spaces, coupling geometric structural alignment with probabilistic belief updating.It grounds inference in semantic evidence, relational structure, and domain-specific priors under incomplete observations.Source: §1 · p. 3
  • Prior-weighted partial alignment supports matching under sparse evidence without full graph correspondence, while Expected Information Gain selects the next most informative query node.Both hypothesis refinement and evidence acquisition use shared semantic-structural constraints to prune candidates.Source: §1 · p. 3
  • SGUR is evaluated for sequential clinical diagnosis over a large-scale hierarchical biomedical knowledge graph, with incomplete evidence revealed iteratively.The setting requires structured hypothesis reasoning and consistency across long-horizon inference.Source: §1 · p. 3
  • SGUR improves Hit@3 by 9.4 percentage points over the strongest GraphRAG baseline while reducing token overhead by approximately 5×.It also enables smaller backbone models to match or exceed substantially larger LLMs.Source: §1 · p. 3

2 Related Work

Related work improves reasoning decomposition, evidence retrieval, and graph-based knowledge organization, but often leaves graph structure outside the inference state. SGUR connects partial structural alignment and active evidence acquisition within a shared belief state.

Reasoning and Retrieval under Weak Structural Control

  • Structured prompting methods improve reasoning decomposition, while RAG and hybrid systems add external evidence or interleave retrieval with reasoning and verification.GraphRAG extends retrieval with explicit graph structures, and hierarchical variants organize knowledge into communities and indexes.Source: §2 · p. 4
  • GraphRAG and hierarchical GraphRAG organize evidence for retrieval, but retrieved graph content is typically flattened into text rather than used to constrain the evolving inference state.The gap is making graph structure part of inference itself, not only improving evidence access.Source: §2 · p. 4

Geometric Alignment and Active Uncertainty Reduction

  • Optimal Transport methods such as Gromov–Wasserstein and Fused Gromov–Wasserstein compare node semantics and relational structure, while partial transport focuses on overlapping substructures under incomplete observations.Scalable approximations improve practicality for larger graphs [Scetbon et al., 2022, Nguyen and Tsuda, 2023].Source: §2 · p. 4
  • Prior work explores sequential uncertainty reduction and medical question selection, but SGUR couples semantic-structural partial alignment with EIG-driven evidence acquisition in one shared belief state.This makes hypothesis refinement and question selection subject to the same structure-grounded objective.Source: §2 · p. 4

3 Methodology: The SGUR Framework

SGUR combines a fixed expert graph database with an online loop that routes observations, prunes candidates, aligns query and candidate graphs, and acquires further evidence. The loop progressively contracts the candidate hypothesis space under partial observation.

3 Methodology: The SGUR Framework

  • SGUR’s online loop routes observations, prunes candidates, aligns query and candidate graphs, ranks beliefs, and acquires further evidence until termination or the turn budget is exhausted.The selected observation is added to the accumulated evidence before the loop repeats.Source: §3 · p. 4, §3.2 · p. 6

3.1 Offline Expert Graph Construction

  • The fixed expert database organizes diagnosis-centered graphs in a shared schema, with observations, localizations, diagnoses, and treatments as semantic layers.An auxiliary L0 layer routes observation embeddings; it is an index, not a diagnostic semantic layer.Source: §3.1 · p. 5

3.2 Online Interactive Inference

  • The online phase routes observations through L0, constructs a query graph, and uses path filtering and specificity-aware priors to retain structurally compatible candidates.Prior-weighted partial alignment permits unmatched query evidence and expert structure to remain uncoupled when observations cover only a sparse substructure.Source: §3.2 · pp. 5–6
  • SGUR converts alignment discrepancies into a normalized candidate belief ranking, then selects unobserved L1/L2 nodes for acquisition to reduce uncertainty.The selected query is presented to the user, and the returned observation is incorporated into the next interaction round.Source: §3.2 · p. 6

4 Experiments

The experiments compare SGUR with graph-RAG baselines on a three-round PrimeKG diagnostic retrieval benchmark and test performance under sparse, progressively revealed evidence. SGUR leads the complex-regime comparison in accuracy and efficiency, while ablations show the value of its components, especially under sparse evidence.

4 Experiments

  • The benchmark contains 1,469 three-round cases, with 854 simple-regime and 615 complex-regime cases; the complex regime emphasizes sparse, ambiguous evidence.The study compares four graph-RAG paradigms adapted to three rounds by reinjecting textual summaries of prior predictions and evidence.Source: §4 · p. 7, §4.1 · p. 7

4.1 Comparison with Graph-RAG Baselines

  • Over 99% Hit@3 retention: smaller-backbone SGUR remains above every graph-RAG baseline using a larger backbone.Retention is the small/large backbone ratio in the complex-regime stress test.Source: Table 2 · p. 8, §4.1 · p. 8
  • 92.50% Hit@3: SGUR beats the strongest graph-RAG baseline by 9.40 percentage points in the complex regime, with the lowest MTTC.It reduces token consumption by 5.70× and latency by 3.90× compared with GraphRAG.Source: §4.1 · p. 8
  • SGUR occupies the upper-left accuracy–efficiency frontier, with the highest final Hit@3 and lowest average token cost over three rounds.The plot’s y-axis is final Hit@3 after round 3, and its x-axis is average total tokens over three rounds.Source: §4.1 · p. 8, Figure 2 · p. 9

4.2 Ablation Study

  • Removing latent routing lowers final Hit@3 from 92.50% to 68.90%, while replacing partial transport with full FGW lowers it to 73.30%.With two observed symptoms, the corresponding variants reach 35.70% and 15.30% Hit@3, respectively; uniform importance yields 83.40% final Hit@3.Source: §4.2 · pp. 8–9, Table 3 · p. 9
  • Semantic-only and structure-only variants remain competitive but underperform full SGUR, which stays strongest as observations increase.The right block of Table 3 reports Hit@3 as observed evidence increases.Source: §4.2 · p. 9, Table 3 · p. 9

5 Discussion and Conclusion

The discussion argues that multi-turn inference depends on how reasoning is organized as well as on backbone scale. SGUR uses structured query states and prior-weighted partial matching, but remains a research framework rather than a clinical decision system.

5 Discussion and Conclusion

  • SGUR turns fragmented observations into a structured query state, matches it to candidate hypothesis graphs through prior-weighted partial matching, and selects evidence nodes to reduce uncertainty.The paper argues that this structural organization can partially substitute for model scale.Source: §5 · pp. 9–10
  • SGUR is a research framework, not a deployed clinical decision system; clinical use requires expert validation, clinician oversight, and regulatory review.The evaluation uses sequential clinical diagnosis as a demanding testbed.Source: §5 · p. 10
  • The framework assumes a meaningful domain hierarchy, standardized evidence-node mappings, and a sufficiently reliable expert graph.Real-world settings may include noisy observations, missing relations, evolving knowledge, and heterogeneous evidence sources.Source: §5 · p. 10

A Additional Experimental Details and Results

The appendix section outlines materials for reproducing the main experiments.

A Additional Experimental Details and Results

  • The section covers benchmark construction, test sampling, hyperparameters, prompts, compute environment, code and data release, and comprehensive results.These are presented as additional implementation details for reproducing the main experimental results.Source: A Additional Experimental Details and… · p. 16

A.1 Source Data, Schema Construction, and Test Case Sampling

The benchmark uses a four-layer PrimeKG schema and completes a missing phenotype–anatomy relation with prompted LLM assistance. Cases are sampled by disease and reveal symptoms over three rounds.

A.1 Source Data, Schema Construction, and Test Case Sampling

  • The benchmark retains disease, phenotype, anatomy, and drug nodes from PrimeKG and removes all other node types and their incident edges.These correspond to the paper’s L3, L1, L2, and L4 hierarchy, respectively.Source: §A.1 · p. 16
  • Because PrimeKG lacks direct phenotype–anatomy links, a prompted LLM supplies phenotype-to-body-part pairs restricted to PrimeKG’s anatomy vocabulary.The paper rejects the indirect gene-to-organ path because it does not identify where a phenotype is clinically observed.Source: §A.1 · p. 16
  • Diseases are sampled uniformly for disjoint simple and complex test sets, and associated symptoms are revealed over three rounds.When a disease has fewer than 3b associated symptoms, symptoms are resampled from its set to complete the observation budget.Source: §A.1 · p. 16

A.2 Hyperparameters

This appendix gives the shared SGUR hyperparameters and additional implementation settings needed for reproduction. It also specifies entity matching and the two baseline-comparison exceptions.

A.2 Hyperparameters

  • The PFGW solver uses Frank–Wolfe with squared-loss coupling and a 2,000-iteration cap; structural costs use specificity-weighted directed shortest paths, jointly rescaled to [0, 1].Partial transport mass is the inherited overlap between query and candidate marginals, leaving unmatched evidence and expert structure uncoupled; α and the feature-mismatch penalty follow Table 4.Source: §A.2 · p. 16
  • The L0 routing layer uses UMAP followed by HDBSCAN, then forwards the 10 most cosine-similar clusters as the query-time observation pool.The pool feeds the schema-constrained extractor and path-compatibility filter.Source: §A.2 · p. 17
  • Entity matching retains the top two L1 observation matches per extracted symptom and the top one L2 localization match per observation.Matches are selected by cosine similarity at threshold τsim.Source: §A.2 · p. 17
  • Baseline comparisons use official implementations and defaults, except that all methods share text-embedding-3-large and GPT-5.2 for corpus indexing.The shared embedding model and indexing LLM are the two stated fairness exceptions.Source: §A.2 · p. 17

A.3 Prompts

The appendix documents prompts for the LLM-invoked components. SGUR extracts observations and localizations using a routed shortlist, while adapted single-shot baselines return ranked diseases with supporting evidence.

A.3 Prompts

  • Prompts for each LLM-invoked component are documented and released in full with the code.Source: §A.3 · p. 17
  • The SGUR extractor receives a shortlist from the L0 pre-filter and returns observations and localizations as JSON, limiting extraction to routed candidates.The extractor is instructed to choose only from the shortlist rather than guess beyond it.Source: §A.3 · p. 17
  • The entity-recognition prompt directs the model to match lay or indirect symptom descriptions semantically to the supplied symptom list and return no more than three symptoms.It also requests one directly affected organ per symptom, from the supplied organ vocabulary.Source: §A.3 · p. 17
  • The single-shot baseline adaptation requests the top 10 diseases with supporting observations and localizations, using retrieved context only.The prompt forbids introducing unsupported diseases, symptoms, or localizations and asks for the best candidates even when evidence is weak or ambiguous.Source: §A.3 · pp. 17–18

A.4 Compute Environment

The experiments ran on one specified laptop-class machine, with LLM inference accessed through an API gateway and graph operations implemented in Python.

A.4 Compute Environment

  • Experiments ran on an 11th Gen Intel Core i7-11370H machine with 16 GB RAM; LLM inference used the TensorBlock API gateway for GPT-5.2 and GPT-5.4-nano.Graph operations and PFGW solving used Python, networkx, and POT; reported end-to-end latency includes API round-trip time under shared network conditions.Source: §A.4 · p. 18

A.6 Full Accuracy, Convergence, and Efficiency Results

SGUR leads the reported multi-round diagnostic comparisons, with its advantage strengthening as evidence accumulates. Its structured query-state updates also keep token use nearly flat and reduce latency across rounds.

A.6 Full Accuracy, Convergence, and Efficiency Results

  • SGUR achieves the best final ranking quality and lowest MTTC in the complex two-symptom regime, with its advantage growing as evidence accumulates.Table 5 reports round-wise Hit@3 for simple and complex regimes; SGUR leads every interaction round in complex cases, while in simple cases it trails HippoRAG2 at R1 but has the best R2 and R3 Hit@3.Source: §A.6 · p. 18, Table 5 · p. 19
  • SGUR keeps total token usage nearly flat across R1–R3 and has the lowest latency in every round.Its output-token use is substantially lower than all baselines; graph-RAG baselines accumulate larger input contexts as prior dialogue is re-injected.Source: Table 6 · p. 19, §A.6 · p. 19
  • The reported trends link grounded alignment and active acquisition with faster candidate-space contraction as newly revealed evidence arrives.Source: §A.6 · p. 19
  • Table 7 describes a complex-regime Large-versus-Small backbone stress test, with retention reported for Hit@k, MRR, and MTTC after round 3.Retention is Small/Large for Hit@k and MRR, and min(L, S)/max(L, S) for MTTC.Source: Table 7 · p. 20

Related papers

Loading 2610.30611v1…