PaperlayerView exact revision on arXiv

Source-linked AI summary

A Storage-Retrieval Gap in Parametric Knowledge Graph Memory

Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Volker Tresp

arXiv:2608.25489v1cs.LGcs.CLcs.IR

TL;DR

Graph RAG repeatedly places source subgraphs in the context, motivating a zero-context-token parametric alternative. The paper compiles entity subgraphs into LoRA adapters and finds that they store generalizable closed-book facts, while similarity-based routing cannot reliably select the correct local adapter. Thus, storing knowledge is feasible, but selecting and composing adapters remains unresolved, with multi-hop composition constrained by the benchmark’s entity coverage.

  • Problem

    The paper asks whether subgraph-trained adapters store facts recoverable without context, and whether the right adapters can be selected and composed from queries alone.

  • Method

    It compiles each entity subgraph offline into a per-entity LoRA adapter and answers queries by injecting selected adapters into a frozen base model without subgraph text.

  • Results

    +0.243 EM on single-valued relations over a near-blind base demonstrates generalizable closed-book knowledge, but embedding and ΔW geometry retrieval perform at chance.

  • Takeaways & Limitations

    Parametric knowledge graph memory is feasible for storing context-free knowledge, but selecting and composing the right local adapters requires a mechanism other than semantic similarity.

  • Takeaways & Limitations

    MetaQA two-hop answers require knowledge absent from both entity subgraphs being merged, limiting the paper’s composition evaluation.

Abstract

from arXiv · show

Graph retrieval-augmented generation places retrieved subgraphs into the model's context window at query time, paying a recurring token cost and exposing source data on every call. We study an alternative: compiling a knowledge graph offline into a bank of LoRA adapters, one per entity, that serve as a parametric knowledge layer queried by injecting weights rather than text, at zero query-time context cost. On the MetaQA dataset, we find that subgraph-trained adapters encode context-free factual knowledge that generalizes to unseen questions: on single-valued relations the adapter gains $+0.243$ exact-match score over a base model that is nearly blind closed-book ($0.007$), and only the correct adapter recovers this knowledge (an oracle gap of $+0.283$ over the base model). However, the stored knowledge is not recoverable by similarity: given a query with no subgraph, embedding-based and weight-space geometry retrieval both perform at chance, because a semantically neighbouring entity's adapter does not contain the answer - knowledge is stored locally and does not transfer. Weight geometry correlates with subgraph semantics ($ρ= +0.329$) but not with functional retrievability. We quantify the byte and context-token costs against graph retrieval-augmented generation and discuss deployment implications. Our results establish that parametric knowledge graph memory is feasible for storing knowledge, and identify selecting and composing the right adapters by a mechanism other than semantic similarity as the central open problem - motivating a learned, query-conditioned composition mechanism.

1. Introduction

The paper proposes compiling entity subgraphs into LoRA adapters as a zero-context-token parametric memory, then tests whether stored knowledge can be recovered and routed. Adapters store generalizable facts, but similarity-based retrieval fails because knowledge remains local to each entity.

  • Motivation: Graph RAG repeatedly serializes retrieved subgraphs into prompts, incurring query-time token and data-exposure costs.The paper targets costs that scale with usage rather than graph size.
  • Approach: Per-entity subgraphs are compiled offline into LoRA adapters and queried through weight injection without subgraph text in context.The resulting parametric memory stores knowledge as weights rather than a text corpus.
  • Findings: +0.243 EM on single-valued relations shows that subgraph adapters store generalizable closed-book knowledge over a near-blind base.Only the correct adapter recovers this knowledge, with a +0.283 oracle gap over base.
  • Findings: Embedding and weight-space geometry retrieval both perform at chance because semantically neighbouring adapters do not contain the query’s answer.The paper characterizes this as a storage–retrievability dissociation.
  • Open problem: Selecting and composing the right adapters remains unsolved, motivating mechanisms beyond similarity-based routing.The paper identifies learning which local adapters to combine for a query as the central open problem.

2. Method

The method extracts each anchor entity’s k-hop neighbourhood, verbalizes it, and fine-tunes a LoRA adapter while keeping the base model frozen. At query time, one or more adapters are retrieved, weighted and injected, and the answer is decoded without subgraph text.

  • Offline compilation: Each anchor entity’s k-hop neighbourhood subgraph is verbalized and used to fine-tune a per-entity LoRA adapter on a frozen base model.The adapters form a bank of entity-specific perturbations.
  • Online inference: At query time, the system retrieves adapters, forms θ = θ0 + ∑ λ_iΔθ_i, and decodes with no subgraph text in context.Single-entity questions use one adapter, whereas multi-entity questions require merging.
  • Verbalization: A fixed fact-dense template renders each subgraph as one declarative clause per triple before training.The template minimizes surface variation while staying close to the underlying facts.
  • Adapter geometry: Weight-space retrieval computes normalized Frobenius distances from truncated SVD factors, contrasting ΔW geometry with sentence embeddings and random selection.The perturbation need not be materialized to compute pairwise distances.
  • Deployment constraint: Because deployment supplies no subgraph, the retrieval signal must come from the question text alone.This constraint makes query-conditioned adapter selection necessary.

3. Experimental setup

Experiments use MetaQA with clean single-film entities, evaluating base and adapters closed-book and open-book across relation types. EM is primary for single-valued relations, while AM and set recall address multi-valued answers.

  • Training: Qwen3.5-2B adapters share a frozen 4-bit QLoRA base and are trained with standard LoRA settings to convergence.The setup uses rank 8, α = 16, 0.05 dropout, AdamW, and a 2 × 10^-4 learning rate.
  • Corpus: MetaQA is filtered to 150 clean single-film entities whose titles map to exactly one release year.Collided titles are excluded because contradictory labels can prevent adapters from reproducing their own training answers.
  • Evaluation: Adapters are evaluated closed-book without subgraphs and open-book with subgraphs, using identical metrics for the base model and adapters.Generations use greedy decoding with a 128-token cap.
  • Metrics: EM treats a prediction as correct when it equals any valid object, while AM and set recall capture partial success on multi-valued relations.Single-valued relations include director, writer, and release year; tags and cast are multi-valued.
  • Reporting: Table 1 reports CB EM gains over base averaged across 150 clean entities with 95% bootstrap confidence intervals.Significance uses a one-sample bootstrap test on paired differences with 50,000 resamples.

4. Experiments

The experiments show that entity-specific LoRA adapters store generalizable closed-book facts, but similarity-based retrieval cannot identify the adapter containing a query’s answer. Weight geometry reflects semantic distance weakly while failing to provide functional retrieval, and multi-hop composition remains unresolved.

  • Weight-level storage: +0.243 EM is recovered on single-valued held-out relations over a nearly blind closed-book base.The base answers only 1% of held-out questions, with 147 of 150 entities scoring zero.
  • Weight-level storage: +0.283 EM over base is achieved only by the correct entity adapter, confirming entity-specific storage.An unrelated trained adapter has gain 0.000, while a random-weight adapter falls below base by −0.017.
  • Weight-level storage: Multi-valued relations score 0.000 EM despite AM 0.854 for tags and set recall 0.793, making set metrics necessary for these relations.The zero EM reflects verbose valid answers failing exact single-token matching, rather than a learning failure.
  • Retrieval: Embedding and ΔW retrieval both produce 0.000 gain over base, exactly matching random selection in closed-book evaluation.The correct adapter still achieves +0.283, so query-driven similarity methods fail to recover the stored knowledge.
  • Retrieval: ΔW geometry correlates with semantic distance at ρ= +0.329, but its retrieval ordering does not separate the single hit from 29 misses.The observed trend is weak, and the sole retrieval hit is interleaved with misses.
  • Composition: Multi-hop composition could not be evaluated because MetaQA answers were absent from the individual or jointly provided subgraphs.The authors leave learning which adapters to combine and how to weight them for future work.

5. Discussion

The results support storage of entity-specific knowledge in adapters but reveal a dissociation between semantic organization and functional retrievability. Deployment benefits are bounded by storage, update, selection, and composition costs.

  • Storage and retrievability: +0.283 CB gain comes from the correct entity adapter, while unrelated adapters provide no recovery and random weights can underperform the base.This supports entity-specific storage and the absence of graceful transfer to other adapters.
  • Storage and retrievability: ρ= +0.329 ΔW correlation with subgraph semantics does not predict answer containment.The geometry organizes adapters semantically, but that organization does not make the needed facts functionally retrievable.
  • Interpretation: Similarity-based retrieval fails because neighbouring adapters may not contain the specific answer, while the experiments cannot distinguish answer non-overlap from storage locality.Separating these mechanisms would require shared-answer cases at a scale unavailable in MetaQA’s single domain.
  • Cost and deployment: Zero context-token cost is the central storage advantage over graph RAG, but adapter injection and swapping costs remain unmeasured for end-to-end latency and energy.The supported claim is context-token saving, not a net speedup or energy reduction.
  • Cost and deployment: 10.4 MiB of adapter weights versus 188 B of serialized triples makes parametric memory attractive mainly when entities are queried often or context budget is binding.A million-entity bank would require roughly 10 TiB of checkpoints, excluding one-time offline training.
  • Cost and deployment: Continuously changing graphs and one-shot queries favor in-context graph RAG because compiled adapters require offline encoding and re-encoding after affected facts change.The deployment case is strongest for stable, frequently queried subgraphs or undesirable raw-subgraph transmission.

6. Related work

The paper extends parametric retrieval work from documents to knowledge-graph subgraphs and directly tests whether their adapters retain recoverable knowledge without context. It contrasts this with graph RAG and frames adapter composition as an open problem.

  • Parametric RAG and LoRA encodings: Parametric RAG fine-tunes a LoRA adapter per document and averages retrieved adapters, while DyPRAG generates adapters through a hypernetwork and Poly-PRAG uses a shared adapter basis.The paper applies the parametric-memory idea to KG subgraphs rather than documents.
  • Evaluation distinction: Closed-book evaluation isolates weight-resident knowledge, and own-versus-other-adapter comparisons test whether recovered knowledge is entity-specific.The reported conclusion is that adapters encode recoverable knowledge, but query-only retrieval cannot find it.
  • KG QA and composition: Graph RAG retrieves and serializes subgraphs into prompts, whereas this approach moves retrieved knowledge into weights and uses an in-context oracle as an upper bound.The paper also probes whether per-entity perturbations can be merged without destroying knowledge or enabling cross-entity reasoning.

7. Conclusion

The paper frames LoRA adapters as a parametric memory that stores graph knowledge in weights without query-time subgraph text. The results support context-free storage, while retrieval and composition remain unresolved under the study’s stated scope.

  • Conclusion: LoRA adapters provide a parametric memory queried by weight injection without subgraph text in context.The system uses one adapter per film and stores knowledge as weights rather than corpus text.
  • Conclusion: +0.243 EM gain on single-valued relations shows that subgraph-trained adapters encode context-free knowledge beyond a near-blind base.The base model’s closed-book performance is described as near-blind, while the adapter generalizes to unseen questions.
  • Conclusion: +0.283 oracle gap over base shows that only the correct adapter recovers the stored knowledge.Embedding-based and ΔW geometry retrieval both perform at chance because knowledge remains local rather than transferring to neighbouring adapters.
  • Conclusion: The study’s scope is limited to one benchmark, one model, film entities, and single-valued relations.Multi-valued relations require a set-valued notion of parametric memory that the paper leaves for future work.
  • Conclusion: Within these bounds, storing knowledge in weights is feasible at zero query-time token cost, but selecting and composing adapters remains open.The paper identifies learning which local adapters to combine for multi-entity queries as the central unresolved problem.

Declaration on Generative AI

The authors used Claude Opus 4.8 during preparation for drafting, paraphrasing, style improvement, and content enhancement, then reviewed and edited the work.

  • Declaration on Generative AI: Claude Opus 4.8 was used to draft, paraphrase, reword, improve writing style, and enhance content.The authors state that they reviewed and edited the generated material afterward.

A.1. SVD representation

The appendix represents each LoRA update through a truncated SVD of its low-rank perturbation rather than forming the dense weight matrix. QR factorizations make this representation efficient, and normalized singular values encode normalized perturbations.

  • A.1. SVD representation: Each adapter modifies W0 with a low-rank perturbation ΔW = BA, and the analysis uses ΔW rather than raw factors.B and A are the low-rank factors defining the perturbation.
  • A.1. SVD representation: Only U, σ, and V⊤ from the truncated SVD are stored, avoiding formation of the full d_out × d_in matrix.This reduces analysis-time memory by roughly a d/r factor and yields a 10.4 MiB LoRA-factor checkpoint.
  • A.1. SVD representation: QR factorizations reduce the SVD computation to the small product R_B R_A⊤, with O(d r^2) rather than O(d^3) time.The method first factorizes B and A⊤ with QR, then applies SVD to the resulting small product.
  • A.1. SVD representation: Normalized perturbations are represented implicitly by σ̃ = σ/‖σ‖2 because the Frobenius norm of ΔW equals the 2-norm of σ.This gives the normalized representation Δ̃W = ΔW/‖ΔW‖F without materializing ΔW.

A.2. Frobenius distance

The appendix defines adapter distance by comparing normalized low-rank SVD representations across selected layer–module pairs. Inner products are computed from small factor matrices, then aggregated with layerwise normalization.

  • A.2. Frobenius distance: For adapter pairs, the normalized Frobenius distance is computed at each selected layer–module pair.The comparison operates on normalized perturbations rather than dense adapter matrices.
  • A.2. Frobenius distance: SVD factors compute the inner product directly, using r × r matrices and avoiding dense ΔW materialization.The order V1⊤V2 preserves the low-rank r × r computation instead of creating a d_in × d_in matrix.
  • A.2. Frobenius distance: O(r^2(d_out + d_in)) per adapter pair makes the factor-based comparison roughly 1000 times cheaper than dense computation.For r = 8, the saving is two to three orders of magnitude at current model layer widths.
  • A.2. Frobenius distance: The aggregate distance is the Euclidean norm over per-layer distances, with each layer normalized independently.Independent normalization prevents layers with larger weight magnitudes from dominating the aggregate.
  • A.2. Frobenius distance: The evaluation template uses an entity document as context, while the CB condition supplies an empty context field.The ground-truth answer is used only during training, and the prompt instructs concise factual answering from provided information.
Loading 2608.25489v1…