PaperlayerView exact revision on arXiv

Source-linked AI summary

FedV-KGQA in Practice: Design Lessons and an Interactive Prototype

Md Saikat Islam Khan Bappy, Oshani Seneviratne

arXiv:2609.13661v1cs.AIcs.CLcs.IR

TL;DR

When organizations hold disjoint relations, no single party may see the full chain needed for multi-hop KGQA. FedV-KGQA enriches and embeds local graphs, then fuses entity views to rank answers. Across benchmarks, federation approaches centralized accuracy, and the experiments show that anchoring, enrichment, and target-dependent encoder cost matter.

  • Problem

    Most KGQA assumes one system can query the whole graph, but vertically partitioned organizations hold disjoint relations and cannot merge triples.

    Source: §1 · p. 1

  • Method

    FedV-KGQA enriches each silo's graph and trains a local embedding model, then concatenates entity views and ranks candidates using a question anchored at its supplied topic entity.

    Source: §2 · p. 2

  • Results

    Federated fusion reached 0.76, 0.65, and 0.54 MRR, trailing the centralized upper bound by 0.04, 0.03, and 0.03 on MetaQA, PathQuestion, and WebQSP, respectively.

    Source: §4 · p. 3

  • Takeaways & Limitations

    The design lessons are to preserve each silo's geometric view, account for silo count and relation composition, anchor questions, and compare communication cost at a target accuracy.

    Source: §6 · p. 5

    The evaluation covers path-shaped questions with supplied topic entities and answers reached by bounded expansion, not aggregation, comparison, counting, or time constraints.

    Source: §2 · p. 2

Abstract

from arXiv · show

Knowledge graph question answering usually assumes that one system can reach the whole graph. In practice, facts are often held by organizations that share entity identifiers but own disjoint relation types, so no single party sees a complete reasoning chain. This poster presents the empirical findings of FedV-KGQA on multi-hop question answering over such vertically partitioned graphs. Each silo enriches its local graph and trains a knowledge graph embedding on its own triples. A server then concatenates the silo-specific entity views, anchors the projected question at the topic entity, and ranks candidates by similarity. Raw triples and relation embeddings never leave a silo. Comparing the FedV-KGQA experiments with one another yields three results. First, federated fusion recovers most of the centralized accuracy, while a single silo recovers little. Second, anchoring and enrichment matter more than the choice of embedding model. Third, the cheapest encoder depends on the target accuracy rather than on parameter count. This poster paper contributes that cross-experiment comparison, four design lessons drawn from it, and an interactive prototype that runs real inference and traces the full pipeline, per question, on released checkpoints.

1. Introduction

Most KGQA assumes one system can access the whole graph, leaving a gap when organizations hold disjoint relations and cannot share triples. This poster compares FedV-KGQA experiments and presents design lessons and an interactive prototype that traces inference per question.

1. Introduction

  • Vertical partitioning leaves each organization with only part of a reasoning chain, while existing federated embedding work targets horizontal partitions and link prediction.Federated SPARQL engines require queryable endpoints and structured queries; FedV-KGQA ranks answers from natural language while keeping triples and relation parameters local.Source: §1 · p. 1
  • The poster compares six FedV-KGQA research questions to show how anchoring, enrichment, silo count, and encoder choice interact across experiments.It adds four design lessons based on that cross-experiment comparison.Source: §1 · p. 1
  • An interactive prototype runs live inference on released checkpoints and traces one question through every pipeline stage.FedV-KGQA itself does not provide this per-question view.Source: §1 · p. 2

2. Framework Overview

FedV-KGQA enriches and embeds each silo's local graph, then combines silo-specific entity views to rank answers to questions anchored at a supplied topic entity. Triples and relation embeddings remain local, and the evaluation is limited to path-shaped questions whose answers are reached by bounded expansion.

2. Framework Overview

  • Each silo enriches its graph with inverse-property and property-chain rules before training a local KGE model, while keeping raw triples and relation embeddings local.Inverse rules restore a useful embedding signal when an answer entity appears only as a tail in its silo.Source: §2 · p. 2
  • The server concatenates silo-specific entity embeddings, projects a question into their joint space, and anchors it at the supplied topic entity.Candidates are ranked by cosine similarity; the MLP is the only trainable component.Source: §2 · p. 2
  • QA training minimizes a margin-ranking loss over precomputed candidates, and gradients for the concatenated entity matrix return as silo-specific slices.The encoder is frozen, relation embeddings receive no gradient, and inference reuses fine-tuned entity matrices and precomputed candidates without graph traversal or cross-silo triple exchange.Source: §2 · p. 2
  • The method covers path-shaped questions with a supplied topic entity and an answer entity reached by bounded expansion, not aggregation, comparison, counting, or time constraints.Accuracy also depends on each silo's structure and alignment of entity identifiers.Source: §2 · p. 2

3. Experimental Design

The evaluation spans three benchmarks, varying relation partitions, embedding models, encoders, and QA settings. It measures ranking accuracy and communication cost, while accounting for differences in per-silo structure and candidate coverage.

3. Experimental Design

  • The study uses MetaQA, PathQuestion, and WebQuestionsSP, spanning movie, person-centric, and open-domain questions at different scales and path lengths.The benchmarks include more than 43,000 movie entities in MetaQA and more than 985,000 entities in WebQSP.Source: §3 · p. 2
  • Twelve configurations pair four KGE models with frozen BERT, DistilBERT, or RoBERTa encoders and evaluate MRR and Hits@K.Experiments cover two- and three-hop accuracy, component ablations, adapted federated and centralized baselines, embedding perturbation, and communication costs to reach target Hits@3.Source: §3 · pp. 2–3
  • Under three-silo splits, relation and triple counts vary substantially by benchmark and silo; using one silo alone covers at most 54%, 35%, and 46% of gold answers on MetaQA, PathQuestion, and WebQSP.With combined silos, coverage is 99%, 100%, and 78%, respectively; WebQSP accuracy is measured against a 0.78 coverage ceiling.Source: §3 · p. 3

4. Empirical Findings

Across the experiments, federated fusion approaches centralized accuracy, while results vary with partitioning and path length. Anchoring and enrichment improve results, and communication efficiency depends on the target accuracy.

Finding 1

  • Federated fusion reaches 0.76, 0.65, and 0.54 MRR on MetaQA, PathQuestion, and WebQSP, trailing centralized results by 0.04, 0.03, and 0.03 MRR, respectively.The best single-silo baselines fall to 0.40, 0.20, and 0.32 on those datasets.Source: §4 · p. 3, Figure 2 · p. 4

Finding 2

  • With BERT+TransE, seven silos yield 0.51 MRR on WebQSP, MetaQA peaks at 0.83 with five silos, and PathQuestion peaks at 0.68 with five before falling to 0.55.From two to three hops, MetaQA changes from 0.76 to 0.74 MRR, while PathQuestion changes from 0.65 to 0.57 MRR and from 0.96 to 0.90 Hits@10.Source: §4 · pp. 3–4, Figure 2 · p. 4

Finding 3

  • Removing anchoring lowers MRR from 0.64 to 0.57 on PathQuestion and from 0.53 to 0.47 on WebQSP, while removing enrichment costs WebQSP 0.19 MRR and 0.28 Hits@10.The reported roles are distinct: anchoring focuses ranking on the topic neighborhood, while enrichment strengthens local structure and candidate reachability.Source: §4 · p. 4
  • At σ=0.05, MRR drops only to 0.79, 0.59, and 0.48, with steeper decline above σ=0.10; this robustness analysis is not a privacy guarantee.The passage presents this as a robustness analysis, not a privacy guarantee.Source: §4 · p. 4, Figure 2 · p. 4
  • DistilBERT reaches Hits@3 of 0.35 on WebQSP at 96.8 versus 121.1 GB, but BERT is cheaper on PathQuestion at Hits@3 of 0.70, using 23.1 versus 31.4 GB.Communication scales with entity vocabulary rather than triple count; sparse candidate-set updates are proposed but unmeasured.Source: §4 · p. 4, Figure 2 · p. 4

5. Interactive Prototype

The prototype runs inference using released checkpoints and lets a viewer follow each question through the pipeline. Its trace exposes silo contributions, ranking outcomes, and where a query succeeds or fails.

Interactive inference

  • Using released TransE-with-BERT checkpoints for three-silo, two-hop MetaQA, the prototype runs real inference and streams the pipeline stages for a user-selected question.The displayed stages include the topic entity, silo triples, candidate filtering, fusion, and ranked answers with per-question MRR and Hits.Source: §5 · p. 4

Per-question trace

  • The John Travis example splits written_by across Silo A and starred_actors across Silo B, so neither silo alone can answer the question.Fusion ranks a gold answer second and recovers all three gold answers in the top six; Table 1 identifies the example and its gold answers.Source: §5 · pp. 4–5, Table 1 · p. 5
  • The trace distinguishes filtering failures from cases where an answer is present but scored too low, a distinction aggregate averages cannot show.The demonstration displays silo triples, but in deployment neither the user nor the server sees them; only entity embeddings leave a silo.Source: §5 · pp. 4–5

6. Cross-Experiment Lessons and Limitations

The cross-experiment lessons are qualified by the evaluation’s assumptions: a static graph, shared aligned entity identifiers, and semantic relation partitions. The server receives entity embeddings and candidate sets, so the boundary is not a formal privacy guarantee.

Design lessons

  • Preserve each silo’s geometric view through concatenation rather than averaging, which removes that view.This lesson is tied to Finding 1.Source: §6 · p. 5
  • Treat silo count and relation composition together: moderate partitioning may aid specialization, while heavy fragmentation can cost accuracy.The lesson reflects Finding 2.Source: §6 · p. 5
  • Anchor questions to the topic entity because language semantics alone do not identify the correct neighborhood.This lesson reflects Finding 3.Source: §6 · p. 5

Design lessons and limits

  • Report communication against an accuracy target, since the cheapest encoder depends on the operating point.The evaluation assumes a static graph, shared and aligned entity vocabulary, and semantic relation partitions; candidate sets also reveal that an entity is within two hops of the topic.Source: §6 · p. 5

7. Conclusion

The conclusion is that multi-hop KGQA remains feasible under vertical relation splits, with federated fusion approaching centralized accuracy. The paper attributes success to mechanisms beyond keeping triples local and leaves changing graphs and imperfect entity alignment for future evaluation.

Conclusion

  • FedV-KGQA reports that multi-hop KGQA remains feasible under vertical relation splits, approaching centralized accuracy, extending to three-hop reasoning, and tolerating moderate embedding noise.The conclusion frames these as findings within the evaluated setting.Source: §7 · p. 5
  • The cross-experiment comparison surfaces design lessons, while the prototype makes the mechanism inspectable one question at a time.The conclusion names enrichment, geometry-preserving fusion, topic anchoring, and cost-aware model selection as factors that decide whether federation works.Source: §7 · p. 5
  • Whether the findings hold for graphs that change over time or have imperfect entity alignment remains for future evaluation.The conclusion identifies both as open directions.Source: §7 · p. 5

Declaration on Generative AI

The authors used Claude and OpenAI Codex for language editing and structural refinement, reviewed all content, and retained responsibility for the submission. They state that no generative AI tool produced research findings, experimental results, tables, or citations.

Declaration on Generative AI

Loading 2609.13661v1…