Source-linked AI summary
Knowledge Homophily in Large Language Models
Utkarsh Sahu, Zhisheng Qi, Mahantesh Halappanavar, Nedim Lipka, Ryan A. Rossi, Franck Dernoncourt, Yu Zhang, Yao Ma, Yu Wang
TL;DR
The paper investigates whether LLM knowledge is structurally organized, a question left open by prior work on knowledge patterns. It measures knowledgeability on graph facts and entities, tests for homophily, and uses neighborhood scores in a GNN estimator. The authors report similar knowledgeability among nearby entities and applications to knowledge injection and multi-hop retrieval.
Problem
Whether LLM knowledge exhibits structural organization remains underexplored, despite prior work identifying other knowledge patterns.
Source: §1 · p. 2Method
The authors map LLM knowledgeability onto knowledge graphs and use a GNN regressor to estimate entity-level scores from neighborhood information.
Source: §3.1 · p. 2, §4.1 · p. 3Results
The authors find that topologically close entities tend to have similar knowledgeability, and report that G-BS improves over M-BS by 4.57% on 2-hop queries and 2.62% on 3-hop queries.
Source: §1 · p. 2, §4.3 · p. 4Takeaways & Limitations
Estimated knowledgeability is used to select less-known triplets for fine-tuning and to improve multi-hop question-answering retrieval.
Source: §5 · p. 4The authors flag that knowledge homophily can facilitate cohort-based knowledge-extraction attacks, posing privacy and copyright risks.
Source: §6 · p. 5
Abstract
from arXiv · showhide
Large Language Models (LLMs) have been increasingly studied as neural knowledge bases for supporting knowledge-intensive applications such as question answering and fact checking. However, the structural organization of their knowledge remains unexplored. Inspired by cognitive neuroscience findings, such as semantic clustering and priming, where knowing one fact increases the likelihood of recalling related facts, we investigate an analogous knowledge homophily pattern in LLMs. To this end, we map LLM knowledge into a graph representation through knowledge checking at both the triplet and entity levels. After that, we analyze the knowledgeability relationship between an entity and its neighbors, discovering that LLMs tend to possess a similar level of knowledge about entities positioned closer in the graph. Motivated by this homophily principle, we propose a Graph Neural Network (GNN) regression model to estimate entity-level knowledgeability scores for triplets by leveraging their neighborhood scores. The predicted knowledgeability enables us to prioritize checking less well-known triplets, thereby maximizing knowledge coverage under the same labeling budget. This not only improves the efficiency of active labeling for fine-tuning to inject knowledge into LLMs but also enhances multi-hop path retrieval in reasoning-intensive question answering.
1 Introduction
The paper asks whether LLM knowledge has structural organization, extending prior work that largely identifies knowledge patterns without examining their structure. It investigates knowledge homophily and proposes using it to improve knowledge-intensive tasks.
1 Introduction
- The work builds on LLM knowledge checking used to study knowledge patterns and guide adaptive retrieval, knowledge editing, and hallucination detection.LLMs are also presented as neural knowledge bases supporting fact checking and question answering.Source: §1 · p. 1
- The paper addresses the underexplored question of whether LLM knowledge exhibits structural organization, hypothesizing that conceptually related entities have similar knowledgeability.The motivation draws an analogy to semantic clustering and homophily in human brain networks.Source: §1 · p. 2
- The authors identify knowledge homophily in LLMs by measuring knowledge at triplet and entity levels, finding that topologically close entities have similar knowledgeability.They propose a GNN estimator and demonstrate applications in knowledge injection and multi-hop retrieval for question answering.Source: §1 · p. 2
2 Related Work
Prior knowledge-checking methods assess LLM knowledge content, while structural studies examine storage and relational reasoning; homophily remains largely unexplored. The supplied prompt passages also describe a relation-template procedure for evaluating triplets.
2 Related Work
- Existing knowledge checks assess factual accuracy, self-awareness, or coverage and consistency, but focus on content rather than structure.Research on structure has examined where facts are stored and properties such as symmetry, hierarchy, and path-following; homophily remains largely unexplored.Source: §2 · p. 2
- The triplet-evaluation prompt selects a relation-specific pattern, fills in subject and object, adds the date when applicable, and records the LLM’s True/False response.The relation pattern is illustrated by “son_of → {SUB} is the son of {OBJ}.”Source: §2 · p. 2
3 Knowledge Homophily Discovery
The section measures LLM knowledgeability for graph facts and entities, then tests whether neighboring entities have similar scores. It reports high homophily across models and graphs, above a degree-matched random baseline, with coherent high- and low-knowledge clusters in a T-Rex visualization.
3.1.2 Calculating Entity Knowledgeability.
- Entity knowledgeability aggregates scores from all triplets involving that entity, and the aggregation also accommodates temporal triplets.Figure 2 summarizes node-level distributions and average homophily across datasets and LLMs, with Citeseer’s 0.74 value shown as a reference.Source: §3.1 · p. 2, §3.1.2 · p. 2, Figure 2 · p. 3
3.1.1 Calculating Triplet Knowledgeability.
- Triplet knowledgeability is a binary score from an LLM’s True/False judgment of a natural-language fact, with temporal prompts including the timestamp.The score represents how well the model knows the relational fact.Source: §3.1.1 · p. 2
3.2 Homophily Computation and Analysis
- Knowledge homophily is one minus the average absolute knowledgeability difference between an entity and its neighbors; smaller differences yield higher homophily.The evaluation covers five representative LLMs and five knowledge graphs spanning general, biomedical, and political knowledge.Source: §3.2 · p. 3
3.2.1 Quantitative Analysis of Node/Graph Knowledge Homophily.
- Across datasets and LLMs, node-homophily distributions are right-skewed and peak near 0.8, while average graph homophily exceeds Citeseer’s high-homophily benchmark.Temporal information in MVPKG causes a slight left shift.Source: §3.2.1 · p. 3, Figure 2 · p. 3
- True-neighborhood homophily significantly exceeds a degree-matched random baseline across datasets (100 trials per dataset, two-tailed z-test, p<0.01).Figure 3(a) presents this comparison; the result supports the claim that knowledge homophily is not a random artifact.Source: §3.2.1 · p. 3, Figure 3 · p. 3
- A T-Rex subgraph shows a high-knowledge geopolitical neighborhood and a lower-knowledge historical football cluster, both with small within-neighborhood deviations.Figure 3(b) encodes entity knowledgeability by node color.Source: §3.2.2 · p. 3, Figure 3 · p. 3
4 Knowledge Homophily Application
The application uses neighborhood-based knowledgeability estimates to identify less-known triplets for fine-tuning and retrieval. Its experiments report improved knowledge injection and multi-hop question answering, with gains for GNN-guided retrieval over homophily-agnostic estimation.
4 Knowledge Homophily Application
- Knowledgeability estimates identify low-knowledge regions so triplets can be prioritized for fine-tuning or retrieval to address knowledge gaps.The application covers knowledge injection and multi-hop question answering.Source: §4 · p. 3
4.1 Homophily-aware Knowledge Estimation
- A GNN regressor uses message passing and feature transformation to estimate unknown entities’ knowledgeability from neighborhood signals.It is trained on entities with knowledgeability scores obtained by prompting LLMs, then used to infer scores for unseen entities.Source: §4.1 · p. 3
4.2 Homophily-guided Knowledge Injection
- The GNN regressor selects the highest proportion of unknown triplets and yields the strongest knowledge generalization among the compared selection methods.The generalization advantage holds across evaluation budgets from 1% to 20%.Source: Table 1 · p. 4, §4.2 · p. 4
- For knowledge injection, 20% of the 4,000-triplet budget supplies queried anchor labels, and predicted low-knowledge entities guide selection from the remaining pool.The selection is compared with Random and an MLP estimator without homophily.Source: §4.2 · p. 4
4.3 Homophily-guided Knowledge Retrieval
- G-BS improves accuracy by 4.57% on 2-hop queries and 2.62% on 3-hop queries, and surpasses homophily-agnostic M-BS.All methods decline on 3-hop queries due to semantic drift; the gains remain consistent across training budgets.Source: Table 2 · p. 4, §4.3 · p. 4
- Retrieval uses beam search that scores candidate paths by semantic similarity and penalizes entities with higher predicted knowledgeability.Evaluation uses 1,000 questions per knowledge graph, split evenly between 2-hop and 3-hop questions.Source: §4.3 · p. 4
5 Conclusion
The paper investigates knowledge homophily in LLMs, builds a GNN regressor that uses local neighborhoods to estimate entity knowledgeability, and evaluates it in knowledge injection and multi-hop retrieval. The authors propose uncertainty-aware verification and dynamic homophily modeling as future directions.
5 Conclusion
- Neighboring entities in knowledge graphs have correlated knowledgeability scores, motivating a GNN regressor that uses local neighborhoods to estimate entity-level knowledgeability.The authors report applications to selecting less-known triplets for fine-tuning and improving multi-hop question-answering retrieval.Source: §5 · p. 4
6 Ethical Considerations
Knowledge homophily may increase exposure to knowledge-extraction attacks because related entities can be queried as cohorts. The paper identifies privacy and copyright risks and lists possible defensive strategies.
6 Ethical Considerations
- Queries over cohorts of related entities may increase unintended disclosure, creating privacy and copyright risks.Proposed defenses include query filtering and sanitization, rate limits, prompt-level heuristics, detection, access controls, and differential privacy.Source: §6 · p. 5
A.1.1 Homophily-guided Knowledge Injection.
The pipeline labels anchor triplets, trains a GNN to estimate knowledgeability for other entities, and uses lower predicted scores to select triplets for fine-tuning. It assesses both selection quality and generalization on a held-out set.
A.1.1 Homophily-guided Knowledge Injection.
- Twenty percent of the triplet budget is allocated to anchor triplets, whose queried knowledgeability labels train the GNN; it predicts scores for the remaining entities, and low-scored entities supply the other 80%.The selected triplets and anchors form the fine-tuning dataset.Source: §A.1.1 · p. 6
- The held-out 2% set excludes fine-tuning triplets and entities overlapping with fine-tuned triplets, and the paper reports improvements in selection quality and generalization.Figure 4(b) presents the retrieval pipeline as a separate method for improving retrieved context in multi-hop question answering.Source: §A.1.1 · p. 6, §A.1.2 · p. 6
A.1.2 Homophily-guided Knowledge Retrieval.
The retrieval method uses estimated entity knowledgeability to steer multi-hop search toward less-known entities. On CoDEx-S, GNN-based search outperforms the MLP-based and semantic alternatives for both 2-hop and 3-hop QA.
A.1.2 Homophily-guided Knowledge Retrieval.
- G-BS scores candidate paths by semantic relevance while penalizing expansion toward entities with higher estimated knowledgeability.The score S × (1 − α·K(u)) is used to prioritize facts with higher information gain.Source: §A.1.2 · pp. 6–7
- Figure 5 reports Mistral knowledge-injection performance on CoDEx-S across test-set sizes, with the GNN-guided approach ahead of other methods.Source: Figure 5 · p. 7
- On CoDEx-S, across varying training budgets, GNN-based search (G-BS) consistently outperforms homophily-agnostic MLP-based search (M-BS) and the semantic baseline for both 2-hop and 3-hop QA.Source: Figure 6 · p. 7
- Table 3 reports correlations between entity knowledgeability scores calculated under full and sparse settings across datasets.Source: Table 3 · p. 7
A.2 Additional Results
The additional results test robustness across data conditions, retrieval budgets, and datasets, and clarify the scope and computational rationale of the probing approach. They report consistent GNN-guided injection and retrieval advantages, stable knowledgeability scores under sparsity, and improving retrieval performance with larger estimator budgets.
A.2 Additional Results
- Under Sparse75%, Pearson correlations exceed 0.91 across datasets; under Sparse50%, they are generally above 0.80.Both Pearson and Spearman correlations remain high, and the authors report that the scores are not simply a byproduct of local graph density.Source: §A.2.1 · p. 7
A.2.2 Sensitivity Analysis: Knowledge Injection.
- GNN-guided injection outperforms MLP, Random, and Baseline across test-set sizes, while performance remains relatively stable across those sizes.The sensitivity analysis uses five CoDEx-S test sets representing 1%, 2%, 5%, 10%, and 20% of the dataset.Source: §A.2.2 · p. 7
A.2.3 Sensitivity Analysis: Knowledge Retrieval.
- G-BS outperforms M-BS and the baseline across estimator training budgets and both 2-hop and 3-hop questions; both search methods improve as the budget grows.Source: §A.2.3 · pp. 7–8
A.2.4 Role of Knowledge Graph Probing.
- The study uses KG structure to examine patterns induced by its chosen probing signal, not to establish the probing method’s superiority or equivalence to alternatives.Source: §A.2.4 · p. 8
- The approach probes only a subset of entities and infers the rest through graph structure, reducing prompts, inference time, token usage, and computational cost.Source: §A.2.5 · p. 8
A.2.5 Scalability and Computational Considerations.
- For a fixed number of layers, the authors characterize GNN inference as linear in graph nodes and edges, and describe their graphs as smaller than million-entity benchmarks.Source: §A.2.5 · p. 8
A.2.6 Datasets.
- Experiments span MVPKG, T-Rex, PharmKG8K, WD50K, and CoDEx-S; MVPKG is also evaluated with temporal information included and excluded.Source: §A.2.6 · p. 8