Source-linked AI summary
A Survey of Large Language Models for Graphs
Xubin Ren, Jiabin Tang, Dawei Yin, Nitesh Chawla, Chao Huang
TL;DR
Graph learning still faces data sparsity and limited generalization, motivating integration with LLMs. This survey reviews LLMs for graph applications through a framework-design taxonomy of four architectures and discusses their strengths, limitations, and future directions. It identifies integration challenges and emerging opportunities for more flexible graph reasoning and new applications.
Problem
Graph learning faces persistent data sparsity and limited generalization, while integrating LLM capabilities with graph learning remains an active research question.
Method
The survey systematically reviews LLMs for graph applications and categorizes methods by inference-framework design into four architectural approaches.
Results
The survey characterizes four framework designs—GNNs as Prefix, LLMs as Prefix, LLMs-Graphs Integration, and LLMs-Only—and discusses representative methods, strengths, and limitations.
Takeaways & Limitations
The survey highlights integration challenges and future opportunities involving interactive graph agents, improved scaling, and applications such as knowledge graph reasoning, molecular modeling, and social network analysis.
Takeaways & Limitations
Current approaches face scalability constraints, limited support for non-text-attributed graphs, and difficulty transforming large graphs into text while preserving structural integrity.
Abstract
from arXiv · showhide
Graphs are an essential data structure utilized to represent relationships in real-world scenarios. Prior research has established that Graph Neural Networks (GNNs) deliver impressive outcomes in graph-centric tasks, such as link prediction and node classification. Despite these advancements, challenges like data sparsity and limited generalization capabilities continue to persist. Recently, Large Language Models (LLMs) have gained attention in natural language processing. They excel in language comprehension and summarization. Integrating LLMs with graph learning techniques has attracted interest as a way to enhance performance in graph learning tasks. In this survey, we conduct an in-depth review of the latest state-of-the-art LLMs applied in graph learning and introduce a novel taxonomy to categorize existing methods based on their framework design. We detail four unique designs: i) GNNs as Prefix, ii) LLMs as Prefix, iii) LLMs-Graphs Integration, and iv) LLMs-Only, highlighting key methodologies within each category. We explore the strengths and limitations of each framework, and emphasize potential avenues for future research, including overcoming current integration challenges between LLMs and graph learning techniques, and venturing into new application areas. This survey aims to serve as a valuable resource for researchers and practitioners eager to leverage large language models in graph learning, and to inspire continued progress in this dynamic field. We consistently maintain the related open-source materials at \url{https://github.com/HKUDS/Awesome-LLM4Graph-Papers}.
1 INTRODUCTION
Graphs represent relationships across domains, while GNNs support graph tasks and LLMs offer language understanding and generalization. This survey reviews LLMs for graphs, proposes a framework-design taxonomy, and identifies integration challenges and future directions.
- Graphs encode relationships across domains including social networks, molecular graphs, recommender systems, and academic networks.
- GNNs aggregate information across nodes and refine node features to support tasks such as node classification and link prediction.
- LLMs have prompted graph-learning research because of their generalization abilities and strengths in language understanding, including prompt-based graph reasoning.
- The survey systematically reviews LLMs for graph applications and distinguishes methods by framework design, especially inference and training processes.
- Its taxonomy covers four designs and discusses their strengths, limitations, integration challenges, and future application areas.
2 PRELIMINARIES AND TAXONOMY
The preliminaries define graph learning and language-model concepts before introducing a taxonomy based on how models process graph data and text. Four designs organize representative methods: GNNs as Prefix, LLMs as Prefix, LLMs-Graphs Integration, and LLMs-Only.
- Definitions: A graph G = (V, E) consists of nodes and edges, while a text-attributed graph assigns sequential text features to nodes.
- Definitions: GNNs update node embeddings by aggregating information from neighboring nodes, producing representations for tasks such as node classification and link prediction.
- Definitions: Masked Language Modeling predicts masked tokens using both left and right context.
- Definitions: Causal Language Modeling predicts the next token using only preceding tokens; GPT and Llama are representative examples.
- Taxonomy: The taxonomy organizes methods by model inference pipelines that process graph data and text with LLMs.
- Taxonomy: GNNs as Prefix supplies structure-aware tokens to LLMs, whereas LLMs as Prefix supplies node embeddings or labels to GNNs.
- Taxonomy: LLMs-Graphs Integration combines LLMs with graphs through fusion, alignment, or graph-interacting agents, while LLMs-Only uses prompting or fine-tuning to process graph information directly.
3 LARGE LANGUAGE MODELS FOR GRAPHS
The survey organizes LLM-based graph-learning methods by framework design into approaches that use GNNs or LLMs as prefixes, integrate both modalities, or rely on LLMs alone. It reviews their mechanisms, applications, strengths, and unresolved challenges.
- 3.1 GNNs as Prefix: GNNs as Prefix methods encode graph structure into node-, edge-, or graph-level tokens that LLMs consume for inference.GNNs generally act as structural encoders or tokenizers, aligning graph information with natural language.
- 3.1 GNNs as Prefix: Node-level tokenization preserves fine-grained structural representations for tasks such as node classification and link prediction.Graph-level tokenization instead pools node representations into unified graph representations for graph-level tasks.
- 3.1 GNNs as Prefix: GraphLLM, MolCA, and InstructMol use graph encoders with projectors, pooling, or alignment stages to provide graph-enhanced inputs to language models.These designs support graph reasoning, molecular understanding, and drug-discovery-related tasks.
- 3.2 LLMs as Prefix: LLMs as Prefix methods use textual content, labels, or embeddings generated by LLMs to improve GNN training.Their decoupled two-stage design makes GNN performance depend on pre-generated LLM outputs or task-specific prompts.
- 3.3 LLMs-Graphs Integration: LLMs-Graphs Integration combines language and graph modeling through fusion, alignment, contrastive learning, or graph-interacting agents.The survey identifies scalability and limited repeated agent interaction as major unresolved challenges.
- 3.4 LLMs-Only: LLMs-Only methods model graphs within text space, but transforming large graphs into prompts while preserving structural integrity remains difficult.The approach explores whether pretrained LLMs can interpret graph data without an explicit graph encoder.
4 FUTURE DIRECTIONS
Future work targets broader graph data and tasks, multimodal reasoning, scalable training, and interactive graph agents. These directions aim to extend LLM-based graph learning beyond current computational and interaction constraints.
- LLMs for Multi-modal Graphs: Multimodal LLMs could process graph nodes containing textual, visual, auditory, and other features for more comprehensive graph reasoning.This direction builds on LLMs’ demonstrated ability to process multimodal data.
- Scalability: Reducing training and inference costs is necessary for applying LLMs and GNNs to large-scale graphs with millions of nodes.The computational burden also makes integrating the two model families increasingly difficult.
- Tackling Different Graph Tasks: LLM-based graph methods should expand beyond link prediction and node classification to graph generation, graph understanding, and graph-based question answering.Suggested applications include molecular structure generation, social relationship analysis, and knowledge graph construction.
- User-Centric Agents on Graphs: Future graph agents should support iterative, feedback-driven interaction rather than one-time execution for single graph tasks.Such agents would dynamically search graph data and adapt their plans to diverse open-ended user questions.
5 CONCLUSION
The survey reviews LLMs for graph data through a taxonomy based on inference-framework design. It characterizes four framework designs and discusses their advantages, limitations, challenges, and opportunities.
- Conclusion: The survey proposes a taxonomy of LLMs for graph data grounded in distinctive inference-framework designs.The taxonomy is intended to deepen understanding of how LLMs address graph-related challenges.
- Conclusion: Four framework designs are categorized, each with its own advantages and limitations.The survey also discusses associated challenges and opportunities.
- Conclusion: The survey aims to guide researchers and inspire further innovation in integrating LLMs with graphs.It is presented as a resource for graph-related applications of large language models.
6 APPENDIX
The appendix includes Table 1, which overviews representative graph learning methods that use large language models. The table is presented as a summary resource for these methods.
- APPENDIX: Table 1 provides an overview of notable graph learning techniques that utilize large language models.
- APPENDIX: Table 1 summarizes representative graph learning methods with large language models.
- APPENDIX: The appendix identifies the table as a summary of representative LLM-based graph learning methods.