PaperlayerView exact revision on arXiv

Source-linked AI summary

Can LLMs Convert Graphs to Text-Attributed Graphs?

Zehong Wang, Sidney Liu, Zheyuan Zhang, Tianyi Ma, Chuxu Zhang, Yanfang Ye

arXiv:2412.10136v2cs.CLcs.AIcs.LG

TL;DR

Cross-graph learning is difficult when graphs have different feature spaces, and obtaining the node descriptions used by TAGs is often impractical. TANS uses LLMs guided by graph topology to generate node descriptions, with reported superiority across text-rich, text-limited, and text-free graphs and several learning settings.

  • Problem

    GNNs struggle to handle multiple graphs with different feature spaces, while TAG-based alignment depends on collecting node descriptions that are often impractical to obtain.

    Source: §1 · pp. 1–2

  • Method

    TANS uses LLMs and node topology to generate node-level textual descriptions for graphs, including graphs without existing text.

    Source: §1 · p. 2

  • Results

    TANS shows superiority across text-rich, text-limited, and text-free graphs in training from scratch, domain adaptation, and transfer learning settings.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The results support using LLM-generated descriptions to align node features across different graphs, including graphs without textual information.

    Source: §6 · p. 8

    Experiments exclude graphs with more than 100,000 nodes because querying GPT for each node makes description generation expensive in time and cost.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Graphs are ubiquitous structures found in numerous real-world applications, such as drug discovery, recommender systems, and social network analysis. To model graph-structured data, graph neural networks (GNNs) have become a popular tool. However, existing GNN architectures encounter challenges in cross-graph learning where multiple graphs have different feature spaces. To address this, recent approaches introduce text-attributed graphs (TAGs), where each node is associated with a textual description, which can be projected into a unified feature space using textual encoders. While promising, this method relies heavily on the availability of text-attributed graph data, which is difficult to obtain in practice. To bridge this gap, we propose a novel method named Topology-Aware Node description Synthesis (TANS), leveraging large language models (LLMs) to convert existing graphs into text-attributed graphs. The key idea is to integrate topological information into LLMs to explain how graph topology influences node semantics. We evaluate our TANS on text-rich, text-limited, and text-free graphs, demonstrating its applicability. Notably, on text-free graphs, our method significantly outperforms existing approaches that manually design node features, showcasing the potential of LLMs for preprocessing graph-structured data in the absence of textual information. The code and data are available at https://github.com/Zehong-Wang/TANS.

1 Introduction

GNNs struggle to learn jointly across graphs with different feature spaces, while collecting the node descriptions used by text-attributed graphs is often impractical. TANS uses LLMs and graph topology to generate node descriptions across text-rich, text-limited, and text-free settings.

1 Introduction

  • A single GNN struggles across graphs with different feature dimensions, limiting cross-graph learning in transfer learning, domain adaptation, out-of-distribution detection, and graph foundation models.The fixed input dimension of message-passing GNNs makes varying feature spaces difficult to accommodate.Source: §1 · pp. 1–2
  • TAGs use textual encoders to align node features across graphs, but collecting high-quality descriptions for every node is often impractical.SVD aligns dimensions but does not ensure consistent feature semantics and cannot handle graphs without node features.Source: §1 · p. 2
  • TANS uses LLMs and each node’s topology to generate descriptions that capture its role, addressing the difficulty of collecting text for TAGs.The method is intended for graphs without text and can also enhance graphs with rich or limited textual data.Source: §1 · p. 2
  • Across five datasets spanning text-rich, text-limited, and text-free settings, TANS shows superior performance in generating node properties and robustness in transfer learning and domain adaptation.The authors present this as evidence for the value of LLM-generated descriptions in aligning node features across graphs.Source: §1 · p. 2

2 Backgrounds

A GNN learns node embeddings through message passing, but its fixed input transformation makes one model difficult to apply across graphs with different feature dimensions. Prior alignment approaches address parts of this problem but face limitations in feature alignment or generalization.

2 Backgrounds

  • A GNN encoder learns node embeddings from a graph through message passing, with node features represented as vectors.The graph is given as G = (V, E), with node set V and edge set E.Source: §2 · p. 2
  • The first GNN layer’s transformation dimensions depend on the input features, so the same transformation cannot accommodate a new graph with different feature dimensions.This fixed-dimensionality mismatch is why a single GNN cannot effectively handle multiple graphs with different feature spaces.Source: §2 · p. 2
  • SVD aligns node-feature dimensions across graphs but may not ensure fine-grained semantic alignment, while graph-specific projectors fail to generalize to unseen graphs.The cited SVD approaches are Yu et al. (2024) and Zhao et al. (2024); the projector limitation is attributed to Yu et al. (2024).Source: §2 · p. 3
  • TAG methods encode node descriptions with textual encoders, but their embeddings may remain in different subspaces and existing-text-dependent approaches cannot handle text-free graphs.TAPE (He et al., 2024) and KEA (Chen et al., 2024a) are examples of LLM methods that rely on existing text.Source: §2 · p. 3

3 Method: TANS

TANS uses graph topology as auxiliary knowledge to guide LLM-generated node descriptions, aiming to capture each node’s role. Its four-step framework is designed for graphs with rich, limited, or no existing text.

3.1 Application Scenarios

  • TANS handles text-rich, text-limited, and text-free graphs, while TAPE and KEA are limited to specific cases.In text-rich graphs it adds topology to existing descriptions; in text-limited graphs it supplements sparse text; in text-free graphs topology is the only node resource.Source: §3.1 · p. 3

3.2 Step 1: Graph Properties

  • TANS encodes topology with degree, betweenness, closeness, triangle clustering, and square clustering to represent node influence and neighborhood structure.These properties capture direct connections, shortest-path brokerage, average distance, triangle-like cohesion, and square-like patterns.Source: §3.2 · p. 4

3.3 Step 2: Generate Basic Node Descriptions

  • Basic descriptions combine graph and node-type context, optional node and five-neighbor text, and ranked node properties, making topology explicit in the LLM prompt.The five neighbors are randomly selected; the prompt uses node-property rankings to convey relative importance.Source: §3.3 · pp. 4–5

3.4 Step 3: Prompting LLMs

  • GPT-4o-mini receives prompts designed for cross-graph-consistent descriptions, with class-conditioned top-k predictions and explanations rather than overly specific outputs.The method sets k to 3 when there are more than three classes, and to 1 otherwise.Source: §3.4 · p. 5

3.5 Step 4: Explanations

  • LLM-generated explanations become final node descriptions, appended to existing text or used as text-free descriptions before textual encoding.Text-rich and text-limited graphs retain their original text alongside the generated descriptions.Source: §3.5 · p. 5

4 Experiments

Experiments assess TANS on text-rich, text-limited, and text-free graphs in single-graph and cross-graph settings. Results include gains over baselines, with ablations examining neighborhood text and textual encoders.

4.1 Experimental Setup

  • Experiments use five citation and airport datasets, evaluate text-rich, text-limited, and text-free settings, and report test accuracy across GCN, GAT, and MLP backbones.Runs use 30 random seeds; training settings include low-label and high-label splits.Source: §4.1 · pp. 5–6, §4.2 · p. 6

4.2 Single-Graph Learning

  • In text-rich Cora and Pubmed, TANS outperforms the compared methods, with its largest improvement using MLP backbones.The authors attribute the gains partly to topological information that captures node roles.Source: §4.2 · p. 6, Table 4 · p. 6
  • On text-limited citation graphs, adding TANS to titles improves performance by about 5% on average over the best baselines.Performance is lower than in the text-rich setting; TAPE and KEA are not applicable in this comparison.Source: §4.2 · pp. 6–7, Table 7 · p. 7
  • For text-free airport graphs, TANS significantly outperforms methods that generate node features from graph properties.The authors point to its broader set of topological properties and LLM analysis of prompts and potential classes.Source: Table 5 · p. 6, §4.2 · p. 7

4.3 Cross-Graph Learning

  • 58.40 average versus 54.21 for the second-best method, with the best result on 5 of 6 datasets, in text-free domain adaptation.The study also reports that TANS-generated descriptions are reliable and show some transferability across text-free graphs.Source: §4.3 · pp. 7–8, Table 6 · p. 7
  • In citation-network pretrain-and-finetune transfer, TANS is more robust than SVD and existing methods, which sometimes perform worse than original textual descriptions.The authors suggest excess generated information may limit generalization and obscure shared patterns; negative transfer also occurs in some experiments.Source: §4.3 · p. 7, Table 8 · p. 7

4.4 Ablation Study

  • Removing neighborhood descriptions drops text-limited performance from 76.21 to 72.92; TANS also performs best across four tested textual encoders on text-rich Cora.The encoder comparison uses the low-label setting and GCN backbone.Source: §4.4 · p. 8, Table 9 · p. 8, Table 10 · p. 8

5 Discussion

The discussion considers extending TANS beyond node classification and converting basic attributed graphs into text-attributed graphs.

5 Discussion

  • The demonstrated tasks are node classification on citation and airport networks; edge-level and graph-level extensions remain future work.The authors describe these results as motivating exploration of LLM understanding of edge and graph properties.Source: §5 · p. 8
  • Feature-engineered attributed graphs may also be convertible when feature dimensions have inherent semantics.The discussion gives keyword-based one-hot Cora features and TF-IDF Pubmed features as examples; exploring this conversion is future work.Source: §5 · p. 8
  • Whether converted graphs are text-rich or text-limited depends on the specific case.Source: §5 · p. 8

6 Conclusion

The conclusion presents TANS as a way to use LLMs to generate node descriptions that incorporate graph topology. Experiments report performance across graph types and training settings.

6 Conclusion

  • TANS generates node descriptions using graph topology, including for graphs with or without existing textual information.Source: §6 · p. 8
  • The experiments cover text-rich, text-limited, and text-free graphs in training-from-scratch, domain-adaptation, and transfer-learning settings.Source: §6 · p. 8
  • The conclusion reports that TANS outperforms the compared methods across those graph types and settings.Source: §6 · p. 8

Limitations

The experiments exclude large-scale graphs, and the authors note limits related to querying cost and the LLM used.

Limitations

  • Graphs with more than 100,000 nodes were excluded because querying GPT for each node makes TANS expensive in time and cost.The cost also restricted more extensive prompt-design ablations.Source: Limitations · p. 9
  • The authors identify more efficient template designs as a possible way to optimize the process.Source: Limitations · p. 9
  • The experiments used GPT-4o-mini; whether GPT-4o would significantly outperform it remains uncertain.Source: Limitations · p. 9

Ethical Considerations

Generated node descriptions can carry bias or harmful content, while the related-work discussion covers feature alignment and topology-based node features.

Ethical Considerations

  • Generated descriptions may include biased or harmful information, so the paper recommends careful prompts and responsible use.It also calls for attention to bias in the data.Source: Ethical Considerations · p. 9

A More Related Works

  • SVD aligns feature dimensions but may not provide fine-grained feature alignment across graphs.The related-work discussion also notes that manually designed features can include node degrees, Laplacian eigenvectors, and node2vec embeddings.Source: A More Related Works · p. 11

B Experimental Setting

The experiments cover text-rich transfer learning, text-free domain adaptation, and additional encoder and prompt-design evaluations. Hyperparameters are tuned and reported across settings.

B.1 Hyper-parameters

  • Hyperparameters are selected through 500 Bayesian-search tuning runs per method, with one attention head fixed.The searched and final hyperparameters are listed in Tables 11–14.Source: §B.1 · p. 11, Table 11 · p. 12

B.2 Pretrain & Finetune Setting

  • The pretrain-and-finetune transfer setting compares Cora and Pubmed despite their different original feature dimensions, using text-encoded node descriptions for alignment.Pretraining uses 100 epochs and a learning rate of 0.001.Source: §B.2 · pp. 11–12

B.3 Domain Adaptation Setting

  • Domain adaptation trains on one text-free airport graph and tests on another, using 20% of target nodes for validation and 80% for testing.USA, Brazil, and Europe are used because their label spaces are aligned.Source: §B.3 · p. 12

C Additional Experimental Results

D.1 Advanced Encoder

  • OFA is evaluated as an advanced encoder on low-label Cora, where its performance is reported below GCN’s.The authors suggest this may reflect OFA’s cross-domain and cross-task design versus GCN’s single-task optimization.Source: §D.1 · p. 12

D.2 Impacts of Topological Information on Prompt Design

  • Removing topological information from prompts lowers performance on low-label Cora in both text-rich and text-limited settings.The paper also notes that useful topological properties may vary by dataset and graph type.Source: §D.2 · p. 12

D.3 Impacts of Topological Information on Node Features

The paper compares LLM-generated descriptions with direct topological features and examines transfer between text-attributed and text-free graphs.

D.3 Impacts of Topological Information on Node Features

  • Directly using degree, centrality, or clustering coefficients as features is limited in cross-graph training unless original features are aligned or excluded.Source: §D.3 · p. 12
  • The TPF baseline concatenates prompt-design topology properties as node features and normalizes them for training stability.The study compares this baseline with TANS in low-label single-graph and domain-adaptation settings.Source: §D.3 · p. 14
  • TANS significantly outperforms Topology Properties as Features in both single-graph and cross-graph training.The comparison uses a GCN backbone in low-label settings.Source: Table 20 · p. 14, Table 21 · p. 14, §D.3 · p. 14

D.4 Advanced Knowledge Transfer

  • TANS supports transfer across text-attributed and text-free datasets and across citation and airport domains, with reasonable performance even across fundamentally different graph types.The authors attribute some marginal transfer drops to shared structural patterns, such as the importance of high-degree nodes.Source: Table 22 · p. 14, §D.4 · p. 14

E Results on Larger Graphs

On a sampled ogbn-products graph, TANS consistently outperforms TAPE; the evaluation does not include KEA.

E Results on Larger Graphs

F Case Studies

The case studies report that topology influences LLM answers and improves generated-text quality, with examples spanning node-class predictions and airport activity descriptions.

F Case Studies

  • Incorporating topological information significantly influences generated answers and improves generated-text quality.Providing neighborhood information allows LLMs to adjust predictions.Source: F Case Studies · p. 15
  • For a node about feature engineering driven by first-order clauses, the generated classes include Rule Learning, Probabilistic Methods, and Theory.The output connects the described algorithm and its stochastic nature to those categories.Source: F Case Studies · p. 16
  • For “How to evolve autonomous robots,” the full-prompt and no-neighborhood outputs propose different classes; the latter includes Reinforcement Learning and Case Based.The full-prompt output instead lists Genetic Algorithms, Neural Networks, and Theory.Source: F Case Studies · p. 17
  • Airport-node examples translate network properties into high, low, and moderate activity assessments.The examples draw on degree, centrality, and clustering measures.Source: F Case Studies · p. 21

Related papers

Loading 2412.10136v2…