PaperlayerView exact revision on arXiv

Source-linked AI summary

TAHB: A Comprehensive Benchmark for Text-Attributed Hypergraph Learning

David Yoon Suk Kang, JungHyun Kim, Juhyun Jeon, Sang-Wook Kim

arXiv:2608.15055v1cs.AI

TL;DR

Text-attributed hypergraph learning lacks a public benchmark combining higher-order structure with raw textual attributes. TAHB introduces such a benchmark and shows that topology and LLM-derived semantics are complementary, with joint use performing best for prediction.

  • Problem

    No public benchmark jointly supports higher-order relational modeling and raw text-aware semantic learning for hypergraph research.

  • Method

    TAHB provides 10 real-world text-attributed hypergraph datasets across e-commerce, academia, movies, and politics, and evaluates LLM-as-Predictor and LLM-as-Enhancer settings.

  • Results

    Jointly leveraging hypergraph topology and textual semantics consistently achieves the best LLM-as-Predictor performance, while LLM-generated semantic augmentation improves existing HGNNs over raw text alone.

  • Takeaways & Limitations

    TAHB establishes a reliable benchmark for studying text-attributed hypergraph learning and the complementarity of LLM-driven semantics with hypergraph topology.

  • Takeaways & Limitations

    Current LLMs still face limitations in multi-step structural reasoning when operating as standalone hypergraph predictors.

Abstract

from arXiv · show

Hypergraphs effectively model higher-order groupwise relationships beyond pairwise interactions, while pretrained language models (PLMs) and large language models (LLMs) provide rich semantic understanding from textual attributes. However, research on combining language models with hypergraph learning remains limited due to the lack of public text-attributed hypergraph benchmarks. To address this limitation, we present TAHB (Text-Attributed Hypergraph Benchmark), the first public benchmark integrating hypergraph structures and raw textual attributes. TAHB contains 10 real-world datasets from four domains - e-commerce, academia, movies, and politics networks - enabling systematic evaluation of text-aware hypergraph representation learning. Experimental results show that TAHB preserves key structural properties of real-world hypergraphs and consistently reproduces performance tendencies observed in existing benchmarks. Furthermore, experiments under both LLM-as-Enhancer and LLM-as-Predictor settings demonstrate that LLM-enhanced textual semantics improve hypergraph learning performance, while structural and textual information jointly provide the best setting for LLM-based prediction. Our benchmark provides a foundation for future research at the intersection of hypergraph learning and language models.

1 Introduction

TAHB addresses the limitations of pairwise graph learning and text-free hypergraph benchmarks by introducing a public benchmark that jointly models hypergraph structures and textual attributes. Its validation and LLM experiments show that the benchmark preserves real-world characteristics and that combining topology, text, and LLM-generated semantics benefits hypergraph learning.

  • Motivation: Structural information alone may miss rich semantic relationships, motivating the incorporation of textual and other auxiliary signals into graph learning.
  • Motivation: Hypergraphs overcome pairwise limitations by connecting multiple nodes within one hyperedge and preserving groupwise relationships.
  • Benchmark: TAHB is the first public benchmark jointly integrating textual attributes and hypergraph structures across 10 real-world datasets from e-commerce, academia, movies, and politics.Its nodes include papers, products, movies, and bills, while hyperedges represent naturally occurring groupwise relationships.
  • Benchmark Validation: TAHB preserves structural characteristics of existing text-free hypergraph benchmarks, exhibits text distributions similar to established TAG datasets, and maintains strong text–node semantic consistency.These validation results support the reliability of the benchmark for downstream evaluation.
  • LLM Integration: In LLM-as-Predictor experiments, jointly using hypergraph topology and textual semantics consistently achieves the best performance across all datasets.LLM-as-Predictor directly performs downstream tasks using topology and textual semantics as input.
  • LLM Integration: In LLM-as-Enhancer experiments, LLM-generated semantic augmentation consistently improves existing HGNN models over using only raw textual attributes.The augmentation is incorporated into existing HGNNs as enhanced input features.
  • Significance: By publicly releasing TAHB with standardized evaluation protocols, the work establishes a foundation for future research on LLM-driven text-attributed hypergraph learning.

2 Related Work

Existing hypergraph representation learning methods mainly model structure through semi-supervised or self-supervised approaches, while PLM/LLM integration remains underexplored. Existing graph and hypergraph benchmarks separately provide textual attributes or higher-order relations, motivating TAHB as the first public benchmark combining both.

  • Hypergraph learning: Existing HRL methods use semi-supervised or self-supervised learning, with semi-supervised approaches extending GNNs through clique, bipartite, message-passing, or set-based hyperedge modeling.Self-supervised methods derive supervision directly from hypergraph topology.
  • Hypergraph learning: Existing HRL methods primarily rely on structural features or shallow node embeddings, leaving PLM and LLM integration largely underexplored.This limits their ability to capture semantic information embedded in real-world higher-order group interactions.
  • Benchmark datasets: Existing graph benchmarks increasingly incorporate semantic information and textual attributes, but they mainly model primitive pairwise relations rather than higher-order interactions.Benchmark datasets can be categorized by textual-attribute availability and whether they represent pairwise or higher-order relations.
  • Benchmark datasets: Existing hypergraph benchmarks capture compressive higher-order relations but generally lack raw textual attributes, leaving no public benchmark supporting both capabilities.TAHB addresses this gap as the first benchmark specifically designed for text-attributed hypergraph learning.

3 TAHB: The Proposed Benchmark

TAHB is a standardized benchmark of 10 real-world text-attributed hypergraph datasets spanning four domains and multiple scales, preserving raw textual semantics alongside higher-order structure. Structural, textual, and task-level analyses show that it retains real-world properties and reproduces established hypergraph-learning performance trends.

  • Benchmark composition: TAHB comprises 10 real-world datasets across E-commerce, Academia, Movies, and Politics, with small, medium, and large node scales supporting diverse evaluations.The scales are small (1,000–10,000 nodes), medium (10,000–20,000 nodes), and large (more than 20,000 nodes).
  • Textual attributes: The benchmark preserves original raw text for every hypergraph node, enabling flexible integration of PLMs and LLMs for textual representation learning and feature extraction.This design addresses the semantic loss caused by numerical metadata or preprocessed shallow features.
  • Reproducible pipeline: Standardized hyperedge and text formats, together with a modular pipeline, directly support node classification and hyperedge prediction under fair, reproducible comparisons.These components establish a systematic foundation for text-attributed hypergraph learning research.
  • Benchmark validation: TAHB preserves real-world hypergraph structure, with most datasets showing giant connected components, effective diameters from 2 to approximately 10, and realistic clustering behavior.Clique expansion is used to analyze structural characteristics, and the reported findings collectively support TAHB’s structural realism.
  • Benchmark validation: Text-based features substantially outperform random features for node classification across all datasets, while HRL rankings correlate with existing benchmarks at PCC 0.97 and hyperedge prediction preserves their task trends.Larger PLMs generally improve performance from Small to Medium, whereas DeBERTa-Large does not consistently improve results; TriCL and HypeBoy encounter OOM limitations on large arXiv datasets.

4 Exploring LLM Integration in Hypergraph Learning

This section systematically examines LLM integration with hypergraph representation learning through LLM-as-Predictor and LLM-as-Enhancer settings. Results show that combining textual semantics with hypergraph topology is most effective, while topology alone remains weaker than text-only reasoning.

  • Overview: TAHB enables the first systematic analysis of integrating LLMs with hypergraph representation learning using real-world hypergraph structures and raw textual attributes.The study addresses the limited prior work connecting LLMs with hypergraph learning.
  • LLM-as-Predictor: In the LLM-as-Predictor setting, hyperedge co-membership is converted into textual prompts and evaluated through node classification.Neighboring information from nodes sharing hyperedges is incorporated alongside textual attributes in LLM inputs.
  • LLM-as-Predictor: Topology+Text consistently achieves the best node-classification performance across datasets and ChatGPT-4o, Llama-3.1, and Gemini-2.5.The result indicates that textual semantics and hypergraph topology provide complementary information.
  • LLM-as-Predictor: Text-only consistently outperforms Topology-only, indicating stronger LLM use of textual semantics than complex structural relationships among nodes connected through shared hyperedges.Current LLMs can use high-order topology as complementary information but may remain limited in complex hypergraph structural reasoning.
  • LLM-as-Enhancer: In the LLM-as-Enhancer setting, LLM-generated class predictions and reasoning augment node text before PLM feature extraction and HGNN-based node classification.The augmented and original textual attributes are jointly fed into PLMs; experiments use BERT-Tiny and HGNN.

5 Future Research Directions

TAHB establishes a foundation for future research combining raw textual attributes with real-world hypergraph structures. Its directions span multi-level prediction, text-aware learning, LLM-based structural reasoning, and hypergraph foundation models.

  • Beyond Node-Level Downstream Tasks: LLM-based hyperedge prediction is an important future direction because existing LLM-based graph learning methods primarily target node classification.Their extension to link prediction remains relatively underexplored.
  • Beyond Node-Level Downstream Tasks: TAHB supports unified node-, hyperedge-, and hypergraph-level learning by combining textual semantics with high-order topology.Future tasks include hyperedge prediction, hyperedge classification, group recommendation, and whole-hypergraph classification.
  • LLM-based Hypergraph Reasoning: LLM-based hypergraph reasoning remains challenging because topology-only settings consistently underperform text-only settings.This indicates that existing LLMs struggle to understand and reason over high-order topology induced by hyperedge co-membership.
  • Text-aware Hypergraph Learning: Text-aware hypergraph learning can combine complementary semantic and structural signals through semantic-aware representation learning and text-guided hyperedge modeling.TAHB enables these directions by jointly providing real-world hypergraph structures and raw textual attributes.
  • Hypergraph Foundation Models: Hypergraph foundation-model research remains at an early stage because existing benchmarks rarely pair raw textual attributes with high-order structural information.This limitation makes it difficult to study models that jointly learn semantics and hypergraph structure.

6 Conclusions

TAHB is introduced as the first publicly available benchmark for text-aware hypergraph learning, connecting high-order groupwise relations with raw textual semantics. Validation shows that it preserves real-world hypergraph topology and informative textual distributions, supporting systematic study of LLM integration under complementary paradigms.

  • Benchmark contribution: TAHB is the first publicly available benchmark specifically designed for text-aware hypergraph learning.It bridges high-order groupwise relations and raw textual semantics.
  • Benchmark validation: TAHB preserves fundamental structural topologies of real-world hypergraphs while retaining informative textual distributions.These properties establish a reliable and realistic evaluation environment.
  • LLM integration: The benchmark enables systematic exploration of LLM integration with hypergraph representation learning under LLM-as-Predictor and LLM-as-Enhancer paradigms.The paper studies these complementary settings to evaluate how language models contribute to hypergraph learning.

GenAI Usage Disclosure

The authors disclose selective use of generative AI in accordance with ACM Authorship Policy. The models supported LLM-as-Predictor and LLM-as-Enhancer implementation and limited writing assistance, but not other research stages.

  • Disclosure scope: ChatGPT-4o, Llama-3.1, and Gemini-2.5 were used to implement the LLM-as-Predictor and LLM-as-Enhancer frameworks.These models were part of the disclosed generative AI usage.
  • Disclosure scope: Generative AI provided limited writing assistance during the research.The passage distinguishes this assistance from the models’ framework-implementation role.
  • Disclosure scope: The models were not used for data collection, preprocessing, experimental evaluation, or result analysis.The disclosure excludes these research stages from generative AI use.
Loading 2608.15055v1…