Source-linked AI summary
Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks
Yufeng Zhang, Xueli Yu, Zeyu Cui, Shu Wu, Zhongzhen Wen, Liang Wang
TL;DR
Existing graph-based text classifiers do not capture contextual word relations within each document and struggle with inductive learning for new words. TextING builds individual document graphs and applies GNN-based local word interaction before aggregating word nodes into document embeddings. It outperforms state-of-the-art methods, including with an average gain of 21.06% under the reported inductive condition.
Problem
Existing graph-based methods neglect contextual word relations within documents and require global structures that make inductive learning for new words difficult.
Method
TextING builds an individual graph for each document, uses GNN message passing to learn local word interactions, and aggregates word nodes into a document embedding.
Results
TextING ranks top on all evaluated tasks and achieves an average gain of 21.06% in the reported inductive condition.
Takeaways & Limitations
Document-specific graph structure supports local word-interaction modelling and generalisation to words absent from training.
Abstract
from arXiv · showhide
Text classification is fundamental in natural language processing (NLP), and Graph Neural Networks (GNN) are recently applied in this task. However, the existing graph-based works can neither capture the contextual word relationships within each document nor fulfil the inductive learning of new words. In this work, to overcome such problems, we propose TextING for inductive text classification via GNN. We first build individual graphs for each document and then use GNN to learn the fine-grained word representations based on their local structures, which can also effectively produce embeddings for unseen words in the new document. Finally, the word nodes are aggregated as the document embedding. Extensive experiments on four benchmark datasets show that our method outperforms state-of-the-art text classification methods.
1 Introduction
TextING addresses two limitations of prior graph-based text classification: missing document-specific word relations and difficulty learning inductively for new words. It uses individual document graphs and GNN-based word interactions, and experimentally outperforms state-of-the-art methods.
- Research Gap: Prior graph-based methods neglect contextual word relations within individual documents by using globally fixed or document-global structures.TextGCN constructs one global document-word graph, while other methods use globally fixed word edges despite text-dependent interactions.
- Research Gap: Global graph structures also require test documents during training, making these methods inherently transductive and limiting inductive learning for new documents and words.The paper contrasts this with inductive settings where embeddings are needed for new document structures.
- Approach: TextING trains a GNN on training documents, builds individual document graphs, and generalises detailed word-word relations to test documents.Its graph construction uses a sliding window inside each document, while word information is propagated and aggregated into document representations.
- Contributions: TextING learns text-level word interactions and generalises to words absent from training, supporting inductive circumstances.These properties are stated as two central contributions of the approach.
- Results: TextING experimentally outperforms state-of-the-art text classification methods.The paper presents this as an experimental contribution.
2 Method
TextING represents each document as an individual word graph, learns contextual interactions with gated message passing, and aggregates updated word nodes into a document representation. A multichannel variant combines local and global graph structures in parallel.
- Architecture: TextING comprises graph construction, graph-based word interaction, and a readout function.The architecture is illustrated in Figure 1.
- Graph Construction: Each document graph uses unique words as vertices and undirected edges for word co-occurrences within a fixed-size sliding window, length 3 by default.The graph is denoted G = (V, E), with V as vertices and E as edges.
- Graph Construction: Tokenisation and stopword removal precede vertex initialisation with word features, which are propagated contextually through each document graph.The vertex embeddings are represented as h ∈R|V|×d, where d is the embedding dimension.
- Graph-based Word Interaction: Gated Graph Neural Networks update each word node from adjacent neighbours and its own representation, with stacked layers enabling interactions up to t hops.The update and reset gates determine how much neighbour information contributes to the current node embedding.
- Readout Function: Updated word nodes are aggregated into a graph-level document representation for final prediction.This aggregation forms the model's readout stage.
- Multichannel Variant: TextING-M runs local-structure and global-structure graphs in parallel and combines their separately trained predictions with a 1:1 vote.The global edges are extracted from TextGCN's corpus-level graph for each document; this variant is not inductive.
3 Experiments
Experiments evaluate TextING across benchmark performance, inductive cold-start conditions, attention behavior, and graph-design sensitivity. Results show strong overall accuracy, substantial inductive gains, and sensitivity to interaction depth and graph density.
- Overall performance: TextING ranks first on all four benchmark tasks, with especially strong performance on MR and smaller improvement on R8.The paper attributes MR’s advantage partly to short-document sparsity in TextGCN and the presence of many unseen test words.
- Inductive condition: 21.06% average gain is reported for TextING over TextGCN under the inductive condition with most test words unseen during training.The evaluation uses 20 labelled documents per class, with 40 labelled MR documents and 460 labelled Ohsumed documents in Table 3.
- Case study: Attention weights in MR are visualized for positive and negative reviews, with highlighted words proportional to their weights and positively correlated with labels.The visualization is used to interpret the readout function in sentiment analysis.
- Parameter sensitivity: Increasing interaction steps initially improves representations through higher-order neighbors, but excessive depth causes over-smoothing; increasing graph density shows a similar trend.The sensitivity analyses vary graph layers on MR and Ohsumed and window size on MR and Ohsumed.
4 Conclusion
TextING is presented as a graph-based method for inductive text classification in which each text owns a structural graph. The conclusion emphasizes learning text-level word interactions and modelling local relations and word significance.
- TextING assigns each text its own structural graph for inductive text classification.
- The method learns text-level word interactions within each document.
- Experiments support modelling local word-word relations and word significance in the text.