Source-linked AI summary
TACO-Graph: Automated Construction of Text-Atributed Graphs
Amiri Hayes, Kristina Wicke
TL;DR
Choosing nodes, edges, and node text can substantially affect downstream graph performance, but systematic guidance for task-specific text-attributed graph construction is limited. TACO-Graph enumerates and scores constructions on small representative subsets, then fits interpretable decision trees. Across five datasets, its trees predict construction quality on separately drawn data, with mean Spearman ρ above 0.9 for node classification and above 0.8 for GraphRAG retrieval.
Problem
Text-attributed graph construction choices can substantially affect downstream performance, yet systematic guidance for selecting nodes, edges, and node text for a given dataset and task is lacking.
Source: §1 · p. 1Method
Results
Mean Spearman ρ exceeds 0.9 for node classification and 0.8 for GraphRAG retrieval across five datasets, predicting construction quality on separately drawn evaluation data.
Source: §5 · p. 9Takeaways & Limitations
TACO-Graph provides a systematic, interpretable, and transferable framework for selecting text-attributed graph constructions for downstream tasks.
Source: §5 · p. 9GraphRAG rankings require the full 300-question budget, raising whole-space evaluation costs to 8.0×–27.5× full-scale runs without significant runtime savings.
Source: §4.3 · p. 9
Abstract
from arXiv · showhide
Graph-structured representations of text data have become foundational across machine learning, yet it remains unclear how to systematically construct a graph from a given dataset for a target task. Choices about which entities become nodes, which relationships become edges, and what text each node contains can substantially affect downstream performance. In practice, finding optimal constructions often requires expert knowledge or manual trial and error. We introduce TACO-Graph, a decision tree framework for learning which graph construction choices are most effective for a given dataset and task. Specifically, we focus on text-attributed graphs (TAGs) constructed from multi-attribute tabular data. TACO-Graph enumerates valid TAG representations, scores each using fast proxy evaluation on small representative subsets, and trains an interpretable decision tree on those scores. Across five datasets spanning citation networks, product reviews, and bibliographic relationships, our decision trees predict construction quality on separately drawn evaluation samples with mean Spearman $\rho$ above $0.9$ for node classification and above $0.8$ for GraphRAG retrieval. Our trees systematically identify which constructions perform better across both GNN and GraphRAG tasks, and the full pipeline is released as an open-source library to support evaluation on new tasks and datasets at https://github.com/AmiriHayes/text_attributed_graphs. TACO-Graph provides a systematic and interpretable guide for determining how to convert tabular data to text-attributed graphs for downstream tasks.
1 INTRODUCTION
TACO-Graph addresses the lack of systematic, task-conditioned guidance for choosing text-attributed graph constructions. It uses small-subset evaluations and interpretable decision trees to predict which constructions work better across GNN classification and GraphRAG retrieval.
1 INTRODUCTION
- Existing TAG methods typically treat nodes, edges, and textual attributes as fixed, despite construction choices substantially affecting downstream performance.The same product-review data can be represented using product or user nodes and different relationships.Source: §1 · p. 1
- TACO-Graph enumerates valid construction choices, evaluates them on representative subsets, and trains interpretable decision trees to rank constructions without full-dataset evaluation.The pipeline characterizes dataset columns, samples data, and validates tree rankings on separately drawn data.Source: §1 · p. 1, Figure 1 · p. 2
- Trees trained on small subsets predict construction quality on separately drawn data for both GNN classification and GraphRAG retrieval, though optimal constructions differ by task.The study spans five datasets across three domains.Source: §1 · p. 2
2 RELATED WORK
Prior TAG learning work generally takes graph entities, relationships, and text as given, while newer relational construction approaches address only parts of the problem. TACO-Graph frames these choices as task-conditioned variables and uses an interpretable predictive model to identify effective combinations.
2.1 LEARNING FROM TEXT-ATTRIBUTED GRAPHS
- TAG learning methods typically treat node entities, edge relationships, and textual attributes as fixed before learning begins.Prior work has studied TAG features, benchmarks, GNNs, and language-model integration, but the graph itself is given.Source: §2.1 · p. 2
- AutoG proposes relational schema transformations, while AutoGrable fixes rows as nodes, omits text, and screens rather than ranks constructions for downstream performance.TACO-Graph instead optimizes node entities, edge relationships, and textual attributes for a dataset and task.Source: §2.1 · p. 3
2.2 CONSTRUCTION OF TEXT-ATTRIBUTED GRAPHS
- Surrogate-model search is established in architecture and AutoML settings, but existing approaches are not designed to determine how tabular data should become a text-attributed graph.TACO-Graph evaluates sampled categorical configurations and fits an interpretable predictive model to identify high-performing constructions.Source: §2.2 · p. 3
- GraphRAG systems typically use fixed or manually designed graph-construction procedures, leaving alternative TAG constructions largely unexplored.The paper investigates whether small-subset evaluation can identify effective constructions before full-scale GraphRAG construction.Source: §2.2 · p. 3
3 METHODOLOGY
The methodology defines a task-conditioned space of text-attributed graph constructions, scores valid variants on sampled data, and fits decision trees to predict their performance. It evaluates GNN node classification and GraphRAG retrieval separately and tests whether learned rankings generalize to independently drawn data.
3 METHODOLOGY
- For a target task, the construction space combines choices of node entities, edge relationships, and node text into configurations in ΩM = N × E × T.The dataset is mapped to a generalized schema and sampled for evaluation.Source: §3 · pp. 3–4
3.1 DEFINING TAG CONSTRUCTION SPACE
- The node axis distinguishes primary, secondary, and aggregate entities; edges use observed relationships or similarity, while text fidelity ranges from full context to no text.The search space uses representative parameterizations and prunes invalid supervision, degenerate topology, label leakage, and empirically degenerate variants.Source: §3.1 · p. 4
3.2 GRAPH SCORING AND EVALUATION
- The study evaluates GNN node classification and GraphRAG question answering across five tabular datasets, recording scores for each dataset, subset, and valid construction variant.The resulting construction-performance tables provide targets for decision-tree training.Source: §3.2 · p. 4
3.2.1 GNN SCORING FOR NODE CLASSIFICATION
- For GNN scoring, each dataset contributes 150 random 1,000-row subsets, and variants are scored using floored McFadden pseudo-R2 on separately drawn test nodes.The fixed GNN architecture and hyperparameters keep performance differences attributable to construction choices; the score is calibration-sensitive and does not rank variants below the class prior against one another.Source: §3.2.1 · p. 5
3.2.2 RAGAS SCORING FOR RETRIEVAL AUGMENTED GENERATION
- For GraphRAG, variants are scored with the mean RAGAS faithfulness, answer-relevance, and context-relevance measures across evaluation questions.Questions pass a faithfulness threshold before inclusion, and retrieved nodes are expanded along each variant’s edges to form answering context.Source: §3.2.2 · p. 5
3.3 DECISION TREE TRAINING AND VALIDATION
- Decision trees map categorical construction choices to per-variant mean training-pool scores, with Spearman correlation assessing ranking agreement and tree edit distance assessing structural agreement on test pools.The pools are independently drawn rather than a strict partition; the study also reports a separate validation using strictly disjoint pools.Source: §3.3 · p. 5
4 RESULTS
Construction choices produce substantial downstream performance differences, and small-sample decision trees predict construction quality across GNN classification and GraphRAG retrieval. Rankings are stable at small budgets for node classification, while retrieval rankings sharpen as the question budget grows.
4 RESULTS
- The results establish that construction choices matter, that small-subset evaluations support quality prediction for both tasks, and that ranking stability improves with larger budgets.Node-classification rankings are stable at the smallest budgets, while retrieval rankings sharpen as the question budget grows.Source: §4 · p. 5, Figure 4 · p. 8
4.1 CONSTRUCTION PERFORMANCE
- Alternative constructions yield substantial performance differences; in Sports, reviewer-centric N2 graphs plateau near 30% for classification while N1 and N3 exceed 50% at 100 epochs.N1 and N3 remain statistically indistinguishable in retrieval, where the gap narrows; results persist across independent samples, and winning constructions also shift across datasets.Source: Figure 2 · p. 6, §4.1 · p. 7
4.2 DECISION TREE META-LEARNER
- The depth-3 trees avoid N2 at the root for both tasks but diverge below it, favoring contextual text for classification and edge semantics for retrieval.For Sports classification, N1/E5a/T6a scores 41.3; the retrieval tree selects E5a, while edge choices vary little among top N1/T6a variants.Source: Figure 3 · p. 6, §4.2 · p. 7
4.3 DECISION TREE VALIDATION
- Mean Spearman ρ exceeds 0.9 for node classification and 0.8 for GraphRAG retrieval across five datasets, with rankings validated on separately drawn data.Within-dataset diagonal correlations exceed 0.90 for GNNs and 0.80 for both tasks; GNN within-dataset tree edit distance averages 0.11 versus 0.46 across datasets.Source: §4 · p. 5, §4.3 · pp. 7–8, Table 3 · p. 7
- GNN rankings stabilize with a small handful of subsets, whereas GraphRAG rankings require the full 300-question budget and do not plateau early.Across datasets, GNN search costs 2.5×–9.3× a full-scale run; GraphRAG whole-space evaluation costs 8.0×–27.5×, providing structured coverage rather than significant runtime savings.Source: §4.3 · pp. 8–9, Figure 4 · p. 8, Table 4 · p. 8
5 DISCUSSION AND CONCLUSION
TACO-Graph finds construction rankings predictable from small samples, while their usefulness varies with the task–dataset score spread. The conclusion frames it as a systematic, interpretable framework and outlines extensions to broader graph and task settings.
5 DISCUSSION AND CONCLUSION
- Mean Spearman ρ > 0.9 for node classification and ρ > 0.8 for GraphRAG retrieval across five datasets; alternative constructions yield substantially different performance.The trees predict construction quality from small representative samples on separately drawn data.Source: §5 · p. 9
- Tree utility depends on task–dataset score spread, not score level: History node-classification variants span 6.3 scoring points, while ArXiv spans 46 points.A small spread leaves little benefit from systematic search; a meaningful spread is needed for decision trees to learn useful rules.Source: §5 · p. 9
- History’s GraphRAG scores span 22.0 points and ArXiv’s retrieval range is 9 points; composite RAGAS aggregation provides the most robust ranking signal.The discriminative power of individual RAGAS axes varies, with context relevance highest and answer relevance lowest.Source: §5 · p. 9
- Future work includes broader GNN tasks and non-tabular corpora, cheaper structural proxies, richer multi-table and semantic-edge construction spaces, and knowledge graphs.For knowledge graphs, the construction choices would concern extraction rather than schema columns.Source: §5 · p. 9
REPRODUCIBILITY STATEMENT
The statement identifies where code, configuration files, and trained trees are available, and discloses generative AI uses and author verification.
REPRODUCIBILITY STATEMENT
- Code, dataset configuration files, and trained decision-tree artifacts are available in the project’s open-source repository.Section 3 gives the sampling procedure and scoring formula; Appendix A gives dataset-specific notes.Source: REPRODUCIBILITY STATEMENT · p. 9
AI USE STATEMENT
- GPT-4o-mini generated evaluation questions and served as the generator and judge for RAGAS scoring.Source: AI USE STATEMENT · p. 10
- Claude and ChatGPT assisted with code generation, literature search, and writing; authors verified all code, checked empirical claims, and take responsibility.Source: AI USE STATEMENT · p. 10
A DATASET-SPECIFIC NOTES
Dataset-specific results show that construction rankings were consistent across datasets, while decision-tree feature preferences varied by task. The evaluation and its practical scope are bounded by sampled data and manual schema choices.
A DATASET-SPECIFIC NOTES
- Construction choices vary across node text and edge rules: T6a adds context, T6b uses primary content, and T6e uses zero-vector features; graphs above 200,000 edges are excluded.E4a links shared aggregate identifiers; E4b/E4c use shared participation identifiers; E5a uses cosine similarity 0.75; E5b links the five closest nodes in degree centrality. Node classification uses floored McFadden pseudo-R2.Source: A DATASET-SPECIFIC NOTES · p. 12
- GraphRAG retrieves three passages by dense similarity, expands one hop along graph edges, and caps retrieval at fifteen passages.Question generation uses three subtypes of 100; subtypes with pilot faithfulness below 0.5 are dropped in full, leaving History with 233 questions.Source: A DATASET-SPECIFIC NOTES · p. 12
- Ranking consistency (node-classification/GraphRAG ρ) was ArXiv 0.958/0.840, Sports 0.939/0.814, History 0.945/0.976, Electronics 0.957/0.805, and Toys 0.873/0.941.Figure 5 displays construction performance for the four datasets not shown in the main body, with datasets as rows and tasks as columns.Source: A DATASET-SPECIFIC NOTES · pp. 12–14, Figure 5 · p. 13
- Classification trees emphasized node type in ArXiv (88%), Sports and Electronics (all importance), and Toys (96%), while History split first on edge type.For retrieval, ArXiv assigned 80% importance to edge type; Sports split importance between edge type (62%) and node type (38%); Electronics and Toys assigned 77% and 80% to edge type, respectively.Source: A DATASET-SPECIFIC NOTES · pp. 12–14
- Classification scores ranged 38.3–84.6 on ArXiv, 9.9–42.1 Sports, 31.0–37.3 History, 6.2–56.7 Electronics, and 1.7–12.5 Toys.History's GNN learned poorly; Toys had an 86.9% majority class, while Electronics had an 81.5% majority class and 74.4% dense ground-truth edges.Source: A DATASET-SPECIFIC NOTES · pp. 12–14
- Variant spaces contained 16 ArXiv variants, 26 each for Sports, Electronics, and Toys, and eight for History, where author and reviewer columns were unavailable.The dataset-specific variant counts exclude the no-text control and empirically degenerate variants removed by the filter.Source: A DATASET-SPECIFIC NOTES · pp. 12–13
- Scores come from 1,000-row subsets drawn from sampled 20k–50k-row pools, so claims concern construction ordering in that regime, not portable gap magnitudes.Attribute-matching edges grow denser with pool size, unlike k-nearest-neighbour rules; absolute score ranges are proxy properties.Source: A DATASET-SPECIFIC NOTES · p. 14
- Practitioners provide a schema column mapping, label space, and valid-variant rules; diagnostic checks flag issues but do not correct inputs or gate execution.Without adjustments, a dataset may produce a flat score spread; scalar and semantic edge sparsity checks require a full build after embeddings exist.Source: A DATASET-SPECIFIC NOTES · p. 14
B DECISION TREE VISUALIZATIONS
The appendix shows decision trees for four datasets across node classification and GraphRAG retrieval, revealing dataset- and task-specific construction rules.
B DECISION TREE VISUALIZATIONS
- The Figure 6 trees are capped at depth three for readability, and their in-sample R2 illustrates learned rules rather than reproducing Table 3's depth-eight trees.The no-text control is excluded because a tree rooted on the control status would encode no construction rule.Source: B DECISION TREE VISUALIZATIONS · p. 14
- Electronics and Toys isolate N2 in both tasks, ArXiv tasks differ at the root, and History splits on different edge types.The penalty for choosing N2 is described as severe for classification but mild for retrieval; History has one node type and no node axis to split on.Source: B DECISION TREE VISUALIZATIONS · p. 14
- Figure 6 shows a node-classification tree and a GraphRAG retrieval tree for each of the four datasets not shown in Figure 3, scaled to 0–100.The trees are fit on train-pool variant means, using the floored McFadden score for classification and the RAGAS composite for retrieval.Source: B DECISION TREE VISUALIZATIONS · p. 14, Figure 6 · p. 15