PaperlayerView exact revision on arXiv

Source-linked AI summary

RelICL: Training-free Relational Learning with Tabular Foundation Models

Simon Forbat, Rainer Gemulla

arXiv:2610.01725v1cs.LG

TL;DR

Deep feature synthesis lets tabular foundation models use relational context, but its growing feature sets and single-column aggregates limit scalability and interaction modeling. RelICL propagates and fuses row embeddings through the schema using a TFM, and its RelBench results were on par with the strongest DFS-based approach.

  • Problem

    DFS-based relational learning faces feature explosion as schemas grow and interaction blindness because its aggregates are column-wise.

    Source: §1 · p. 2, §2.4 · p. 4

  • Method

    RelICL uses a TFM to extract row embeddings, propagate and fuse them through the schema, and make predictions from an extended task table.

    Source: §1 · p. 2, §3.2 · p. 7

  • Results

    On RelBench, RelICL was on par with the strongest DFS-based approaches and outperformed pretrained relational approaches.

    Source: §4 · p. 7

  • Takeaways & Limitations

    The underlying TFM used by RelICL can be easily swapped.

    Source: §1 · p. 2

    The experiments considered only RelBench classification and regression tasks, and unreproduced baseline protocols may differ enough that Tab. 4 wins may not be statistically significant.

    Source: RelICL (TabPFN) · p. 9

Abstract

from arXiv · show

Tabular foundation models achieve state-of-the-art performance on single-table tasks without any training. Recent work suggests that they are also well-suited for relational learning via deep feature synthesis (DFS), which flattens a relational schema into a single table by adding aggregates of the other tables' columns as features. This approach is appealing because it directly benefits from improvements to or customization of the underlying tabular foundation model. In this paper, we identify two key problems with DFS: feature explosion and interaction blindness. The first problem arises because the number of DFS features grows quickly as the schema becomes more complex, limiting scalability and performance. The second problem arises because column-wise aggregates do not account for feature interactions, limiting performance. We propose and explore an alternative method termed RelICL, which keeps the benefits of DFS but alleviates these two problems. At its heart, RelICL propagates and fuses information step by step through the schema graph, using the same tabular foundation model that is eventually used for prediction to do so. In our experimental study using RelBench tasks, RelICL was on par with the strongest approach based on deep feature synthesis.

1 INTRODUCTION

The paper asks how tabular foundation models can use information across relational tables while retaining training-free prediction. It proposes RelICL to address deep feature synthesis's scalability and interaction limitations, and reports results on par with the strongest DFS-based approach.

1 INTRODUCTION

  • Tabular foundation models perform state-of-the-art single-table tasks without further training, motivating their use for relational learning.DFS is one existing route: it adds context-table column aggregates to the task table before prediction.Source: §1 · p. 1
  • DFS feature counts grow with schema complexity, limiting scalability, while single-column aggregates cannot capture interactions between context-table columns.The introduction identifies both as limitations of DFS for relational learning.Source: §1 · p. 2
  • RelICL uses a TFM for feature extraction and step-by-step fusion of row embeddings, then feeds an extended task table to a TFM for prediction.Its additional feature count depends on the local neighborhood rather than the full database.Source: §1 · p. 2
  • Across 21 RelBench entity tasks, RelICL was on par with the strongest DFS-based approach and outperformed many alternative approaches; its underlying TFM could be swapped.The study also reports that the TFM used by RelICL can be easily swapped.Source: §1 · p. 2

2 DEEP FEATURE SYNTHESIS AND ITS LIMITATIONS

The section defines relational prediction with a task table and connected context tables, then describes DFS as aggregating context-table columns along meta-paths. It identifies feature explosion and interaction blindness as DFS limitations, illustrated by tasks where depth-bounded or single-column aggregates miss the signal.

2.1 PRELIMINARIES

  • A relational prediction task uses a task table with training and query rows plus context tables linked by key references; this paper's setting uses a single pretrained TFM.For relational deep learning, the section describes temporal filtering and pooling to avoid using future rows.Source: §2.1 · pp. 2–3

2.2 DEEP FEATURE SYNTHESIS

  • DFS extends the task table with aggregates of context-table columns reached along meta-paths, repeating the process for each path.A meta-path is a sequence of linked tables starting at the task table; its endpoint rows' columns are aggregated individually.Source: §2.2 · p. 3

2.3 FEATURE EXPLOSION

  • DFS feature count grows with schema complexity, and every fixed depth bound leaves some tasks no better than random guessing when the signal lies farther away.Post-hoc feature selection still requires computing all DFS features and may remove relevant information.Source: §2.3 · pp. 3–4
  • 50.8% AUROC for depth-2 RDBLearn versus 100.0% for RelICL with either TabICL or TabPFN on the length-3 signal task.DFS feature-table width grows as O(#meta-paths ⋅ #attributes ⋅ #aggregates); the rel-f1 database has nine tables and 67 columns, while RDBLearn generates 7,715 columns before pruning on driver-top3.Source: §2.3 · pp. 3–4, Table 1 · p. 4

2.4 INTERACTION BLINDNESS

  • Single-column aggregates make DFS blind to interactions; when the signal lies only in the relationship among context-row values, DFS cannot distinguish the cases, unlike RelICL.The illustrated task gives each case the same values in each column but different correlations; feature engineering can be difficult and may cause further feature explosion.Source: §2.4 · p. 4, Table 2 · p. 4

3 RELICL

RelICL uses a tabular foundation model (TFM) both to extract and fuse context-table row embeddings and to predict on the resulting extended task table. Its design relies on row embeddings as features and uses outward and inward traversals to propagate relevant information through the schema.

3 RELICL

  • RelICL uses the TFM for prediction and for extracting and fusing context-table row embeddings into an extended task table.The number of added features depends on directly referenced tables, while row embeddings capture feature interactions.Source: §3 · pp. 4–5, §3.2 · p. 7

3.1 ROW EMBEDDINGS ARE GOOD FEATURES

  • RelICL assumes that replacing table features with TFM row embeddings preserves model performance; on all but philippine and jasmine, TabICL showed no statistically significant loss.The comparison covered ten single-table binary classification datasets from OpenML.Source: §3.1 · pp. 5–6

3.2 FEATURE EXTRACTION AND FEATURE FUSION

  • An outward traversal marks relevant context rows by propagating keys from the task table, then an inward topological traversal fuses child information toward the task table.Pruning irrelevant rows reduces the rows processed by tabular models, whose attention cost is asymptotically quadratic in row count.Source: §3.2 · pp. 6–7
  • Encoding each table row produces embeddings whose size is independent of table width and whose elements depend on the entire row.These properties support fixed-size propagation and preserve information about feature interactions.Source: §3.2 · p. 7
  • For each child table, RelICL pools the embeddings of rows referenced by each parent row and adds an explicit count of referenced rows.Pooling can use sum, average, or max.Source: §3.2 · p. 7
  • RelICL predicts by applying the full tabular model to the extended task table, whose feature count depends on the task table’s children rather than the database’s total number of tables.It uses no depth bound to avoid feature explosion.Source: §3.2 · p. 7

4 EXPERIMENTS

Across RelBench experiments, RelICL was on par with the strongest DFS-based approaches and outperformed relationally pretrained approaches. Results with TabPFN also indicate that RelICL was not tied to the TabICL backbone.

4.1 EXPERIMENTAL SETUP

  • The evaluation covered 21 RelBench v2 entity tasks across seven databases, including classification and regression.The databases ranged from under 100 k to over 44 M rows.Source: §4.1 · p. 7

4.2 RESULTS

  • RelICL either outperformed or matched all compared models, with TabPFN-Rel the closest DFS-based competitor.The comparison counts head-to-head wins only on tasks and metrics shared by each model pair.Source: §4 · p. 7, Table 4 · p. 8, §4.2 · p. 8

RelICL (TabPFN)

  • RelICL outperformed RT and PluRel without relational pretraining, mostly on classification, despite not being tuned for their R2 metric.The comparison is reported for the TabPFN-backed RelICL results.Source: RelICL (TabPFN) · p. 9
  • With TabPFN v3, TabICL v2 led on most tasks of both types, but the classification margin was small.The authors conclude that RelICL did not depend specifically on TabICL.Source: RelICL (TabPFN) · p. 9, Table 6, Table 8
  • The study covers only RelBench classification and regression, and baseline numbers from prior papers may differ in tuning, seeds, and evaluation protocol.The authors note that some apparent wins may not be statistically significant.Source: RelICL (TabPFN) · p. 9

5 CONCLUSION

RelICL uses a tabular foundation model for feature extraction, fusion, and final prediction, and the paper reports strong empirical performance without DFS’s two identified limitations. The related-work discussion distinguishes approaches trained on relational data from the paper’s training-free setting and situates its benchmark choice.

5 CONCLUSION

  • RelICL uses a tabular foundation model for feature extraction, feature fusion, and final prediction, and the authors report strong empirical performance.They characterize it as training-free and as avoiding feature explosion and interaction blindness.Source: §5 · p. 10

A EXTENDED RELATED WORK

  • Relational Transformers and PluRel are examples of relational foundation models trained on relational data, unlike RelICL’s frozen-backbone, no-training approach.Earlier schema-message-passing approaches used a tabular transformer trained end-to-end per database.Source: A EXTENDED RELATED WORK · p. 15
  • RelBench pairs seven databases with 30 tasks and provides splits and an evaluation protocol; the paper uses it because its comparison baselines report results on it.RelBench tasks are evaluated at a cutoff time so models see only data available at prediction time.Source: A EXTENDED RELATED WORK · p. 15

B.1 PROP. 1: INSUFFICIENCY OF BOUNDED DEPTH

The bounded-depth proposition shows that a target signal beyond the depth limit can be independent of all reachable features. The appendix also describes a DFS case where aggregates collapse distinct interaction patterns, leaving the predictor to guess.

B.1 PROP. 1: INSUFFICIENCY OF BOUNDED DEPTH

  • With one-to-one references and a random target bit beyond depth d, every feature following at most d references depends only on keys independent of the target.Thus bounded-depth features cannot recover that target signal.Source: §B.1 · p. 15
  • In the three-table-chain construction, the label bit sits three references from the task table, beyond the d = 2 bound.The task uses balanced, disjoint train and test rows, with 1,000 training and 300 test rows.Source: §B.1 · p. 15, Figure 2 · p. 16

B.2 PROP. 2: INSUFFICIENCY OF DFS FEATURES

  • In the DFS interaction example, all nine generated aggregates were constant and discarded, leaving RDBLearn only the join key and random-guessing labels.The two classes differ in which b value is paired with each a value, not in either column’s set of values.Source: §B.2 · p. 16, Figure 3 · p. 16

C METHOD DETAILS

RelICL rewrites the schema before fusion by removing tables. Its rewrites inline leaf-table data and handle junction tables by creating two copies that pass neighboring attributes in one step.

C.1 SCHEMA REWRITES

  • A schema rewrite removes tables before fusion, providing a view on the database.Source: §C.1 · p. 16
  • RelICL inlines leaf tables into each referencing table, adding columns without adding rows; entity tasks inline the entity table into the task table.Source: §C.1 · p. 16
  • For a junction table referencing two tables, RelICL creates two copies, each carrying the other neighbor’s columns so pooling passes those attributes in one step.This follows RelGNN’s single-step message passing between the junction table’s neighbors.Source: §C.1 · p. 16

C.2 ASM. 1: ROW EMBEDDINGS ARE GOOD FEATURES

The appendix evaluates replacing single-table feature columns with row embeddings. On ten OpenML datasets, most replacements incurred little or no measurable AUROC loss, though two datasets were outliers.

C.2 ASM. 1: ROW EMBEDDINGS ARE GOOD FEATURES

  • The appendix identifies this experiment as support for Assumption 1.Source: §C.2 · p. 16
  • The comparison used ten single-table OpenML binary-classification datasets, with three seeds determining stratified 80/20 splits and model randomness.It compared TabICL on the original table with replacement by 512-dimensional embeddings and by embeddings reduced to 32 dimensions.Source: §C.2 · p. 17, Table 9 · p. 17
  • Eight of ten datasets lost at most 1.5 AUROC points after replacing feature columns with row embeddings; four showed no measurable cost.Philippine lost 8.9 points and jasmine lost 3.0; bioresponse, the widest table at 1776 columns, lost nothing measurable.Source: §C.2 · p. 17

D.1 DATASET AND TASK STATISTICS

The appendix documents the RelBench task scope, published-baseline comparisons, hyperparameter search, and configuration ensembling. It reports methodological details and comparison boundaries rather than new performance outcomes.

D.1 DATASET AND TASK STATISTICS

D.2 BASELINES

  • Comparisons with published models are rough because the papers differ in tuning, seeds, and evaluation protocol.The appendix compares RelICL with collected published regression and classification results.Source: §D.2 · p. 17, Table 16 · p. 20

D.3 HYPERPARAMETER OPTIMIZATION

  • Hyperparameter search ran on six classification and six regression tasks, selecting each setting by its mean validation rank across trials and tasks that used that value.The other nine entity tasks were not used to choose defaults.Source: §D.3 · p. 19
  • The search considered 11 hyperparameters and selected the reported configuration; target transformation was the exception, fixed to none for classification.The search consumed 121 GPU-hours across 1200 completed trials.Source: §D.3 · p. 19, Table 15 · p. 20

D.4 CONFIGURATION ENSEMBLING

  • Reported results ensemble the searched default with fixed variants shared across tasks of each type: two members for regression and four for classification.Variant selection was joint across tasks of the same type, not individualized per task.Source: §D.4 · p. 19
  • The regression variant adds windowed counts; classification variants remove reference-count features, disable schema rewrites, or reduce each backbone’s embeddings separately.Source: §D.4 · pp. 19–20

E ADDITIONAL RESULTS

The appendix extends model comparisons to all collected published RelBench results and reports pairwise head-to-head win counts. Regression comparisons use different metrics depending on which models report them, and ties count for neither model.

E.1 ALL PUBLISHED MODELS

  • Tables 16 and 17 compare RelICL with every model for which published RelBench results were collected; Table 17 covers classification, measured by AUROC (%), where higher is better.Source: §E.1 · p. 20, Table 17 · p. 21

E.2 HEAD-TO-HEAD WINS

  • Head-to-head win tables compare regression by MAE when both models report it and otherwise by R2; classification comparisons use AUROC.An asterisk marks cells where R2 decided at least one regression task.Source: §E.2 · p. 20, Table 18 · p. 21, Table 19 · p. 21
  • A tie counts as a win for neither model, so paired win counts can sum to fewer than the shared tasks.The appendix provides pairwise tables for selected baselines and all published models.Source: §E.2 · p. 21, Table 20 · p. 22, Table 21 · p. 22
Loading 2610.01725v1…