PaperlayerView exact revision on arXiv

Source-linked AI summary

InRTL: Effective Intra-Inter Interaction Learning for Relational Tables

Weichen Li, Ken Zhong, Zheng Wang, Li Pan, Jianhua Li

arXiv:2609.12712v1cs.LGcs.AI

TL;DR

Relational table learning lacks a principled framework for modeling dependencies across PK-FK-connected tables while preserving within-table structure. InRTL explicitly learns intra- and inter-table interactions with attention-based modules and scalable simplifications, and experiments report consistent superiority over state-of-the-art baselines across diverse relational tasks.

  • Problem

    Relational table learning remains underexplored in principled modeling frameworks for dependencies within and across PK-FK-connected tables.

  • Method

    InRTL uses a column-aware encoder, Transformer self-attention and cross-attention, linearized attention, and HGNNs to model and scale intra- and inter-table learning.

  • Results

    InRTL consistently outperforms state-of-the-art baselines across diverse relational table tasks.

  • Takeaways & Limitations

    Explicitly modeling intra-table and inter-table interactions provides a unified approach for relational table learning across diverse tasks.

  • Takeaways & Limitations

    The evaluation focuses on well-curated relational databases with explicit table relationships, while open heterogeneous data lakes remain challenging because inter-table relationships can be weak and noisy.

Abstract

from arXiv · show

Relational table learning has recently emerged as an important research direction for modeling multiple tables connected through primary key-foreign key (PK-FK) relationships. Despite recent advances, a principled modeling framework tailored to this task remains underexplored. In this paper, we propose Intra-Inter Relational Table Learning (InRTL), a unified framework that explicitly models dependencies both within and across relational tables. Specifically, InRTL formalizes two complementary interaction patterns: intra-table interactions, describing associations among rows within the same table, and inter-table interactions, describing dependencies between rows across PK-FK-linked tables. To model these dependencies, we develop a column-aware table encoder to generate initial row representations, followed by Transformer-based self-attention and cross-attention modules for intra-table and inter-table learning, respectively. To further improve scalability, InRTL incorporates linearized attention and heterogeneous graph neural networks to simplify the self-attention and cross-attention operations. Extensive experiments on ten datasets covering 24 real-world tasks demonstrate the effectiveness of our approach. Code is available at https://github.com/W1nterFloW/InRTL.

1 Introduction

Relational table learning addresses the limitations of flattening PK-FK-connected tables by explicitly modeling dependencies within and across tables. InRTL combines attention-based interaction modeling with scalable simplifications and is evaluated broadly across relational tasks.

  • Flattening multiple tables through manual feature engineering is labor-intensive, sacrifices relational structure, and can create performance bottlenecks.
  • Existing relational methods independently encode tables with TNNs and then use GNN message passing to model inter-table relationships, but lack a principled relational framework.
  • InRTL explicitly models intra-table row dependencies and inter-table dependencies across PK-FK-linked tables using self-attention and cross-attention.
  • Linearized attention and HGNNs simplify self-attention and cross-attention, enabling scaling with table rows and inter-table PK-FK relations.
  • InRTL connects linear attention with HGNN message passing while avoiding positional encodings and auxiliary training objectives.
  • Experiments cover ten datasets and 24 real-world tasks to validate InRTL's effectiveness.

2 Preliminaries

The preliminaries define relational tables through PK-FK-linked table collections and distinguish row associations within a table from those across linked tables. They also introduce HGNN and Transformer components used to represent heterogeneous relational structures and sequence features.

  • 2.1 Problem Definition: Relational table data comprises a set of tables and PK-FK relations, where foreign keys reference primary keys that uniquely identify rows.
  • 2.1 Problem Definition: Intra-table interaction denotes explicit or implicit contextual and structural associations between different rows in the same table.
  • 2.1 Problem Definition: Inter-table interaction denotes row-level associations induced by PK-FK links between tables and provides complementary contextual information.
  • 2.2 Heterogeneous Graph Neural Networks: An HGNN computes node representations by aggregating neighboring features separately by edge type and updating nodes according to node type.
  • 2.3 Transformer Architecture: A Transformer projects input features into query, key, and value matrices before computing attention, with the formulation extendable to cross-attention.

3 Methodology

InRTL encodes relational tables column-wise, models within-table and PK-FK-linked cross-table interactions, and combines both representations for prediction. It improves scalability by replacing quadratic Transformer operations with linearized attention and heterogeneous graph message passing.

  • 3.1 Column-aware Table Encoder (ColATE): InRTL first encodes each table into row representations, then applies self-attention for intra-table context and masked cross-attention for PK-FK-linked inter-table dependencies.The column-aware encoder weights columns before residual fusion; cross-attention masks unrelated row pairs.
  • 3.1 Column-aware Table Encoder (ColATE): Column-aware encoding embeds cells, computes Softmax-normalized column importance weights, and uses a residual network to capture higher-order column interactions.The resulting aggregated representation E has shape R^(n×d).
  • 3.2 Modeling Intra- and Inter-table Interactions with Transformer: The intra- and inter-table representations are fused with a learnable weighting coefficient before an MLP produces the final target-table prediction.The framework is optimized end-to-end with task-specific loss functions.
  • 3.3 Simplifying Transformer for Table Interactions: Classical self- and cross-attention require O(n^2) pairwise computation, motivating simplification strategies for large relational tables.The paper introduces linearized attention for self-attention and HGNN-based message passing for cross-attention.
  • 3.3.1 Intra-table Interaction Simplification: Linearized attention uses a first-order Taylor approximation whose error is second-order in query-key dot products and is typically negligible for normalized high-dimensional vectors.The approximation yields a linear-complexity self-attention formulation.
  • 3.3.2 Inter-table Interaction Simplification: HGNN message passing is equivalent to inter-table cross-attention on a PK-FK bipartite graph, with attention weights acting as edge weights and distinct relations represented as edge types.This sparse reformulation avoids computations between rows without PK-FK relationships.

4 Algorithm Analysis

This section analyzes InRTL’s computational efficiency and contrasts its deterministic Taylor linearization with prior kernel-based and low-rank attention methods.

  • 4.1 Time Complexity Analysis: O(nmd^2 + nd^2 + ed) is InRTL’s overall time complexity, scaling linearly with table rows and PK-FK-induced edges when n,e ≫ m,d.The three terms arise from the column-aware encoder, linearized intra-table attention, and HGNN message passing, respectively.
  • 4.2 Relation to Prior Linear Attention Methods: Unlike Linear Transformer’s empirically chosen kernel mapping, InRTL derives its formulation from a first-order Taylor expansion of the Softmax exponential term.The resulting formulation is deterministic, interpretable, and requires no additional hyper-parameters.
  • 4.2 Relation to Prior Linear Attention Methods: Unlike Performer, InRTL avoids random features and their Monte Carlo variance through deterministic first-order Taylor linearization.Both approaches target linear complexity, but InRTL does not approximate the Softmax kernel by averaging sampled Gaussian features.
  • 4.2 Relation to Prior Linear Attention Methods: Taylor-expanded linear attention reaches O(nd^2) without a low-rank assumption, preserving full-rank pairwise interactions while remaining comparable in efficiency to Linformer.The method directly linearizes the exponential kernel rather than reducing attention through a low-rank projection.
  • 4.2 Relation to Prior Linear Attention Methods: Prior linear attention methods may introduce uncertainty through heuristic kernels, stochastic random features, or structural low-rank constraints, whereas InRTL emphasizes stability and theoretical completeness.This comparison frames InRTL’s deterministic inner-product-space linearization as its principal methodological distinction.

5 Experiments

Experiments evaluate InRTL against diverse baselines across ten datasets and 24 real-world tasks, including benchmark performance, ablations, hyper-parameter effects, and scalability. InRTL generally outperforms single-table approaches, while its gains vary by dataset and task and its components jointly contribute to performance.

  • 5.2 Performance Evaluation on Benchmarks: Relational table learning methods generally outperform both shallow and deep single-table approaches across the evaluated tasks.The comparison covers SJTUTables and RelBench benchmark settings.
  • 5.2 Performance Evaluation on Benchmarks: InRTL achieves consistently strong performance across datasets and tasks, with larger gains on TACM12K, rel-f1, and rel-event than on rel-amazon and rel-stack.The reported variation is associated with dataset-specific differences in table size, sparsity, and PK–FK graph topology.
  • 5.3 Ablation Study: Removing the column-aware encoder, linear attention, or HGNN causes noticeable degradation on representative small-, medium-, and large-scale classification tasks.The ablations cover rel-f1, TML1M, and rel-amazon, and support jointly modeling intra- and inter-table interactions.
  • 5.4 Hyper-parameter Analysis: Two HGNN layers provide the best depth trade-off, whereas increasing linear-attention depth consistently degrades performance.The authors attribute the latter to noise from deeper intra-table layers and the former to receptive-field limits and over-smoothing in graph networks.
  • Effect of Order Number in Taylor Expansion: First-order Taylor attention remains close to original Softmax attention while retaining linear complexity, motivating its use in InRTL.Higher Taylor orders approach Softmax performance but have quadratic complexity.
  • 5.5 Scalability Analysis: InRTL’s training time grows approximately linearly with node count from 10K to 200K, with the table encoder and linear attention dominating runtime.HGNN contributes a relatively small runtime fraction and grows moderately under fixed average node degree.

6 Related Work

Related work covers deep learning for single-table data and emerging approaches for multi-table data. These lines of work differ in whether they model within-table feature interactions or exploit relationships across tables and datasets.

  • Multi-table modeling: The related-work discussion organizes prior multi-table and single-table methods by their assumptions and limitations.This organization distinguishes explicit relational modeling from approaches that do not use table relationships.
  • Single-table modeling: Single-table deep learning methods learn expressive feature representations and complex feature interactions, often using attention over feature tokens.Representative approaches include TabTransformer and FT-Transformer.
  • Multi-table modeling: Multi-table deep learning research includes transfer learning across independent tables without explicit relational structure.These methods seek shared representations or inductive biases that improve generalization to new tasks.

7 Conclusion

The paper concludes that InRTL explicitly models relational dependencies and uses theoretically grounded simplifications to improve scalability. Its evaluation supports strong performance on curated relational databases, while open heterogeneous data lakes remain outside the demonstrated scope.

  • 7 Conclusion: InRTL formalizes intra-table and inter-table interactions with Transformer-based self-attention and cross-attention, then simplifies them for scalable relational learning.The conclusion characterizes these mechanisms as a task-specific objective for relational tables.
  • 7 Conclusion: Extensive experiments demonstrate that InRTL consistently outperforms state-of-the-art baselines on diverse relational table tasks.This is the paper’s principal reported conclusion about empirical effectiveness.
  • 7 Conclusion: The evaluation mainly covers well-curated relational databases with explicit table relationships, leaving open heterogeneous data lakes as a challenging future setting.The authors identify weak and noisy inter-table relationships as the reason this extension remains challenging.

A.1 Proof of Theorem 1

The proof derives linear attention by applying a first-order Taylor approximation to Softmax attention under normalized queries and keys. It establishes lower complexity than standard Transformer attention when the number of rows greatly exceeds feature dimension.

  • A.1 Proof of Theorem 1: The proof applies a first-order Taylor expansion of exp(x) at x=0 to approximate the Softmax exponential in intra-table attention.The approximation is analyzed under normalized query, key, and value vectors.
  • A.1 Proof of Theorem 1: Normalized queries and keys bound their inner product by 1, supporting the approximation and non-negative approximated attention weights.The non-negativity follows from the resulting inner-product approximation.
  • A.1 Proof of Theorem 1: The linearized attention matrix is D^-1A, where D is formed from row sums of the approximate attention matrix.Substituting this matrix into the attention output yields the linear-attention formulation.
  • A.1 Proof of Theorem 1: The approximation has complexity O(nd^2), compared with O(n^2d) for standard Transformer attention, becoming linear in n when n≫d.The proof concludes the validity of the linear attention approximation before deriving this complexity comparison.

A.2 Proof of Corollary 1

The proof derives an error bound for replacing standard softmax attention with a first-order linearized approximation under bounded inputs.

  • A.2 Proof of Corollary 1: The proof compares standard softmax attention weights with their linearized counterparts through their respective denominators.It introduces D_exp and D_lin and decomposes the difference between the two attention forms.
  • A.2 Proof of Corollary 1: Applying a first-order Taylor approximation to exp yields a quadratic error term bounded by 2 exp(ε)ε2.The derivation assumes |x_i| ≤ ε < 1 and uses this bound to control the approximation error.
  • A.2 Proof of Corollary 1: The proof establishes the denominator lower bound needed to complete the approximation-error analysis.The final steps combine the denominator bounds and conclude the proof.

A.3 Proof of Proposition 1

The proof shows that message passing on a masked bipartite heterogeneous graph reproduces the Transformer cross-attention computation for target nodes.

  • A.3 Proof of Proposition 1: The proof models the setting as a bipartite graph with target and auxiliary node types connected by one edge type.Only target nodes are considered, and a masking matrix is defined from the graph’s edge set.
  • A.3 Proof of Proposition 1: Graph aggregation collects features from neighboring auxiliary nodes, while the update operator is defined as the identity mapping.The resulting target-node representation is obtained after one message-passing round.
  • A.3 Proof of Proposition 1: The masked matrix formulation is exactly equivalent to Transformer cross-attention, excluding layer normalization and temperature scaling.The equivalence follows after substituting the graph mask into the message-passing expression.

C Implementation details

The implementation evaluates InRTL across relational benchmarks using reproducible training procedures, broad baselines, and its three-component architecture. Unified text-embedding tests show that InRTL remains best on all three SJTUTables datasets.

  • Data preparation: The evaluation covers SJTUTables and RelBench, using benchmark-prescribed graph construction, full-batch or mini-batch training, and original train-validation-test splits.SJTUTables contains three datasets, while RelBench contains seven datasets and 21 tasks.
  • Baselines: Baselines span decision trees, single-table deep models, and relational table methods, with flattening used for methods unable to process multi-relational inputs.The study re-implements baselines, searches key hyperparameters, and reports the best performance under the experiment settings.
  • Model details: InRTL comprises a column-aware table encoder, linear attention for intra-table interactions, and an HGNN for inter-table interactions.The encoder specializes representations by column type, while the interaction modules model within-table and PK–FK-linked dependencies.
  • Comparison with Unified Text Embeddings: InRTL remains best on all three SJTUTables datasets under both GloVe and all-MiniLM-L6-v2 unified text embeddings.This indicates that the reported performance gain is not solely due to text embedding choice.
  • Training procedure: The training procedure initializes InRTL, samples relational data each epoch, computes intra- and inter-table representations, and returns the trained model.The pseudocode specifies relational tables, the PK–FK graph, layer count, labels, and a sampling function as inputs.
Loading 2609.12712v1…