PaperlayerView exact revision on arXiv

Source-linked AI summary

STEER: Reducing Inference Cost in Relational Foundation Models through Semantically Informed Sampling

Abdalla Mohamed, Ashraf Aboulnaga

arXiv:2610.00907v1cs.DBcs.LG

TL;DR

Reducing RFM inference cost requires smaller sampled contexts, but random pruning can discard informative rows. STEER uses an LLM to rank schema edges by task relevance and probabilistically guide traversal. Across three RFMs, it reduces context size by about 40% on average while maintaining or sometimes improving prediction accuracy.

  • Problem

    RFM inference cost grows with context size, while random row dropping ignores schema semantics and can discard informative rows.

    Source: §1 · pp. 1–2

  • Method

    STEER prompts an LLM to rank schema edges by task relevance and uses those rankings to probabilistically filter traversal.

    Source: §1 · p. 2, §3 · p. 4, §5 · p. 6

  • Results

    About 40% smaller samples on average while matching, and sometimes slightly improving, full-size baseline accuracy across RT, RT-J, and Griffin.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across three RFMs, STEER reduces inference context size while preserving, and in several cases improving, prediction accuracy; random pruning causes substantial accuracy degradation.

    Source: §8.2 · p. 9, §10 · p. 13

    The optional column filter applies only to transformer-based RFMs and is less robust than STEER’s traversal element.

    Source: §7 · p. 7

Abstract

from arXiv · show

Relational foundation models (RFMs) are pretrained once on a collection of relational databases and prediction tasks, and then applied zero-shot to previously unseen databases and tasks. To make a prediction for a target row, an RFM samples a neighborhood of rows linked to that row through foreign keys and uses this neighborhood as its inference context. Lowering inference cost is an important goal for any foundation model, and for RFMs this cost grows with the size of the context. The simplest ways to shrink the context is to drop some of the sampled rows, but this ignores the semantics of the database schema, so it is as likely to discard informative rows as uninformative ones. We propose STEER, a sampling approach that shrinks the inference context by concentrating it on the tables most relevant to the prediction task at hand. STEER obtains relevance information by prompting a large language model to rank the foreign-key edges of the database schema into relevance tiers for the given task, and then maps each tier to a probability of following that edge during traversal. Because the ranking uses only the schema, it is computed once per task and reused across all subsequent predictions, amortizing its cost. We evaluate STEER on three state-of-the-art RFMs (RT, RT-J, and Griffin) and show that it reduces inference context size by about 40% on average while maintaining, and in some cases improving, prediction accuracy.

1 INTRODUCTION

RFMs enable zero-shot prediction across unseen databases and tasks, but inference cost depends on the sampled context, and random row dropping can discard informative data. STEER uses schema semantics to prioritize relevant traversal edges, reducing sample size while maintaining prediction accuracy.

1 INTRODUCTION

  • RFMs are pretrained once and make zero-shot predictions on unseen databases and tasks, unlike earlier RDL models trained for a specific database and task.RDL represents rows as graph nodes and foreign-key references as edges.Source: §1 · p. 1
  • Random row dropping ignores task relevance and can substantially degrade accuracy when it removes informative rows.Inference cost grows with context size, so reducing context matters for recurring prediction requests.Source: §1 · pp. 1–2
  • STEER uses schema-derived semantic information to rank traversal edges and choose which tables to sample, rather than randomly dropping rows after broad sampling.The schema can signal likely table usefulness without inspecting database rows.Source: §1 · p. 2
  • An LLM ranks each edge in both traversal directions into four task-specific relevance tiers, which determine the probability of following it.The paper also presents an optional mechanism for identifying task-relevant columns.Source: §1 · p. 2
  • About 40% smaller samples on average while matching, and sometimes slightly improving, full-size baseline accuracy across RT, RT-J, and Griffin.The evaluation uses RelBench and relevance rankings generated by Claude and GPT.Source: §1 · p. 2

2 BACKGROUND: SAMPLING IN RFMS

RFM samplers build a bounded inference context by traversing foreign-key links from a task-table seed row. Traversal directions differ in fan-out, and random child selection can make the resulting context depend on chance.

2 BACKGROUND: SAMPLING IN RFMS

  • The sampled neighborhood supplies the inference context, and STEER modifies sampling by incorporating semantic information into traversal decisions.The end-to-end depiction connects the seed task row, sampled rows, and model prediction.Source: §2 · p. 2, Figure 1 · p. 3
  • A task table can include example entities with known outcomes exposed to the RFM, providing information about the prediction task.Its rows are graph nodes connected to database entities through foreign-key edges.Source: §2 · pp. 2–3
  • A task-table row is the seed for sampling a bounded portion of the database graph, which becomes the RFM input for predicting its label.The task table can be specified through PQL or as a table linked to the entity being predicted.Source: §2 · pp. 2–3
  • F→P traversal reaches at most one parent, whereas P→F traversal may reach many child rows and is typically randomly subsampled.This makes the particular child rows included depend on chance rather than expected task relevance.Source: §2 · p. 3
  • Samplers bound their contexts and exclude rows timestamped after the seed row to prevent temporal leakage.Griffin samples a fixed number of neighbors per node for a fixed number of hops; RT uses a token budget and width bound.Source: §2 · p. 3

3 CONTROLLING THE TRAVERSAL

STEER intervenes after an RFM sampler selects candidate neighbors, probabilistically filtering traversal edges according to their task-specific relevance. Filtering an edge can prune the entire branch rooted at its destination.

3 CONTROLLING THE TRAVERSAL

  • STEER intervenes when selected neighbors enter the frontier, steering traversal toward rows considered more informative for the prediction task.F→P reaches at most one parent, while P→F may reach many children.Source: §3 · p. 4
  • STEER filters candidate traversal edges after the sampler selects neighbors; a rejected edge prevents its destination and the rows reachable through it from being visited.The same filtering process can be added to different RFM samplers.Source: §3 · p. 4
  • Implementing the modified samplers required substantial model-specific effort because the three RFMs perform sampling in Python rather than through a relational DBMS.The authors propose a unified SQL sampler with a common API as future work.Source: §3 · pp. 4–5
  • For each task, direction, and schema edge, an inclusion probability determines whether a Bernoulli trial keeps that edge.Higher-relevance edges are more likely to be retained, while lower-relevance edges are more likely to be pruned.Source: §3 · p. 5
  • Probabilistic filtering preserves some lower-relevance edges because relevance is graded, allowing them to contribute occasionally rather than being discarded outright.In the Formula 1 example, standings and qualifying may still provide some signal despite results being more directly informative.Source: §3 · p. 5

4 DETERMINING RELEVANCE

STEER estimates task-specific semantic relevance from the schema and asks an LLM to rank foreign-key edges. Its prompt uses staged reasoning and graded relevance tiers, with configurations reused across predictions for the task.

4.1 Estimating Semantic Relevance with LLMs

  • The LLM judges schema edges using table and column names and their relation to the prediction task, without inspecting database rows.STEER uses a structured prompt to obtain relevance judgments for traversal edges.Source: §4.1 · p. 5
  • STEER ranks edges rather than tables because the relevance of reaching a table can depend on the edge and the table from which traversal arrives.Each foreign-key column defines an edge that can be traversed in F→P or P→F directions.Source: §4.1 · p. 5

4.2 Prompting Strategy

  • The prompt asks the LLM to reason first about the schema, then the task, and then the relation of each edge to them.The authors expect this ordering to help with judgments that require combining schema and task information.Source: §4.2 · p. 5
  • Instead of binary relevance decisions, the prompt assigns each edge to one of four ordered tiers, allowing graded distinctions in how often edges should be followed.The tiers are Very Strong, Strong, Moderate, and Weak.Source: §4.2 · p. 5
  • Two prompts, one per traversal direction, generate a task configuration that is stored and reused for later predictions rather than queried each time.The pipeline consults stored task configuration before generating a new one.Source: §4.2 · p. 5, Figure 2 · p. 6

5 INCLUSION PROBABILITIES

STEER converts LLM-assigned relevance tiers into edge-inclusion probabilities, using probabilistic filtering to preserve graded relevance and reduce the impact of ranking errors.

5 INCLUSION PROBABILITIES

  • STEER assigns an inclusion probability to each relevance tier in each traversal direction, so edges in the same tier share a probability.The four tiers and two directions yield eight probabilities per task.Source: §5 · p. 6
  • The default probability profile maps Very Strong to 1.0, Strong to 0.6, Moderate to 0.2, and Weak to 0.0.The paper reports this fixed mapping yields good results and that tuning is warranted only in limited cases.Source: §5 · p. 6
  • Probabilistic filtering makes LLM ranking errors less consequential than binary inclusion or exclusion, because neighboring tiers change how often an edge is followed rather than deciding it outright.The probability of following an edge increases with its relevance tier.Source: §5 · p. 6

6 STEER PIPELINE

STEER separates task configuration from row-level sampling: it obtains a configuration once per database and task, then reuses it as predictions are made.

6 STEER PIPELINE

  • The STEER pipeline has two frequencies: LLM configuration occurs once per database and task, while sampling runs for every task row on each invocation.Source: §6 · p. 6
  • For a previously seen task, STEER looks up and reuses its stored configuration; for a new task, it generates and saves one before sampling.The configuration depends on the schema and task, not on the rows being predicted.Source: §6 · pp. 6–7
  • On a task’s first encounter, the generator makes two LLM calls—one for each traversal direction—and assigns relevance tiers to schema edges.For subsequent requests, the configuration lookup is the stated STEER cost.Source: §6 · p. 7

7 EXTENDING STEER: COLUMN FILTER

The optional column filter probabilistically withholds less-relevant cells to shrink transformer context, but it does not apply to graph-based RFMs and is less robust than traversal filtering.

7 EXTENDING STEER: COLUMN FILTER

  • The column filter removes selected cells from already visited rows, shrinking context and leaving room for more informative columns or additional rows.Unlike edge filtering, it changes what is written after a row has been reached rather than which rows are visited.Source: §7 · p. 7
  • The column filter applies to transformer-based RFMs such as RT and RT-J, not graph-based Griffin, and is an optional extension less robust than traversal filtering.It is independent of edge filtering and can be used with or without it.Source: §7 · p. 7
  • An LLM ranks each table column into one of four tiers, and STEER uses the tier’s probability to decide independently whether to write each visited column.Source: §7 · p. 7
  • Because column rankings use names rather than values, a name-only ranking can recommend columns with many nulls; STEER demotes columns above a fixed null-rate threshold to Moderate.The correction is based on each column’s fraction of null rows; Moderate reflects that such a column may still carry signal when present.Source: §7 · p. 7

8 EXPERIMENTS

Across 24 tasks on 8 databases and three RFMs, STEER reduces inference sample size while maintaining or sometimes improving accuracy. Ablations show that filtering both traversal directions performs best overall, while column filtering and task-specific probability tuning offer additional but variable benefits.

8.1 Experimental Setup

  • The evaluation covers 24 classification and regression tasks across 8 databases, using RT, RT-J, and Griffin with baseline, naive-pruning, and two STEER configurations.Results average 1,280 evaluation items per task; classification uses AUROC and regression uses R².Source: §8.1 · pp. 7–8

8.2 Main Result: Effectiveness of STEER

  • Naive pruning’s 50% sample reduction substantially degrades accuracy; for RT-J, classification AUROC falls 4.10 points and regression R² falls 16.09 points versus baseline.The comparison indicates that shrinking the sample uniformly is not equivalent to selecting less useful information.Source: §8.2 · p. 8
  • STEER reduces sample size by 32.8–56.7% across the three RFMs and task types, with little or no accuracy loss and occasional gains.For RT, accuracy improves by up to 1.48 percentage points on classification and 3.89 on regression; Griffin’s largest decrease is 0.78 percentage points.Source: §8.2 · pp. 8–9

8.3 Ablation: Direction of Traversal

  • Filtering both traversal directions reduces sample size by 40.3–56.7%, more than either P→F filtering (32.2–35.0%) or F→P filtering (14.5–29.3%).The two directions target different parts of sample redundancy; using both yields the largest reduction and highest accuracy in the ablation.Source: §8.3 · p. 9

8.4 Ablation: Prompting Strategy

  • STEER’s smart prompting outperforms simple prompting in every tested setting, with the largest reported gap on RT-J regression, where simple prompting is about 6 R² points below baseline.Claude is slightly more accurate than GPT on regression, while high and default reasoning effort show no large difference.Source: Table 1 · p. 10, §8.4 · p. 10

8.5 Ablation: Column Filter

  • Adding the column filter reduces sample size by an additional 5.6–9.8 percentage points, but its accuracy effects vary by model and task.It raises RT regression R² by 5.39 points and RT-J classification by 0.56, while lowering RT-J regression accuracy by 3.11 points.Source: §8.5 · p. 10

8.6 Ablation: Probability Tuning

  • Task-specific probability tuning has mixed, task-dependent accuracy effects and incurs search overhead; the default profile is a robust, effective, inexpensive choice across tasks and models.Tuning improves average accuracy for three of four model/task-type combinations, but not RT-J classification.Source: §8.6 · pp. 10–11
  • Tuning can produce large task-level gains or losses: RT’s rel-ratebeer/userrating-churn AUROC rises from 73.02 to 82.75, while RT-J’s rel-trial/study-outcome AUROC falls from 52.68 to 47.11.These comparisons are default versus tuned probability profiles; validation-based tuning can fit the validation split without generalizing to test data.Source: §8.6 · p. 10

9 RELATED WORK

Prior relational deep learning work developed graph-based models and sampling strategies, while relational foundation models seek transfer across unseen databases. STEER differs from earlier LLM applications by using schema semantics to rank traversal edges for inference-time sampling.

Relational Deep Learning

  • Fey et al. introduced an end-to-end relational database modeling framework, with rows represented as nodes and primary–foreign key relationships as edges.Initial approaches applied heterogeneous GNNs directly to these graphs, followed by more expressive message-passing architectures [5].Source: §9 · p. 11
  • RDL represents databases as graphs and has used GNN message passing and transformer architectures, but these approaches are schema-specific and do not transfer to unseen databases.Transformer approaches are reported to achieve stronger relational benchmark performance than GNN message passing [7].Source: §9 · pp. 11–12

Relational Foundation Models

  • Relational foundation models address schema specificity through zero-shot generalization, with Griffin [33] and RT [29] representing distinct open-source architectures.Griffin combines table-level encoders with graph neural network propagation, while RT uses cell-level tokens and Relational Attention; RT-J and RT-PluRel [18] change pretraining approaches.Source: §9 · p. 12

Sampling Strategies for Graph-Based Models

  • Prior sampling methods guide neighborhoods through uniform, importance, path-constrained, SQL-based, or semantic strategies, while STEER targets inference-time sampling for zero-shot RFMs without task-specific training data.The cited work includes GraphSAGE [11], PinSage [39], path-constrained walks, SQL-based sampling, LLM-generated query templates [2], and collaborative sampling [6].Source: §9 · p. 13

LLMs for Relational Data

  • Earlier LLM work applies models to relational prediction, schema matching, or schema exposure, whereas STEER uses an LLM as a semantic oracle to rank traversal edges for sampling.Related approaches include flat-text prediction [36], Rel-LLM [35], schema matching [24], and text-to-SQL [21].Source: §9 · p. 13

10 CONCLUSION

STEER uses schema-edge relevance to guide sampling and reduce RFM inference context, reusing each task’s relevance configuration across predictions. Across three RFMs, it reduces context size while preserving or sometimes improving accuracy; future work includes more robust column filtering and cheaper probability adaptation.

10 CONCLUSION

  • Across three RFMs, STEER reduces inference context size while preserving, and in several cases improving, prediction accuracy.The relevance configuration is reused across predictions, amortizing the LLM-call cost.Source: §10 · p. 13
  • Uninformed random pruning causes substantial accuracy degradation, unlike STEER’s relevance-guided context reduction.The conclusion states that effective context reduction requires reasoning about which relational structure is relevant to the task.Source: §10 · p. 13
  • Future work includes more robust column filtering, inexpensive task-specific traversal-probability adaptation, and applying semantic guidance to additional RFM architectures and samplers.The column filter’s mixed results motivate combining schema semantics with lightweight data statistics.Source: §10 · p. 13

A PROMPT TEMPLATES

The appendix gives smart and simple prompt templates for ranking schema traversal edges with respect to a prediction task. The smart template uses stepwise semantic reasoning and tiers; the simple template makes a binary useful-edge selection.

A PROMPT TEMPLATES

  • The templates are applied in both traversal directions and end with task-specific context including the task, dataset, task table, its columns, and the database definition.The smart prompt is used once for P→F and once for F→P; the simple prompt also specifies its F→P variant.Source: A PROMPT TEMPLATES · p. 15, §A.1 · p. 16, §A.2 · p. 16

A.1 Smart Prompting Template

  • The smart template first asks the LLM to understand database tables, columns, keys, and connections, then identify the predicted entity and target.It frames the task as estimating or classifying the target value.Source: §A.1 · p. 15
  • The smart template explicitly lists each foreign-key traversal edge as source→destination:fk_col, with each foreign-key column generating a distinct edge.For P→F, the source is the parent table and the destination is the child table.Source: §A.1 · p. 15
  • Smart ranking judges each edge by its task relevance from the current source and by whether its destination rows form a targeted, useful subset.The prompt says edges sharing a destination can differ in value because their source-table contexts differ.Source: §A.1 · p. 15
  • Smart prompting assigns every listed edge to one of four relevance tiers, requires at least one edge, and forbids invented edges or altered database names.If none qualify, the prompt directs selection of the single most relevant edge for the very_strong tier.Source: §A.1 · pp. 15–16
  • The F→P smart variant reverses traversal source and destination, places the foreign-key column on the source, and explicitly includes self-referential edges.It reframes relevance around whether a parent row characterizes the prediction entity.Source: §A.1 · p. 16

A.2 Simple Prompting Template

  • Simple prompting returns useful edges without a reasoning scaffold; all returned edges enter very_strong, leaving the other tiers empty.This creates a binary include-or-exclude decision rather than a graduated four-tier ranking.Source: §A.2 · p. 16

B FULL PER-TASK RESULTS

The appendix reports full per-task results under the same evaluation procedure as Section 8.1. Its tables cover the main comparisons and ablations, but the supplied captions do not provide cell values or establish winners.

B FULL PER-TASK RESULTS

  • The appendix reports full per-task experiment results using the evaluation procedure described in Section 8.1.The supplied captions do not include the table cells, so they support scope and layout rather than result values.Source: B FULL PER-TASK RESULTS · p. 16
  • Tables 5 and 6 report classification AUROC and regression R^2 results for baseline sampling, naive pruning, and STEER rankings from Claude and GPT.They include per-model means and changes relative to the baseline.Source: Table 5 · p. 17, Table 6 · p. 18
  • Tables 7 and 8 compare baseline sampling, P→F-only, F→P-only, and combined STEER traversal for classification and regression tasks.These direction ablations use RT and RT-J with Claude rankings.Source: Table 7 · p. 19, Table 8 · p. 20
  • Tables 9 and 10 compare baseline accuracy with simple prompting and smart prompting across four Claude and GPT reasoning-effort variants.The ablation covers RT and RT-J on classification and regression tasks.Source: Table 9 · p. 21, Table 10 · p. 22
  • Tables 11 and 12 report classification and regression accuracy and token counts for baseline sampling, STEER without column filtering, and STEER with filtering.The column-filter ablations use RT and RT-J with Claude.Source: Table 11 · p. 23, Table 12 · p. 24

Related papers

Loading 2610.00907v1…