PaperlayerView exact revision on arXiv

Source-linked AI summary

LoGIC: Budgeted Context Construction for Node-Level Graph In-Context Learning with Tabular Foundation Models

Mingqi Yang, Zidong Guo, Jihui Yang, Wenming Zuo

arXiv:2609.05955v1cs.LG

TL;DR

Full-table prompts in graph ICL create quadratic attention costs and memory bottlenecks, while the choice of labeled and auxiliary unlabeled nodes for each query remains an open context-construction problem. LoGIC allocates labeled evidence and adapter support through graph-aware, cluster-shared retrieval without test labels. Across GraphLand backbones, budgeted contexts retain competitive performance while reducing resource burdens, including at million-node scale.

  • Problem

    Full-context graph ICL uses the entire training table, making attention quadratic in prompt size and creating memory bottlenecks for graph inference.

    Source: B. Cost Model and the Memory Wall · p. 4

  • Method

    LoGIC shares graph-local query-cluster contexts selected through structural, feature-based, and coverage channels, with separate labeled and unlabeled-halo budgets.

    Source: §I · p. 2

  • Results

    Across GraphLand, bounded contexts retain competitive performance while reducing computational and memory burdens relative to full-context and full-graph inference, including at million-node scale.

    Source: §I · p. 2, §VII · p. 11

  • Takeaways & Limitations

    Retrieval-channel benefits vary with graph properties, supporting context construction that adjusts to measurable properties rather than relying on one rule across datasets.

    Source: §VII · p. 11

    The evaluation covers transductive node classification and regression on graphs with tabular node attributes; inductive and streaming graphs and link- or graph-level tasks remain future directions.

    Source: B. Scope and Future Work · p. 11

Abstract

from arXiv · show

Tabular foundation models have become powerful graph learners. Systems such as G2T-FM and GraphPFN encode each node as a feature row and make predictions through in-context learning (ICL), with labeled rows serving as the prompt. Current protocols employ the complete training table as context, causing attention to scale quadratically with the labeled pool and introducing preprocessing and memory bottlenecks. We investigate context construction for node-level graph ICL: which labeled nodes and auxiliary unlabeled nodes should constitute the prompt for specified queries. We formulate this allocation in terms of two resources: a labeled-context budget for predictive evidence and an unlabeled-halo budget for adapter message passing without using label capacity. We present LoGIC, which retrieves labeled nodes via structural, feature-based, and coverage channels, shares each context across the queries in a graph-local cluster, incorporates an unlabeled halo for adapter backbones, and chooses the channel and context budget without test labels. Across three backbone configurations drawn from two model families on GraphLand, budgeted contexts maintain locally runnable full-context performance, stay competitive with published large-dataset results, and markedly lower peak memory requirements compared with full-context and whole-graph inference. They further permit frozen graph ICL on million-node graphs without retraining. Our analysis identifies when retrieval channels work best and connects their behavior with graph properties.

I. INTRODUCTION

Node-level graph ICL uses labeled rows as prompts, but standard systems use the entire training table, making context construction a costly and underexamined design choice. LoGIC allocates labeled evidence and unlabeled structural support under separate budgets, and its GraphLand evaluation connects retrieval behavior to graph properties.

I. INTRODUCTION

  • Existing graph ICL systems use the entire training table, while random prompt subsampling limits capacity without selecting rows pertinent to the queries.The paper identifies which labeled and auxiliary unlabeled nodes should enter the prompt for specified queries as its central question.Source: §I · p. 1
  • Full-table prompts make attention scale quadratically with labeled rows, while adapter backbones also incur graph-residency costs that depend on graph inputs and execution settings.Whole-graph GraphPFN uses 19.59 GB on artnet-exp and 7.71 GB on city-roads-M, versus LoGIC’s 8.24 GB and 5.73 GB, respectively.Source: §I · p. 1
  • Across GraphLand, three backbone configurations from two model families show that budgeted contexts retain competitive performance while lowering memory burden, including at million-node scale.The best retrieval channel varies with graph properties, and the channel and context budget are chosen without test labels.Source: §I · p. 2
  • The paper frames context construction as allocating a labeled-context budget for predictive evidence and an unlabeled-halo budget for adapter message passing.The halo does not use label capacity.Source: §I · p. 2
  • LoGIC retrieves labeled nodes through structural, feature-based, and coverage channels, shares contexts across graph-local query clusters, and adds an unlabeled halo for adapter backbones.It leaves backbone parameters and training unchanged and supports row-based and graph-adapter inference.Source: §I · p. 2
  • The evaluation covers eight GraphLand datasets and three backbone configurations spanning two model families, with performance, efficiency, scalability, and ablations examined under a consistent protocol.The contribution list also reports connections between channel behavior, context budget, and measurable graph properties.Source: §I · p. 2

II. RELATED WORK

Prior work addresses tabular context retrieval, graph foundation models, and graph-node selection, but these directions do not jointly provide graph-aware prompt selection for frozen tabular graph models. LoGIC distinguishes labeled PFN evidence from unlabeled structural support for adapters.

A. Tabular Foundation Models and Context Retrieval

  • Tabular retrieval methods select rows using feature-space or distributional signals, whereas LoGIC selects labeled graph nodes and, for adapters, unlabeled nodes that preserve local receptive fields.Chunked attention changes how a PFN processes a context, not which graph nodes fill it.Source: A. Tabular Foundation Models and… · p. 2

B. Tabular Foundation Models as Graph Foundation Models

  • Tabular foundation models have been applied to graph tasks, but their standard inference protocols still use the complete training context; TAG’s random subsampling controls capacity without graph-aware retrieval.GraphPFN also identifies whole-graph processing as a primary limitation.Source: B. Tabular Foundation Models as Graph… · p. 3

C. Selecting Nodes on Graphs

  • Graph-ICL, active-learning, and GNN-sampling methods select demonstrations, annotation candidates, or computational subgraphs, rather than allocating labeled PFN prompts separately from adapter support nodes.Graph samplers chiefly regulate message-passing cost, and partitioners could still supply candidate subgraphs or replace LoGIC’s query order.Source: C. Selecting Nodes on Graphs · p. 3

D. Positioning

LoGIC is positioned as a context-allocation layer for frozen tabular graph models, distinct from methods that alter attention processing or graph computation. The paper’s graph-ICL setup predicts from a query representation and a labeled context.

D. Positioning

  • LoGIC addresses which labeled rows support queries and which unlabeled nodes support adapter message passing; chunked attention and graph samplers address different decisions.The methods can be integrated, with LoGIC choosing evidence and support before memory-efficient attention processes the bounded context.Source: D. Positioning · p. 4
  • The paper compares context construction using matched full-context or full-graph runs and isolates retrieval signals with fixed PPR, feature-kNN, and coverage policies.Changing attention kernels, query batching, graph partitioning, and retrieved labels together would confound those factors.Source: D. Positioning · p. 4

A. Graph In-Context Learning

  • A graph tabular foundation model represents each node as a row and applies a frozen PFN; full-context protocols use all labeled rows, and GraphPFN also materializes the whole graph.G2T-FM rows include raw attributes, neighborhood aggregates, and structural encodings, while GraphPFN processes raw features with graph-attention adapters.Source: A. Graph In-Context Learning · p. 4

B. Cost Model and the Memory Wall

Full-context inference incurs quadratic attention interactions in context size, and adapter backbones retain graph tokens in memory. Budgeted context construction separates supervised evidence from label-masked structural support and bounds the materialized input.

B. Cost Model and the Memory Wall

  • A forward pass has Θ(|S|^2 + q|S|) attention-score interactions per layer, while adapter backbones additionally retain all graph tokens.Actual adapter VRAM also depends on feature width, edge structure, batching, and implementation.Source: B. Cost Model and the Memory Wall · p. 4
  • Whole-graph residency grows with node count, whereas LoGIC limits materialized input through labeled, query, and halo budgets; full-feature preprocessing can also exceed memory and require PCA truncation.The cited G2T-TabPFNv2 protocol uses PCA truncation on two of eight benchmarks.Source: B. Cost Model and the Memory Wall · p. 4

C. Problem: Budgeted Context Construction

  • The labeled budget k controls supervised evidence, while the halo budget h controls label-masked adapter support; increasing either does not substitute for the other resource.Halo nodes provide features and edges but no supervision, while selected labeled nodes need not complete local receptive fields.Source: B. Cost Model and the Memory Wall · p. 4, C. Problem: Budgeted Context… · p. 4
  • A context policy selects at most k labeled nodes and h label-masked halo nodes for each query cluster, then predicts on their induced subgraph with the queries.Full-context and full-graph protocols are special cases; budgeted construction occupies an intermediate regime without changing the frozen backbone.Source: C. Problem: Budgeted Context… · p. 4
  • The stated design goals include matching or exceeding full-context accuracy with k much smaller than the labeled pool, bounding inference memory, and amortizing retrieval across queries without extra training.The method is intended to work with both pure-tabular and adapter backbones.Source: C. Problem: Budgeted Context… · p. 4

IV. LOGIC: GRAPH-AWARE CONTEXT CONSTRUCTION

LoGIC constructs bounded contexts for graph ICL by sharing retrieved labeled rows across graph-local query clusters and selecting among structural, feature-based, and coverage channels. For adapter backbones, an unlabeled halo can supply graph context without consuming labeled-row budget.

A. Graph-Local Query Clustering

  • RCM orders graph-adjacent queries near one another, then groups them so each cluster shares one retrieved context and one backbone forward.This reduces forwards from |Q| to ⌈|Q|/q⌉; the cached RCM order can be replaced by a balanced graph partitioner.Source: A. Graph-Local Query Clustering · p. 5

B. Retrieval Channels

  • The PPR channel ranks labeled nodes by personalized PageRank mass from the query cluster, encoding the assumption that labels are locally smooth across edges.Its scores depend only on the graph and cluster, so they can be cached and reused across policies and budgets.Source: B. Retrieval Channels · p. 5
  • Feature-kNN ranks labeled nodes by negative distance to their closest query in the backbone-transformed feature space.It transfers feature-space retrieval from tabular ICL and serves as the feature-space control in channel ablations.Source: B. Retrieval Channels · p. 5
  • Coverage sampling selects labeled rows uniformly or stratifies them by class or target quantile, favoring global representativeness over graph locality.If a channel cannot fill a cluster's budget, coverage supplies the remainder.Source: B. Retrieval Channels · p. 5

C. The Unlabeled Halo

For adapter backbones, LoGIC adds a label-masked halo around the query cluster and retrieved context to preserve local message passing. The halo uses memory but no labeled-context capacity, and larger halo budgets can improve performance before saturation.

C. The Unlabeled Halo

  • The halo comprises nodes in the 1-hop closure of the query cluster and retrieved context, capped at h and uniformly sampled beyond the cap.Halo nodes remain label-masked even when they belong to the global labeled pool; only their feature- and edge-dependent adapter messages can reach predictions.Source: C. The Unlabeled Halo · p. 5
  • The adapter subgraph materializes query, context, and halo nodes, but only retrieved context rows are labeled demonstrations and only query nodes are evaluated.When the full 1-hop closure fits within h, the closure is retained exactly; otherwise, sampling provides a bounded approximation.Source: C. The Unlabeled Halo · pp. 5–6
  • On tolokers-2, the halo lifts subgraph performance from substantially below the full-graph result to parity; on artnet-exp, it raises accuracy 0.27 AP beyond the published full-graph result.Performance rises with halo size before reaching saturation.Source: C. The Unlabeled Halo · p. 5

D. Channel and Budget Configuration

LoGIC selects a channel and labeled budget using held-out data without test labels or parameter fitting. Its shared-context and retrieved-subgraph protocols reduce repeated inference work and resident graph size relative to full-context or whole-graph protocols.

D. Channel and Budget Configuration

  • The deployed configuration evaluates a coarse ×4 budget ladder on held-out data and keeps one channel-budget pair per dataset, without using test labels or fitting parameters.Near-tied candidates are resolved using edge-label assortativity when distinctly positive or negative; a clear score margin takes precedence.Source: D. Channel and Budget Configuration · p. 6

E. Complexity Analysis and Implementation

  • Preprocessing runs once per graph; per-cluster retrieval costs O(iters · m) for PPR or O(|L| d) for feature-distance scans.PPR scores are reused across budgets, and cached sparse adjacency supports induced-subgraph construction without reconstructing the whole graph for every cluster.Source: E. Complexity Analysis and… · p. 6
  • Shared contexts use C forwards with k labeled rows, versus |Q| forwards for per-query retrieval; retrieved adapter subgraphs retain at most k + q + h nodes.Whole-graph adapters retain all n nodes and edges, while measured accelerator memory also depends on induced edges, row width, and model activations.Source: E. Complexity Analysis and… · p. 6, Table I · p. 6

V. EXPERIMENTS

The experiments assess predictive performance, efficiency and scalability, and the effects of retrieval channels, budgets, halos, and graph properties. The primary evaluation spans eight GraphLand benchmarks and three frozen backbone configurations from two model families.

V. EXPERIMENTS

  • The evaluation is organized around predictive comparisons, resource and scalability comparisons, and analyses of retrieval components and graph properties.These are the paper's three stated research questions.Source: §V · p. 6

A. Experimental Setup

  • Eight GraphLand benchmarks cover 12k–168k nodes, classification and regression, and assortative and disassortative targets.The dataset table reports labeled-pool size, undirected edge counts, and train-edge label assortativity.Source: A. Experimental Setup · p. 6, Table II · p. 7
  • Predictive evaluation uses three frozen configurations across TabPFNv2, LimiX with G2T-FM featurization, and GraphPFN-1.3; resource studies focus on TabPFNv2 and GraphPFN.The study also includes a 1.63M-node pokec-regions experiment; the eight GraphLand benchmarks use five seeds and random-label splits.Source: A. Experimental Setup · p. 7
  • Within-backbone comparisons change only context while reusing preprocessing artifacts, report mean±std across five seeds, and measure runtime end-to-end and peak accelerator memory.Resource comparisons are restricted to matched backbones; Table III compares published predictive baselines, while Table IV compares original protocols with LoGIC.Source: A. Experimental Setup · p. 7

B. Predictive Performance on GraphLand (RQ1)

Across backbone families, budgeted LoGIC contexts generally retain full-context or full-graph predictive performance. Local measurements show reduced resource use, and the study demonstrates frozen inference at million-node scale.

B. Predictive Performance on GraphLand (RQ1)

C. Efficiency, Scalability, and Cross-Backbone Comparison (RQ2)

  • On artnet-exp, GraphPFN LoGIC achieves 52.06±0.18 AP using 8.24 GB, versus 51.86±0.10 AP and 19.59 GB under full-graph inference.With TabPFNv2, LoGIC lowers artnet-exp end-to-end time from 117.78±1.37 to 65.68±0.80 seconds per seed and peak VRAM from 6.9 to 5.8 GB.Source: C. Efficiency, Scalability, and… · pp. 7–8, Table IV · p. 8
  • On 1.63M-node Pokec-regions, LoGIC predicts 76.8k held-out queries with 36.3% accuracy in 40 minutes using 4.2 GB peak VRAM.Whole-graph residency is estimated at approximately two terabytes; a random context with the same k=4096 achieves 8.6%.Source: C. Efficiency, Scalability, and… · p. 8

D. Ablation and Mechanistic Analysis (RQ3)

The ablations show that no retrieval channel wins across all datasets, and that channel behavior varies with graph properties. Geometry, budget, halo, and calibration analyses further characterize budgeted contexts.

Retrieval channels

  • No fixed retrieval channel performs best across all datasets; PPR is strong with positive assortativity, while coverage/random is preferred on negatively assortative twitch-views.The association is not deterministic: near-zero assortativity does not explain PPR's advantage on artnet-exp, and feature-kNN is best on avazu-ctr when node attributes are strong.Source: D. Ablation and Mechanistic Analysis… · pp. 8–9

Retrieved-context geometry

  • PPR contexts have a median distance of two hops and no nodes at radius five or above, while feature-kNN and random have a three-hop median and 1% at radius five or above.Figure 3 shows one deterministically selected artnet-exp query cluster; its angle is only a node hash for separating markers.Source: D. Ablation and Mechanistic Analysis… · p. 9, Figure 3 · p. 10

Budget regimes

  • Every channel deteriorates below approximately 512 rows; at larger budgets, channel rankings vary by dataset and ultimately approach saturation.The PPR sensitivity analysis reports artnet-exp stability across tested teleport α values, greater city-reviews sensitivity, and better RCM than natural query ordering.Source: D. Ablation and Mechanistic Analysis… · p. 9, Table VII · p. 9, Figure 4 · p. 10

Unlabeled halo

  • On tolokers-2, adding the 1-hop halo at k=4096 raises AP from 54.20 to 61.09, matching the full-graph reference; artnet-exp reaches 52.06±0.18 AP.The halo retains first-layer neighborhoods when the 1-hop closure fits within its cap; a finite cap provides a sampled approximation.Source: D. Ablation and Mechanistic Analysis… · p. 9

Posterior calibration

  • On tolokers-2, feature-kNN shifts Brier score from 12.17 to 12.15 and ECE from 1.60 to 1.59; on artnet-exp, PPR raises AP from 45.68 to 45.94 while ECE falls from 1.51 to 0.85.These representative comparisons show no apparent calibration penalty from budgeting, but do not establish consistent calibration gains across datasets or channels.Source: D. Ablation and Mechanistic Analysis… · p. 9, Table VIII · p. 9

VI. DISCUSSION

Deployment choices concern the labeled-context budget, retrieval channel, and—when using adapter backbones—the unlabeled halo. The discussion gives dataset-dependent guidance for each choice.

A. Deployment Considerations

  • With a runnable local full-context protocol, k=4096 maintains its performance using bounded memory and comparable or lower latency; it also surpasses the local artnet-exp result by 0.26 AP.For larger labeled pools, budgeted contexts remain competitive with published full-context results; budgets below the current backbones' approximate 512-row capacity floor perform weakly across policies.Source: A. Deployment Considerations · p. 10
  • Distinctly positive train-edge label assortativity favors PPR as a strong default, while distinctly negative assortativity typically favors coverage sampling.Feature-kNN is another candidate when raw attributes are informative; assortativity helps resolve candidates within one standard deviation of the best held-out score without overriding a clear margin.Source: A. Deployment Considerations · pp. 10–11
  • For adapter backbones, allocating memory to the unlabeled halo can recover local receptive fields: it matches full-graph inference on tolokers-2 and exceeds the published GraphPFN result by 0.27 AP on artnet-exp.Figure 5 presents the artnet-exp halo-cap ablation against the published full-graph score.Source: A. Deployment Considerations · p. 11, Figure 5 · p. 11

B. Scope and Future Work

The evaluation and conclusions position LoGIC as an inference-time context-construction layer for frozen tabular graph foundation models. Its demonstrated scope is transductive node classification and regression, with several graph settings left for future study.

B. Scope and Future Work

  • The evaluation covers transductive node classification and regression on graphs with tabular node attributes, with graph neighborhoods available at inference time.Inductive and streaming graphs, as well as link- and graph-level tasks, are identified as natural next steps.Source: B. Scope and Future Work · p. 11
  • Budgeting bounds the input presented to the backbone, not total workload, because runtime still grows with the number of query clusters.For graph-adapter backbones, the halo restores 1-hop neighborhoods of query and context tokens within its cap.Source: B. Scope and Future Work · p. 11
  • Larger backbones may accommodate larger contexts and make the observed capacity floor less restrictive, but prompts will still require selecting specific rows.The paper anticipates extensions to inductive and dynamically evolving graphs and points to learned retrieval mixtures, cluster-adaptive budgets, and multi-hop halos as future directions.Source: B. Scope and Future Work · p. 11

VII. CONCLUSION

  • LoGIC allocates labeled evidence and label-masked structural support across bounded budgets, sharing graph-local contexts and selecting a channel–budget configuration without test labels.It leaves the host backbone and training procedure unchanged while constraining PFN attention and peak input residency.Source: §VII · p. 11
  • Across GraphLand and three backbone configurations from two model families, bounded contexts preserve competitive predictive performance while lowering the burden of full-context and full-graph inference, including at million-node scale.The experiments find different benefits from structural locality, attribute similarity, and global coverage under different graph regimes.Source: §VII · p. 11
Loading 2609.05955v1…