TabICLv2: A better, faster, scalable, and open tabular foundation model

• Tabular foundation models can match or surpass gradient-boosted trees on tables with up to 100K samples while performing training and inference in one Transformer forward pass.Their capabilities also support downstream causal inference, generative modeling, joint predictive distributions, and simulation-based inference.Source: §1 · p. 1
• The field needs fully open-source tabular foundation models that rival closed-source systems, democratize top-tier performance, and clarify how they are built.Source: §1 · p. 1
• TabICLv2 is introduced as a state-of-the-art tabular foundation model.Source: §1 · p. 1
• Its contributions span architectural innovations, pretraining improvements, a novel synthetic data generator, extensive evaluations, and ablation studies.Source: §1 · pp. 1–2

2.1. Tabular foundation models

• PFN-style tabular foundation models directly predict test-label distributions from training data and test inputs in one forward pass, enabling gradient-free in-context learning.Pretraining samples datasets from a prior distribution and minimizes predictive loss.Source: §2.1 · p. 2
• Architectures evolved from row-token TabPFN to cell-based TabPFNv2 and two-stage TabICL, which reduces complexity from O(n2m + nm2) to O(n2+nm2).TabICL first constructs fixed-dimensional row embeddings with column-then-row attention before performing in-context learning over them.Source: §2.1 · p. 2
• Synthetic priors span structural causal models, tree-based mixtures, richer DAGs, hierarchical SCMs, and temporal generators, while real-data pretraining or continued pretraining can also be competitive.These designs target inductive biases, controllable difficulty, distribution shift, or improved TabPFNv2 performance.Source: §2.1 · p. 2
• Beyond pretraining, TFM adaptation uses fine-tuning, retrieval-based context selection, and distillation into compact MLPs or trees.These strategies respectively shift learned priors, improve compute-constrained scalability, or compress models.Source: §2.1 · p. 2

2.2. Attention struggles with long-context generalization

• Standard softmax attention suffers attention fading as context length grows, flattening attention distributions and limiting models trained on short sequences when applied to longer ones.The softmax denominator increases with context length, preventing sharp focus on relevant tokens.Source: §2.2 · p. 2
• Proposed remedies include softmax alternatives, fixed or dynamic temperature scaling, SSMax’s s log n scaling, ASEntmax’s content-aware scaling, and query-dependent temperatures.Theory establishes log n scaling as necessary for maintaining attention sharpness as context length n grows.Source: §2.2 · p. 2

Core architecture

• TabICLv2 repeatedly groups features via circular shifts, injects target embeddings before compression, then applies column-wise, row-wise, and dataset-wise transformers for prediction.Repeated grouping preserves the effective feature count while breaking feature symmetries; target information helps determine feature relevance before compression.Source: §3 · p. 3

Query-aware scalable softmax

• 100% accuracy with 15K negatives: QASSMax maintains low attention entropy and stable performance, whereas attention without scalable softmax collapses and SSMax degrades substantially.This toy needle-in-a-haystack task evaluates attention fading as negative samples increase.Source: Figure 3 · p. 4, §3 · p. 4

Many-class classification

• Mixed-radix ensembling combined with hierarchical classification enables TabICLv2 to handle an arbitrary number of classes despite pretraining with at most 10.Each class label is decomposed into digits with bases no larger than 10, and TFcol outputs are averaged across digit-wise groupings.Source: §3 · p. 4

Quantile predictions for regression

• Quantile regression with 999 quantiles outperforms MSE and TabPFNv2’s bin-based approach under preliminary RMSE evaluation.Point predictions average the quantiles, while probabilistic predictions reconstruct distributions using monotonicity enforcement and tail extrapolation.Source: §3 · pp. 4–5

4.1. Pretraining setup

• The three-stage schedule expands pretraining datasets from 1,024 samples to 60K samples while using about 35M datasets, versus 83M for TabICL and 130M for TabPFNv2.Stages use 500K, 40K, and 10K steps, respectively, with maximum learning rates of 8e-4, 1e-4, and 2e-5; stages 2 and 3 progressively improve performance, especially on large datasets.Source: §4.1 · p. 5
• Muon replaces AdamW, using orthogonalized gradient updates and a higher stage-1 maximum learning rate of 8e-4 versus TabICL’s 1e-4.The setup also uses 0.01 weight decay, stronger gradient clipping in stages 1 and 2, variable train/test sizes per micro-batch, and cosine learning-rate scheduling.Source: §4.1 · p. 5
• Pretraining costs 24.5 H100 GPU-days per model, lower than TabICL’s 60 A100-days.Stages 1, 2, and 3 take around 20, 2.5, and 2 GPU-days, respectively, on 80GB H100 GPUs.Source: §4.1 · p. 5

4.2. Inference optimizations

• Disk offloading and selective Q/K/V projections process 1M-sample, 500-feature tables in 450 seconds using under 24 GB CPU and 50 GB GPU.Combined with QASSMax, these optimizations support million-scale tables natively without retrieval or distillation.Source: §4.2 · p. 5

Prior design

• The new prior combines the structural causal model framework of TabPFN with innovations from TabICL and TabPFNv2, while targeting diverse datasets, useful inductive biases, and computational efficiency.Its development relies mainly on general design principles rather than fine-grained ablations, and TabICL-prior pretraining performs substantially worse.Source: §5 · p. 5

High-level structure

• Synthetic datasets sample global properties, generate a directed acyclic graph with random parent–child functions, propagate root-node vectors, and extract columns from randomly assigned nodes.Figure 4 summarizes this generation pipeline and its rejection-based filtering stage.Source: §5 · p. 5, Figure 4 · p. 6

Feature extraction

• Features use partially observed node dimensions to introduce noise; numerical columns extract single dimensions, whereas categorical columns use multiple dimensions with nearest-neighbor or softmax discretization.Unlike prior work, Gaussian noise is not added at the node level.Source: §5 · p. 6

Sampling mechanisms

• The prior adds random Cauchy graphs for varied global and local connectivity and correlated scalar sampling to reproduce shared categorical cardinalities across columns.The Cauchy-graph mechanism overcomes the tree-structure limitation of earlier random graph sampling.Source: §5 · pp. 6–7

Random functions

• The prior uses eight random-function types spanning smoothness and inductive biases, with parent matrices combined by concatenation or aggregation through sum, product, max, or logsumexp.The function set includes MLPs, tree ensembles, discretization, Gaussian processes, linear and quadratic functions, EM-inspired plateaus, and products.Source: §5 · p. 6, Figure 4 · p. 6

Data filtering

• Filtering removes datasets that fail bootstrap-tested ExtraTrees improvement or lack shared ancestors between features and targets, improving pretraining convergence.In stage 1, filtering removes roughly 35% of classification and 25% of regression datasets.Source: §5 · p. 6

Experimental setup

• Experiments evaluate 51 TabArena and 300 TALENT datasets using repeated cross-validation or fixed train/validation/test splits, with benchmark-specific classification and regression metrics.TabICLv2 uses 8 estimators on TabArena and 32 on TALENT, matching the comparison settings; TabICL uses the TabICLv1.1 checkpoint (Qu et al., 2025).Source: §6 · p. 7

Benchmark results

• TabICLv2 is state-of-the-art on both TabArena and TALENT, surpassing tuned and ensembled RealTabPFN-2.5 and heavily tuned CatBoost and XGBoost.A statistical test confirms that TabICLv2’s win rate versus RealTabPFN-2.5 on TALENT exceeds 50%, despite requiring orders of magnitude less training time than traditional methods.Source: §6 · p. 7

Runtime

• TabICLv2 is faster than TabPFN-2.5 across hardware, reaching 10.6× speedup on an H100 at 50K samples and 11.8× on CPU at 10K samples.The comparison uses 8 estimators, 100 features, classification, and 500 test samples.Source: §6 · p. 7, Figure 6 · p. 7

Many-class classification

• TabICLv2 substantially outperforms all baselines on TALENT datasets with more than 10 classes using either ECOC or native mixed-radix ensembling.ECOC is slightly better but 3× slower than native handling.Source: §6 · p. 7, Figure 7 · p. 7

Scaling

• TabICLv2 maintains top rankings from 10^3 to 10^5 samples, outperforms RealTabPFN-2.5 above 20K samples, and remains strong at approximately 600K samples.On the largest TALENT-extension datasets, TabPFN-2.5 produced out-of-memory errors, showing TabICLv2’s native large-scale handling.Source: §6 · pp. 7–8, Figure 8 · p. 8, Figure 9 · p. 8

Other experiments

• Additional experiments show that eight estimators capture most ensemble benefits, TabICLv2 ranks second to RealTabPFN-2.5 with 16 training samples, and it remains robust to label noise.It outperforms untuned classical baselines and Claude Opus 4.6 and Qwen3.5-112B in few-shot settings, while not benefiting from TabPFNv2’s random-forest extension.Source: §6 · p. 8

• The study varies one architectural, prior, or pretraining component at a time against a reference model across 60 validation datasets and two train/test splits.Training sets use up to 2,048 samples per dataset, with fewer used when datasets are smaller.Source: Figure 10 · p. 8, §7 · p. 8
• TabICLv2 matches RealTabPFN-2.5 after ≈200K steps in log-loss and in < 100K steps for normalized accuracy, despite the ablated model using only 280K pretraining steps.The reference checkpoint additionally uses longer pretraining and 8 rather than 4 attention heads in TFcol and TFicl.Source: §7 · p. 8
• TabICLv2 with the TabICL prior fails during pretraining, whereas the TabICL architecture with the TabICLv2 prior only matches TabICL, demonstrating an architecture–prior interaction.The results suggest that TabICLv2 requires higher prior diversity, consistent with the possibility that weak synthetic generators can break scaling laws (Ma et al., 2025).Source: §7 · p. 8
• The prior has the largest effect, while early target inclusion, Muon, and QASSMax each provide ≈100 Elo and 64% win rate gains; feature grouping and prior filtering help less.The ablation ordering is consistent across normalized accuracy, Elo, and log-loss.Source: §7 · pp. 8–9

8. Limitations

• TabICLv2 does not natively use semantic information from column names or textual features, although its feature scalability may support combining it with text embeddings.Semantic information has been shown valuable (Spinaci et al., 2025).Source: §8 · p. 9
• Datasets with millions of samples remain challenging despite improved scalability, while multi-output regression and distribution-shift handling remain future work.Distribution-shift handling is cited as an extension (Helli et al., 2024).Source: §8 · p. 9
• Distributional regression is evaluated only on toy datasets because established benchmarks are lacking, and missing values are currently mean-imputed.Missing indicators (Le Morvan & Varoquaux, 2025) or pretraining missingness are unexplored.Source: §8 · p. 9
• Hyperparameter tuning and fine-tuning (Rubachev et al., 2025) could improve performance but are not explored because they would increase runtime.Source: §8 · p. 9

• TabICLv2 achieves state-of-the-art performance while redefining the native scalability of tabular foundation models.Source: §9 · p. 9
• Its open-source prior, pretraining, and inference code democratize access to state-of-the-art tabular foundation models.Source: §9 · p. 9
• Moderate pretraining and inference costs make TabICLv2 a basis for adaptations to time-series, causality, and relational databases.Source: §9 · p. 9
• TabICLv2 emphasizes out-of-the-box performance and principled innovations over real-data fine-tuning (Garg et al., 2025) or deeper (Grinsztajn et al., 2025) and wider (Ma et al., 2025; Zhang et al., 2026) architectures.Source: §9 · p. 9

• JQ developed the regression framework and multiclass extensions, designed QASSMax, optimized implementation for large n, led experiments, and prepared production-ready code.Source: Contribution Statement · p. 9
• DH conceived and implemented the new generative prior and other architectural and pretraining enhancements, while also conducting small-scale experiments and implementing nanotabicl.Source: Contribution Statement · p. 9
• MLM and JQ managed larger-scale pretraining and systematic model-variant evaluation, while all authors contributed to steering, experimental design, and manuscript refinement.Source: Contribution Statement · p. 9

A.1. Repeated feature grouping

• Repeated feature groups generalize from triples to k columns, with m ≥ 2k ensuring that no pair of columns appears together in two groups.Although larger k may increase expressiveness, the model uses k = 3 because larger values were not beneficial in initial experiments.Source: §A.1 · p. 16

A.2. Compression then ICL

• TabICLv2 compresses columns with induced self-attention, aggregates features using four [CLS] tokens, and performs dataset-wise in-context learning with test-to-training attention.Induced self-attention reduces complexity from O(n^2) to O(nk), while QASSMax is applied to column-wise and dataset-wise attention for long-context generalization.Source: §A.2 · p. 16

A.3. Many-class classification

• Mixed-radix ensembling creates multiple at-most-10-class label views, and combined with hierarchical classification enables arbitrary-class tasks without retraining.The method operates at the column-wise embedding stage, while hierarchical classification operates during ICL.Source: §A.3 · pp. 17–18

A.4. Model configuration

• The classification model uses separate TFcol, TFrow, and TFicl components with 128-dimensional column and row representations, while regression replaces class embeddings with continuous-target layers and predicts 999 quantiles.The classification configuration includes three induced-attention blocks, four [CLS] tokens, and a 12-layer ICL transformer.Source: §A.4 · p. 18

B.1. Three pretraining stages

• Stage 3 reaches average rank 5.41 overall and 4.71 on datasets exceeding 10K samples, outperforming all baselines including RealTabPFN-2.5 at 6.35.Stage 1, Stage 2, and Stage 3 use progressively larger synthetic datasets; large-dataset rank improves from 14.91 to 5.50 to 4.71.Source: §B.1 · p. 19

B.2. Speed and memory optimization

• Automatic mixed precision is always enabled; stage 3 adds gradient checkpointing above 20K samples, while stages 2 and 3 use FlashAttention-3 for 1.3× average speedup over FlashAttention-2.Gradient checkpointing avoids out-of-memory errors during large-sample stage-3 training.Source: §B.2 · p. 19

C. Additional Ablation Results

• Across log-loss, normalized accuracy, and Elo, ablation rankings remain consistent; increased depth offers no clear log-loss gain, while added prior noise has negligible impact.The deeper model shows only a slight late-pretraining improvement on normalized accuracy and Elo, likely because it had not fully converged.Source: C. Additional Ablation Results · p. 21

D. Other Things We Tried

• Exploratory experiments found Muon outperformed AdEMAMix and AdamW variants, full row-wise attention was important, and several embedding, normalization, and residual alternatives provided little benefit.The authors caution that these smaller-scale pretraining observations are noisy and anecdotal.Source: D. Other Things We Tried · p. 21
• Other discarded options included ISQF regression, convolutional prior functions, missing-column masking, and increased attention heads, whose benefits were absent, unclear, or limited to small-scale runs.TabPFNv2-type architectures performed well but were discarded because of higher runtime complexity and per-step time.Source: D. Other Things We Tried · pp. 21, 23

E. Details on the Prior

• The prior is modularized into samplers for datasets, functions, points, matrices, and related objects, enabling diverse synthetic-data generation components.Some components are visualized in Appendix F.Source: E. Details on the Prior · p. 23

E.1. Differences to previous priors

• Compared with TabPFNv2, the prior adds correlated scalar and graph sampling, node and feature importances, richer converters, multiple-parent function handling, diversified functions, activations, matrices, weights, points, and filtering.New function families include CatBoost-style symmetric-tree ensembles, extended multivariate Gaussian-process functions, linear, quadratic, clustering-based, and product functions.Source: §E.1 · p. 23

E.2. Sampling correlated scalars

• Correlated scalar sampling assigns variables names and draws shared Beta-distribution parameters, then transforms sampled values into the required numeric, logarithmic, or integer ranges.Categorical correlated sampling was intended but unused because of a bug.Source: §E.2 · p. 24

E.3. Random dataset

• Synthetic datasets sample configurable column counts, categorical structure, random graphs, node functions, and train-test splits, with graph connectivity generated to produce diverse patterns.The random graph uses heavy-tailed Cauchy variables to permit both typical and exceptional connectivity patterns.Source: §E.3 · p. 24, §E.4 · p. 24

E.5. Random node function

• Random node processing standardizes and rescales multi-function outputs, while numerical and categorical converters provide multiple warping, discretization, neighbor-based, and softmax-based representations.Multiple parent nodes are combined either by concatenation or by sum, product, max, or logsumexp aggregation.Source: §E.5 · p. 24, §E.6 · p. 25, §E.7 · p. 25

E.8. Random functions

• The prior combines diverse random functions, including matrix-based constructions, EM-inspired assignments, and products of random functions.Inputs are subsampled to at most 20 dimensions when necessary, with linear and constant terms included by appending 1.Source: §E.8 · p. 27

E.9. Random activations

• Random activations standardize, randomly rescale, activate, and standardize again, sampling from fixed or parametric activation families.Fixed choices include nonlinear, discontinuous, trigonometric, ranking, and logarithmic functions; parametric choices randomize exponents or powers.Source: §E.9 · p. 27

E.10. Random matrix

• Random matrices use Gaussian, weighted, singular-value, kernel, or activation-based constructions, followed by noise addition and row normalization.The activation-matrix variant omits standardization and rescaling because its batch size is one; postprocessing prevents all-zero rows.Source: §E.10 · p. 28

E.11. Random weights

• Random positive weights model feature importances, singular-value decays, or unnormalized probability distributions by sampling power-law-shaped vectors and normalizing and shuffling them.The sampled exponent range permits both broadly distributed weights and vectors dominated by a few near-zero components.Source: §E.11 · p. 28

E.12. Random points

• Random points begin with several base distributions, including Gaussian, uniform, unit-ball, and covariance-point samples, before applying random functions.Covariance points are generated by elementwise random weighting followed by multiplication with a random Gaussian matrix.Source: §E.12 · p. 28

E.13. Postprocessing; E.14. Filtering; F. Plots for the Prior

• Datasets undergo single-value column removal, categorical encoding, outlier removal, standard scaling, and permutation, while ExtraTrees filtering rejects datasets whose out-of-bag predictions fail to beat mean-label MSE.The filter uses n estimators=25, bootstrap=True, max depth=6, and requires improvement over mean-label MSE on at least 95% of 200 bootstrap subsamples.Source: §E.13 · pp. 28–29, §E.14 · p. 29

G. Path Smoothness for Gaussian Processes

• For every s < (q − d)/2, a modification of X has paths in Hs(B) almost surely, whereas every modification has paths outside Hs(B) almost surely for s ≥ (q − d)/2.Source: G. Path Smoothness for Gaussian… · pp. 35–36

H.1. Efficient attention computation via selective query-key-value projections

• Selective query-key-value projections reduce final row-wise attention from O((c + m)^2 · d) to O(c · (c + m) · d) and avoid test-sample key-value projections.Layer normalization is computed once before slicing, and dataset-wise key-value projections are restricted to training samples.Source: §H.1 · p. 37

H.2. Offloading technique

• Hierarchical CPU-and-disk offloading addresses the approximately 250 GB intermediate embedding tensor required for a table with 1 million samples and 500 features.Disk offloading uses memory-mapped files, asynchronous transfers, periodic flushing, and automatic resource-based mode selection.Source: §H.2 · pp. 37–38
• 115s versus 450s: CPU offloading is faster, while disk offloading uses under 24 GB CPU memory and under 50 GB GPU memory for million-scale inference.The comparison processes 1 million samples and 500 features; disk offloading additionally uses 250 GB of disk space.Source: §H.2 · p. 38

I.1. Problem setup

• TabICLv2 constructs a valid regression distribution from 999 quantiles by enforcing monotonicity, extrapolating tails, and providing closed-form PDF, CDF, and moments.The predicted probability levels are uniformly spaced from 0.001 to 0.999; raw quantiles can cross, omit extreme tails, and lack analytical statistics.Source: §I.1 · p. 40

I.2. Quantile function

• The complete quantile function combines exponential left and right tails with a piecewise linear interior spline whose corrected slopes are non-negative.Tail models are designed for sub-exponential distributions such as Gaussian ones and are joined continuously at the boundary quantiles.Source: §I.2 · p. 40, §I.4 · p. 42

I.3. Quantile crossing correction

• Sorting enforces monotonicity in O(K log K) and is used by default because GPU sorting outperforms PAVA while quantile crossings are rare.Numba-compiled PAVA provides competitive CPU performance and exceeds SciPy’s implementation at large batch sizes, but remains slower than CUDA sorting.Source: §I.3 · pp. 41–42, Figure I.1 · p. 42

I.4. Tail parameter estimation

• Tail scales βL and βR are estimated by log-space ordinary least squares from boundary quantiles and clamped to [0.01, 100] for stability.The default tail region uses 20 quantiles.Source: §I.4 · p. 42

I.5. Cumulative distribution function (CDF)

• The resulting distribution supports inverse-CDF evaluation, derivative-based PDF computation, and analytical CRPS integration across spline and tail regions.CRPS is computed analytically by integrating over each region, while log densities avoid numerical issues for very small values.Source: §I.5 · p. 42, §I.6 · p. 43, §I.7 · p. 43

I.7.1. CRPS CONTRIBUTION FROM SPLINE REGION

• CRPS is computed by summing contributions from each linear spline segment using clamped CDF values.The section separately formulates the first and second integrals over spline regions before giving the total spline CRPS.Source: §I.7.1 · p. 44

I.7.2. CRPS CONTRIBUTION FROM EXPONENTIAL LEFT TAIL

• CRPS contributions for the left and right exponential tails use clamped CDF values and piecewise expressions conditioned on whether z lies beyond the corresponding tail boundary.The left-tail expression uses βL and bL, while the right-tail expression uses aR and bR.Source: §I.7.2 · p. 44, §I.7.3 · p. 44

I.8.1. MEAN

• The distributional mean and variance are computed from the reconstructed quantile distribution, including spline-region contributions for variance.The variance derivation explicitly includes the linear-segment term involving adjacent quantiles.Source: §I.8.1 · p. 45, §I.8.2 · p. 45

I.9. Empirical validation on synthetic regression tasks

• Four synthetic regression datasets test distributional reconstruction against known ground-truth quantities, PDFs, and CDFs.The datasets cover homoscedastic Gaussian noise, heteroscedastic uncertainty, discontinuities, and heavy-tailed Gaussian-mixture noise.Source: §I.9 · p. 46, §I.9.1 · p. 46

I.9.1. SYNTHETIC REGRESSION DATASETS

• The quadratic dataset provides a homoscedastic baseline whose predictive distributions should have uniform spread and symmetric Gaussian PDFs around the quadratic mean.The conditional distribution is N(0.15x^2 − 0.5, 0.25^2).Source: §I.9.1 · p. 46
• The heteroscedastic sinusoidal dataset requires wider predicted quantile intervals at larger |x| than near the center.Its noise scale is σ(x) = 0.12 + 0.1|x|.Source: §I.9.1 · p. 46
• The step-function dataset tests whether predicted distributions handle the abrupt transition between two modes at x = 0.Its conditional distribution is Gaussian around sign(x) with variance 0.3^2.Source: §I.9.1 · p. 46
• The heavy-tailed linear dataset uses a 90% inlier and 10% outlier Gaussian mixture to test extreme-quantile tail extrapolation.The component scales are σ1 = 0.2 and σ2 = 0.8, producing heavier tails than a pure Gaussian.Source: §I.9.1 · p. 46

I.9.2. VISUALIZATION AND ANALYSIS

• Predicted quantile curves and functions closely match ground truth, capturing heteroscedastic widening toward |x| = 3 and smoothly extrapolating beyond [0.001, 0.999].The quantile-function visualization covers representative inputs x ∈ {−2, 0, 2} and shows crossing correction.Source: §I.9.2 · p. 46
• Predicted PDFs and CDFs align with ground truth, reproducing conditional-distribution location, spread, and smooth cumulative behavior.For the heteroscedastic dataset, predicted PDFs are narrower at x = 0 and broader at x = ±2.Source: §I.9.2 · p. 46
• Density heatmaps recover the heteroscedastic funnel and the step function’s two horizontal bands separated at x = 0.The heatmaps visualize conditional density f(y|x) across the input domain on a log scale.Source: §I.9.2 · p. 47
• Inverse-transform resampling produces synthetic points that closely match the original training data.Samples are drawn using y = Q(U) with U ∼ Uniform(0, 1).Source: §I.9.2 · p. 47

J.1. Aggregation metrics

• TabArena computes per-dataset errors using 1 − ROC AUC for binary classification, Log-Loss for multiclass classification, and RMSE for regression.Errors are averaged over all outer cross-validation folds before aggregation.Source: §J.1 · p. 49
• Elo compares pairwise wins, ties, and losses with equal dataset contribution, whereas improvability measures magnitude-sensitive gaps from the best method and average rank summarizes mean dataset rank.A 400-point Elo gap implies a 10:1 expected win rate; improvability ranges from 0% optimal to 100%, and lower average rank is better.Source: §J.1 · p. 49
• TabICLv2 consistently achieves state-of-the-art performance across TabArena’s Elo, improvability, and average-rank metrics.The main paper primarily reports improvability, while Elo and rankings are provided for completeness.Source: §J.1 · p. 49

J.2. Results on all datasets

• TabICLv2 achieves average rank 4.82 across 51 TabArena datasets, outperforming AutoGluon 1.4 (extreme, 4h) at 5.24 and RealTabPFN-2.5 (T+E) at 5.88 without tuning.This result uses a single forward pass, whereas AutoGluon 1.4 uses extensive tuning, ensembling, and a 4-hour budget.Source: §J.2 · p. 49
• TabICLv2 wins 57% of pairwise comparisons against AutoGluon 1.5 and 59% against RealTabPFN-2.5 (T+E), despite AutoGluon 1.5’s better average rank of 3.88 versus 4.82.The discrepancy reflects average rank’s sensitivity to occasional poor dataset performance, suggesting TabICLv2 may struggle outside its pretraining distribution.Source: §J.2 · p. 50, Figure J.2 · p. 51
• TabICLv2 dominates the Pareto fronts for improvability versus runtime and Elo versus runtime, being faster and more accurate than competing tabular foundation models.The comparison includes RealTabPFN-2.5, TabPFN-2.5, TabICL, LimiX, and Mitra.Source: §J.2 · p. 50, Figure J.3 · p. 51, Figure J.4 · p. 52

J.3. Results on binary classification datasets

• On 24 binary-classification datasets, TabICLv2 ranks 4.43, behind AutoGluon 1.5 at 3.83 but ahead of RealTabPFN-2.5 (T+E) at 6.90, and remains strongest among tabular foundation models on Pareto fronts.TabArena binary classification results are presented across the section’s figures.Source: §J.3 · p. 53, Figure J.6 · p. 53, Figure J.7 · p. 53

J.4. Results on multiclass classification datasets

• On 14 multiclass datasets, TabICLv2 ranks 6.75, behind AutoGluon 1.5 at 4.00, RealTabPFN-2.5 (T+E) at 4.25, and AutoGluon 1.4 at 4.50, while beating TabPFNv2 (T+E) at 8.62 and LimiX at 9.31.RealTabPFN-2.5 uses tuning and ensembling, whereas TabICLv2 uses neither.Source: §J.4 · p. 54

J.5. Results on regression datasets

• On 13 regression datasets, TabICLv2 ranks 4.54, tying RealTabPFN-2.5 (T+E) and trailing AutoGluon 1.5 at 3.92.TabDPT ranks 5.31 on regression, prompting questions about possible training-corpus leakage with TabArena regression datasets.Source: §J.5 · p. 56

K.1. Benchmark overview

• TALENT contains 300 datasets—120 binary-classification, 80 multiclass-classification, and 100 regression datasets—split 64%/16%/20% into training, validation, and test sets.Classification uses accuracy and regression uses RMSE; aggregated metrics include improvability, Elo, and average rank, with development datasets excluded from the main-paper comparison.Source: §K.1 · p. 57

K.2. Results on all datasets

• 4.66 average rank and 62%/65% pairwise win rates establish TabICLv2’s overall lead over RealTabPFN-2.5 and TabPFN-2.5 on TALENT.TabICLv2 also substantially outperforms LimiX and TabPFNv2, whose average ranks are 8.34 and 8.82.Source: §K.2 · p. 57, Figure K.1 · p. 58, Figure K.2 · p. 58

K.3. Results on binary classification datasets

• TabICLv2 leads on binary-classification AUC and log-loss—3.31 vs. 4.62 and 2.83 vs. 3.78 against RealTabPFN-2.5—while remaining essentially tied on accuracy, 4.98 vs. 4.82.The probabilistic-metric advantage indicates better probability estimates for uncertainty quantification and decision-making beyond fixed-threshold predictions.Source: §K.3 · p. 60

K.4. Results on multiclass classification datasets (≤10 classes) / K.5. Results on multiclass classification datasets (> 10 classes)

• TabICLv2 leads RealTabPFN-2.5 across multiclass tasks, with accuracy 4.58 vs. 5.64, AUC 3.38 vs. 5.20, and log-loss 2.67 vs. 4.48.For datasets with more than 10 classes, both native mixed-radix ensembling and the ECOC wrapper outperform RealTabPFN-2.5-ECOC.Source: §K.4 · p. 63, §K.5 · p. 66

K.6. Results on regression datasets

• TabICLv2 outperforms TabPFN-2.5 on regression RMSE and R2, while TabPFN-2.5 has a small MAE edge, 4.43 vs. 4.63.Its 999-quantile pinball-loss training targets the full conditional distribution, supporting variance and extreme-value modeling rather than only median prediction.Source: §K.6 · p. 68

K.7. Results on small datasets with less than 10K samples / K.8. Results on large datasets with more than 10K samples / K.9. Model rankings with respect to meta-features

• TabICLv2 performs comparably to RealTabPFN-2.5 on small datasets, while demonstrating a clear advantage over both RealTabPFN-2.5 and TabPFN-2.5 on large datasets.The supplied figures show model rankings by meta-features across all TALENT datasets, classification datasets, and regression datasets.Source: §K.7 · p. 71, §K.8 · p. 72, Figure K.21 · p. 74

L.1. Effect of ensemble size / L.2. Direct statistical comparison to RealTabPFN-2.5

• Most ensemble-size gains are captured by E = 8, although performance continues improving with diminishing returns; direct binomial and Wilcoxon tests find statistically significant TabICLv2 comparisons.The binomial comparison tests whether TabICLv2’s win rate exceeds 50% against RealTabPFN-2.5 and TabPFN-2.5.Source: §L.1 · p. 76, §L.2 · p. 76, Table L.1 · p. 77

L.3. Few-shot benchmark / L.4. Few-shot comparison to LLMs

• In the 16-sample few-shot benchmark, TabICLv2 ranks second only to RealTabPFN-2.5 and beats all traditional methods despite no pretraining below 300 training samples; from 16 shots onward, it dominates the tested LLMs.The LLM comparison uses 9 classification datasets with train sizes of 8, 16, and 64; LLMs are competitive only at 8 shots and TabICLv2 is orders of magnitude more efficient.Source: §L.3 · p. 76, §L.4 · p. 76, Table L.3 · p. 77

L.5. Benchmarking TabPFN’s Random Forest extension / L.6. Noise resilience

• The random-forest extension degrades TabICLv2 by fragmenting useful context and adding latency, whereas replacing up to 50% of training labels with random values causes less degradation than for tree-based methods.The random-forest approach reduces peak GPU memory, but TabICLv2 already handles large datasets with QASSMax; the noise study uses six low-noise binary TabArena datasets.Source: §L.5 · p. 76, §L.6 · pp. 77, 79, Figure L.1 · p. 78