TabICLv2: A better, faster, scalable, and open tabular foundation model

Motivation

• Tabular foundation models have recently matched or surpassed gradient-boosted decision trees on tables with up to 100K samples, while tabular data spans applications from healthcare to fraud detection.Their single-forward-pass Transformer design also supports causal inference, generative modeling, joint predictive distributions, and simulation-based inference.Source: §1 · p. 1
• Fully open-source tabular foundation models are needed to democratize access to top-tier performance and demystify the recipes behind leading closed-source systems.The paper frames openness as a way to foster research on downstream adaptations.Source: §1 · p. 1

Contributions

• TabICLv2 is introduced as a state-of-the-art tabular foundation model combining architectural innovations, pretraining improvements, a novel synthetic data generator, extensive evaluations, and ablation studies.These contributions are covered in Sections 3–7.Source: §1 · pp. 1–2

2.1. Tabular foundation models

• PFN-style tabular foundation models predict test-label distributions directly from training data and test inputs, enabling in-context learning without gradient updates.Pretraining samples datasets from a prior distribution and minimizes predictive loss.Source: §2.1 · p. 2
• Architectures evolved from row-token attention in TabPFN to cell-based alternating row and column attention in TabPFNv2, while TabICL reduces complexity to O(n^2+nm^2).TabPFNv2 has O(n^2m + nm^2) complexity, whereas TabICL first constructs fixed-dimensional row embeddings before in-context learning.Source: §2.1 · p. 2
• Synthetic-prior research mixes structural causal models, tree-based priors, DAGs, hierarchical SCMs, and temporal modulation, while real-data pretraining can also be competitive or beneficial.TabDPT reports competitive large-scale real-data pretraining, and Real-TabPFN reports gains from continued real-data pretraining on TabPFNv2.Source: §2.1 · p. 2
• Beyond pretraining, tabular foundation models use fine-tuning, retrieval-based context selection, and distillation into compact MLPs or trees for adaptation and scalability.These strategies shift learned priors, select contexts under compute constraints, or compress models.Source: §2.1 · p. 2
• LLMs adapted through table serialization and continued pretraining are promising but underperform tabular foundation models when sufficient training data is available.This comparison concerns table-native TFMs versus LLM-based tabular models.Source: §2.1 · p. 2

2.2. Attention struggles with long-context generalization

• Standard softmax attention fades as context length grows, flattening attention distributions and limiting generalization from shorter training sequences to longer ones.Length-aware alternatives include SSMax scaling by s log n, theoretically necessary log n scaling, ASEntmax content-aware scaling, and query-dependent temperature.Source: §2.2 · p. 2

Input embeddings

• TabICLv2 uses repeated feature grouping and target-aware embedding to break feature symmetries while preserving fine-grained information and injecting target signals into every feature.Each column is placed into multiple circularly shifted groups, and target embeddings are added directly to all features rather than appended as a column.Source: §3 · p. 3

Compression then ICL

• The architecture compresses column representations, aggregates features row-wise, and performs dataset-wise in-context learning, applying QASSMax in inducing-point aggregation and ICL attention.TFcol uses a set transformer, TFrow collapses feature embeddings with [CLS] tokens, and TFicl lets test samples attend to training samples.Source: §3 · p. 3

Query-aware scalable softmax

• QASSMax preserves 100% accuracy with low attention entropy at 15K negatives, whereas attention without scalable softmax collapses and SSMax largely degrades.QASSMax combines log n scaling with bounded, query-aware element-wise gating to mitigate attention fading and improve long-context generalization.Source: §3 · pp. 3–4, Figure 3 · p. 4

Many-class classification

• Mixed-radix ensembling combined with hierarchical classification enables TabICLv2 to handle an arbitrary number of classes.For more than 10 classes, labels are decomposed into balanced radix digits, TFcol is run per digit, and outputs are averaged.Source: §3 · p. 4

Quantile predictions for regression

• TabICLv2 predicts 999 quantiles with pinball loss, outperforming MSE and TabPFNv2’s bin-based approach in preliminary RMSE experiments.Averaging predicted quantiles provides fast point estimates, while monotonicity enforcement supports full probabilistic distributions and derived moments.Source: §3 · pp. 4–5

4.1. Pretraining setup

• Stages 2 and 3 progressively improve performance, especially on large datasets.The three-stage schedule expands dataset sizes from 1,024 samples to as many as 60K samples while retaining up to 100 features.Source: §4.1 · p. 5
• The revised setup uses Muon instead of AdamW, with higher learning rates, cautious weight decay, stronger gradient clipping, varied micro-batch sizes, and cosine scheduling.Stage 1 uses a maximum learning rate of 8e-4 versus 1e-4 for AdamW in TabICL.Source: §4.1 · p. 5
• Pretraining totals 24.5 H100 GPU-days, lower than TabICL’s 60 A100-days.The stages require approximately 20, 2.5, and 2 GPU-days, respectively.Source: §4.1 · p. 5

4.2. Inference optimizations

• Disk offloading processes 1M-sample, 500-feature tables within 450 seconds using under 24 GB CPU and 50 GB GPU memory.Combined with QASSMax, this supports million-scale tables natively without retrieval or distillation; selective Q/K/V projections reduce redundant computation.Source: §4.2 · p. 5

Prior design

• The new synthetic prior is key to final performance, as pretraining with the TabICL prior yields substantially lower performance.It maximizes dataset diversity while encoding useful inductive biases and preserving computational efficiency.Source: §5 · p. 5

Dataset generation

• The prior samples global dataset properties, causal graphs, random functions, and feature extractions to generate varied numerical and categorical datasets.Random vectors propagate through graph nodes, while selected node dimensions become observed features and unobserved dimensions introduce noise.Source: §5 · pp. 5–6, Figure 4 · p. 6

New sampling mechanisms

• A random Cauchy graph mechanism extends prior sampling beyond tree-structured graphs by modeling diverse global and local node connectivities.Eight random-function types span different smoothness properties and inductive biases, with multiple-parent relationships combined through several aggregation strategies.Source: §5 · p. 6, Figure 4 · p. 6

Data filtering

• Filtering removes datasets that fail bootstrap-based ExtraTrees improvability tests or lack causal connectivity between features and targets, improving pretraining convergence.During pretraining stage 1, roughly 35% of classification and 25% of regression datasets are filtered.Source: §5 · p. 6

Benchmark setup

• Evaluation uses 51 TabArena datasets and 300 TALENT datasets, with repeated cross-validation for TabArena and fixed train/validation/test splits for TALENT.TabArena uses ROC AUC, log-loss, and RMSE by task; TALENT selects hyperparameters using validation accuracy or RMSE.Source: §6 · p. 7

Overall results

• TabICLv2 dominates the improvability–runtime Pareto fronts on TabArena and TALENT, surpassing tuned, ensembled RealTabPFN-2.5 and heavily tuned CatBoost and XGBoost without tuning.The comparisons are reported for the two benchmark suites using improvability, which measures the average relative error gap to the best method per dataset.Source: §6 · p. 7

Runtime

• For 100 features, TabICLv2 is faster than TabPFN-2.5 across all hardware, reaching 10.6× speedup on an H100 at 50K samples and 11.8× on CPU at 10K samples.Both models use 8 estimators in the runtime comparison.Source: §6 · p. 7, Figure 6 · p. 7

Many-class classification

• On TALENT datasets with more than 10 classes, both ECOC and native mixed-radix ensembling substantially outperform all baselines, while ECOC is slightly better but 3× slower.The comparison covers normalized accuracy across 12 many-class datasets.Source: §6 · p. 7, Figure 7 · p. 7

Large-scale datasets

• TabICLv2 maintains top rankings from 10^3 to 10^5 samples, outperforms RealTabPFN-2.5 above 20K samples, and remains strong on 600K-sample datasets.These results extend the demonstrated scale of tabular foundation models to larger datasets.Source: §6 · p. 7

• TabICLv2 matches RealTabPFN-2.5 in log-loss after approximately 200K steps and in normalized accuracy in fewer than 100K steps.The comparison concerns the reference checkpoint versus an ablation trained for 280K steps; the official model uses 500K stage-1 steps plus stages 2 and 3.Source: §7 · p. 8
• Using the TabICL prior with the TabICLv2 architecture fails, whereas using the TabICLv2 prior with the TabICL architecture only matches TabICL.The results indicate that TabICLv2 requires higher prior diversity, while TabICL has limited ability to exploit it.Source: §7 · p. 8
• The prior produces the largest ablation effect, while early target inclusion, Muon, and QASSMax each add approximately 100 Elo and a 64% win rate.Repeated feature grouping and prior filtering provide smaller gains; the ordering is consistent across normalized accuracy, Elo, and log-loss.Source: §7 · p. 8, Figure 10 · p. 8
• The ablation study evaluates architectural, prior, and pretraining choices on 60 validation datasets using two train/test splits and up to 2,048 training samples per dataset.Figure 10 compares official checkpoints with models pretrained for 280K steps and includes an AdamW configuration using regular weight decay and a 1e-4 learning rate.Source: §7 · p. 7, Figure 10 · p. 8

8. Limitations

• TabICLv2 does not natively use semantic information from column names or textual features, although text embeddings may remain feasible given its feature scalability.Semantic information has been shown valuable (Spinaci et al., 2025).Source: §8 · p. 8
• Datasets with millions of samples remain challenging despite improved scalability, while multi-output regression and distribution-shift handling remain future work.The latter extension is associated with Helli et al. (2024).Source: §8 · p. 8
• Distributional regression is not evaluated beyond toy datasets because established benchmarks are lacking.This limitation is documented in Appendix I.9.Source: §8 · p. 8
• Missing values are currently mean-imputed, while missing indicators or pretraining missingness remain unexplored.Adding missing indicators is associated with Le Morvan & Varoquaux (2025).Source: §8 · p. 8
• Hyperparameter tuning or fine-tuning (Rubachev et al., 2025) could improve performance but may increase runtime and is not explored.The paper leaves this performance–runtime trade-off untested.Source: §8 · pp. 8–9

• TabICLv2 achieves state-of-the-art performance and redefines the native scalability of tabular foundation models.Source: §9 · p. 9
• The model combines moderate pretraining and inference costs with a commitment to fully open-source the system, enabling future adaptations and broader access.Source: §9 · p. 9
• TabICLv2 prioritizes out-of-the-box performance and principled innovations over fine-tuning on real data (Garg et al., 2025) or scaling deeper (Grinsztajn et al., 2025) and wider (Ma et al., 2025a; Zhang et al., 2025a) architectures.Source: §9 · p. 9

• JQ developed the regression framework and multiclass extensions, designed QASSMax, enabled large-n scaling, led most experiments and benchmarks, and prepared production-ready code.JQ and DH contributed equally as co-first authors.Source: Contribution Statement · p. 9
• DH conceived and implemented the new generative prior and other core architectural and pretraining enhancements, and implemented nanotabicl.DH also conducted small-scale experiments.Source: Contribution Statement · p. 9
• MLM and JQ managed larger-scale pretraining runs and systematically evaluated model variants.Source: Contribution Statement · p. 9
• All authors jointly contributed to weekly technical steering, experimental design, and iterative manuscript refinement.Source: Contribution Statement · p. 9

A.1. Repeated feature grouping

• Column groups of size k have pairwise intersections of at most one when m ≥2k, although the model uses k = 3 because larger values were not beneficial initially.The grouping construction is intended to increase architectural expressivity while avoiding repeated column pairs under the stated condition.Source: §A.1 · p. 15

A.2. Compression then ICL

• TabICLv2 compresses columns with induced self-attention, aggregates rows using four [CLS] tokens, and performs dataset-wise in-context learning where test samples attend only to training samples.The column-wise stage reduces complexity from O(n^2) to O(nk), while QASSMax is applied to column and dataset-wise attention for long-context generalization.Source: §A.2 · p. 15

A.3. Many-class classification

• Mixed-radix ensembling represents many-class labels through multiple views with at most 10 classes, and combined with hierarchical classification supports arbitrary class counts without retraining.Balanced mixed-radix views are combined at the embedding level, while hierarchical classification operates during in-context learning.Source: §A.3 · pp. 15–16

A.4. Model configuration

• The classification model uses three induced-attention blocks with 128 inducing vectors and d = 128, while its row and ICL transformers use 3 and 12 layers, respectively.Separate classification and regression checkpoints are trained; regression replaces class embeddings with continuous-target linear embeddings and predicts 999 quantiles.Source: §A.4 · p. 17

B.1. Three pretraining stages

• The three-stage curriculum uses 500K, 40K, and 10K steps while increasing synthetic dataset sizes from 1,024 to as many as 60,000 samples.Stage 2 samples 400–10,240 examples log-uniformly, and Stage 3 samples 400–60,000 examples log-uniformly, with batch size 64.Source: §B.1 · p. 18
• Stage 3 reaches average rank 5.41 overall and 4.71 on datasets above 10K samples, outperforming RealTabPFN-2.5 at 6.35.Average rank improves from 9.94 in Stage 1 to 5.69 in Stage 2 and 5.41 in Stage 3; on large datasets, Stage 1 is 14.91 and Stage 2 is 5.50.Source: §B.1 · p. 18

B.2. Speed and memory optimization

• Automatic mixed precision is always enabled, gradient checkpointing is added above 20K samples in Stage 3, and FlashAttention-3 delivers an average 1.3× speedup over FlashAttention-2.These measures target memory safety and faster large-scale pretraining.Source: §B.2 · p. 18

B.1. Three pretraining stages

• Figure B.1 summarizes TabICLv2’s critical-difference performance across the three pretraining stages on the TALENT benchmark.The detailed stage-wise results show progressively improving average ranks, especially for datasets with more than 10K samples.Source: §B.1 · p. 18, Figure B.1 · p. 19

C. Additional ablation results

• Across log-loss, normalized accuracy, and Elo, ablation ordering remains consistent, supporting the same conclusions about each component’s contribution.The additional metrics are reported in Figure C.1a and Figure C.1b alongside the main-text log-loss results.Source: C. Additional ablation results · p. 20, Figure C.1 · p. 21
• Deeper TFcol, TFrow, and TFicl models show no clear log-loss improvement, with only slight late-pretraining gains in normalized accuracy and Elo.The marginal gains may reflect insufficient pretraining for the larger model, so the authors did not pursue scaling model size further.Source: C. Additional ablation results · p. 20
• Adding Gaussian edge noise to the prior, following TabPFNv2, has negligible impact across all metrics.Source: C. Additional ablation results · p. 20

D. Other things we tried

• The authors caution that observations from smaller, incompletely analyzed pretraining experiments are noisy and should be treated as anecdotal evidence.They identify reducing pretraining noise as a potential direction for future research.Source: D. Other things we tried · p. 20
• Muon outperformed AdEMAMix in tested runs, while schedule-free AdamW, label-smoothing decay, and weight-decay variants showed no clear advantage over selected alternatives.AdEMAMix helped versus AdamW in small-scale runs, but combining them did not help; lowering AdamW β2 improved shorter runs, and decaying label smoothing matched baseline performance.Source: D. Other things we tried · p. 20
• Full row-wise attention is important, whereas induced self-attention performed considerably worse; MLP embeddings, added log(n), mixed attention stages, and separate y embeddings were not beneficial.Normalization changes also showed no improvement, while bias-free or parameter-free layernorms appeared similarly effective but faster with limited confidence.Source: D. Other things we tried · p. 20

E. Details on the prior

• The prior decomposes synthesis into modular samplers for datasets, functions, points, matrices, and related objects.This modular organization structures the implementation details described across the prior subsections.Source: E. Details on the prior · p. 22

E.1. Differences to previous priors

• Compared with TabPFNv2, the prior adds correlated scalars, random graphs, node and feature importances, random converters, diverse functions, activations, matrices, weights, points, and filtering.Its function family includes CatBoost-style symmetric-tree ensembles, extended multivariate Gaussian processes, and random linear, quadratic, clustering-based, and product functions.Source: §E.1 · p. 22

E.8. Random functions; E.9. Random activations; E.10. Random matrix; E.11. Random weights; E.12. Random points

• The prior combines random functions, activation functions, matrices, weights, and point distributions to increase synthetic-data diversity.Random functions include compositions and products; matrices use five sampling types, while points may be Gaussian, uniform, unit-ball, or covariance-transformed.Source: §E.8 · p. 26, §E.9 · p. 26, §E.10 · p. 27

E.9. Random activations

• Random activations standardize inputs, apply random rescaling and a sampled activation, then standardize again, mixing fixed and parametric choices.With probability 2/3, the method selects a fixed activation; otherwise it uses a parametric activation with randomly sampled parameters.Source: §E.9 · p. 26

E.10. Random matrix

• Random matrices are sampled from one of five types, including weighted, singular-value, kernel, and activation-based constructions, with row normalization applied in the described procedures.The activation-based variant adds Gaussian noise with standard deviation 10−3. Postprocessing adds 1e-6 times a random Gaussian matrix before normalizing each row, preventing all-zero rows.Source: §E.10 · p. 27

E.13. Postprocessing

• Datasets are cleaned by removing constant columns, rejecting invalid class or split configurations, encoding categoricals, standard-scaling columns, and permuting column and class orders.These operations are applied after removing outliers from all columns and, for regression, the target y.Source: §E.13 · p. 27

E.14. Filtering

• The ExtraTrees filter rejects a dataset unless out-of-bag predictions beat the mean-label MSE on at least 95% of 200 bootstrap subsamples.Classification is converted to regression with one-hot encoding; the filter uses 25 estimators, bootstrap=True, and max depth=6.Source: §E.14 · p. 28

G. Path smoothness for Gaussian processes

• For Gaussian processes with spectral decay exponent q > 2d, sample paths lie almost surely in Hs(B) for every s < (q −d)/2 and in no Hs(B) for s ≥(q −d)/2.The result also identifies the associated kernel through the inverse Fourier transform of an integrable, even spectral density.Source: G. Path smoothness for Gaussian… · p. 34

H.1. Efficient attention computation via selective query-key-value projections

• Selective query, key, and value projections reduce attention computation by restricting outputs to required positions and omitting test-sample keys and values.Row-wise attention reduces the final-block query cost from O((c + m) · d^2) to O(c · d^2), while dataset-wise attention reduces each key/value cost to O(ntrain · d^2).Source: §H.1 · p. 36

H.2. Offloading technique

• Hierarchical disk offloading extends TabICL’s CPU offloading to make tables with millions of samples accessible under constrained hardware resources.The method addresses an intermediate embedding tensor requiring approximately 250GB for 1M samples and 500 features by streaming it through memory-mapped files.Source: §H.2 · pp. 36–37
• 115s CPU offloading versus 450s disk offloading enables million-sample inference with 24GB CPU and 50GB GPU memory, compared with 250GB CPU memory for CPU offloading.The comparison uses a 1M-sample, 500-feature table with 80% training and 20% test data; disk offloading uses 250GB of disk space.Source: §H.2 · p. 37, Figure H.2 · p. 38, Figure H.1 · p. 38

I.1. Problem setup

• TabICLv2 converts 999 quantiles spanning α = 0.001 to 0.999 into a valid full distribution by enforcing monotonicity, extrapolating tails, and providing analytical statistics.These steps address quantile crossing, undefined extreme tails, and the absence of direct PDF, CDF, and moment representations.Source: §I.1 · p. 39

I.2. Quantile function

• The complete quantile function combines a piecewise-linear spline on [αL, αR] with parametric exponential models in the left and right tails.The spline slopes must be non-negative, while tail parameters are chosen for continuity at the boundaries and suitability for sub-exponential distributions.Source: §I.2 · p. 39, §I.4 · p. 41

I.3. Quantile crossing correction

• Sorting is the default monotonicity correction because quantile crossing is rare and GPU sorting outperforms the Numba-based PAVA implementation.PAVA offers an O(K) solution and competitive CPU performance, but its sequential dependencies limit vectorization; tail scales are clamped to [0.01, 100].Source: §I.3 · pp. 40–41, §I.4 · p. 41

I.5. Cumulative distribution function (CDF)

• The constructed quantile function yields the CDF by inversion and the PDF from the inverse-function derivative, with region-specific formulas for spline and tail regions.Log-density computation is used to avoid numerical issues with very small densities.Source: §I.5 · pp. 41–42, §I.6 · p. 42

I.7. Continuous ranked probability score (CRPS)

• CRPS is computed analytically in quantile space by integrating the pinball-loss decomposition across the spline and tail regions.For each spline segment, the contribution uses the CDF value clamped to that segment’s probability interval.Source: §I.7 · p. 42, §I.7.1 · p. 43

I.7.1. CRPS CONTRIBUTION FROM SPLINE REGION

• CRPS is decomposed into spline, left exponential-tail, and right exponential-tail contributions using clamped CDF values and piecewise expressions.The section also provides formulas for the distribution mean and variance, including spline contributions.Source: §I.7.1 · p. 43, §I.7.2 · p. 43, §I.7.3 · p. 43

I.9. Empirical validation on synthetic regression tasks

• Four synthetic datasets test whether QuantileDistribution reconstructs predicted and true distributional quantities, PDFs, and CDFs.The tasks cover homoscedastic Gaussian, heteroscedastic, discontinuous, and heavy-tailed behavior.Source: §I.9 · p. 45

I.9.1. SYNTHETIC REGRESSION DATASETS

• The quadratic dataset provides a constant-variance Gaussian baseline with uniformly spread predictive uncertainty around a quadratic mean.Its conditional distribution is N(0.15x^2 −0.5, 0.25^2).Source: §I.9.1 · p. 45
• The sinusoidal dataset tests input-dependent uncertainty, requiring wider quantile intervals at larger |x| than near the center.Its noise scale is σ(x) = 0.12 + 0.1|x|.Source: §I.9.1 · p. 45
• The step-function dataset tests distributional behavior at a discontinuity where the conditional mean switches abruptly at x = 0.The conditional distribution is N(sign(x), 0.3^2).Source: §I.9.1 · p. 45
• The heavy-tailed linear dataset uses a 90%/10% Gaussian mixture to test whether exponential tails capture extreme quantiles.The mixture combines scales σ1 = 0.2 and σ2 = 0.8 around mean 0.3x.Source: §I.9.1 · p. 45

I.9.2. VISUALIZATION AND ANALYSIS

• Predicted distributions reproduce conditional location, spread, and heteroscedasticity: quantile bands widen toward |x| = 3, while PDFs narrow at x = 0 and broaden at x = ±2.These patterns match the known ground-truth behavior in the heteroscedastic synthetic dataset.Source: §I.9.2 · p. 45
• Predicted quantile functions closely match ground truth across representative inputs, with crossing correction and smooth exponential-tail extrapolation beyond [αL, αR] = [0.001, 0.999].This visualization is part of Figure I.2, which validates QuantileDistribution on four synthetic regression tasks with known distributions.Source: §I.9.2 · p. 45, Figure I.2 · p. 47
• Predicted CDFs closely follow the true smooth S-curves across the evaluated synthetic datasets.The CDF comparison is shown as the fourth row of the six-row visualization.Source: §I.9.2 · p. 45
• Density heatmaps capture dataset-specific structure, including a funnel-shaped heteroscedastic density and two step-function bands separated at x = 0.The heatmaps visualize conditional density f(y|x) across the input domain on a log scale.Source: §I.9.2 · p. 46
• Inverse-transform resampling produces synthetic points that closely match the original training data.Samples are generated using y = Q(U) with U ∼Uniform(0, 1), and are compared visually with the training points.Source: §I.9.2 · p. 46

J.1. Aggregation metrics

• TabArena computes per-dataset errors by averaging across outer cross-validation folds, using 1 − ROC AUC for binary classification, Log-Loss for multiclass classification, and RMSE for regression.These task-specific errors form the basis for cross-dataset aggregation.Source: §J.1 · p. 48
• Elo compares pairwise wins, ties, and losses, whereas improvability measures the relative error gap to the per-dataset best method and average rank averages error-based ranks across datasets.A 400-point Elo gap corresponds to a 10:1 expected win rate; improvability ranges from 0% optimal to 100%.Source: §J.1 · p. 48
• TabICLv2 consistently achieves state-of-the-art performance across TabArena’s aggregation metrics, with improvability primarily reported for interpretability and Elo and rankings provided for completeness.The metrics emphasize different properties: Elo treats datasets equally, improvability captures performance gaps, and rank-based metrics are robust to outliers.Source: §J.1 · p. 48

J.2. Results on all datasets

• TabICLv2 (default) achieves average rank 4.82, outperforming AutoGluon 1.4 (extreme, 4h) at 5.24 and RealTabPFN-2.5 (T+E) at 5.88 without tuning.TabICLv2 dominates the Pareto fronts for improvability versus runtime and Elo versus runtime among all tabular foundation models.Source: §J.2 · pp. 48–49

J.3. Results on binary classification datasets

• TabICLv2 ranks second on binary classification at 4.43, behind AutoGluon 1.5 at 3.83 but ahead of RealTabPFN-2.5 at 6.90.It remains the strongest tabular foundation model on the binary-classification Pareto fronts.Source: §J.3 · p. 52, Figure J.6 · p. 52, Figure J.7 · p. 52

J.4. Results on multiclass classification datasets

• On multiclass classification, TabICLv2 ranks 6.75, behind AutoGluon 1.5 at 4.00, RealTabPFN-2.5 at 4.25, and AutoGluon 1.4 at 4.50.It nevertheless substantially outperforms TabPFNv2 at 8.62 and LimiX at 9.31; unlike RealTabPFN-2.5, it uses no tuning or ensembling.Source: §J.4 · p. 53, Figure J.9 · p. 53, Figure J.10 · p. 54

J.5. Results on regression datasets

• On regression, TabICLv2 achieves average rank 4.54, tying RealTabPFN-2.5 and trailing only AutoGluon 1.5 at 3.92.TabDPT ranks 5.31 on regression, raising questions about possible training-data leakage because it uses real-world pretraining datasets.Source: §J.5 · p. 55, Figure J.12 · p. 55, Figure J.13 · p. 55

K.1. Benchmark overview

• TALENT comprises 300 datasets: 120 binary-classification, 80 multiclass-classification, and 100 regression datasets, split 64%/16%/20% into training, validation, and test sets.Hyperparameters are selected on validation data, and final performance is reported on held-out test data.Source: §K.1 · p. 56
• TALENT aggregates classification with accuracy and regression with RMSE using improvability, Elo, and average rank, while also reporting supplementary metrics such as log-loss, AUC, MAE, and R2.Development datasets used for TabICLv2’s development are excluded from the main-paper comparison for fairness.Source: §K.1 · p. 56

K.2. Results on all datasets

• TabICLv2 achieves the best average rank at 4.66, ahead of RealTabPFN-2.5 (5.11) and TabPFN-2.5 (5.45), with pairwise win rates of 62% and 65%.It also substantially outperforms LimiX and TabPFNv2, ranked 8.34 and 8.82, respectively.Source: §K.2 · p. 56

K.3. Results on binary classification datasets

• On binary classification, TabICLv2 leads AUC (3.31) and log-loss (2.83), while essentially tying RealTabPFN-2.5 on accuracy (4.98 vs. 4.82).The probabilistic-metric advantage indicates better probability estimates despite similar thresholded accuracy.Source: §K.3 · p. 59

K.4. Results on multiclass classification datasets (≤10 classes)

• On multiclass tasks, TabICLv2 achieves clear superiority over RealTabPFN-2.5 across all evaluated metrics: accuracy (4.58 vs. 5.64), AUC (3.38 vs. 5.20), and log-loss (2.67 vs. 4.48).Source: §K.4 · p. 62

K.5. Results on multiclass classification datasets (> 10 classes)

• For datasets with more than 10 classes, both TabICLv2 with ECOC and its native mixed-radix ensembling outperform RealTabPFN-2.5-ECOC.ECOC is the error-correcting output codes wrapper from TabPFNv2.Source: §K.5 · p. 65

K.6. Results on regression datasets

• On regression, TabICLv2 outperforms TabPFN-2.5 on RMSE and R2, while TabPFN-2.5 has a small MAE advantage (4.43 vs. 4.63).TabICLv2’s quantile regression uses pinball loss across 999 quantiles to model the full conditional distribution, favoring variance and extreme-value metrics.Source: §K.6 · p. 67

K.7. Results on small datasets with less than 10K samples

• TabICLv2 and RealTabPFN-2.5 achieve comparable performance on datasets with fewer than 10K samples.This is the regime where tabular foundation models were originally designed to excel.Source: §K.7 · p. 70

K.8. Results on large datasets with more than 10K samples

• On datasets with more than 10K samples, TabICLv2 demonstrates a clear advantage over both RealTabPFN-2.5 and TabPFN-2.5.Source: §K.8 · p. 71

K.9. Model rankings with respect to meta-features

• Model rankings are further examined with respect to meta-features across all, classification, and regression datasets in TALENT.Source: Figure K.21 · p. 73, Figure K.22 · p. 74, Figure K.23 · p. 74