PaperlayerView exact revision on arXiv

Source-linked AI summary

TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models

Léo Grinsztajn, Klemens Flöge, Oscar Key, Felix Birkel, Philipp Jund, Brendan Roof, Benjamin Jäger, Dominik Safaric, Simone Alessi, Adrian Hayler, Mihir Manium, Rosen Yu, Felix Jablonski, Shi Bin Hoo, Anurag Garg, Jake Robertson, Magnus Bühler, Vladyslav Moroshan, Lennart Purucker, Clara Cornu, Lilly Charlotte Wehrhahn, Alessandro Bonetto, Bernhard Schölkopf, Sauraj Gambhir, Noah Hollmann, Frank Hutter

arXiv:2511.08667v2cs.LGstat.ML

TL;DR

TabPFN-2.5 addresses limitations of traditional tabular models by scaling tabular foundation-model inference to larger datasets. It combines architectural, data, calibration, and distillation advances, and reports state-of-the-art benchmark performance against tuned tabular methods. Its current scope is constrained by a non-commercial, non-production license and a focus below million-row datasets.

  • Problem

    Traditional tabular models require extensive dataset-specific tuning, often have unreliable uncertainty estimates, and lack modern foundation models’ generalization and transferability.

  • Method

    TabPFN-2.5 combines deeper architectures, richer synthetic priors, calibration and inference modules, expanded scalability, and distillation into MLP or tree-ensemble predictors.

  • Results

    TabPFN-2.5 sets a new state-of-the-art for tuning-free tabular models, matching complex four-hour-tuned ensembles and outperforming tuned models on TabArena.

  • Takeaways & Limitations

    The distilled MLP and tree-ensemble outputs preserve most of TabPFN’s accuracy while providing low-latency, low-memory, production-pipeline deployment.

  • Takeaways & Limitations

    The model, its derivatives, and outputs cannot be used for commercial or production purposes, and the next scaling target is datasets with millions of rows.

Abstract

from arXiv · show

The first tabular foundation model, TabPFN, and its successor TabPFNv2 have impacted tabular AI substantially, with dozens of methods building on it and hundreds of applications across different use cases. This report introduces TabPFN-2.5, the next generation of our tabular foundation model, built for datasets with up to 50,000 data points and 2,000 features, a 20x increase in data cells compared to TabPFNv2. TabPFN-2.5 is now the leading method for the industry standard benchmark TabArena (which contains datasets with up to 100,000 training data points), substantially outperforming tuned tree-based models and matching the accuracy of AutoGluon 1.4, a complex four-hour tuned ensemble that even includes the previous TabPFNv2. Remarkably, default TabPFN-2.5 has a 100% win rate against default XGBoost on small to medium-sized classification datasets (<=10,000 data points, 500 features) and a 87% win rate on larger datasets up to 100K samples and 2K features (85% for regression). For production use cases, we introduce a new distillation engine that converts TabPFN-2.5 into a compact MLP or tree ensemble, preserving most of its accuracy while delivering orders-of-magnitude lower latency and plug-and-play deployment. This new release will immediately strengthen the performance of the many applications and methods already built on the TabPFN ecosystem.

1 Introduction

TabPFN-2.5 extends tabular foundation modeling to substantially larger datasets while targeting strong, tuning-free performance and faster deployment. It is presented as outperforming tuned tree models, matching AutoGluon 1.4, and supporting compact distilled models.

  • Traditional tabular models require extensive dataset-specific tuning and often provide unreliable uncertainty estimates without significant modification.
  • Tabular foundation models use synthetic-task pretraining and in-context learning to provide training-free predictors with strong calibration.
  • TabPFN-2.5 is introduced as the next release of the TabPFN tabular foundation model.
  • In a forward pass, TabPFN-2.5 outperforms tuned tree-based models and matches the accuracy of AutoGluon 1.4 tuned for four hours.
  • TabPFN-2.5 scales in-context learning to 50,000 samples and 2,000 features, respectively 5x and 4x the TabPFNv2 limits.
  • A proprietary output engine produces MLP or tree-ensemble models that combine most of TabPFN’s accuracy with low-latency inference and easier deployment.

2 Ecosystem & Adoption

TabPFN has broad adoption across industries and is increasingly used as a foundation for extensions beyond direct tabular prediction. Its ecosystem spans applications, structured-data research domains, and practical tooling.

  • TabPFNv2 has become a widely used baseline, with almost 400 citations and more than 2,000,000 PyPI downloads within the reported period.
  • Adoption spans healthcare, finance, insurance, retail, manufacturing, energy, utilities, geoscience, agriculture, materials, and engineering.
  • TabPFN-2.5 is expected to boost performance across applications and methods built on the existing ecosystem.
  • TabPFN extensions apply the model to time-series forecasting, graph node classification, evolving data streams, reinforcement learning, Bayesian optimization, multimodal learning, and causal inference.
  • The TabPFN–Extensions repository supports interpretability, unsupervised tasks, advanced modeling, lightweight optimization, ensembling, and integration with tree or forest baselines.

3 Model Overview

TabPFN-2.5 retains TabPFNv2’s general design while expanding its data, architecture, calibration, preprocessing, and deployment capabilities. The model also introduces synthetic-data improvements, real-data fine-tuning, and distillation into deployable predictors.

  • TabPFN-2.5 retains the general TabPFNv2 design while adding deeper architectures, richer synthetic priors, and new calibration and inference modules.
  • Data: The model is trained on expanded synthetic data, with Real-TabPFN-2.5 additionally fine-tuned on 43 curated, deduplicated real-world datasets.
  • Architecture: The architecture increases depth to 18 regression layers and 24 classification layers, uses feature groups of 3, and adds 64 learned thinking rows.
  • Calibration and optimization: Post-processing tunes classification thresholds for metrics such as F1-score and applies temperature scaling for multiclass probability calibration.
  • Inference: Optimized preprocessing, larger feature groups, FlashAttention-3, and multi-GPU evaluation make TabPFN-2.5 between 1x and 2.3x faster than TabPFNv2.
  • Deployment: A distillation engine outputs dataset-specific MLP or tree-ensemble classifiers with low latency, low memory use, and production-pipeline compatibility.
  • Model variants: Table 1 summarizes model variants using recommended maximum rows and features, while noting that larger datasets are not built and evaluated for those settings.

4 Experimental Results

The experiments evaluate TabPFN-2.5 on TabArena, internal classification and regression benchmarks, inference speed, distillation, and causal inference. Across these settings, it combines strong predictive performance and scalability with lower-latency deployment options.

  • TabArena: TabPFN-2.5 outperforms existing tabular prediction methods on TabArena-Lite classification datasets with up to 10,000 samples and 500 features.In a forward pass, it outperforms AutoGluon 1.4, a four-hour-tuned ensemble that includes TabPFNv2.
  • TabArena: On TabArena datasets with up to 100,000 rows and 2,000 features, TabPFN-2.5 ranks as the best default model.When tuned, it outperforms AutoGluon 1.4 on regression and approaches it on classification.
  • TabArena: TabPFN-2.5 clearly outperforms TabPFNv2 on almost all compatible TabArena datasets and is never much worse.The comparison reports a substantial improvement in default performance.
  • Internal benchmarks: On internal benchmarks, TabPFN-2.5 outperforms all tuned baselines in one forward pass on datasets with up to 50,000 data points.The benchmark covers classification and regression datasets, including settings with 500 to 2,000 features.
  • Inference and deployment: TabPFN-2.5-as-MLP and TabPFN-2.5-as-TreeEns offer competitive accuracy while reducing inference cost relative to the standard TabPFN-2.5 model.The distillation engine outputs a dataset-specific MLP or tree ensemble for high-throughput or resource-constrained deployment.
  • Causal inference: PFN-based meta-learners achieve the strongest overall performance on the RealCause causal-estimation benchmark.TabPFN-2.5 also performs better out-of-the-box than TabPFNv2 and HPO5 across the three meta-learners.

5 How to Get Optimal Fit + Predict Speed from TabPFN-2.5

The section recommends hardware, parallelization, batching, software, caching, and parameter choices to optimize TabPFN-2.5 fit and prediction speed. It also notes that causal estimation benefits from strong propensity and outcome models, with TabPFN-2.5 reported as a strong base model.

  • Hardware: Dedicated NVIDIA H100 or A100 GPUs are recommended, while CPUs, integrated GPUs, and Apple MPS are suitable only for small datasets.Other PyTorch-supported dedicated GPUs are compatible, but memory or speed may limit larger workloads.
  • Parallel inference: Multiple GPUs can dramatically reduce fit and prediction time for larger datasets by parallelizing inference.The device parameter of TabPFNClassifier and TabPFNRegressor enables this setup.
  • Causal inference: TabPFN-2.5 is described as the strongest base model for all metalearners, and PFN-based CATE estimators outperform specialized tree- and deep-learning-based causal methods.The passages also state that propensity and outcome model choice is important for CATE estimation.
  • Inference strategy: Batch inference is faster than separate prediction calls because the model is retrained each time .predict() runs unless fitted-model caching is enabled.When memory is insufficient, test points can be split into batches of 1000 to 10000.
  • Software: PyTorch 2.8 or above is recommended because earlier versions may deliver lower performance.
  • Caching and deployment: For small datasets, fitted-model caching makes later predictions fast through a KV-Cache, but its memory cost limits current suitability to small training datasets.For larger datasets and CPU inference, the section recommends TabPFN-as-MLP/TreeEns.

6 License and Availability

The license permits research, testing, evaluation, and internal benchmarking, but prohibits commercial or production use of the model, derivatives, and outputs. Commercial users can instead obtain an enterprise license or use the managed cloud endpoint and APIs.

  • License scope: The TABPFN-2.5 License v1.0 is designed to be permissive for research and internal evaluation.
  • Permitted use: Testing, evaluation, and internal benchmarking are explicitly allowed, including downloading the model for preliminary assessment on an organization’s datasets.
  • Restriction: Commercial or production use of the model, derivatives, and outputs is prohibited, including revenue-generating products and internal commercial decision-making.
  • Commercial access: A Commercial Enterprise License provides production access to a proprietary high-speed inference engine, support, integration tooling, and other internal models.
  • Cloud availability: The managed TabPFN-2.5 cloud endpoint is recommended for users without a dedicated local GPU or those seeking commercial use without a full on-premise license.
  • Integration: The API supports integration into non-commercial and commercial applications through a Python SDK or standard REST API.

7 Conclusion and The Road Ahead

The conclusion presents TabPFN-2.5 as a state-of-the-art tuning-free tabular model that matches complex tuned ensembles and outperforms tuned models on TabArena. Future work targets million-row datasets and broader tabular-like reasoning problems.

  • Conclusion: TabPFN-2.5 sets a new state of the art for tuning-free tabular models across public and private benchmarks.
  • Conclusion: TabPFN-2.5 matches complex 4-hour-tuned ensembles, including ensembles containing TabPFNv2, while outperforming other tuned models in a forward pass on TabArena.The unrestricted public TabArena benchmark contains datasets with up to 100,000 training data points.
  • Road ahead: The next scaling target is datasets with millions of rows, with retrieval, fine-tuning, and novel architectures under active development.
  • Road ahead: The broader vision extends to time series, multimodal tabular data, causal inference, unsupervised tasks, domain knowledge, and decision support.
  • Existing ecosystem: TabPFN has already been applied across healthcare and life sciences, including diagnosis, prognosis, treatment response prediction, and data-scarce multimodal settings.The listed domains include oncology, neurology, cardiology, psychiatry, nephrology, and pharmacology.

C Data Contamination and Deduplication for Real-TabPFN-2.5

Real-TabPFN-2.5 uses a deduplication and filtering pipeline to reduce data contamination before fine-tuning. The curated fine-tuning data comes from identified OpenML and Kaggle datasets, with training data checked against internal and public evaluation resources.

  • Deduplication: The Real-TabPFN-2.5 pipeline deduplicates training datasets against internal benchmarks, an in-house validation suite, and the public TabArena benchmark.It extends the methodology used for Real-TabPFN and combines dataset identifiers, feature schemas, and additional cross-referencing.
  • Dataset curation: A table lists the datasets selected for fine-tuning together with their sources and access links.
  • Dataset curation: The listed fine-tuning sources include OpenML and Kaggle datasets spanning classification and other tabular tasks.Examples in the listings include breast cancer, tic-tac-toe, employee turnover, medical appointments, and cardiovascular disease datasets.

D Details on Causal Inference Results

This section frames causal inference around estimating treatment effects under unconfoundedness and highlights the role of the predictive base model. It also situates the results within the RealCause benchmark and TabPFN–Extensions workflow.

  • Causal Inference: CATE estimation asks how an individual’s outcome would change under treatment versus withholding treatment.
  • Assumption: Unconfoundedness assumes that no unobserved feature influences both treatment and outcome.
  • Base Model: Under unconfoundedness, CATE estimation can be framed as AutoML because meta-learners require classification or regression models for treatment and outcome likelihoods.
  • Base Model: Prior studies identify TabPFN as a strong base model for X-, T-, and S-Learner meta-learners.
  • Benchmark: The RealCause benchmark includes a table describing its causal inference datasets.

F.2 Additional results on many features

The additional many-feature experiment evaluates TabPFN-2.5 on datasets spanning 500 to 2,000 features and reports strong default performance.

  • Evaluation Scope: 500 to 2,000 features are covered by the internal many-feature dataset set.
  • Evaluation Scope: The experiment focuses on TabPFN-2.5’s default performance.
  • Result: The reported results show strong default performance across the many-feature datasets.

G Detailed TabArena Results

The detailed TabArena results report pairwise model win rates across both TabPFNv2-compatible datasets and the broader benchmark. They also compare TabPFN-2.5 with TabICL and LimiX on subsets matching those models’ supported settings.

  • TabArena Win Rates: Pairwise win rates are reported for classification and regression on TabArena datasets.
  • TabArena Win Rates: The restricted evaluation covers datasets with fewer than 10,000 rows and 500 features, while the full evaluation reaches 100,000 rows and 2,000 features.
  • Foundation-Model Comparisons: TabPFN-2.5 outperforms TabICL on TabArena datasets for which TabICL is designed.
  • Foundation-Model Comparisons: TabPFN-2.5 shows much better performance than LimiX on datasets with fewer than 50,000 samples and 2,000 features.

H Results with Tuned Decision Thresholds

This section examines metric-specific threshold tuning, performance across feature counts and benchmark subsets, and computational behavior. It reports strong comparisons against other models, substantial F1-score gains in some imbalanced datasets, and faster execution than TabPFNv2.

  • Many Features: TabPFN-2.5 performs well by default on datasets with 500 to 2,000 features for both classification and regression.
  • TabArena Comparisons: On TabArena-Lite datasets below 10,000 samples and 500 features, pairwise win rates compare TabPFN-2.5 with competing models using classification and regression tasks.
  • TabArena Comparisons: On the broader TabArena-Lite datasets reaching 100,000 samples and 2,000 features, pairwise win rates provide the corresponding classification and regression comparison.
  • Decision Thresholds: Threshold tuning can substantially improve macro F1-score on several imbalanced datasets.
  • Runtime: 1x to 2.3x speedup is reported for TabPFN-2.5 over TabPFNv2 depending on dataset size.
  • Runtime: Inference time scales linearly with test-set size, while datasets above 500 features use feature subsampling to 500 features per estimator.
Loading 2511.08667v2…