TabPFN-3.5: Technical Report

1 Introduction

• Prior evaluations found that even the strongest TFMs fell behind tuned conventional models on grouped or temporal splits, large or wide tables, and tables with text or high-cardinality features.These are data settings practitioners encounter in everyday work.Source: §1 · p. 3
• TabPFN-3.5 sets new state-of-the-art results across standard, non-i.i.d., multimodal, predictive-distribution, and relational benchmarks.The report names TabArena, BeyondArena, STRABLE, MulTaBench, ScoringBench, and RelArena.Source: §1 · p. 3
• TabPFN-3.5 retains support for up to one million training rows, many-class classification, calibrated predictive distributions, and cached inference that scores test rows in well under a second.Training rows are processed once for cached inference.Source: §1 · p. 3
• Its reported gains draw on revised per-cell encodings, doubled model width with unchanged cache size, joint classification-and-regression training, and a synthetic prior tuned for high-cardinality, wide, and grouped data.The report says Section 2 quantifies these gains.Source: §1 · p. 3
• The release includes a core model and TabPFN-3.5-Fast, while Plus expands text and date processing and Thinking scales inference-time computation for peak performance.Fast is described as an alpha checkpoint for latency-sensitive applications; Plus and Thinking are available through the Prior Labs API.Source: §1 · p. 3

2 Experimental Results

• TabPFN-3.5 ranks first on TabArena, BeyondArena’s non-large datasets, STRABLE, MulTaBench, ScoringBench, and task-specific harnesses.The task-specific results include relational data and time-series forecasting.Source: §2 · pp. 3–4

2.1 TabArena

• 84 Elo points over TabFM at one-tenth its total time, and 130 Elo points over AutoGluon 1.6 extreme in one-fifth the time.The three TabPFN variants form the Elo-versus-time Pareto frontier, so one reaches the highest Elo at every time budget.Source: §2.1 · pp. 4–5, Figure 2 · p. 5

2.2 BeyondArena

• TabPFN-3.5 establishes a new state of the art on BeyondArena and improves over TabPFN-3 in every slice, but tuned MLPs remain strongest on grouped, temporal, and large datasets.On non-large datasets, TabPFN-3.5 matches the best baselines on grouped and temporal data. Thinking adds 20 Elo points over TabPFN-3.5, while Fast is 3× faster and scores 40 Elo points lower.Source: §2.2 · p. 5, Figure 3 · p. 6, Figure 4 · p. 6

2.3 STRABLE

• All TabPFN-3.5 variants significantly outperform prior published baselines on STRABLE’s tabular-string tasks.The comparison uses published results without rerunning baselines; TabPFN-3.5 and Fast use TF-IDF, while Plus and Thinking receive raw tables.Source: §2.3 · p. 6

2.4 MulTaBench

• The TabPFN-3.5 family significantly outperforms prior published methods on MulTaBench’s tables paired with text or image fields.Plus and Thinking process raw text directly while using frozen embeddings for image features.Source: §2.4 · pp. 6–7, Figure 5 · p. 7

2.5 RelArena-α

• TabPFN-Rel built on TabPFN-3.5-Plus ranks first on both RelArena-α leaderboards and outperforms RT-PluRel while requiring less runtime.Updating the base model improves performance by nearly 90 Elo points; the latest internal harness adds another 36 Elo points.Source: §2.5 · p. 7, Figure 6 · p. 8

2.6 Inference benchmarks

• TabPFN-3.5-Fast runs up to 3× faster than TabPFN-3; the core model can run up to 2× slower on large training sets, while optimized Plus has 1.25× its latency.The Plus variant’s faster FP8 attention reduces latency; Fast with optimized Plus can run in a quarter of the time.Source: §2 · p. 4, §2.6 · pp. 7–8
• Cached single-row prediction times align between TabPFN-3 and TabPFN-3.5, while TabPFN-3.5-Fast is around three times faster.Cache size remains largely unchanged between TabPFN-3 and TabPFN-3.5 despite roughly quadrupled parameter count and substantially improved prediction quality.Source: §2.6 · p. 8

3 Model Design and Inference

• On a single test row, TabPFN-3.5 matches TabPFN-3’s inference speed when predicting against a training-row KV cache.Source: Figure 7 · p. 9

3.1 Architecture

• Per-cell encoding combines standardized values, learned Fourier features, and in-context ECDF ranks to represent both magnitudes and column-relative distributions.ECDF midranks are invariant to strictly increasing transformations; Fourier features are especially advantageous for ordinal-encoded, high-cardinality categorical variables.Source: §3.1 · p. 9, Figure 8 · p. 10
• TabPFN-3.5 widens the in-context transformer from 512 to 1024 dimensions and shares most model components across classification and regression.Task-specific components are the label encoder and output head; the shared model uses task type as input.Source: §3.1 · p. 9

3.2 Preprocessing

• TabPFN-3.5 uses no additional feature transformations or SVD augmentation, and aggregates eight estimators by default; Fast aggregates four.Source: §3.2 · p. 11

3.3 Synthetic Prior

• The synthetic prior emphasizes realistic high-cardinality categorical variables, large feature counts, and grouped data where test rows come from different groups than training rows.Source: §3.3 · p. 11

3.4 TabPFN-3.5-Plus and Thinking mode

• TabPFN-3.5-Plus extends multimodal capabilities and uses FP8 attention to accelerate inference on datasets with many rows.The Plus and Thinking variants are available through the API and enterprise deployments as drop-in replacements for the open-source interface.Source: §3.4 · p. 11
• TabPFN-3.5-Thinking adds inference-time computation to Plus and supports mixed numerical, categorical, date, and text columns in one call.The report says it uses no LLMs, real data, internet search, or other model.Source: §3.4 · p. 11
• 12 times faster than TabPFN-3-Thinking, TabPFN-3.5-Thinking benefits from a stronger base model and implementation improvements.The implementation details of Plus and Thinking are proprietary and not described in the report.Source: §3.4 · p. 11

4 License and Availability

• The open-source release has two parts—the code and model weights—under two licenses; its code, including the tabpfn package and Python client SDK, is released under Apache License 2.0.Source: §4 · p. 11
• The weights license permits academic use, research, evaluation, and internal benchmarking, but prohibits commercial or production use of the models, derivatives, and outputs.Examples include revenue-generating products, procurement benchmarking, client deliverables, and commercial internal decision-making.Source: §4 · p. 12
• Commercial production access is offered through enterprise licensing, including managed API, cloud, on-premises, and custom deployments.The license includes the proprietary high-speed inference engine and Plus and Thinking variants.Source: §4 · p. 12

4.1 Distribution

• The open-source distribution supports evaluation and non-commercial use, while managed API and listed marketplace options support commercial workloads.Source: §4.1 · p. 12

A Contributors

• Benjamin Jäger led TabPFN-3.5 development; the remaining model-development team members were listed in randomized order.Source: A Contributors · p. 17

C.1.1 Tuning trajectories and pairwise win rates on TabArena

• Figures 9 and 11 plot improvability against total training-plus-inference time per 1,000 rows, with medians over 51 datasets and 15 medium-sized datasets, respectively.Figure 9 also shows classical-baseline tuning trajectories; Figure 11 uses the same layout.Source: §C.1.1 · pp. 17–18, Figure 9 · p. 18, Figure 11 · p. 18

C.1.2 TabArena leaderboard tables

• Pairwise win-rate figures compare each row method against each column method by the share of tasks won, across 816 tasks overall and 135 medium-view tasks.Source: Figure 10 · p. 18, Figure 12 · p. 18
• TabPFN-3.5-Thinking ranks first in Elo across all five views, while TabPFN-3.5 ranks second except on TabArena-small, where TabFM+ places between them.Source: §C.1.2 · p. 18, Table 2 · p. 19, Table 3 · p. 20
• At a fraction of TabFM’s full-benchmark training and prediction time, TabPFN-3.5-Fast reaches TabFM’s Elo.Source: §C.1.2 · p. 19, Table 2 · p. 19
• The leaderboard views cover the full 51-dataset benchmark, small and medium dataset subsets, and separate classification and regression tasks.The corresponding tables report full-view task counts of 816, 681, 135, 594, and 222, respectively.Source: Table 2 · p. 19, Table 3 · p. 20, Table 4 · p. 21

• ROC AUC above 0.997 on two nearly perfectly solved datasets inflates small residual differences, dominating the remaining improvability gap against TabPFN-3.5.The datasets are hazelnut-spread-contaminant-detection and polish-companies-bankruptcy.Source: §C.1.3 · p. 23

C.1.4 How the time axis of the TabArena Pareto frontier is computed

• TabArena’s timing protocol measures an 8-fold bagged fit and refitted-model prediction on one node, while non-default configurations are called once per split.Fits use 24 CPU cores and one NVIDIA RTX Pro 6000 GPU; the once-per-split configurations may appear to have better fit times than bagged configurations.Source: §C.1.4 · p. 23
• Figure 13 plots TabArena improvability against total time, with lower values better.It uses the same methods and time axis as Figure 2.Source: Figure 13 · p. 24
• Per split, fit and prediction times are normalized per 1,000 training and test rows, respectively, and their sum defines total time.Rates are averaged over folds per dataset and reported as the median over 51 datasets.Source: §C.1.4 · p. 25
• Foundation-model wrappers were adjusted to remove timing overhead, and models were re-timed on 153 splits; their predictions and full-evaluation Elo scores were unchanged.TabPFN-3-Thinking retains its earlier published timing, which the passage describes as an upper bound.Source: §C.1.4 · p. 25

C.2.1 Radar view of the BeyondArena slices

• Figure 15 presents the Figure 3 and Figure 4 BeyondArena Elo scores as radar charts, with one axis per data slice.The group identifier is exposed to TabPFN models on label-per-sample grouped datasets.Source: Figure 15 · p. 25, §C.2.1 · p. 25

C.2.2 Comparison to recent TFMs

• The comparison covers 116 BeyondArena core datasets without large tables, after excluding five datasets unsupported by one or more methods or exceeding available memory.Every method is scored on the same splits.Source: §C.2.2 · pp. 25–26
• TabPFN-3.5 leads TabFM and EXAONE-Tabular overall and across every data slice except small datasets, where TabFM leads.Wide confidence intervals make some slice-level differences uncertain.Source: §C.2.2 · p. 26, Figure 16 · p. 26

C.2.3 Exposing the group identifier on label-per-sample grouped tasks

• BeyondArena drops group identifiers for grouped data, and label-per-sample tasks receive no other grouped-specific preprocessing, so group structure is not passed to models.Tuned and early-stopped baselines can still exploit group structure through group-aware validation splits.Source: §C.2.3 · p. 26
• Passing the group id as a categorical variable helps TabPFN models, while TabFM is evaluated with the id as a float and unseen test groups marked missing.Other model families showed no gains from the tested encodings; only default models were tested, so benefits for tuned and ensembled versions are not excluded.Source: §C.2.3 · p. 26
• 80 Elo points raise TabPFN-3.5 from 1187 to 1267 Elo and from rank 9 to rank 4 on grouped data including large tables.Without large tables, TabPFN-3.5 moves from 1240 to 1288 Elo and rank 6 to rank 1; TabPFN-3 and TabFM also gain Elo in that setting.Source: §C.2.3 · p. 26

C.3 Additional Results on STRABLE

• STRABLE Elo ratings are jointly computed across 200 methods on 108 datasets with three folds each, including four TabPFN-3.5-family pipelines.The added pipelines include TabPFN-3.5 and Fast with TF-IDF, TabPFN-3.5 with e5-small-v2, Plus, and Thinking.Source: §C.3 · pp. 26–27
• Figure 17 displays selected STRABLE baselines, while preserving the Elo values from the joint fit used for the main-paper figure.The selection focuses on end-to-end models or stronger string-encoding methods; cited examples include Ridge, XGBoost, TabICLv2, and raw-text CatBoost.Source: §C.3 · p. 27, Figure 17 · p. 27
• TabPFN-3.5-Thinking and TabPFN-3.5-Plus retain the first and second overall leaderboard positions by a substantial margin with raw text.With e5-small-v2 embeddings, TabPFN-3.5 narrowly outperforms TabPFN-2.5, by less than the margin observed with TF-IDF.Source: §C.3 · p. 27
• TF-IDF generally proves the stronger text representation on STRABLE, whose strings are mostly short entries rather than long-form free text.The relative rankings with e5-small-v2 embeddings remain consistent, though TabPFN-3.5’s margin over TabPFN-2.5 is smaller.Source: §C.3 · p. 27

C.4 Additional Results on MulTaBench

• On text-focused MulTaBench datasets, TabPFN-3.5-Plus and TabPFN-3.5-Thinking ranked first and second by a substantial margin.TabPFN-3.5 and Fast use the same frozen e5-small-v2 features as non-native-text baselines, while the performance gap widens on text-focused datasets.Source: §C.4 · p. 28

C.5 TALENT

• Average rank was 2.95 for TabPFN-3.5 on 274 TALENT datasets, ahead of TabPFN-3 at 4.48 and TabICLv2 at 5.35.This view follows the TabICLv2 protocol and excludes 26 base datasets used during TabPFN-2 and TabICLv2 development.Source: §C.5 · p. 29, Figure 19 · p. 29
• Average rank was 2.13 for TabPFN-3.5 across 337 TALENT datasets, versus 3.20 for TabPFN-3.This view covers datasets with up to one million training rows and compares against classical and deep tabular baselines.Source: §C.5 · p. 29, Figure 20 · p. 30

C.6 fev-bench

• On fev-bench, TabPFN-3.5 placed sixth, 1.1 points above TabPFN-TS-3 at 2.4× its speed.TabPFN-3.5 is the general tabular checkpoint without time-series finetuning; TabPFN-TS-3 is time-series-specific.Source: §C.6 · p. 30, Figure 21 · p. 31

C.7 ScoringBench

• CRPS mean rank was 2.85 for TabPFN-3.5, ahead of TabPFN-3.5-Fast at 5.76 and TabPFN-3 at 7.64.TabPFN-3.5 improved over TabPFN-3 on 85 of 101 datasets.Source: §C.7 · p. 31, Figure 22 · p. 32

C.8 Details on the benchmark overview table

• Each benchmark uses its best evaluated family member: Thinking on the first four, TabPFN-Rel (3.5) on RelArena-α, and TabPFN-3.5 on TALENT and ScoringBench.On STRABLE and MulTaBench, Thinking is evaluated inside TabPFN-3.5-Plus, which handles raw text columns.Source: §C.8 · p. 32
• Rank is that member’s position on the benchmark’s headline ranking, with denominators excluding other TabPFN-3.5 family members.The fields range from 11 official RelArena-α baselines to 88 TabArena entries, with tuning variants counted separately.Source: §C.8 · p. 32
• Competitor columns select the highest-placed other TFM and non-foundation model, excluding systems containing foundation models from both columns.For the 337-dataset TALENT view, TabICLv2’s entry comes from the 274-dataset protocol view because no other foundation model appears in the larger view.Source: §C.8 · p. 32
• Win rates compare each dataset-and-fold split, count ties as half wins, then average within and across datasets so each dataset weighs equally.RelArena-α and TALENT have one split per dataset; the reported benchmark counts range from 21 RelArena-α tasks to 816 TabArena splits.Source: §C.8 · p. 33