PaperlayerView exact revision on arXiv

Source-linked AI summary

Mitra-v2 Technical Report

Yefan Tao, Xiyuan Zhang, Xinyi Liu, Boran Han, Danielle Maddix, Haoyang Fang, Zhen Han, Jiading Gai, Xuanqing Liu, Michael Bohlke-Schneider, Yuyang, Wang, Gerald Friedland, Kevan Mah, Chris Lee, Chris Kong

arXiv:2609.04540v1cs.LG

TL;DR

Mitra-v2 asks whether broader synthetic task coverage and prior diversity can improve tabular transfer without a deeper backbone. It expands synthetic pretraining and optimization while retaining a small 2D Transformer, and achieves leading benchmark performance with far fewer parameters than TabFM.

  • Problem

    Mitra-v1 showed benefits from combining synthetic priors, leaving open how much broader task coverage and within-prior diversity could improve transfer without a deeper backbone.

    Source: §1 · p. 3

  • Method

    Mitra-v2 retains a small 2D-attention backbone while scaling and diversifying synthetic pretraining and improving optimization and downstream adaptation.

    Source: §1 · p. 3

  • Results

    Across full TabArena and TALENT, Mitra-v2 matches TabFM and EXAONE Tabular while surpassing TabPFN-3 by a wide margin; its 77M parameters compare with TabFM’s 1.6B.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The reported results point to synthetic-task-distribution and optimizer scaling, rather than increased depth or parameter count, as the route to leading performance.

    Source: §6 · p. 25

    The outer-mixture weights are hand-chosen and may be suboptimal; prediction time is not separately instrumented, and TALENT results use a frozen benchmark snapshot.

    Source: §2.2 · p. 6, §5 · p. 25

Abstract

from arXiv · show

We introduce Mitra-v2, a tabular foundation model that delivers state-of-the-art performance on real-world classification and regression problems, from credit-risk scoring and clinical prediction to equipment-failure detection and house-price estimation. Mitra-v2 is trained only on synthetic data, with a pretraining distribution that is much larger and more diverse than Mitra-v1's. Built on a small 2D Transformer backbone, Mitra-v2 supports longer contexts and larger feature spaces. Improved optimization lets it learn from this larger task distribution. We evaluate Mitra-v2 on the TabArena and TALENT benchmarks, comprising more than 300 real-world datasets under two evaluation protocols. On the full TabArena benchmark, Mitra-v2 delivers state-of-the-art performance at the level of the industry-scale TabFM and EXAONE Tabular models, while surpassing TabPFN-3 by a wide margin in both classification and regression. Mitra-v2 matches the 1.6B-parameter TabFM with only 5% of its size (77M parameters), delivering frontier performance at a fraction of the cost. On TALENT, Mitra-v2 remains among the leading models, clearly outperforming TabPFN-3 and TabICLv2. It also ranks first on classification tasks with more than ten classes, even though it was pretrained only on tasks with at most ten classes. These results make Mitra-v2 one of the strongest and most broadly applicable open tabular foundation models released to date. We release the model weights, the inference and fine-tuning code, and our evaluation results under the Apache-2.0 license.

1 Introduction

Mitra-v2 tests whether a broader, more diverse synthetic task distribution can improve transfer without deepening Mitra-v1’s small 2D-attention backbone. It reaches leading performance across TabArena and TALENT, matching industry-scale models while using far fewer parameters than TabFM.

1 Introduction

  • Mitra-v2 asks whether expanding task range and synthetic-prior diversity can improve transfer without making the backbone deeper.Mitra-v1 found that combining structural causal models with complementary tree-based generators improved over a single synthetic prior and transferred across row-wise and element-wise architectures.Source: §1 · p. 3
  • Mitra-v2 retains Mitra-v1’s small 2D-attention backbone while expanding the scale and diversity of synthetic pretraining and improving optimization and downstream adaptation.These changes support longer contexts, wider tables, and more complex prediction tasks, with lightweight extensions for tables beyond native feature and class limits.Source: §1 · p. 3
  • Across full TabArena and TALENT, Mitra-v2 matches TabFM and EXAONE Tabular while surpassing TabPFN-3 by a wide margin in classification and regression.The reported improvements over Mitra-v1 hold across most task groups and are largest in regression.Source: §1 · p. 3
  • 77M parameters match TabFM’s top-benchmark performance despite TabFM’s 1.6B parameters.Mitra-v2’s released classification and regression models have 75.7M and 76.7M parameters, respectively; default deployment fine-tunes and bags eight copies, so wall-clock cost is higher than zero-shot forward-pass models.Source: §1 · p. 3

1.1 From Mitra-v1 to Mitra-v2

  • Elo values are omitted from the Mitra-v1-to-v2 comparison because the reports use different task sets, method pools, scorer revisions, and evaluation protocols.Elo is pool-dependent and cannot be compared across those settings.Source: §1.1 · p. 4
  • The recipe broadens support tables to 160–5,120 rows and 1–50 features, adds Hybrid SCM, and replaces uniform AdamW with geometry-aware composite optimization.The architecture changes only modestly in parameter count; the central hypothesis is that a wider synthetic task distribution can improve downstream transfer.Source: §1.1 · pp. 4–5

2 Model and synthetic pretraining

Mitra-v2 combines a 2D-attention backbone with a broader synthetic pretraining mixture, including Hybrid SCM, and task-specific heads. It also uses deployment-time wrappers to handle feature and class counts beyond pretraining caps.

2.1 Tab2D backbone

  • The 12-layer, 512-dimensional backbone applies attention across rows and columns, with separate feed-forward blocks, and no positional encoding.Scalar cells are embedded independently; support labels are added to their rows, while unknown query labels receive a learned mask token. Predictions are invariant to row and column ordering.Source: §2.1 · p. 5

2.2 Outer prior mixture

  • The outer mixture draws one generator per table, while Hybrid SCM mixes mechanisms within a table; its weights are hand-chosen and may not be optimal.Experiments varying the weights produced little change in downstream accuracy, and systematic mixture optimization is left to future work.Source: §2.2 · p. 6

2.3 Hybrid SCM

  • Hybrid SCM combines neural, tree-based, Gaussian-process, convolutional, and autoregressive mechanisms within one causal graph, widening the functional relationships its synthetic prior can express.Its vector-valued nodes transform concatenated parent outputs and, for multiple parents, aggregate the mechanism output with projected parent outputs; final features and targets are selected from node outputs.Source: §2.3 · pp. 6–7

2.4 Task shapes and preprocessing

  • Classification pretraining samples 160–5,120 support rows, 1,280 query rows, 1–50 features, and up to ten classes; features and class order are shuffled.Synthetic tables are generated online, and preprocessing is fitted on support rows before being applied to query rows.Source: §2.4 · p. 7

2.5 Optimization and distributed training

  • The selected release checkpoint uses about 14,000 optimizer updates and a hybrid stack with Muon for most matrix weights and AdamW for remaining parameters.Training used four nodes with eight GPUs each and a nominal effective batch of 2,048 tasks; the report estimates about 28.7 million nominal synthetic tasks.Source: §2.5 · p. 7

2.6 Handling tables beyond the pretraining caps

  • For wide tables, train-only reduction sets a 256-feature budget before fine-tuning and bagging; classification uses SelectKBest, while regression uses truncated SVD.The same frozen checkpoint handles tables beyond the 50-feature and ten-class pretraining caps without additional pretraining.Source: §2.6 · p. 8
  • On TALENT’s twelve many-class datasets, Mitra-v2 ranks first in the comparison pool under both context policies despite pretraining on at most ten classes.The wrapper decomposes labels into balanced groups of at most ten classes, routes queries through a shallow tree, and recursively handles more than 100 classes.Source: §2.6 · pp. 8–9

3 Evaluation on TabArena

The TabArena evaluation assesses Mitra-v2 across classification and regression, finding it reaches the state-of-the-art level of TabFM and EXAONE Tabular and substantially surpasses TabPFN-3. It also examines deployment tradeoffs, with a roughly nine-minute operating point rating above more costly entries within the one-hour per-fit budget.

3.1 Setup and protocol

  • TabArena evaluates 38 classification datasets over 594 units and 13 regression datasets over 222 units, using a frozen reference pool and a one-hour per-unit fit budget.Mitra-v2 has 0% imputation across the benchmark; Mitra-v1 is imputed on 32% of classification datasets and 46% of regression datasets. TabFM’s reference artifacts include fits exceeding the one-hour budget, and Elo differences near the leaders are treated as ties given wide bootstrap intervals.Source: §3.1 · pp. 9–10

3.3 Classification results

  • On classification, Mitra-v2 surpasses TabPFN-3 by roughly 128 Elo and is statistically tied with TabFM and EXAONE Tabular.It has lower error than TabPFN-3 on 31 of 38 datasets (p=0.0001), while winning 22 against EXAONE and 13 against TabFM. On the 10 datasets with at least 10,000 training rows, its mean rank is 5.3 versus 6.8 on smaller datasets; it scores 1,756.3 Elo versus Mitra-v1’s 1,350.2 and has lower error on all 26 classification datasets where Mitra-v1 produced real results.Source: §3.3 · pp. 11, 13, 14

3.2 Overall standing

  • On the 51-dataset, 816-unit board, Mitra-v2 scores 1,774.6 Elo, statistically level with TabFM and roughly 25 Elo above EXAONE Tabular and 137 above TabPFN-3.Only a four-hour AutoGluon portfolio ranks above it; Mitra-v2 places first on about 9 of 51 datasets. Its 7.1% average gap to the per-task best is similar to AutoGluon’s 7.4%, while TabFM has the smallest gap at 5.3%.Source: §3.2 · pp. 10–11, Table 3 · p. 12

3.4 Regression results

  • On regression, Mitra-v2 is statistically level with TabFM and leads TabPFN-3 by roughly 187 Elo and EXAONE Tabular by roughly 106 Elo on point estimates.It wins on 3.6 of 13 datasets, more often than any entry in the pool. The released 1,985.6 Elo result uses a 1,000-bin distributional head, up to 32,768 prediction-time support rows, and heldout-in-support prediction; the latter adds roughly 11 Elo.Source: §3.4 · pp. 14–15

3.5 Deployment-time configuration as a tuning axis

  • On the full 222-unit regression protocol, raising the prediction support cap from 8,192 to 32,768 rows moves Mitra-v2 from 1,848.4 to 1,935.9 Elo, a gain of 87.5.On a single-split classification diagnostic, feature selection adds roughly 42 Elo and raising the support cap adds a further 22, about 64 Elo total. Feature selection is train-only and dtype-gated; the useful deployment settings depend on table shape and require no retraining.Source: §3.5 · pp. 15–16

3.6 Accuracy–time trade-off

  • Increasing prediction support from 512 to 32,768 rows cuts error by 28% for classification and 21% for regression in geometric mean, while adding 25% and 9% end-to-end time, respectively.At cap 8,192, the model already achieves 94% of classification and 92% of regression error reduction at essentially no extra cost; returning to that cap costs about 2% relative error and requires no retraining.Source: §3.6 · pp. 17–18
  • 43 classification and 40 regression configuration entries with higher recorded costs rate below Mitra-v2, whose operating point costs roughly nine minutes per dataset.No entry spending more per dataset than Mitra-v2 achieves a higher rating.Source: §3.6 · p. 19
  • EXAONE Tabular fits in seconds per unit, TabFM in two to three minutes, and Mitra-v2 in about nine to ten minutes at release defaults.Every model in the band remains inside the protocol’s one-hour per-fit budget, making cost an operational choice rather than a protocol constraint.Source: §3.6 · pp. 19–20
  • Lowering Mitra-v2’s support cap to 8192 removes most added prediction time at about a 2% relative-error cost.This adjustment requires a single runtime setting.Source: §3.6 · p. 20

4 A broader benchmark: TALENT

On TALENT, Mitra-v2 remains in the leading group under both matched context policies, with especially strong results on many-class classification. The section also reports how context policy, dataset selection, and training-data provenance qualify those comparisons.

4.1 Protocol and matched-policy design

  • TALENT’s headline main274 preset covers 274 datasets after excluding 26 TabPFN-2/TabICLv2 development datasets, with separate slices: 173 classification datasets at most ten classes, 12 above ten classes, and 89 regression datasets.The full300 preset retains all 300 datasets and serves as a robustness check.Source: §4.1 · p. 20
  • Folding validation rows into context improved Mitra-v2 on 105 of 173 at-most-ten-class datasets and 70 of 89 regression datasets.The corresponding average-accuracy gain was 0.29 points; geometric-mean RMSE decreased by 2.1%.Source: §4.1 · p. 21

4.2 Results

  • On both boards, Mitra-v2 leads TabPFN-3 by 65–85 Elo and TabICLv2 by 80–96 Elo, while all foundation models outrate the best classical baseline by over 225 Elo.ModernNCA is the best classical baseline at 1,166.9 Elo.Source: §4.2 · pp. 21–22
  • Mitra-v2, TabFM, and EXAONE Tabular form a statistically indistinguishable leading group on the main274 joint board under both context policies.The train-only board rates Mitra-v2 at 1,479.7 Elo; with train+val context it rates 1,490.2 Elo.Source: §4.2 · p. 21, Table 7 · p. 22
  • The full300 train+val results reproduce the leading-group picture, but 26 datasets are TabPFN-2/TabICLv2 development sets, making those models’ full-suite comparisons non-like-for-like.Mitra-v2 and EXAONE Tabular have zero imputed cells across all 300 datasets and synthetic-only pretraining; TabFM’s training-data provenance is undocumented.Source: §4.2 · p. 22, Table 9 · p. 23
  • Mitra-v2 ranks first across the pool on more-than-ten-class classification under both policies, despite pretraining only on tasks with at most ten classes.Its ratings are 2,070.1 Elo with train-only context and 2,204.2 Elo with train+val context; the 12-dataset slice has wide intervals.Source: §4.2 · pp. 22, 24, Table 8 · p. 23
  • On regression, Mitra-v2 has the highest point estimate with train-only context, while the three leading models form a statistical tie with train+val context.The train-only point estimates are 1,499.3 for Mitra-v2, 1,480.2 for TabFM, and 1,466.7 for EXAONE Tabular.Source: Table 8 · p. 23, §4.2 · p. 24
  • At-most-ten-class classification is Mitra-v2’s weakest slice, where it trails TabFM by 75 Elo with train-only and 81 Elo with train+val context.These gaps remain within per-slice interval widths of roughly ±70–80.Source: Table 8 · p. 23, §4.2 · p. 24

5 Limitations

The reported rankings have overlapping uncertainty and depend on frozen, evolving benchmarks. The evaluation also has compute, training-provenance, and timing constraints, while the largest tables retain some in-context scaling headroom.

5 Limitations

  • Intervals overlap across the leading group, so classification and overall-board gaps are statistical ties rather than strict orderings.The regression board has only 13 datasets and the widest intervals; its separations are point-estimate orderings.Source: §5 · p. 24
  • The headline system uses fine-tuning and eight-fold bagging, making it much slower than forward-pass foundation models rather than zero-shot or Pareto-dominant.Figure 6 presents the accuracy–time comparison.Source: §5 · p. 24
  • Experiments change context length, feature range, prior mixture, optimizer, distributed recipe, and checkpoint together, so they do not isolate each change’s contribution.Accelerator hours, energy, and realized prior-mixture frequencies are also unavailable from the metadata.Source: §5 · p. 24
  • Two of the three largest tables still improve at the top of the in-context support range, leaving headroom that further support scaling may recover.The released configuration’s mean rank on large-sample classification datasets is slightly better than on smaller ones.Source: §5 · p. 25
  • Prediction time is not separately instrumented, so the accuracy–time analysis reports end-to-end wall-clock rather than isolated prediction latency.TALENT and TabArena results use frozen revisions; future public rankings may differ as benchmarks and baselines evolve.Source: §5 · p. 25

6 Conclusion

Mitra-v2 reaches the top tier on TabArena and TALENT, matching leading models while surpassing TabPFN-3 across classification and regression. It achieves this with synthetic-task and optimization scaling rather than a deeper backbone, alongside practical limitations in deployment and runtime.

6 Conclusion

  • Mitra-v2 matches TabFM and EXAONE Tabular statistically at the top of TabArena and TALENT, while surpassing TabPFN-3 on classification and regression.On TabArena classification, the three leading models are in a statistical tie; Mitra-v2 is clearly ahead of TabPFN-3.Source: §6 · p. 25
  • On TALENT many-class classification, Mitra-v2 ranks first despite pretraining on at most ten classes, a regime TabFM cannot handle.Source: §6 · p. 25
  • Regression is Mitra-v2’s largest improvement over Mitra-v1, with 3.6 outright dataset wins out of 13 versus 2.5 for the four-hour TabFM+ ensemble.Source: §6 · p. 25
  • The 12-layer model reaches this level with 77M parameters versus TabFM’s 1.6B, using synthetic-task and optimizer scaling rather than added depth.The released models have 77M parameters; TabFM has 1.6B.Source: §6 · p. 25
  • The system is fine-tuned and bagged, its standing lies within overlapping intervals, and its runtime exceeds that of forward-pass models.Accuracy is still rising on the very largest tables as in-context support increases.Source: §6 · p. 25

A Reproducibility and release artifacts

The report pins its results to a frozen evaluation, releases model weights, inference and fine-tuning code, and evaluation results, but not the pretraining pipeline. Its appendix provides complete TabArena comparison pools and a per-task classification diagnostic.

A.1 Frozen evidence

  • All reported numbers use frozen released weights, including the 3 September 2026 one-hour TabArena boards and the TALENT boards in Section 4.TabArena evaluation used a 3,600-second per-unit limit and a 250-second per-bag-child fine-tuning budget.Source: §A.1 · p. 27

A.2 Release contents

  • The Apache-2.0 release includes classifier and regressor weights, inference and fine-tuning code, and per-unit evaluation results; the pretraining pipeline is excluded.Source: §A.2 · pp. 27–28

B Complete TabArena leaderboards

  • Complete TabArena boards cover 51 datasets and 816 units overall, 38 datasets and 594 units for classification, and 13 datasets and 222 units for regression.The respective boards list 43, 41, and 40 distinct methods.Source: B Complete TabArena leaderboards · p. 28, Table 10 · p. 30, Table 11 · p. 31
  • Figure 7 compares per-task mean metric error differences against TabPFN-3, TabFM, and EXAONE Tabular; orange favors Mitra-v2, and rows are sorted by training-set size.Column headers count datasets where Mitra-v2 has lower error; the plotted magnitudes mix ROC-AUC error and log-loss and are descriptive only.Source: Figure 7 · p. 29

C Negative results

Development tests found no improvement over the released configuration from several alternative regression objectives, longer or continued training, prior adjustments, and deployment-time changes. Per-dataset tuning scored higher only as a benchmark oracle, so reported results use a uniform configuration.

C Negative results

  • Alternative regression objectives and heads—including HL-Gauss, CRPS, quantile prediction, and target-binning variants—did not improve accuracy over the released binned cross-entropy head.Source: C Negative results · p. 32
  • Further training did not help: continued pretraining washed out or left the board flat, while correlation-rebalanced prior variants lost ground across regression.Source: C Negative results · p. 33
  • Log-uniform support-row sampling and reweighting the outer synthetic-prior mixture left downstream accuracy essentially unchanged.Source: C Negative results · p. 33
  • Longer classification support caps reduced accuracy despite chunked attention; raising the fine-tuning support budget gained under one Elo on the full multi-split evaluation.One-hot categorical expansion hurt, and alternative feature selectors, support-row selection strategies, and support-scaled temperatures provided no reliable gain or sometimes hurt.Source: C Negative results · p. 33
  • Per-dataset configuration selection scored higher only as an oracle that tunes on the benchmark; reported results instead use one training-data-computable configuration.Source: C Negative results · p. 33

D 2D visualizations of the Hybrid SCM prior

The Hybrid SCM prior spans varied dataset geometries and feature dimensions, including dimensions beyond Mitra-v1’s range. The appendix also reports broad independent use of the Mitra family across published studies.

D 2D visualizations of the Hybrid SCM prior

E Mitra use case overview

  • Since July 2025, the Mitra family has been downloaded more than 12 million times and independently evaluated in published studies spanning energy, healthcare, transportation, finance, and manufacturing.Source: E Mitra use case overview · p. 36

E.1 Healthcare and life sciences

Mitra was applied to clinical survival prediction in both static and longitudinal settings, and was also evaluated for censoring-aware survival prediction.

E.1 Healthcare and life sciences

  • MITRA-CF ranked best overall among 14 survival models on 43 static clinical datasets and achieved the best raw dynamic-prediction performance on 5 longitudinal datasets.For dynamic prediction, integrated AUC was 0.693 versus 0.642 for the runner-up.Source: §E.1 · p. 36
  • Mitra was evaluated as one of five tabular foundation-model backbones for censoring-aware survival prediction using a pseudo-RMST regression target on SurvSet clinical benchmarks.Source: §E.1 · p. 36

E.2 Financial services, banking, and insurance

The cited applications span zero-shot credit-risk prediction and day-ahead electricity-price forecasting, with a further accuracy gain reported from temporal hierarchy reconciliation.

E.2 Financial services, banking, and insurance

  • Zero-shot Mitra reached an average AUC of 0.724 across 14 credit-risk datasets and 29 methods without dataset-specific tuning.The evaluation covered discrimination, calibration, and decision-oriented metrics relevant to capital allocation, provisioning, and pricing.Source: §E.2 · p. 36

E.3 Energy and utilities

  • Mitra was the most accurate of four base forecasters for day-ahead electricity prices on German EPEX and Spanish OMIE markets at every temporal aggregation level.Temporal hierarchy reconciliation improved accuracy further.Source: §E.3 · p. 36

E.4 Transportation and mobility

The applications cover transportation, procurement, customer churn, real estate, and scientific prediction, including adapters that combine Mitra predictions with domain-specific modeling.

E.4 Transportation and mobility

  • Mitra produced the most consistent and balanced crash-injury-severity predictions among four tabular deep-learning models, reaching 93.3% accuracy on 18,073 glare-related Texas traffic records.The study used a SHAP-based explainable safety analysis.Source: §E.4 · p. 36
  • A constrained multinomial-logit adapter using Mitra in-context predictions gained 4 to 14 accuracy points over classical logit on Swissmetro, LPMC, and IoT-Wearables.The adapter restored economic guarantees such as cost monotonicity.Source: §E.4 · p. 36
  • On Swissmetro, Mitra through AutoGluon achieved 77.7% accuracy versus 63.7% for multinomial logit, while a two-stage econometric adapter retained the gain and restored valid value-of-time estimates.Source: §E.4 · p. 36

E.5 Industrial, manufacturing, and supply chain

  • Zero-shot Mitra outperformed TabICLv2 among foundation-model baselines for supplier lead-time estimation on real anonymized procurement datasets from Kinaxis deployments.Source: §E.5 · p. 37

E.6 Retail, marketing, and customer analytics

  • Across nine public customer-churn datasets spanning seven industry sectors, tabular foundation models collectively and decisively outperformed classical ML, deep learning, and tree-ensemble alternatives.Mitra was one of the six foundation models benchmarked.Source: §E.6 · p. 37

E.7 Real estate and geospatial

  • Mitra was evaluated for estimating urban location values from rents in Dortmund, performing within a narrow margin of a stacked classical ensemble and producing spatially coherent maps local experts judged plausible.The evaluation included zero-shot and AutoGluon fine-tuned settings.Source: §E.7 · p. 37

E.8 Science and engineering

  • For multifidelity regression on aerodynamics and CFD simulations and on hyperparameter-optimization benchmarks, Mitra outperformed the strongest Gaussian-process baseline.It was evaluated zero-shot as one of four tabular foundation-model backbones.Source: §E.8 · p. 37

F Automated checkpoint search with LLMZero

Because validation Elo can decline or stall as pretraining continues, the authors search over continuation trajectories; guided search improved an independent checkpoint, but did not exceed the released model.

F Automated checkpoint search with LLMZero

  • The search adapts LLMZero and ExTS for continued pretraining by launching short continuation segments and proposing trajectory-conditioned hyperparameter adjustments from an initial checkpoint.The motivation is that validation Elo does not increase monotonically with pretraining steps, making naive continuation inefficient.Source: F Automated checkpoint search with… · p. 37
  • +17.1 validation Elo came from a 750-step continuation discovered in eight guided-search iterations; unguided continuation along the original trajectory yielded no improvement at any tested length.Source: F Automated checkpoint search with… · pp. 37–38
  • Search gains are bounded by the initialization: the improved checkpoint did not surpass the released model, which came from a stronger pretraining run.The search resumes from the source checkpoint’s weights and optimizer states; applying it to stronger available checkpoints is left for future work.Source: F Automated checkpoint search with… · p. 38
Loading 2609.04540v1…