Source-linked AI summary
EXAONE Tabular 1.0 : Technical Report
Moonjung Eo, Min-Kook Suh, Hye-Seung Cho, Jiwon Kim, Seoyoon Kim, Sangjun Nam, Soonyoung Lee
TL;DR
Dataset-specific tabular training requires labeled data and computation for each new task, motivating reusable in-context prediction. EXAONE Tabular uses synthetic SCM pretraining and CAST’s interleaved feature- and item-axis processing while retaining cell-level representations. Across four benchmarks, it delivers strong classification, point-regression, and probabilistic-regression performance with compact models and efficient inference.
Problem
Dataset-specific training requires labeled data and computational resources for every new tabular task, making rapid adaptation across many datasets costly and time-consuming.
Method
EXAONE Tabular uses synthetic SCM pretraining and CAST, which interleaves feature-axis attention within items with support-conditioned item-axis attention within features while preserving cell-level representations.
Results
Across four public benchmarks, EXAONE Tabular performs strongly across classification, point regression, and probabilistic regression, with TabArena regression reaching TabFM’s performance regime at roughly 1/11 the inference cost.
Takeaways & Limitations
EXAONE Tabular is a compact and highly competitive foundation model family across classification, point regression, and probabilistic regression.
Takeaways & Limitations
The native classification head supports up to 10 classes, while larger label spaces require ECOC decomposition that increases inference cost as class count grows.
Abstract
from arXiv · showhide
EXAONE Tabular is a compact tabular foundation model family for classification and regression via in-context learning, producing predictions without dataset-specific gradient updates. Pretrained exclusively on a synthetic structural-causal-model (SCM) prior, its central contribution is an architecture-centered redesign of tabular in-context learning. Rather than compressing features into a fixed row embedding before a separate row-level learner, EXAONE Tabular interleaves feature-axis attention within each item with support-conditioned item-axis attention within each feature at every Transformer layer, mediated by item-summary and feature-summary tokens. Across four public benchmarks, EXAONE Tabular combines strong predictive performance with high efficiency. On TabArena, its 20.81M-parameter classification model ranks first overall, surpassing tuned ensembles and 4-hour AutoML pipelines, while regression reaches the performance regime of the 1.64B-parameter TabFM at roughly 1/11 the inference cost. On BCCO and TALENT, EXAONE Tabular ranks second in classification and first in regression. On ScoringBench, it achieves the best mean rank for both point-estimation and predictive-distribution quality, leading the $R^2$, RMSE, and CRPS evaluations. Together, these results establish EXAONE Tabular as a state-of-the-art compact tabular foundation model family, combining strong predictive performance across classification, point regression, and probabilistic regression with an efficient model design.
A Synthetic-Prior Foundation Model for Tabular Classification and Regression
EXAONE Tabular is presented as a compact tabular foundation model family for classification and regression, evaluated through a performance–size comparison on TabArena.
- EXAONE Tabular lies on the TabArena performance–size Pareto front in its default setting.The comparison covers classification and regression, with sizes counting weights stored in evaluated checkpoints.
- ≤100M parameters defines EXAONE Tabular as the strongest compact tabular foundation model in the Figure 1 comparison.It outperforms every shown model except TabFM.
- ∼1.3% of TabFM’s parameters is the scale at which EXAONE Tabular is statistically tied with TabFM.
- 113 to 488 Elo is the reported advantage over every shown model except TabFM.
1 Introduction
Tabular foundation models address the cost of retraining conventional models for each dataset by using pretrained in-context prediction. EXAONE Tabular combines synthetic SCM pretraining with CAST, native missing-data handling, and strong benchmark results.
- Motivation: Dataset-specific training of gradient-boosted trees requires labeled data and computational resources for every new task.This makes rapid adaptation across many datasets costly and time-consuming.
- In-Context Learning: Tabular foundation models use labeled support sets and unlabeled query sets to predict query targets without updating pretrained parameters.They are pretrained across broad families of data-generation mechanisms for forward inference on unseen datasets.
- Results: 20.8M and 21.1M parameters are reported for the classification and regression models, respectively, with state-of-the-art or near-state-of-the-art performance across four public benchmarks.On TabArena, classification ranks first overall without dataset-specific tuning or post-hoc ensembling; regression reaches TabFM’s performance regime at roughly 1/11 the inference cost.
- Architecture: EXAONE Tabular introduces CAST, which interleaves feature-axis processing within items with support-conditioned item-axis processing within features.Item-summary and feature-summary pathways connect the two operations while preserving cell-level representations.
- Missing Data: EXAONE Tabular consumes missing values natively without imputation and ranks second in classification and first in regression on BCCO.BCCO includes a substantial number of datasets with missing entries.
- Related Work: EXAONE Tabular follows the synthetic-prior, in-context-learning paradigm while occupying a distinct position from staged row-embedding architectures.Related work includes TabPFN, TabICL, TabPFN-3, TabFM, TabDPT, and LimiX.
2 Modeling
EXAONE Tabular uses a Cross-Axis Summary Transformer that preserves cell-level representations while interleaving feature-axis and support-conditioned item-axis processing. Separate task heads support classification and distributional regression, with synthetic SCM episodes supplying the pretraining data.
- Architecture: CAST jointly processes tables along feature and item axes while retaining cell-level representations across all 12 Transformer layers.Each layer applies two feature-axis attention operations followed by support-conditioned item-axis interaction.
- Architecture: Item-summary and feature-summary tokens couple feature processing with support-set contextualization without collapsing features into a fixed row representation.Item-summary tokens are concatenated along the feature axis, while feature-summary tokens are concatenated along the item axis.
- Feature encoding: Missing-value handling jointly processes finite values and missing or infinite indicators, allowing representations to use observed values and missingness patterns.
- Inference: Query items cross-attend to support-derived item-axis context but neither attend to one another nor update support states or feature-summary tokens.Thus, each query prediction is determined exclusively by the labeled support set and incoming feature-summary states.
- Synthetic prior: Pretraining uses synthetic SCM-generated tables, with observed features and targets selected from evaluated latent-variable graphs before division into support and query examples.Task settings, DAGs, node noise, mechanisms, observed features, and targets are sampled within the synthetic generation pipeline.
3 Training
Classification and regression share EXAONE Tabular’s architecture but use separately configured training recipes. Training relies on synthetic tables, task-specific optimization settings, WSD-style schedules, and EMA weights for evaluation and inference.
- Training data: Classification processes approximately 30 million synthetic table instances, whereas regression processes approximately 10 million during optimization.The count refers to total synthetic table instances processed across the released models’ training lineage.
- Optimization: Selected matrix-valued parameters use Muon, while remaining parameters use AdamW.Muon applies a momentum-based update followed by matrix orthogonalization.
- Learning-rate schedule: Training primarily uses Warmup-Stable-Decay schedules with warmup, stable, and terminal decay phases.Continuation and adaptation stages additionally use related cosine or constant-learning-rate schedules depending on the objective.
- EMA: Both tasks maintain exponential moving-average model weights, and evaluation and inference use the EMA weights.A decay of γ = 0.999 is used for most training, with later classification stages making adjustments.
4 Evaluation
Across TabArena, BCCO, TALENT, and ScoringBench, EXAONE Tabular delivers strong classification, regression, point-estimation, and predictive-distribution performance with competitive efficiency. Its results remain strong across diverse benchmarks, including incomplete-data settings and comparisons against much larger or tuned alternatives.
- TabArena: EXAONE Tabular reaches TabFM’s regression accuracy regime at roughly 1/11 the inference cost, while achieving the highest classification Elo on TabArena.Its default single-forward-pass inference takes 0.605 s per 1,000 samples; TabFM requires approximately 7.0 s per 1,000 samples.
- TabArena: EXAONE Tabular’s single untuned configuration ranks in the top tier on both TabArena tasks, and tuning or ensembling does not close the gap against it.Every tuned-and-ensembled baseline variant falls short of its single untuned configuration on both classification and regression.
- BCCO: 0.799 mean R2 is EXAONE Tabular’s leading BCCO regression result, ahead of TabPFN-3 at 0.795 and TabFM at 0.792.The comparison covers 50 BCCO regression datasets.
- BCCO: EXAONE Tabular remains competitive on BCCO, ranking second in classification and first in regression across datasets with diverse real-world characteristics.Its second-best classification performance remains competitive under BCCO’s substantial missing-value burden.
- TALENT: On TALENT, EXAONE Tabular achieves 0.857 mean classification accuracy and 0.736 mean R2, ranking near the top in classification and first in regression.TabFM leads TALENT classification with 0.863, while EXAONE Tabular leads the regression comparison.
- ScoringBench: EXAONE Tabular achieves the best mean rank for both point estimation and predictive distributions on ScoringBench, leading R2, RMSE, and CRPS.Its leading CRPS result accompanies leading predictive means, indicating strong aggregate distributional performance as well as point prediction.
5 Limitations and Future Work
EXAONE Tabular has practical inference limits for large class spaces and oversized support sets. The report identifies higher cost, memory trade-offs, and context reduction as areas for future work.
- Class-Count Handling: Up to 10 classes are supported natively; larger label spaces use ECOC decomposition, increasing inference cost as class count grows.A class-count-independent prediction head is identified as a potential future direction.
- Large-Context Inference: Caching can improve large-context inference efficiency but adds memory cost, with benefits depending on support size, query size, and execution environment.Query chunking controls peak query-side memory, while support-side reuse is possible when applicable.
- Large-Context Inference: Support sets beyond the configured inference limit are currently subsampled, creating a boundary for context available at inference.Future options include context compression, representative-context selection, clustering, retrieval, and adaptive sampling.
6 License and Permitted Use
EXAONE Tabular distributes its inference software and model weights under separate licenses. The software permits commercial use under its license, whereas the released weights are restricted to non-commercial research absent separate authorization.
- Software: The inference runtime and associated source code are provided under the BSD-3-Clause-LG AI Research License.The license permits use, modification, and redistribution, including for commercial purposes, subject to its terms.
- Model Weights: The released model weights are licensed under the EXAONE AI Model License Agreement 1.2 - NC.The weights permit non-commercial research use; commercial use requires separate authorization from the licensor.
- License Scope: The authoritative license terms are distributed with the software and model weights through the official GitHub and Hugging Face repositories.The report’s high-level summary does not replace or supersede those terms.
7 Conclusion
EXAONE Tabular is a compact in-context tabular foundation model family whose CAST architecture preserves cell-level representations while refining feature and support-context interactions. Across four benchmarks, it delivers strong classification, point-regression, and probabilistic-regression results with substantially lower inference cost than a much larger regression model.
- Conclusion: CAST interleaves within-row feature processing with support-conditioned across-row contextualization throughout the Transformer hierarchy.Item-summary and feature-summary tokens mediate these interactions while preserving cell-level representations.
- Conclusion: The separately trained classification and regression models use synthetic SCM tasks and predict without dataset-specific gradient updates.They support native missing-value handling, task-specific heads, test-time ensembling, and support-conditioned chunked inference.
- Conclusion: Across four public benchmarks, EXAONE Tabular performs strongly across classification, point regression, and probabilistic regression.On TabArena, classification ranks first with an untuned default configuration, while regression reaches the 1.64B-parameter TabFM performance regime at roughly 1/11 the inference cost.