Source-linked AI summary
When, Where, and How: Adaptive Binning for Tabular Self-Supervised Learning
Daehwan Kim, Haejun Chung, Ikbeom Jang
TL;DR
Medical tabular self-supervised learning remains limited by fixed, feature-agnostic binning despite heterogeneous clinical data and costly labels. Adaptive Binning introduces learning-coupled, feature-wise refinement with type-aware supervision, yielding stronger representations across tasks and datasets while establishing a unified benchmark for reproducible evaluation.
Problem
Existing tabular SSL binning objectives use fixed global discretization and feature-agnostic supervision despite heterogeneous clinical data and costly labels.
Method
Adaptive Binning couples tabular SSL discretization to learning through feature-wise coarse-to-fine refinement, representation-aware splits, and type-aware ordinal supervision.
Results
Adaptive Binning yields stronger representations across tasks and datasets, with benefits persisting under fine-tuning and a single default configuration reducing dataset-specific tuning.
Takeaways & Limitations
The method provides a transferable learning-driven discretization bias and a unified medical tabular SSL benchmark for reproducible comparisons.
Takeaways & Limitations
Evaluation is limited to in-dataset transfer and a small set of downstream protocols.
Abstract
from arXiv · showhide
Medical tabular data are ubiquitous in clinical research, but deep learning for tables remains underexplored because reliable labels often require costly expert adjudication, even though structured clinical variables are routinely available in tabular form. Self-supervised learning can leverage these unlabeled tables, and recent binning-based pretexts offer a promising inductive bias, but existing objectives fix a single global quantile discretization and apply feature-agnostic supervision. We propose Adaptive Binning, a training-adaptive discretization pretext for tabular SSL that couples discretization to learning through a feature-wise coarse-to-fine curriculum. Motivated by the spectral bias of neural networks and the principles of curriculum learning, our method progressively refines discretization per feature upon plateau detection and selects representation-aware splits to jointly improve value-space concentration and representation-space coherence. A heterogeneity-aware objective unifies categorical reconstruction with ordinal supervision for numerical features, and experiments on public medical tabular datasets under unified evaluation protocols show consistent gains for linear probing and fine-tuning without dataset-specific discretization tuning. We further introduce a medical tabular SSL benchmark with standardized protocols to support reproducible progress in this underexplored domain. Our code is available at https://github.com/labhai/Adaptive-Binning.
1 Introduction
Clinical research relies heavily on tabular data, yet mixed feature types, non-smooth interactions, and the absence of spatial or sequential structure make deep learning underexplored. Adaptive Binning addresses fixed, feature-agnostic discretization with a training-adaptive coarse-to-fine pretext and evaluates it through a unified medical tabular SSL benchmark.
- Motivation: 99% of reviewed comparative clinical trials reported baseline or outcome measures in at least one table, while 85% reported both.Clinical trials, registries, and epidemiological studies routinely tabulate characteristics, laboratory panels, graded findings, and outcomes.
- Motivation: Tabular data remain underexplored in deep learning because tables mix categorical and numerical variables, exhibit non-smooth interactions, and lack spatial or sequential structure.These properties favor tree ensembles such as XGBoost and CatBoost, whose recursive partitioning produces piecewise-continuous functions.
- Limitations: Existing binning-based tabular SSL uses one globally fixed quantile discretization and feature-agnostic reconstruction, with numerical targets trained by squared-error regression on integer indices.A single bin count and static quantile boundaries persist throughout training, limiting adaptation to feature-specific learning dynamics.
- Method: Adaptive Binning replaces global binning with feature-wise, coarse-to-fine refinement that specifies when, where, and how discretization evolves during pretraining.Its design combines curriculum learning with spectral bias, progressively fitting coarse structures before fine details; HORD handles mixed-type reconstruction, while FPT and DIGS refine numerical bins using plateau-triggered, representation-aware splitting.
- Evaluation: The method is evaluated across medical tabular datasets covering binary, nominal, and ordinal multiclass classification and regression using linear probing and fine-tuning with multiple tabular encoders.The evaluation uses the unified protocols reported in Tables 2 and 4.
- Benchmark: The paper establishes a medical tabular SSL benchmark with unified evaluation protocols to provide a reproducible foundation for clinical tabular self-supervised learning.The benchmark is reported in Table 1.
2 Method
Adaptive Binning replaces fixed global quantile discretization with feature-wise, learning-driven refinement during tabular SSL. It combines plateau-triggered adaptive splits, representation- and value-aware scoring, and type-aware ordinal reconstruction for mixed schemas.
- Adaptive Binning: The method extends autoencoding tabular SSL with feature-wise adaptive discretization and type-aware ordinal supervision, contrasting with fixed quantile binning.The baseline uses a single global bin count and static boundaries, whereas Adaptive Binning refines discretization during pretraining.
- When: Each numerical feature maintains its own bin count, initialized at Tinit and capped at Tmax, under the Feature-Wise Adaptation schedule.This produces per-feature targets y(n) over feature-specific bin sets rather than a shared resolution.
- When: Feature-Wise Plateau Trigger monitors numerical reconstruction separately and refines features independently when their learning plateaus.The schedule avoids globally synchronized refinement because numerical features differ in complexity and convergence speed.
- How: HORD unifies categorical nominal reconstruction with numerical ordinal, distribution-aware supervision using soft-target cross entropy and mean–variance regularization.Numerical predictions are distributions over ordered adaptive bins, while categorical features use nominal cross-entropy.
3 Experiments and Results
Experiments on a diverse medical tabular benchmark use a standardized 1000-epoch pretraining protocol and show that adaptive, feature-wise discretization consistently improves downstream evaluation. Ablations, fine-tuning, and hyperparameter sweeps indicate complementary component benefits, transferable initialization, and reduced need for dataset-specific tuning.
- Benchmark and protocol: The benchmark spans public medical datasets covering binary, multiclass, nominal, ordinal, and regression tasks with heterogeneous schemas and categorical–numerical compositions.Evaluation follows the fixed-binning baseline’s 1000-epoch pretraining protocol for comparability.
- Linear evaluation: Adaptive feature-wise refinement remains superior to fixed-binning BinRecon across masking settings, while masking functions as a complementary regularizer.The persistent margin and no-mask advantage indicate that training-adaptive refinement, rather than input corruption, drives the main gains.
- Ablation study: Removing any single component degrades linear probing, confirming complementary contributions from refinement, splitting, and heterogeneity-aware ordinal supervision.Even when FPT never triggers in the HF setting, removing HORD causes a marked drop, demonstrating the value of type-aware supervision independently of refinement.
- Hyperparameter robustness: A single default configuration provides a reliable starting point across tasks and datasets, while broad sweeps show that deviations from default settings consistently reduce performance.The tested controls include loss weights, FPT patience, and the DIGS threshold.
- Fine-tuning evaluation: Adaptive discretization provides the most reliable SSL initialization after fine-tuning and typically improves over MR and fixed-binning BR, though supervised training can remain optimal for specific model–task pairs.Its advantage persists under downstream optimization, supporting a transferable inductive bias rather than a probe-specific artifact.
4 Conclusion
The paper introduces Adaptive Binning, a learning-coupled, feature-wise coarse-to-fine self-supervised pretext for medical tabular data, and establishes a benchmark with unified evaluation protocols.
- 4 Conclusion: Adaptive Binning replaces fixed discretization with a learning-coupled, feature-wise coarse-to-fine curriculum for medical tabular self-supervised learning.The method uses plateau-triggered refinement, representation-aware split selection, and heterogeneity-aware ordinal supervision.
- 4 Conclusion: The approach yields stronger representations across tasks and datasets.
- 4 Conclusion: The authors establish a benchmark of medical tabular datasets with unified evaluation protocols.The benchmark is intended to enable reproducible progress in medical tabular self-supervised learning.