PaperlayerView exact revision on arXiv

Source-linked AI summary

TabEmbed: Benchmarking and Learning Generalist Embeddings for Tabular Understanding

Minjie Qiang, Mingming Zhang, Xiaoyi Bao, Xing Fu, Yu Cheng, Weiqiang Wang, Zhongqing Wang, Ningtao Wang

arXiv:2605.04962v1cs.CLcs.IR

TL;DR

Tabular understanding lacks a shared embedding space because existing methods either do not produce retrieval-compatible vectors or fail to represent tabular structure and numerical semantics. The paper introduces TabBench and TabEmbed, which reformulate classification and retrieval as language-to-row matching with contrastive learning and hard negative mining. Experiments show that TabEmbed achieves state-of-the-art performance and establishes a baseline for generalist tabular embeddings, while evaluation omits commercial closed-source APIs and extremely wide tables may exceed context windows.

  • Problem

    Existing methods lack a unified tabular representation: LLM approaches do not produce retrieval-compatible dense vectors, while text embeddings often miss tabular structure and numerical semantics.

  • Method

    The paper introduces TabBench and trains TabEmbed with task-adaptive language-to-row contrastive matching and positive-aware hard negative mining across classification and retrieval tasks.

  • Results

    TabEmbed achieves state-of-the-art performance on TabBench and significantly outperforms state-of-the-art text embeddings across the evaluated tabular understanding tasks.

  • Takeaways & Limitations

    The work establishes a baseline for generalist tabular embeddings and reports that domain alignment is more effective than parameter scaling alone for tabular intelligence.

  • Takeaways & Limitations

    The evaluation excludes commercial closed-source embedding APIs, and serialization may exceed backbone context windows for extremely wide tables.

Abstract

from arXiv · show

Foundation models have established unified representations for natural language processing, yet this paradigm remains largely unexplored for tabular data. Existing methods face fundamental limitations: LLM-based approaches lack retrieval-compatible vector outputs, whereas text embedding models often fail to capture tabular structure and numerical semantics. To bridge this gap, we first introduce the Tabular Embedding Benchmark (TabBench), a comprehensive suite designed to evaluate the tabular understanding capability of embedding models. We then propose TabEmbed, the first generalist embedding model that unifies tabular classification and retrieval within a shared embedding space. By reformulating diverse tabular tasks as semantic matching problems, TabEmbed leverages large-scale contrastive learning with positive-aware hard negative mining to discern fine-grained structural and numerical nuances. Experimental results on TabBench demonstrate that TabEmbed significantly outperforms state-of-the-art text embedding models, establishing a new baseline for universal tabular representation learning. Code and datasets are publicly available at https://github.com/qiangminjie27/TabEmbed and https://huggingface.co/datasets/qiangminjie27/TabBench.

1 Introduction

Tabular data lacks a shared embedding space for classification and retrieval because existing approaches either remain schema-bound, lack retrieval-compatible vectors, or miss tabular structure and numerical semantics. TabBench and TabEmbed address this gap through unified evaluation and language-to-row contrastive learning with hard negative mining.

  • Existing tabular methods treat classification and retrieval as distinct problems, leaving no shared embedding space for diverse tabular understanding tasks.
  • Tree-based models are constrained by fixed schemas, while LLM-based methods do not produce dense fixed-dimensional vectors for retrieval applications.
  • General-purpose text embedding models generate vectors but often treat serialized tables as unstructured text, limiting their ability to capture tabular structure and numerical semantics.
  • TabBench evaluates numerical reasoning and retrieval capabilities, while TabEmbed unifies classification and retrieval within a shared semantic space.
  • TabEmbed reformulates diverse tabular tasks as semantic matching using task-adaptive natural-language queries and positive-aware hard negative mining.

2 The Tabular Embedding Benchmark

TabBench evaluates tabular embeddings through classification and retrieval, using serialized rows, frozen representations, and natural-language queries over heterogeneous datasets. Its retrieval queries test categorical, numeric, and mixed constraints with symbolic validity checks.

  • Benchmark scope: TabBench assesses tabular representation through linear separability in classification and semantic alignment in retrieval.
  • Data serialization: Tabular rows are represented as ordered feature-value pairs and serialized into natural-language sequences through string concatenation.
  • Data serialization: Rows exceeding the predefined maximum sequence length are filtered to preserve token efficiency and compatibility with embedding-model context constraints.
  • Tabular Classification: Classification evaluates linear separability by training an independent logistic-regression classifier on frozen row embeddings for each dataset.
  • Tabular Classification: Datasets with more than 50 labels or a label-to-sample ratio above 0.1 are excluded, and qualified datasets receive stratified train-test splits.
  • Tabular Retrieval: Retrieval ranks serialized rows from a heterogeneous corpus using cosine similarity between natural-language queries and candidate documents.
  • Tabular Retrieval: Queries contain one to three categorical, numeric, or mixed constraints, and symbolic verification retains only valid queries with at least five matching rows.

3 TabEmbed: Unified Tabular Embedding Learning

TabEmbed casts tabular representation learning as language-to-row semantic matching, unifying retrieval and classification in a shared embedding space. It uses self-supervised query generation, hard negatives, and contrastive learning to preserve fine-grained tabular structure.

  • Unified Framework: TabEmbed unifies diverse tabular tasks by casting them into a shared contrastive semantic space.The framework is designed as a generalist embedding model rather than separate task-specific architectures.
  • Contrastive Triplet Formulation: Natural-language queries serve as anchors matched to serialized positive rows and contrasted against hard negatives.Queries express tabular constraints or class intent, while positives satisfy the query and negatives do not.
  • Task-Adaptive Query Generation: Retrieval queries encode explicit numerical or categorical constraints that the corresponding row satisfies.The formulation aligns language constraints with specific attribute values in the serialized input.
  • Task-Adaptive Query Generation: Classification queries describe hidden target labels, requiring rows to imply those labels from correlations among remaining features.This encourages clustering by latent predictive features rather than surface-level token overlap.
  • Training Objective: Positive-aware hard-negative mining retains candidates that resemble a query semantically but violate its retrieval condition.The training objective combines these mined negatives with in-batch negatives using cosine similarity and a temperature parameter.

4 Experiments

Experiments evaluate TabEmbed across model scales and backbone architectures on TabBench. TabEmbed consistently improves tabular performance, including strong gains over its backbone and competitive aggregate results at small scale.

  • Experimental Setup: TabEmbed is initialized from Qwen3-Embedding backbones at 0.6B, 4B, and 8B parameters and evaluated on TabBench.Training uses a balanced mixture of retrieval and classification contrastive triplets.
  • Main Results: TabEmbed achieves state-of-the-art performance across all parameter scales and surpasses existing text embedding models.The reported improvements cover both tabular retrieval and classification metrics.
  • Main Results: Over 35 points: TabEmbed-0.6B surpasses its Qwen3 backbone in MRR@10 for Tabular Retrieval.This result is reported as a substantial retrieval improvement from unified contrastive learning.
  • Main Results: TabEmbed-0.6B outperforms all baselines on the aggregate metric, including models in the 7B and 8B regimes.Scaling TabEmbed from 0.6B to 8B nevertheless yields consistent performance gains.
  • Performance on Diverse Backbones: 48.91 to 70.71: Qwen3-Embedding-4B achieves the largest reported improvement after applying the training paradigm.The framework also improves Jina-Embeddings-v3 from 41.48 to 61.57, indicating gains across diverse architectures.

5 Analysis and Discussion

TabEmbed improves retrieval, numerical sensitivity, embedding-space organization, and robustness to irrelevant columns across the analyzed tabular tasks. The analyses show stronger handling of numerical constraints, better clustering, and greater stability under structural noise.

  • Fine-grained retrieval: 84.61 average nDCG@10 for categorical queries, followed by 65.96 for mixed and 46.37 for numeric queries, revealing a task-difficulty hierarchy.Performance generally decreases as logical constraints increase, while TabEmbed remains robust on multi-condition intersections.
  • Fine-grained retrieval: TabEmbed substantially improves numeric retrieval, where the baseline often fails to capture magnitude and range relationships.The reported gains bridge text-based retrieval and numerical reasoning across query scenarios.
  • Numerical sensitivity: ρ > 0.8 in many numerical-sensitivity cases, with TabEmbed shifting most comparisons into the improved region against Qwen3-Embedding.The test compares similarity scores with ground-truth logical satisfaction for inequalities, equality, and range queries.
  • Embedding-space visualization: 3.26 Cluster Ratio for TabEmbed classification embeddings versus 1.04 for the baseline, indicating more separated class clusters.The visualization attributes the improved geometry to disentangled regions and linear separability.
  • Embedding-space visualization: TabEmbed retrieval embeddings improve Cluster Ratio from 21.28 to 24.79 and reduce intra-cluster distance from 0.60 to 0.57.Queries are anchored closer to corresponding document groups, producing more compact and separable retrieval clusters.
  • Robustness analysis: TabEmbed maintains MRR@10 above 75% after adding 30 irrelevant columns, while the baseline declines from approximately 64% to below 55%.The experiment injects noise columns while keeping queries fixed, testing robustness to structural dilution.

6 Conclusion

The paper concludes that TabEmbed bridges tabular classification and retrieval through a unified contrastive embedding approach. Its results support domain alignment as a more effective route to tabular intelligence than parameter scaling alone.

  • Conclusion: TabEmbed bridges tabular classification and retrieval using task-adaptive query generation and hard negative mining within a unified contrastive paradigm.The approach learns discriminative representations in a shared embedding space.
  • Conclusion: The 0.6B TabEmbed model surpasses significantly larger baselines and achieves state-of-the-art performance in the reported experiments.The conclusion presents this as evidence for the effectiveness of domain alignment rather than parameter scaling alone.
  • Conclusion: TabBench and TabEmbed establish a baseline for generalist tabular embeddings by jointly addressing numerical semantics, tabular structure, classification, and retrieval.The conclusion contrasts these capabilities with the limitations of standard text embeddings.

Limitations

The study’s main limitations concern evaluation coverage and serialization scalability. Commercial closed-source embedding APIs were not benchmarked, and very wide tables may exceed backbone context windows.

  • Evaluation scope: The evaluation excludes commercial closed-source embedding APIs because TabBench contains over 300 datasets and budget constraints limited benchmarking scope.The authors identify comprehensive comparison with systems such as Google Gemini Embedding as future work.
  • Serialization scalability: Serializing tabular data into natural-language sequences may truncate information for extremely wide tables with hundreds of columns.The authors propose token-efficient serialization or long-context architectures as future directions.

A.2 Benchmarks for Text and Tabular Tasks

The paper positions TabBench as a unified evaluation framework for structured-data embeddings, addressing the separation between text retrieval benchmarks and tabular prediction benchmarks. It evaluates semantic alignment, numerical reasoning, and linear separability while TabEmbed is trained with contrastive ranking objectives.

  • Evaluation gap: Existing evaluation protocols remain split between unstructured text embedding tasks and supervised tabular classification.This bifurcation leaves structured scenarios involving numerical and categorical constraints insufficiently covered.
  • TabBench: TabBench simultaneously assesses numerical reasoning, semantic alignment, and linear separability across diverse tabular tasks.It is introduced to bridge evaluations that otherwise treat tabular learning as isolated prediction.
  • Training setup: TabEmbed training uses contrastive Multiple Negatives Ranking Loss with cosine similarity, in-batch negatives, and mined hard negatives.The reported configuration uses τ = 0.05, a global batch size of 256, and 500,000 retrieval plus 100,000 classification triplets.
  • Model family: TabEmbed is released at 0.6B, 4B, and 8B parameters on the Qwen3-Embedding architecture, whose underlying context capacity reaches 32K tokens.The reported fine-tuning protocol uses a 1,024-token context length.

B.3 Evaluation Protocols

TabBench evaluates embeddings through dataset-specific linear probing for classification and exact dense retrieval, using a benchmark spanning diverse classification sources and a large retrieval corpus. TabEmbed serializes heterogeneous fields into standardized text before embedding, with experiments supported by distributed large-model training infrastructure.

  • TabBench aggregates datasets from four diverse sources for classification and constructs a large-scale retrieval corpus.
  • Tabular Classification (Linear Probing): Classification uses frozen dense vectors and an independent Logistic Regression classifier trained separately on each dataset’s training split.Performance is evaluated on the corresponding test split, with no shared classifier across datasets.
  • Tabular Retrieval (Dense Retrieval): Retrieval uses exact Faiss IndexFlatIP search over L2-normalized embeddings, equivalent to cosine-similarity ranking.The main metrics are MRR@10 and nDCG@10, with additional Recall and Precision evaluated across multiple cutoffs.
  • The serialization pipeline converts heterogeneous tabular fields into a standardized natural-language format before concatenation.Numeric values are rounded to a specified precision, while categorical, textual, temporal, and applicable binary fields receive modality-specific conversions.
  • The unified text context lets TabEmbed use the LLM backbone’s pretrained semantic knowledge without modality-specific feature engineering.
  • Figure 9 compares TabBench macro-average performance across four metrics against average inference delay per 10,000 samples.The Y-axis is overall average performance and the X-axis is average inference delay in seconds.

C Inference Efficiency Analysis

TabEmbed improves the performance–latency frontier across model scales: the 0.6B model is efficient, while larger variants achieve higher performance at substantially greater inference cost.

  • Figure 10 compares aggregate TabBench performance on the Y-axis with average inference delay on the X-axis across model scales.
  • The latency analysis uses one accelerator, batch size 64, maximum sequence length 1024, and three runs after warm-up.
  • 65.27% performance at approximately 94 seconds per 10,000 samples makes TabEmbed-0.6B an efficient low-latency model.
  • 71.62% is reached by TabEmbed-8B, but latency exceeds 1,000 seconds per 10,000 samples.
  • TabEmbed-4B provides near-peak performance at approximately half the inference cost of the 8B model.
  • The theoretical Oracle point marks the remaining gap between current models and a system combining minimal delay with maximum accuracy.

D Numeric Sensitivity Curves

TabEmbed’s numerical sensitivity curves are designed to align similarity with logical truth conditions, producing operator-specific responses that contrast with weak baseline numerical behavior.

  • Figure 10 generates 101 candidate documents per numerical query and compares cosine similarity across candidate values against valid logical ranges.
  • Baseline Qwen3-Embedding-8B shows flat or erratic similarity for numerical inequalities, indicating weak correlation with magnitude and range.
  • TabEmbed-8B models inequalities with step-like transitions at decision boundaries and high similarity within valid ranges.
  • For equality queries, TabEmbed produces a sharp similarity peak at the exact target value.
  • For conjunctions, TabEmbed maintains high similarity only across the intersection interval where both numerical conditions hold.
  • Spearman correlation improves from 0.64 to 0.87 in an annotated example, quantitatively verifying alignment with mathematical logic.
  • The robustness visualization tests 12 syntactic variations per intent across Numeric, Categorical, and Mixed retrieval tasks.
  • Distinct query formats form tight clusters around corresponding documents despite differences between SQL, JSON, and natural-language styles.

F Training Convergence Analysis

Training gains arrive quickly and then plateau, while larger TabEmbed models achieve higher and more stable performance than the smallest variant.

  • The convergence study monitors macro-average performance across 0.6B, 4B, and 8B models from 400 to 2800 steps.
  • Most performance gains occur within the first 800 steps, with the 4B model rising from roughly 67% to over 70%.
  • Performance curves generally plateau after approximately 1600 steps across the tracked model scales.
  • TabEmbed-8B maintains higher performance than the 4B and 0.6B variants throughout training.
  • The 4B and 8B models improve smoothly and monotonically, whereas the 0.6B model shows volatility around step 2000.
Loading 2605.04962v1…