PaperlayerView exact revision on arXiv

Source-linked AI summary

Improving Robustness of Tabular Retrieval via Representational Stability

Kushal Raj Bhandari, Adarsh Singh, Jianxi Gao, Soham Dan, Vivek Gupta

arXiv:2604.24040v2cs.CLcs.AIcs.IRcs.IT

TL;DR

Table retrieval is sensitive to serialization because equivalent linearizations can produce different embeddings and rankings. The paper averages serialization views into centroid targets and trains a lightweight adapter to approximate them from one format, improving robustness mainly for dense retrievers while remaining less compatible with sparse retrieval.

  • Problem

    Equivalent table serializations can yield different embeddings and retrieval results, but their impact on table retrieval has not been systematically measured.

  • Method

    The paper treats serialization embeddings as noisy views of a shared signal, uses their centroid as a canonical target, and trains a residual bottleneck adapter over a frozen encoder.

  • Results

    Centroid representations rank strongest overall, while the adapter consistently improves robustness across dense retrievers and is less compatible with SPLADE.

  • Takeaways & Limitations

    Serialization should be treated as a modeling choice with geometric consequences rather than as a negligible preprocessing detail.

  • Takeaways & Limitations

    Centroid recovery can fail when format-specific shifts contain shared table-independent components, and the dense adapter is substantially less compatible with sparse lexical geometry.

Abstract

from arXiv · show

Transformer-based table retrieval systems flatten structured tables into token sequences, making retrieval sensitive to the choice of serialization even when table semantics remain unchanged. We show that semantically equivalent serializations, such as $\texttt{csv}$, $\texttt{tsv}$, $\texttt{html}$, $\texttt{markdown}$, and $\texttt{ddl}$, can produce substantially different embeddings and retrieval results across multiple benchmarks and retriever families. To address this instability, we treat serialization embedding as noisy views of a shared semantic signal and use its centroid as a canonical target representation. We show that centroid averaging suppresses format-specific variation and can recover the semantic content common to different serializations when format-induced shifts differ across tables. Empirically, centroid representations outrank individual formats in aggregate pairwise comparisons across $\texttt{MPNet}$, $\texttt{BGE-M3}$, $\texttt{ReasonIR}$, and $\texttt{SPLADE}$. We further introduce a lightweight residual bottleneck adapter on top of a frozen encoder that maps single-serialization embeddings towards centroid targets while preserving variance and enforcing covariance regularization. The adapter improves robustness for several dense retrievers, though gains are model-dependent and weaker for sparse lexical retrieval. These results identify serialization sensitivity as a major source of retrieval variance and show the promise of post hoc geometric correction for serialization-invariant table retrieval.

1 Introduction

Table retrieval must linearize two-dimensional structure into token sequences, making serialization a consequential source of embedding and ranking variation. The paper measures this instability and proposes centroid targets plus a lightweight adapter to improve format robustness.

  • Flattening tables into sequences imposes ordering and syntax that are not intrinsic to their relational content.
  • Semantically equivalent serializations produce different embeddings and downstream rankings across retriever families.
  • Serialization effects remain under-researched despite extensive architectural and serialization advances in table understanding.
  • The paper treats serialization views as an orbit and targets their centroid as a canonical, format-agnostic representation.
  • A residual bottleneck adapter transports single-format embeddings toward centroid targets without re-indexing or additional inference encoding passes.

2 The Serialization Bottleneck in Structured Retrieval

Serialization is a first-order retrieval variable because flattening tables creates format-dependent geometry. This instability changes nearest neighbors and is especially consequential when queries provide weak lexical evidence.

  • Flattening a table imposes an ordering and syntactic scaffolding absent from the table’s intrinsic row-column structure.
  • Serialization substantially changes retrieval effectiveness across formats, datasets, and BGE-M3, MPNet, ReasonIR, and SPLADE.Table 1 summarizes Recall@1 variation using standard deviation and range across WTQ, WikiSQL, and NQ-Tables.
  • Different serializations of the same WTQ table occupy distinct regions in ReasonIR embedding space.
  • NQ-Tables shows the strongest effect because its larger query-table lexical gap increases reliance on serialization-induced structural cues.
  • Embedding spread across meaning-equivalent serializations changes nearest neighbors and reorders top-K retrieval results.

3 Centroid-Based Representations as Stable Anchors

Centroid averaging treats serialization embeddings as views of a shared semantic signal and suppresses format-specific shifts. Its empirical advantage depends on how those shifts vary across tables.

  • Centroid recovery can degrade when serialization shifts contain table-independent components shared across formats.
  • Centroid averaging approximates the stable semantic signal when format-specific shifts cancel across serialization views.
  • The centroid is the least-squares center of all serialization embeddings and the average over meaning-preserving transformations.
  • Centroid targeting reduces retrieval-score variation by canceling format-specific shifts that would otherwise move scores.
  • Centroid-based representations occupy the strongest aggregate ranking region, with CENTROID ALL performing best overall against individual formats.

4 Post-Hoc Centroid Transport with a Residual Bottleneck Adapter

The paper uses a frozen encoder plus a lightweight residual bottleneck adapter to approximate centroid robustness from one serialization. Its objective aligns embeddings while preserving useful representation geometry.

  • The adapter corrects serialization-specific shifts directly in frozen dense embeddings, moving them toward the shared semantic signal.
  • The bottleneck limits correction capacity, while the residual path preserves the original representation as the dominant signal.
  • A VICReg-inspired objective avoids unstable regression by combining centroid alignment with identity, variance, and covariance terms.
  • The invariance term aligns serializations to centroids, identity preserves query-compatible geometry, variance prevents collapse, and covariance reduces dimensional redundancy.
  • Adapted embeddings cluster more tightly around centroids while retaining inter-table separation across ten tables.

5 Serialization-Invariant Adapter

The adapter reduces serialization sensitivity and improves ranking robustness mainly for dense retrievers, while transfer to unseen data is useful but model- and format-dependent. Sparse SPLADE retrieval is substantially less compatible and can degrade under adaptation.

  • Reduced standard deviation and range of Recall@1 across serialization formats indicate that the adapter lowers sensitivity to serialization choice overall.Across dense retrievers, adapter effects are mostly positive or neutral; negative changes are concentrated in already strong base serializations.
  • 0.59, 0.69, and 0.70 ∆log-rank gains occur for MPNet on xml for WTQ and WikiSQL, and html for NQ-Tables, respectively, under joint training.
  • 0.22 to 0.26 on WTQ and 0.22 to 0.24 on WikiSQL are the Recall@1 increases for ReasonIR with shuffled rows under joint training.On NQ-Tables, shuffled rows and columns achieve ∆log-rank values of 0.71 and 0.69, although Recall@1 gains are modest.
  • Useful transfer to unseen NQ-Tables is possible when the adapter is trained on WTQ and WikiSQL, with performance varying by serialization quality.For ReasonIR, subset training improves shuffled cols from 0.08 to 0.11 while leaving ddl at 0.31, outperforming joint training on both.
  • Mixed serialization consistently degrades base retrieval, while the adapter mitigates the effect and generalizes despite training on single-format inputs.
  • 0.44 to 0.39 on WTQ-tsv and 0.33 to 0.16 on NQ-Tables-csv show SPLADE degradation under joint adaptation.Subset adaptation produces 0.38 and 0.19, respectively, for these two cases.
  • Sparse degradation is consistent with a dense residual correction that densifies sparse activations and blends lexical, structural, and markup signals.
  • The adapter improves robustness primarily in a model- and format-dependent manner rather than as a universal fix.Joint supervision is often beneficial, while subset training can still transfer meaningfully to unseen data for dense retrievers.

6 Conclusion

The paper treats serialization as a geometric source of retrieval bias and uses centroid targets with a lightweight adapter to approximate serialization-invariant representations from one serialization. Robustness gains are strongest for dense retrievers, whereas SPLADE is substantially less compatible with the correction.

  • Serialization can materially alter retrieval quality because linearizing tables introduces representation bias across formats, models, and benchmarks.
  • A frozen-encoder residual bottleneck adapter learns from a single serialization to approximate centroid-level robustness at inference time.The approach uses a centroid-based view of serializations as transformed views and a VICReg-inspired objective.
  • Robustness gains are especially strong for dense models, while SPLADE is substantially less compatible with the transport mechanism.

A.1 Dataset

The study evaluates three table-retrieval benchmarks, four retriever families, and heterogeneous serialization transformations, including format, layout, and schema variants. It also constructs centroids that average embeddings within representation categories and across all formats.

  • Datasets: Three benchmarks—WTQ, WikiSQL, and NQ-Tables—provide increasing structural difficulty for the empirical analysis.WTQ has heterogeneous tables and challenging semantics; WikiSQL is cleaner and structurally shallower; NQ-Tables has a large lexical gap.
  • Retriever Embedding Models: Four retrievers—MPNet, BGE-M3, ReasonIR, and SPLADE—span dense, multilingual or multi-function, reasoning-oriented, and learned sparse retrieval.
  • Serialization Formats: The serialization suite spans flat linearizations, standard exchange formats, shuffled or transposed layouts, and schema-oriented representations.Examples include pipe, token, space, csv, tsv, html, markdown, latex, dict, json, xml, mschema, macschema, and ddl.
  • Serialization Formats: Custom serializations differ in structural explicitness: pipe is a single flat string without row delimiters, while token serialization marks headers and cells explicitly.Space serialization uses space-separated headers and values and is labeled externally as none serialized in the code.
  • Centroid Representations: Centroids average embeddings within categories to capture prototypical semantic signatures of representation groups.
  • Centroid Representations: The data centroid averages eight content-complete formats, while popular, structural, schema, and all centroids summarize their respective format groups.The data category combines delimited text, markup, and structured interchange formats; the all centroid averages every serialization.

B Evaluation Metrics

The evaluation combines rank-based, score-based, and statistical measures to capture both top-position success and finer serialization preferences. It also quantifies format-specific shifts and tests whether observed rank differences are reliable.

  • Retrieval Metrics: Recall@1 measures whether the gold table is ranked first, providing a strict top-position retrieval criterion.It reflects whether downstream question answering can proceed without re-ranking.
  • Retrieval Metrics: Raw gold-table rank captures retrieval quality beyond binary Recall@1 and remains sensitive to near-miss failures.Unlike Recall@1, rank distinguishes different positions below the top result.
  • Retrieval Metrics: Mean pairwise score difference compares two serializations using their average gold-table retrieval-score difference across questions.Positive values favor the first serialization, while the measure retains score resolution even when both fail to rank the gold table first.
  • Adaptation Metric: Log-rank improvement compares adapted and base gold-table ranks, with positive values indicating improvement and logarithmic scaling reducing tail influence.It captures rank-change magnitude that Recall@1 treats equally.
  • Shift Analysis: Format-specific shifts are decomposed into table-independent and table-dependent components, whose magnitude ratio connects to the centering assumption.Centroid averaging can more effectively cancel shifts that vary across tables than shifts with a common direction.
  • Statistical Testing: Pairwise Wilcoxon signed-rank tests with Benjamini–Hochberg correction assess rank differences at α = 0.01.The tests operate over per-question ranks and account for multiple comparisons.

C Scope of the Format-Specific Shift

Format-specific shifts differ in their dependence on serialization format, table content, and retriever family. Dense retrievers often expose systematic schema or markup biases, while structural perturbations are more amenable to centroid averaging.

  • Dense Retrievers: Schema-based formats sit above the diagonal for MPNet, with mschema and macschema reaching |µδs| ≈ 0.65 and ratios of about 1.45 to 1.59.Their shifts remain similar across table contents rather than behaving as harmless noise.
  • Dense Retrievers: BGE-M3 generally shows smaller shift magnitudes, while ddl remains above ratio 1 across datasets.The smaller magnitudes correspond to weaker serialization sensitivity, whereas ddl occupies a distinct embedding region.
  • Retriever Differences: ReasonIR is more formatting-sensitive than BGE-M3, with several data-oriented formats near or above the boundary on WTQ.Shuffled variants remain below the diagonal, indicating table-specific noise that averaging can reduce.
  • Retriever Differences: SPLADE places every format above the diagonal, with systematic directional effects larger than random variation.Under this sparse lexical setting, the centering condition does not hold.
  • Structural Variants: Shuffled rows and columns show low systematic bias and higher residual variation, matching perturbations that centroid averaging can suppress.Transpose lies slightly higher because it introduces a more regular structural change.
  • Cross-Dataset Stability: Format layouts remain highly stable across datasets within each retriever, suggesting biases mainly reflect retriever–format interactions.This supports partial transfer of bias correction learned on one dataset to another.

D Proofs

The proofs establish the centroid as the least-squares representative of serialization embeddings and show when it also approximates the shared semantic component. The adapter objective operationalizes transport toward centroids while preserving useful geometry.

  • Centroid Optimality: The centroid is the unique Euclidean point minimizing squared distance to all serialization embeddings.Strict convexity makes this least-squares representative the unique global minimizer.
  • Shared-Component Recovery: The centroid equals the shared component plus the average format-specific shift, c(T) = µ(T) + ¯δ(T).When average perturbations are small, the centroid lies correspondingly close to µ(T).
  • Geometric Constraints: Identity regularization keeps adapted document vectors compatible with frozen query vectors and discourages large geometric drift.Only document-side table embeddings are adapted, so preserving the original space is essential.
  • Adapter Objective: The transport objective maps each view-specific embedding toward its table’s centroid using a stop-gradient centroid target.This explicitly suppresses view-specific noise by collapsing views toward a shared semantic anchor.
  • Geometric Constraints: Variance regularization keeps feature spread above a threshold, while covariance regularization reduces redundancy and feature entanglement.Together they constrain collapse and preserve an expressive retrieval geometry.

F Training Procedure

The adapter is trained from aligned multi-view cached embeddings and then applied to a single serialized representation for indexing. Training prioritizes cross-view alignment and identity preservation while maintaining non-collapsed, expressive representations.

  • Training Dynamics: λinv = 100 and λid = 100 dominate training, balancing cross-view alignment with preservation of the original embedding geometry.The invariance loss drops sharply, while identity loss plateaus and adapted embeddings retain high cosine similarity to originals.
  • Training Dynamics: λvar = 25 makes variance preservation a secondary constraint, while λcov = 1 lightly suppresses redundancy.The variance loss remains stable and covariance loss stays small throughout training.
  • Training Dynamics: The resulting dynamics show fast early loss reduction, stable non-collapsed training, and conservative adaptation that improves view agreement.The objective assigns distinct roles to invariance, identity, variance, and covariance terms.
  • Training Procedure: Training aligns cached embeddings by table ID, retains tables with at least two views, and samples up to max views per table.Each minibatch groups view embeddings by table and computes adapted embeddings and batch centroids.
  • Training Procedure: The optimization combines centroid invariance, feature-wise variance, covariance, and identity-preservation losses, updated with AdamW and gradient clipping.The loss components respectively align views, prevent collapse, reduce redundancy, and preserve the frozen space.
  • Inference Procedure: At inference, one base serialization is encoded, adapted, and indexed while queries remain encoded by the frozen retriever.This avoids encoding every serialization at inference time.
  • Geometric Effect: PCA projections show adapted embeddings tightening around their centroids across ten tables and four retrieval models.Frozen views are shown as circles, adapted views as crosses, and centroids as diamonds.

H Additional Results

Additional results show that adapter benefits depend on retriever geometry, serialization complexity, and dataset transfer: dense models often gain where representations are unstable, while sparse retrieval can degrade.

  • Additional Results: The detailed Recall@1 results expose absolute performance differences across datasets, retriever families, and serialization groups beyond aggregate rank movement.Table 7 reports base, full-adapter, and subset-adapter results under mixed-format perturbation, with percentage change relative to the base model.
  • Retriever-dependent gains: ReasonIR improves on structural perturbations, with WTQ shuffled rows rising from 0.22 to 0.26 and then 0.29 across the base, joint, and subset settings.Similar gains occur for shuffled columns and on WikiSQL shuffled rows.
  • Retriever-dependent gains: SPLADE usually degrades under adaptation, with its WTQ overall centroid score falling from 0.44 to 0.38 and 0.36.This contrasts with the clearer improvements observed for several dense retrievers.
  • Sensitivity to serialization complexity: MPNet gains most on syntactically heavy formats, rising from 0.09 to 0.18 on WTQ html and from 0.11 to 0.17 on WikiSQL html.Cleaner formats such as WTQ tsv leave less room for improvement because the base model is already comparatively strong.
  • Cross-dataset robustness and transfer: The subset-trained adapter transfers partially from WTQ and WikiSQL to unseen NQ-Tables, but transfer remains incomplete and retriever-specific.ReasonIR’s overall centroid score rises from 0.29 to 0.32, whereas BGE-M3 transfer is mixed and SPLADE remains substantially worse than its base model.
  • Robustness under Table Perturbation: Mixed-format perturbation assigns each row a randomly sampled serialization, removes global structural consistency, and applies the adapter post hoc without re-encoding.Retrieval uses cosine similarity over corrected representations.
  • Robustness rather than universal improvement: Adapter improvements concentrate where base representations are least stable, while relatively stable baselines show mild changes such as MPNet on WTQ at −1.1%.The subset adapter matches or exceeds full adaptation in several cases, but BGE-M3 on NQ declines by −5.7% under the subset setting.
Loading 2604.24040v2…