Polaris: Learning to Generate Table Descriptions from Retrieval Feedback

• A single LLM call achieves comparable or better performance than AutoDDG’s more complex, multi-call pipeline.Source: §1 · p. 1
• Expanding cryptic table and column names into full English phrases gives the LLM richer input and substantially improves generated descriptions.The expansion solution is attributed to Cai et al. (2025).Source: §1 · p. 2
• Polaris ranks candidate descriptions using labeled query-table relevance data, then fine-tunes the LLM with DPO rather than human preference rankings.Source: §1 · p. 2
• Using generated descriptions together with table metadata, such as name expansions and contexts, can significantly improve keyword-search accuracy.Source: §1 · p. 2
• The authors assemble a six-dataset benchmark of table-query relevance judgments and propose retrieval benchmarks as supervision for training LLMs to generate retrieval-oriented metadata.They describe it as the largest benchmark to date for evaluating keyword search over tables.Source: §1 · p. 2

AutoDDG

• AutoDDG generates table descriptions through a multi-step pipeline requiring at least four LLM calls, using table and column names, available metadata, and sample tuples.The pipeline produces table statistics, topics, and column summaries before generating a description geared toward keyword search.Source: §2 · p. 2
• AutoDDG ranks tables by BM25 similarity between descriptions and queries; Zhang et al. (2026) report it outperforms six solutions, including Pneuma and two pretrained data-to-text models.The comparison includes methods by Tang et al. (2023), Zhao et al. (2022), and Balaka et al. (2025).Source: §2 · p. 2

Table Description Generation in Polaris

• Polaris generates descriptions from table and column names and available context, omits tuples that may be unavailable for privacy or legal reasons, and fine-tunes on labeled tables, queries, and relevance scores.The paper states that Polaris without tuples outperforms AutoDDG with tuples; it defers the supporting results to Section 4.Source: §2 · p. 3

Keyword Search in Polaris

• Polaris uses BM25 TF/IDF to rank tables by description-query similarity and also considers a setting that scores descriptions together with other table metadata.Embedding and hybrid similarity scores are left for future work.Source: §2 · p. 3

3.1 Table/Column Name Expansion

• Polaris expands short, cryptic table and column names into English phrases to reduce the vague descriptions that can hurt keyword search, using Columbo (Cai et al., 2025).For example, it expands “DTPh” to “daytime phone” and “russ_ind” to “russell index.”Source: §3.1 · p. 3

3.2 Initial Table Description Generation

• For each gold table, an untuned LLM generates at least two diverse candidate descriptions; each non-gold table receives one description.The prompt includes name expansions, available context, and a one-shot example; descriptions are JSON objects with six fields, including dataset overview, themes, use cases, synonyms, keywords, and additional context.Source: §3.2 · p. 4

3.3 Ranking the Table Descriptions

• For each gold table, Polaris ranks candidate descriptions by mean BM25 over relevant queries in a unified corpus and prefers the highest- over the lowest-scoring candidate.The unified-corpus approximation incurs negligible BM25-score loss; non-target tables receive random descriptions, and the procedure runs once per gold table because repeated runs are time-consuming.Source: §3.3 · pp. 4–5

3.4 DPO Training with the Rankings

• Polaris uses one winner-loser preference pair per gold table to fine-tune the base LLM with DPO, increasing the winner’s likelihood relative to the loser without a separate reward model.The base model is Llama 3.1 8B Instruct, fine-tuned with 4-bit quantization and LoRA.Source: §3.4 · p. 5

3.5 Description Generation for New Tables

• For a new table corpus, Polaris expands table and column names, then feeds the expansions and other metadata to the trained LLM to generate each table’s description.It uses the same prompt as for initial description generation.Source: §3.5 · p. 5

Evaluation Setup

• Each leave-one-out run fine-tunes on five of six labeled datasets, evaluates retrieval on the held-out dataset, and reports NDCG@K and Recall@K.The measures assess ranking quality with graded relevance and recall, respectively.Source: §4 · pp. 5–6

Overall Performance

• 0.356 → 0.515 NDCG@100 on LTER (+44.6% relative) and 0.555 → 0.731 NDCG@5 on AW (+31.7% relative); Polaris beats AutoDDG on nearly all cutoffs for five datasets, while WikiTables is similar.Polaris also reduces average description length by 47–59%.Source: Table 2 · p. 6, Table 3 · p. 6, §4 · p. 6

Ablation Studies: Prompt Quality

• Without DPO or name expansion, Polaris still beats AutoDDG on four datasets and performs similarly on two, suggesting its simpler prompt is competitive.The authors suggest AutoDDG’s multi-step prompting can lose information when intermediate summaries and topics are imperfect or length-limited.Source: §4 · p. 6

Ablation Studies: Effect of DPO Training

• DPO improves @5 accuracy on AW (+9.4%), LTER (+10.4%), and WTR (+5.0%), but lowers ECIR accuracy (.495 vs. .570).The authors attribute ECIR’s exception to high relevant-table density, which leaves little discrimination for DPO’s preference signal.Source: §4 · p. 6

Ablation Studies: Effect of Table/Column Name Expansion

• Name expansion improves @5 accuracy on LTER (+21.2%), AW (+5.4%), Arctic (+6.6%), and WTR (+8.0%); gains are small on ECIR and WikiTables.The latter datasets rely more on external context than table names.Source: §4 · p. 7

Sensitivity Analysis

• With Qwen, Polaris beats AutoDDG on every dataset; compared with Llama, it improves AW, Arctic, and WikiTables, underperforms on LTER, and is mixed on ECIR and WTR.Raising β from 0.1 to 0.5 improves AW, hurts LTER, and has negligible effects elsewhere; β = 0.1 balances datasets better.Source: §4 · p. 7
• Using K = 5 candidates improves @5 on AW (+9.7%) and WikiTables (+5.3%) but hurts LTER (−11.8%) and Arctic (−4.1%); three epochs boost Arctic (+13.3%) but hurt ECIR, WTR, and WikiTables.More candidates may add noisy preferences, while extra epochs help datasets with many training pairs but risk overfitting smaller training sets; one epoch is the default.Source: §4 · p. 7

Multi-Field KWS with Polaris

• Multi-field Polaris outperforms single-field Polaris on five datasets and is competitive on one, using the best-matching field’s BM25 score with other fields as tie-breakers.The description is the most critical field; name expansions matter for datasets with abbreviations, while table context helps web and government data such as WTR.Source: Figure 4 · p. 7, §4 · p. 7

Metadata Generation for Tables

• Polaris differs from table-metadata generators such as AutoDDG by training on retrieval feedback and expanding table and column names.Prior work generates table and dataset descriptions, column and database descriptions, name expansions, semantic column types, and spatial or temporal summaries for tasks including KWS and NL2SQL.Source: §5 · pp. 7–8

Keyword Search over Tables

• Polaris generates keyword-search-optimized table descriptions that can serve as input to existing retrieval models.Earlier keyword-search approaches improve matching and ranking through sparse and dense representations, BERT-based ranking with content selection, structural encoding, or schema labels.Source: §5 · p. 8

Retrieval Benchmarks

• Table-retrieval benchmarks provide the labeled relevance data for Polaris’s DPO training, while PIPER generates pseudo-queries for dense retrieval without optimizing its generator from retrieval feedback.TARGET reports that dense retrievers outperform BM25 but remain sensitive to metadata quality.Source: §5 · p. 8

Preference Learning and DPO

• Unlike DPO applications using human or AI preferences, Polaris derives preference pairs from benchmark relevance judgments and BM25 retrieval effectiveness to optimize table-description generation.The closest concurrent work, Coelho et al. (2025), optimizes query generation using retrieval feedback; related work also optimizes query generation or document rewriting.Source: §5 · p. 8

• Polaris uses retrieval performance on existing table retrieval benchmarks to construct preference pairs for DPO fine-tuning, combined with table and column name expansion to better match users’ search vocabulary.Source: §6 · p. 8
• Across six datasets spanning enterprise, scientific, government, and web domains, Polaris consistently outperforms the state of the art while using a simpler generation pipeline.Source: §6 · p. 8
• The authors suggest retrieval benchmarks can train metadata generators beyond keyword search, including for dense retrieval, NL2SQL, table question answering, and other downstream tasks.Source: §6 · p. 8

7 Limitations

• Polaris’s DPO training requires query–table relevance judgments, limiting retrieval-guided optimization where such supervision is unavailable.The untrained variant remains applicable in those settings, but does not benefit from retrieval-guided optimization.Source: §7 · p. 9
• Because preferences derive solely from BM25 scores, Polaris optimizes descriptions for lexical retrieval and may not transfer to dense or hybrid ranking.Extending preference construction to dense or hybrid objectives is identified as future work.Source: §7 · p. 9
• Polaris optimizes metadata generation rather than retrieval models, making it complementary to advances in dense retrieval and learned ranking.Such retrieval models could potentially benefit from improved table descriptions.Source: §7 · p. 9
• Polaris uses only schemas and metadata, so incorporating representative rows could yield richer descriptions when table contents are available.The schema-only design targets settings where contents are unavailable for privacy, legal, or computational reasons.Source: §7 · p. 9
• Experiments cover six datasets across four domains and show encouraging cross-domain generalization, but retrieval-guided training benefits vary by dataset.The authors identify understanding when retrieval feedback is most informative as future work.Source: §7 · p. 9

A.1 Table/Column Name Expansion

• The expansion guidelines require resolving abbreviations, preserving token order, leaving numbers and unabbreviated tokens unchanged, and producing one short expansion per token without additions or hedging.Chemical symbols and units are included among abbreviations to expand.Source: §A.1 · p. 12
• Column expansion uses four few-shot examples that reason from table context, split column names into tokens, and return token-level mappings concatenated into one expanded string per column.Target columns are prompted in batches of 10; the prompt has a system message, task statement, and examples.Source: §A.1 · p. 12, Table 4 · p. 13
• Table expansion reuses the column-expansion instructions and examples, but supplies the table name with its columns so column names can clarify the table’s meaning.The output is a token-level mapping concatenated into one expanded table name; tables without names skip this step.Source: §A.1 · p. 12, Table 5 · p. 14
• At inference, the system supplies already-expanded column names and includes at most the first 25 to bound prompt length.This limit applies to the column context used for table-name expansion.Source: §A.1 · p. 12

A.2 Initial Table Description Generation

• The sample describes quarterly SSDI language-preference data from fiscal year 2016 onward and frames analyses of trends across quarters and years.It also lists comparing claimant language preferences across fiscal years as a use case.Source: §A.2 · p. 16
• The output adds related concepts and synonyms, retrieval-oriented keywords, and context that the data is organized by fiscal year and quarter.Examples include “Language Preference” with “Spoken Language” and “Preferred Language,” and “Disability Insurance” with “SSDI” and “Disability Benefits.”Source: §A.2 · p. 16
• The example input schema contains fiscal-year, quarter, and language-preference columns for Disability Insurance claimants; the prompt marks input placeholders in red.Source: Table 6 · p. 15, Table 7 · p. 16

A.3 Ranking the Table Descriptions

• For each gold table, Polaris compares K corpus copies that differ only in candidate description, alongside one copy of every other table.Each other gold table is represented by one randomly selected candidate description.Source: §A.3 · p. 16
• Polaris averages retrieval scores across relevant queries, giving each query equal weight, then ranks descriptions from highest to lowest score.Higher-ranked descriptions make the correct table score more highly for its relevant queries.Source: §A.3 · p. 16

A.4 DPO Training with the Rankings

• Polaris retains the most separated pair from the ranked descriptions to construct each preference-training example.The ranking is dπ1 ≻ dπ2 ≻ dπ3.Source: §A.4 · p. 16
• DPO increases the winning description’s likelihood relative to the losing one without fitting a separate reward model.The objective uses the table prompt, a frozen reference model, and β to control regularization.Source: §A.4 · p. 17
• Training uses 4-bit Llama 3.1 8B Instruct with LoRA, updating 167M of 8.2B parameters and holding out each dataset in turn.The setup uses r = 64, α = 64, one epoch, β = 0.1, learning rate 5 × 10^-6, effective batch size 8, and an AdamW 8-bit optimizer; each fold trains on preference pairs from the other five datasets.Source: §A.4 · p. 17

A.5 Datasets and Experimental Settings

• Evaluation covers six datasets across four domains; BM25 downsampling retrieves the top-50 tables per query using available metadata, then unions them with all gold-relevant tables, retaining competitive hard negatives.Three datasets are curated from prior work and three are newly created.Source: §A.5 · p. 17

A.6 Overall Performance

• Figure 6 reports average description length for AutoDDG and Polaris on each dataset.The provided passage and caption specify the comparison but not the lengths.Source: §A.6 · p. 18, Figure 6 · p. 19

A.7 Ablation Studies

• The ablations compare prompt quality for AutoDDG versus Polaris without DPO or name expansion, and measure the effects of DPO training and name expansion.Tables 9–12 cover NDCG@K and Recall@K; the supplied passages do not state the results.Source: §A.7 · p. 18, Table 9 · p. 19, Table 10 · p. 19

A.8 Sensitivity Analysis with Different LLMs

• The LLM sensitivity study evaluates Polaris with Qwen2.5-7B and compares Polaris with AutoDDG when both use Qwen2.5-7B.Tables 13–16 report NDCG@K and Recall@K comparisons.Source: §A.8 · p. 19, Table 13 · p. 20, Table 14 · p. 20

A.9 Sensitivity Analysis with Different DPO β

• The DPO sensitivity study compares Polaris at β = 0.5 with its default β = 0.1.Tables 17–18 report NDCG@K and Recall@K.Source: §A.9 · p. 19, Table 17 · p. 21, Table 18 · p. 21

A.10 Sensitivity Analysis with Different Numbers of Descriptions

• The candidate-count sensitivity study compares Polaris with five candidate descriptions per gold table against its default of three.Tables 19–20 report NDCG@K and Recall@K.Source: §A.10 · p. 19, Table 19 · p. 21, Table 20 · p. 21

A.11 Sensitivity Analysis with Different Numbers of Training Epochs

• The training-epoch sensitivity study compares Polaris trained for three epochs with its one-epoch default.Tables 21–22 report NDCG@K and Recall@K.Source: §A.11 · p. 19, Table 21 · p. 21, Table 22 · p. 21

A.12 Multi-Field Keyword Search

• The multi-field study compares multi-field Polaris with single-field Polaris and AutoDDG, then ablates description, name-expansion, and context fields.The field ablation covers ECIR, WTR, and WikiTables; Tables 23–24 report NDCG@K and Recall@K, while Table 25 reports NDCG@K.Source: §A.12 · p. 19, Table 23 · p. 22, Table 24 · p. 22