Source-linked AI summary
Table2Vec: Neural Word and Entity Embeddings for Table Population and Retrieval
Li Deng, Shuo Zhang, Krisztian Balog
TL;DR
Prior work had not trained table embeddings specifically for population and retrieval. Table2Vec learns embeddings from table elements and uses them in three table-related tasks; the authors report improvements across these tasks, with different results by embedding type.
Problem
Although prior work considered table embeddings, none had trained them specifically for table population and retrieval.
Source: §1 · p. 1Method
Table2Vec uses neural language modeling to embed table elements and incorporates the embeddings into existing task models as semantic matching signals.
Source: §1 · p. 1, §2.1 · p. 2, §3.3 · p. 3Results
Table2VecE* significantly beat row-population baselines, Table2VecH improved column population, and Table2VecW beat a state-of-the-art retrieval baseline by over 10% in NDCG@10.
Source: §1 · p. 1Takeaways & Limitations
The authors conclude that table-derived embeddings significantly improve effectiveness across the three studied tasks.
Source: §5 · p. 4The approach assumes table entities are already recognized and disambiguated by linking them to a knowledge base.
Source: §3 · p. 2
Abstract
from arXiv · showhide
Tables contain valuable knowledge in a structured form. We employ neural language modeling approaches to embed tabular data into vector spaces. Specifically, we consider different table elements, such caption, column headings, and cells, for training word and entity embeddings. These embeddings are then utilized in three particular table-related tasks, row population, column population, and table retrieval, by incorporating them into existing retrieval models as additional semantic similarity signals. Evaluation results show that table embeddings can significantly improve upon the performance of state-of-the-art baselines.
1 INTRODUCTION
Table2Vec trains embeddings from table elements for table-related tasks, addressing the lack of table-specific embedding training for table population and retrieval.
1 INTRODUCTION
- The study targets row population, column population, and keyword-query table retrieval over relational tables.Row population adds entities to the core column, while column population adds headings.Source: §1 · p. 1
- The paper proposes neural language modeling to map different table elements into semantic vector spaces, targeting table population and retrieval.Source: §1 · p. 1
- Table2VecW outperformed a state-of-the-art baseline by over 10% in NDCG@10; Table2VecE* significantly outperformed row-population baselines, and Table2VecH improved column population.For larger seed-heading sets, Table2VecH achieved 40% relative improvement in NDCG@10 over the baseline.Source: §1 · p. 1
- The resources developed in the work are made publicly available.Source: §1 · p. 1
2 TRAINING TABLE2VEC EMBEDDINGS
The paper trains table embeddings with a Word2Vec skip-gram model, varying which table elements count as terms and which elements supply training sequences.
2.1 Neural Model for Training Embeddings
- Word2Vec skip-gram learns term vectors from raw sequences by maximizing the average log probability of context terms around each training term.A softmax calculates context-term probabilities from input and output vectors; similar terms have more similar representations and higher dot products.Source: §2.1 · p. 2
- Negative sampling is used to make training the embedding models more computationally efficient.Source: §2.1 · p. 2
2.2 Four Variants
- Table2VecW uses words from page titles, section titles, captions, headings, and cells, while Table2VecH treats each heading as a term.Source: §2.2 · p. 2
- Table2VecE uses entities from table cells, whereas Table2VecE* uses only entities in the core column.Source: §2.2 · p. 2
3 UTILIZING TABLE2VEC EMBEDDINGS
The section extends population and retrieval methods by incorporating Table2Vec embeddings into relational-table tasks, covering row population, column population, and table retrieval.
3.1.1 Baselines.
- The three [14] baselines rank candidates by relational similarity, Wikipedia outgoing-link relatedness, or Jaccard similarity of outgoing links.Candidate selection follows [14].Source: §3.1.1 · p. 2
3.1 Row Population
- Row population ranks candidate entities by similarity to seed-table entities, combining knowledge-base similarity with Table2Vec-based similarity.The method uses Table2VecE*, trained on core-column entities, and averages cosine similarity between each candidate and the seed entities.Source: §3.1.2 · p. 2, §3.1 · p. 2
3.1.2 Using Table2Vec embeddings.
- Table2VecE* similarity is linearly mixed with knowledge-base similarity to estimate candidate probability given the seed entities.The embedding similarity is the average cosine similarity between a candidate entity and the seed entities.Source: §3.1.2 · p. 2
3.2 Column Population
- Column population ranks candidate headings by relevance to the seed headings, while its corpus baseline estimates relevance from heading occurrences in retrieved tables.The baseline first retrieves relevant tables from a table corpus, then estimates candidate-heading probability from those tables.Source: §3.2 · p. 3, §3.2.1 · p. 3
3.2.1 Baseline.
- Heading relevance combines a baseline estimate with embedding-based similarity between a candidate heading and the seed headings.The embedding-based estimate uses cosine similarity between the candidate heading and each seed heading.Source: §3.2.2 · p. 3
3.3 Table Retrieval
- Table retrieval adds Table2VecW and Table2VecE semantic-matching features to the LTR baseline, with four features contributed by each embedding type.The embedding approach compares query and table vectors using early fusion or late fusion with max, sum, and average aggregators.Source: §3.3 · p. 3
4 EVALUATION
The evaluation tests whether Table2Vec improves table population and retrieval, using relational-table data and task-specific test sets. Results show gains for row and column population, while retrieval favors word embeddings over entity embeddings.
4.1 Research Questions
- The research questions ask whether Table2Vec improves population, whether table-trained word embeddings affect retrieval, and whether word or entity embeddings work better for retrieval.Source: §4.1 · p. 3
4.2 Experimental Setup
- The evaluation uses Wikipedia Tables for embedding training and retrieval, with test sets covering 1,000 tables each for row and column population and 60 retrieval queries.The corpus contains 1.6 million high-quality relational tables; retrieval assessments comprise 3,120 query-table pairs.Source: §4.2 · p. 3
4.3 Row Population
- Combined row-population methods significantly outperform their respective baselines (p < 0.01), and BL1 + Table2VecE* achieves the best MAP.Table 3 reports row-population performance and tests significance against each respective baseline.Source: §4.3 · p. 3, Table 3 · p. 4
4.4 Column Population
- Table2VecH significantly improves column-population MAP when |L| > 1; at |L| = 3, it significantly improves both MAP and MRR.Unlike the baseline, the combined method continues improving with larger heading inputs; its best α is 0.01.Source: Table 4 · p. 4, §4.4 · p. 4
4.5 Table Retrieval
- Table2VecW and Word2Vec perform comparably, outperform other retrieval methods, and significantly improve over the baseline (p < 0.01).Graph2Vec and Table2VecE improve over the baseline, but not significantly; results show no retrieval difference between table-trained and news-trained word embeddings.Source: Table 5 · p. 4, §4.5 · p. 4
4.3 Row Population
- With more seed entities, BL1 + Table2VecE* improves more than BL1, which reaches its peak after two seed entities.The authors say this indicates the combined method better utilizes seed entities.Source: §4.3 · p. 4
5 CONCLUSION AND FUTURE WORK
Table2Vec trains word and entity embeddings from different table elements and applies them to three table-related tasks. The study reports significant improvements in retrieval effectiveness, with embeddings derived from a corpus of high-quality relational tables.
5 CONCLUSION AND FUTURE WORK
- Table2Vec trains word and entity embeddings from table elements and uses them in three table-related tasks; the study reports significant retrieval-effectiveness improvements.The embeddings are derived particularly from a Wikipedia corpus containing high-quality relational tables; future work will extend to other corpora and table types.Source: §5 · p. 4