PaperlayerView exact revision on arXiv

Source-linked AI summary

Text-to-Table: A New Way of Information Extraction

Xueqing Wu, Jiacheng Zhang, Hang Li

arXiv:2109.02707v2cs.CL

TL;DR

Text-to-table addresses extracting complex, document-level information into tables without explicitly defined schemas. The paper formalizes this as seq2seq generation and adds table constraint and table relation embeddings; experiments report that the proposed approach outperforms conventional IE methods and improves on vanilla seq2seq. The task also faces challenges in obtaining paired data, representing some structures as tables, and evaluation.

  • Problem

    Text-to-table addresses extraction from entire documents into complex tables without explicitly defined schemas, a setting unlike traditional IE.

    Source: §3 · p. 4

  • Method

    The paper formulates text-to-table as seq2seq generation and adds table constraint and table relation embeddings to the approach.

    Source: §1 · p. 2

  • Results

    The proposed approach outperforms conventional IE methods, and its two techniques further improve extraction accuracy over vanilla seq2seq.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    Text-to-table outputs can serve applications such as document summarization and text mining.

    Source: §1 · p. 2

    Challenges include costly construction of text-table pairs, structures that may not convert readily into tables, and evaluation across headers, content, and structure.

    Source: §3 · p. 4

Abstract

from arXiv · show

We study a new problem setting of information extraction (IE), referred to as text-to-table. In text-to-table, given a text, one creates a table or several tables expressing the main content of the text, while the model is learned from text-table pair data. The problem setting differs from those of the existing methods for IE. First, the extraction can be carried out from long texts to large tables with complex structures. Second, the extraction is entirely data-driven, and there is no need to explicitly define the schemas. As far as we know, there has been no previous work that studies the problem. In this work, we formalize text-to-table as a sequence-to-sequence (seq2seq) problem. We first employ a seq2seq model fine-tuned from a pre-trained language model to perform the task. We also develop a new method within the seq2seq approach, exploiting two additional techniques in table generation: table constraint and table relation embeddings. We consider text-to-table as an inverse problem of the well-studied table-to-text, and make use of four existing table-to-text datasets in our experiments on text-to-table. Experimental results show that the vanilla seq2seq model can outperform the baseline methods of using relation extraction and named entity extraction. The results also show that our method can further boost the performances of the vanilla seq2seq model. We further discuss the main challenges of the proposed task. The code and data are available at https://github.com/shirley-wu/text_to_table.

1 Introduction

Text-to-table learns to turn text into tables, including document-level extraction of complex information without explicitly defined schemas. The paper formalizes it as a seq2seq task, adds table-generation techniques, and reports improvements over conventional IE baselines.

1 Introduction

  • The system learns from text-table pairs and uses the learned model to extract information from new text into one or more tables.The example converts a basketball-game report into tables of team and player scores.Source: §1 · pp. 1–2
  • Unlike traditional IE, text-to-table can extract complex information from an entire document, with schemas implicitly included in training data rather than explicitly defined.The authors say this reduces manual effort for schema design and annotations.Source: §1 · p. 2
  • Text-to-table is framed as the inverse of table-to-text, with potential uses in text summarization and text mining.The paper gives sports score tables and Wikipedia infoboxes as examples of summaries of original documents.Source: §1 · p. 2
  • The paper formalizes text-to-table as seq2seq generation over a table representation and adds table constraint and table relation embeddings to help produce well-formulated tables.Table constraint controls row creation; relation embeddings affect alignment between cells and row or column headers.Source: §1 · p. 2
  • The proposed seq2seq method with table constraint and table relation embeddings improves extraction accuracy over vanilla seq2seq, which itself outperforms conventional IE baselines.The experiments use four datasets derived from existing table-to-text datasets.Source: §1 · p. 2
  • Text-to-table is proposed as a new IE task, with four datasets derived from existing datasets and a seq2seq method using table constraint and table relation embeddings.The authors also report experiments verifying the approach’s effectiveness.Source: §1 · p. 2

2 Related Work

Prior IE work includes schema-based extraction for individual and joint tasks, as well as schema-free OpenIE and document-level methods. Seq2seq has been used for IE and data-to-text, but the cited approaches do not directly address text-to-table’s combination of long texts, complex tables, and implicit schemas.

2 Related Work

  • Traditional IE methods use pre-defined schemas and often target short texts and simple structures, so they cannot be directly applied to text-to-table.OpenIE avoids explicit schemas but extracts simple relation tuples from short texts; document-level IE typically does not model complex relations among many items.Source: §2 · p. 3
  • Pre-trained BART and T5 have boosted seq2seq performance in tasks including machine translation and text summarization.Seq2seq transforms one text into another text.Source: §2 · p. 3
  • Data-to-text generates natural-language descriptions from structured inputs represented as tables, cell sets, semantic representations, or relation triples.Such models select salient information, organize it, and generate accurate, fluent descriptions, commonly using encoder-decoder architectures.Source: §2 · p. 3

3 Problem Formulation

Text-to-table takes a text and produces one or more tables, potentially extracting complex information from an entire document without explicitly specified schemas. The task also presents challenges in acquiring paired data, representing some structures as tables, and evaluating extraction.

3 Problem Formulation

  • Text-to-table uses a text as input and produces one or more tables that summarize its content.The formalization represents table cells as word sequences arranged in rows and columns.Source: §3 · p. 3
  • Tables may have both row and column headers, only column headers, or only row headers.The player-table example includes player names as row headers and statistics as column headers.Source: §3 · p. 4
  • The task extracts complex information from an entire document and does not require explicitly defined schemas.The example covers multiple types of team and player scores.Source: §3 · p. 4
  • Key challenges include costly construction of text-table pairs, structures that may not convert readily into tables, and evaluation across headers, content, and structure.Source: §3 · p. 4

4 Our Method

The method casts text-to-table as seq2seq generation, encoding tables as token sequences and training on text-table pairs. It adds decoding constraints and table-relation embeddings to improve table generation.

4.1 Vanilla Seq2Seq

  • Vanilla seq2seq represents each table row as cells separated by ⟨s⟩ tokens and separates rows with ⟨n⟩ tokens.When generating multiple tables, captions delimit their sequences.Source: §4.1 · p. 4
  • The model generates one token at a time from prior output tokens and is trained on text-table pairs by minimizing cross-entropy loss.The model is autoregressive at inference.Source: §4.1 · p. 4
  • Vanilla seq2seq does not guarantee well-formed tables, so post-processing uses the first row's width to trim or pad later rows.Extra cells are deleted and empty cells inserted at the ends of other rows.Source: §4.1 · p. 4

4.2 Techniques

  • The method adds a decoding constraint that forces every row after the first to contain the same number of cells.It uses the first generated row's cell count as the constraint.Source: Table Constraint · p. 5
  • Table-relation embeddings are incorporated into decoder self-attention to represent alignments between non-header cells and their row and column headers.During inference, the generated sequence is parsed into a partial table to identify those headers.Source: Table Relation Embeddings · p. 5

5 Experiments

Experiments use four text–table datasets, compare seq2seq methods with IE baselines, and evaluate extraction with F1 measures. The proposed method leads on Rotowire, while results on simpler two-column datasets show no further gain over vanilla seq2seq from table constraint and relation embeddings.

5.1 Datasets

  • The four datasets pair text with tables; Rotowire uses long basketball reports and team/player tables, while the other datasets cover restaurants and Wikipedia.The datasets are Rotowire, E2E, WikiTableText, and WikiBio; table content absent from the text is filtered out.Source: §5.1 · pp. 5–6

5.2 Procedure

  • The procedure compares seq2seq with dataset-specific RE or NER baselines and evaluates header and non-header-cell F1 using exact match, chrF, and BERTScore.The reported evaluation also includes an error rate for output sequences that cannot represent well-formed tables.Source: §5.2 · pp. 6–7

5.3 Results on Rotowire

  • On Rotowire, our method performs best, followed by vanilla seq2seq on most measures; both outperform document- and sentence-level RE baselines.Document-level RE performs better than sentence-level RE because some information requires cross-sentence context.Source: §5.3 · p. 7, Table 3 · p. 7

5.4 Results on E2E, WikiTableText and WikiBio

  • On E2E, WikiTableText, and WikiBio, seq2seq outperforms NER, while our method and vanilla seq2seq are comparable on these two-column tables.NER has high precision but low recall, mainly because it makes the right decision only when the text is clear.Source: §5.4 · p. 7, Table 4 · p. 7

5.5 Additional Study

  • Ablations show that TC and TRE improve Rotowire performance, while the pre-trained language model boosts performance on all datasets, especially Rotowire and WikiTableText.BART-large further boosts performance on all four datasets. Vanilla seq2seq makes more formatting errors, especially in wide Rotowire player tables; TC can eliminate incorrectly formatted output and TRE helps align cells with headers.Source: §5.5 · pp. 8–9, Table 6 · p. 8

5.6 Discussions

  • The discussion identifies challenges in text diversity, redundant input, large tables, open-domain background knowledge, and reasoning when information is implicit.Rotowire tables remain challenging even for TC and TRE; the authors also say further improvement is needed for redundant texts and suggest more powerful pre-trained models or external knowledge bases for background-knowledge demands.Source: §5.6 · p. 9

6 Conclusion

The conclusion presents text-to-table as a new information-extraction task, formalizes it as seq2seq, and proposes table constraint and table relation embeddings. It also identifies challenges for future study.

6 Conclusion

  • Text-to-table extracts information from short or long texts into simple or complex tables without explicitly defining schemas.The approach is formalized as seq2seq, with table constraint and table relation embeddings as additional techniques.Source: §6 · p. 9
  • The paper reports experiments on four datasets derived from existing table-to-text datasets and says its approach outperforms conventional IE methods.The conclusion also notes analysis of text-to-table challenges for future study.Source: §6 · p. 9
  • The identified challenges include text diversity and redundancy, large tables, background knowledge, and reasoning.Source: §6 · p. 9

A Hyper-parameters

This appendix section points to tables listing the pretrained-model and BART-base training hyperparameters used in the experiments.

A Hyper-parameters

  • Table 7 lists hyper-parameters of the pre-trained models, while Table 8 lists training hyper-parameters for the BART-base model in vanilla seq2seq and our method.Source: A Hyper-parameters · p. 12

B Table Constraint Algorithm

The appendix provides pseudocode for decoding with table constraint.

B Table Constraint Algorithm

C Our Method with Multiple Tables

The method can generate multiple tables, illustrated with Rotowire, where captions delimit the team and player tables. It can generate an unspecified number of tables and applies table constraints during table generation.

C Our Method with Multiple Tables

D Information Extraction Baselines

The Rotowire relation-extraction baseline treats team names, player names, and score numbers as entities, with score types as relations. Synthetic training data is created by matching these items to text using fuzzy-match rules.

D.1 Relation Extraction

  • For Rotowire, the baseline treats team names, player names, and score numbers as entities and score types as relations.A relation can be represented as (Al Horford, Points, 15).Source: §D.1 · p. 14
  • The baseline defines 38 relation types, including Points.Source: §D.1 · p. 14
  • Synthetic training data is created by matching player names, team names, and score numbers to text using adapted fuzzy-match rules from Wiseman et al. (2017).Source: §D.1 · p. 14

D.2 Named Entity Recognition

For E2E, WikiTableText, and WikiBio, the named-entity-recognition baseline treats two-column table headers as entity types and other cells as entity mentions. Its synthetic training data matches cell contents to text, though many cells cannot be matched.

D.2 Named Entity Recognition

  • For E2E, WikiTableText, and WikiBio, the NER baseline treats row headers as entity types and non-header cells as entity mentions.Entity types are collected from headers in the training set.Source: §D.2 · p. 14
  • The baseline uses 7 entity types for E2E, 2262 for WikiTableText, and 2272 for WikiBio.Source: §D.2 · p. 14
  • Synthetic matching covers 85% of E2E non-header cells, 74% of WikiTable, and 69% of WikiBio.The texts may paraphrase or abstract table contents, preventing some matches.Source: §D.2 · p. 14

E Detailed Cases for Challenges

The cases illustrate text-to-table challenges involving varied expressions, redundant text, large tables, background knowledge, and information that requires reasoning rather than direct extraction.

Text Diversity

Text Redundancy

Large Table

  • Rotowire tables are large: team tables average 2.71 rows and 4.84 columns, while player tables average 7.26 rows and 8.75 columns.The example has a 3-by-4 team table and an 8-by-8 player table; even the method using TC and TRE finds extraction challenging.Source: E Detailed Cases for Challenges · p. 14, Figure 9 · p. 16

Background Knowledge

  • Open-domain WikiTableText and WikiBio require substantial background knowledge, such as knowledge of the French political system to extract Philippe Adnot’s constituency.More powerful pre-trained language models or external knowledge bases are suggested as possible ways to address this challenge.Source: E Detailed Cases for Challenges · p. 14, Figure 8 · p. 16

Reasoning

Related papers

Loading 2109.02707v2…