PaperlayerView exact revision on arXiv

Source-linked AI summary

TEAR: Table Extraction with Attribute Recommendation from Texts via Large Language Models

Tong Li, Shuye Ding, Jiachuan Wang, Yongqi Zhang, Shuangyin Li, Lei Chen, Bo Li

arXiv:2609.15205v1cs.DBcs.AI

TL;DR

Naturally occurring texts can have ambiguous attribute scopes, such as whether “Victim status” includes only direct casualties or also relocation. TEAR uses dynamically selected demonstrations for table extraction and recommends new text-derived attributes to complement the existing schema. TEAR consistently outperforms state-of-the-art baselines on both table extraction and attribute recommendation.

  • Problem

    Naturally occurring texts can have ambiguous attribute scopes, such as whether “Victim status” includes only direct casualties or also relocation.

    Source: §1 · p. 2

  • Method

    TEAR uses dynamically selected demonstrations for table extraction and recommends new text-derived attributes to complement the existing schema.

    Source: §1 · p. 3

  • Results

    TEAR consistently outperforms state-of-the-art baselines on both table extraction and attribute recommendation.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Text-driven attribute recommendation supports exploratory schema design for table extraction.

    Source: §1 · p. 3, §7 · p. 13

    The paper evaluates single-round recommendation quality; multi-round interaction would require modeling relationships among attributes across rounds and a more complex benchmark.

    Source: §8 · p. 13

Abstract

from arXiv · show

Table extraction from texts is an important task for information systems, and recent approaches that prompt large language models (LLMs) with instructions have drawn great attention for their strong performance. Existing works have assumed the input texts to be table descriptions or specialized documents. However, these efforts have largely overlooked another prevalent category of texts, commonly found in news reports and social media: naturally occurring texts. Extracting tabular information from such texts poses two distinct challenges. First, high variability and the absence of explicit structural cues make fixed heuristic LLM prompts limited in precisely delineating extraction boundaries. Second, manually predefined schemas cannot capture open-ended, unseen attributes in naturally occurring text. In this paper, we propose a framework, TEAR, to address these challenges. It comprises two synergistic workflows: a Table Extraction Workflow that dynamically adapts instructions to overcome the limitation of heuristic instructions, and an Attribute Recommendation Workflow that discovers new attributes from texts to complement the heuristic schema. To our knowledge, TEAR is the first framework that supports automated text-driven attribute recommendation, enabling exploratory schema design for table extraction. To evaluate TEAR, we establish the benchmark for table extraction and attribute recommendation on naturally occurring texts, including two real-world datasets, manual annotations, appropriate metrics, and baseline comparisons. Experiments show that TEAR achieves state-of-the-art performance on both tasks, and the recommended attributes effectively enhance extraction performance in exploratory scenarios.

1 Introduction

Naturally occurring texts lack stable structures and schemas, challenging extraction boundaries and attribute coverage. TEAR combines adaptive demonstrations with text-driven attribute recommendation, and outperforms state-of-the-art baselines on both tasks.

1 Introduction

  • Naturally occurring texts are unscripted, fluid narratives without stable templates, unlike table descriptions and specialized documents.Prior work focused on descriptions generated around existing tables or specialized documents with conventions that provide structural cues.Source: §1 · pp. 1–2
  • High variability makes fixed LLM instructions insufficient to specify all extraction boundaries, while exhaustive instructions risk diluting model attention.“Victim status” could include direct casualties or people transported, requiring task specification.Source: §1 · p. 2
  • Heuristic schemas may miss relevant attributes because naturally occurring texts lack stable informational templates and users face an exploratory schema-design problem.Users may begin with obvious attributes but still seek attributes that emerge from the texts.Source: §1 · pp. 2–3
  • TEAR’s Proactive Demonstration Module retrieves informative examples for each text to adapt instructions to likely table contents and clarify extraction boundaries.It first makes a fuzzy prediction of likely cell contents, then retrieves examples suited to the specific extraction.Source: §1 · p. 3
  • TEAR recommends text-derived attributes by grounding proposals in texts, consolidating differently worded discoveries, and ranking candidates by salience and schema coherence.Its components include a Discovery Mechanism, Hybrid Integration Strategy, and Schema Coherence Score.Source: §1 · p. 3
  • The paper establishes an evaluation methodology for both tasks, including extraction metrics that do not rely on record indices and a recommendation protocol.The benchmark uses two datasets and 3,375 manually annotated ground-truth tables.Source: §1 · p. 3
  • TEAR consistently outperforms state-of-the-art baselines on both table extraction and attribute recommendation.The evaluation includes two naturally occurring-text datasets with 3,375 manually annotated ground-truth tables.Source: §1 · p. 3

2 Related Works

Prior table-extraction work spans supervised generation and LLM in-context learning, while related extraction tasks differ in their targets and input assumptions.

2.1 Table Extraction from Texts

  • Supervised methods generate tables from paired text-table data, while LLM in-context learning uses instructions without updating model parameters.Supervised methods fit controlled table descriptions but can struggle with naturally occurring texts; ICL methods often rely on structural conventions absent from those texts.Source: §2.1 · pp. 3–4

2.2 Open Information Extraction

  • Open information extraction includes novel slot detection, schema induction, event-schema learning, and ontology learning, but these differ from discovering named attributes for known tuples.Novel slot detection identifies new types without revealing their semantic identity through canonical naming.Source: §2.2 · p. 4

2.3 Other Table Extraction Tasks

  • Format mining, information integration, and on-demand extraction target different tasks from extracting atomic values in free-form texts into schema-based tables.On-demand methods answer individual queries; pre-extracted tables instead support broad exact SQL-style operations.Source: §2.3 · p. 4

3 Problem Definition

The paper formalizes extraction from naturally occurring text as producing schema-aligned tables from labeled and unlabeled texts, and attribute recommendation as proposing relevant new schema attributes.

3 Problem Definition

  • Extracted values are lists of non-overlapping text spans, allowing mentions that are non-consecutive; the basic setting retains raw strings.Raw strings may be normalized in post-processing for specialized uses.Source: §3 · p. 4
  • A schema is a set of attributes, and its names use ordinary words or common acronyms understandable to people and general LLMs.Meaningless or obscure symbols such as “Column_1” or “REV” do not qualify as attribute names.Source: §3 · p. 4
  • A table is a rectangular structure with unique attribute names as headers and records containing corresponding values.Empty values can indicate attributes not mentioned in the text, but valid tables contain no entirely empty row or column.Source: §3 · p. 4
  • Table extraction uses labeled samples and unlabeled texts to produce schema-following tables, aiming to maximize table similarity to the ground truth.The task targets stable, task-specific predictions rather than merely subjectively reasonable outputs.Source: §3 · p. 4
  • Attribute recommendation takes an inadequate schema, labeled samples, unlabeled texts, and a budget k to return k new attributes maximizing recall against relevant attributes.The framework performs this additional task under exploratory scenarios with a specified recommendation budget.Source: §3 · pp. 4–5

4 The TEAR framework

TEAR combines adaptive table extraction with ranked attribute recommendation for naturally occurring texts. Its extraction workflow uses preview-guided demonstrations, while its recommendation workflow proposes, deduplicates, and ranks schema extensions.

4.2 Table Extraction Workflow

  • A lightweight dataset-specific surrogate forecasts likely headers and value spans, making preview generation easier and more noise-tolerant than producing a complete table.The forecasts guide demonstration selection rather than directly supplying the final extraction.Source: §4.2 · p. 5

4.2.2 Extraction demonstration.

  • The Proactive Demonstration Module retrieves labeled examples by header, value, and semantic utility, then selects top-scoring samples across the utility types.The number retrieved per utility type is a tunable hyperparameter.Source: §4.2.2 · p. 5, §4.2.3 · p. 6
  • Header utility favors examples containing previewed headers and breaks ties toward samples with additional headers; value utility compares previewed spans with labeled-table spans using chrFβ-based F1.When no value span is previewed, the original input text substitutes for the value preview.Source: §4.2.2 · p. 5

4.2.3 Extraction prompting.

  • The Extraction Instruction wraps the text, schema, previews, and retrieved demonstrations into one prompt for the LLM to produce the table.Figure 2 depicts the template, with a complete instantiation in Appendix A.1.Source: §4.2.3 · p. 6, Figure 2 · p. 6

4.3 Attribute Recommendation Workflow

  • For exploratory use, the Attribute Recommendation Workflow returns a prioritized list of new text-driven attributes rather than a conclusive attribute set.Its components discover proposals, resolve duplicates, and rank candidates by schema relevance.Source: §4.3 · p. 6

4.3.1 Discovery Mechanism.

  • The Discovery Mechanism prompts the LLM to expand an extracted table with new columns, using analogies to existing columns to ground proposed attributes in context.Demonstration examples pair a text and known table with a table containing new columns.Source: §4.3.1 · pp. 6–7

4.3.2 Hybrid Integration Strategy.

  • The Hybrid Integration Strategy first uses diversity checks to find plausible duplicate groups, then asks the LLM to assess them in context and prunes confirmed duplicates.Its breadth-first scheduling updates candidate groups to avoid unnecessary nested-set analysis; the stated call bound decreases as duplicates increase.Source: §4.3.2 · p. 7

4.3.3 Schema Coherence Score.

  • The Schema Coherence Score estimates how naturally a candidate attribute completes the known schema and ranks deduplicated proposals for presentation to the user.The score is computed with an LLM forward pass rather than language generation.Source: §4.3.3 · pp. 7–8

5 Evaluation

The evaluation introduces naturally occurring text datasets and metrics suited to extraction without record indices. It also defines exploratory schema settings for assessing attribute recommendation and resulting table extraction.

5.1 Datasets and Splits

  • The benchmark adds Incidents and Weather, two real-world news datasets that together provide 3,375 manually completed text–table pairs with realistic linguistic variability.Incidents covers gun-violence reports; Weather covers forecasts for multiple countries.Source: §5.1 · p. 8
  • Conversation adapts MultiWoz2.4 dialogues as a controlled stress test for verbosity and noise, with final agreed-upon attribute values as labels.Its service-oriented closed ontology makes it less variable than the new news datasets, and value drifts introduce semantic noise.Source: §5.1 · p. 8
  • All methods use 300 labeled samples; exploratory recommendation settings hide low-frequency attributes so the heuristic schema omits 20%, 35%, or 50% of attributes.The system is not told the exploratory level, and extraction is also evaluated under these settings.Source: §5.1 · p. 8

5.2.1 Table extraction metrics.

  • Structured-F1 compares predicted and reference tables as sets of self-contained header-to-value records, matching records without imposing logical indices.The metric aligns headers and aggregates corresponding value similarities; header-F1 separately measures attribute identification.Source: §5.2.1 · pp. 8–9
  • Attribute recommendation is evaluated with recall-AUC, which rewards recommendation lists that reach higher recall earlier and uses chrF or BERTScore for open-ended matches.Shorter recall curves are padded with their final value for normalization.Source: §5.2.2 · p. 9

6 Experiments

Experiments compare TEAR with supervised and in-context extraction baselines and attribute-recommendation strategies. TEAR leads extraction, and it is best in 48 of 54 attribute-recommendation comparisons.

6.1 Setup; 6.1.1 Baselines.

  • The evaluation compares supervised and in-context extraction baselines, three attribute-recommendation strategies, and three instruction-tuned open-source LLM backbones.The backbones are Llama-3-8B-Instruct, Qwen2.5-14B-Instruct, and Llama-3-70B-Instruct.Source: §6.1.1 · pp. 9–10, §6.1.2 · p. 10

6.2 Q1: Table Extraction Results; 6.3 Q2: Attribute Recommendation Results

  • TEAR is consistently the best table-extraction method, while attribute recommendation is best in 48 of 54 comparisons.The extraction comparison includes both supervised and in-context methods; the recommendation baselines have no clear runner-up.Source: §6.2 · p. 10, Table 2 · p. 10, §6.3 · p. 11

6.2 Q1: Table Extraction Results

  • In-context extraction methods outperform the supervised method, especially on structured-F1 for values; larger LLMs are generally better, and TEAR particularly boosts weaker models.RAG also beats fixed-demonstration MapMake, supporting adaptive example selection; the new benchmarks are more challenging than Conversation.Source: §6.2 · pp. 10–11

6.3 Q2: Attribute Recommendation Results

  • Attribute-recommendation trends across exploratory levels differ by dataset: Weather is easier with a more complete initial schema, while Incidents becomes harder as rare attributes remain difficult to discover.The Incidents example is “Number of rounds fired.”Source: §6.3 · p. 11

6.4 Q3: Table Extraction with Text-Driven Attributes

  • With Qwen, TEAR*—the pipeline after attribute recommendations update the schema—adds extraction gains; on Incidents it stays stable or improves as exploratory levels increase.The simulation adopts recommendations matching ground-truth attribute names exactly, a conservative choice; Llama shows similar trends.Source: §6.4 · p. 11

6.5 Q4: Efficiency

  • For table extraction, the efficiency order is TRE>RAG>Ours>MapMake; for attribute recommendation it is CFR>MSS>DIRECT>Ours, while TEAR uses fewer labels or resources than fine-tuning or extensive annotation.The comparison measures local-server LLM runtime; LLM overhead primarily comes from the backbone.Source: §6.5 · p. 11

6.6.1 Labeled Pool Size, Split and Intialization.; 6.6.2 Proactive Demonstration Module.

  • Labeled-pool growth gradually saturates on Incidents but continues improving Weather; alternative pool initializations give similar results, and splitting learning from retrieval underperforms the shared-pool default.For retrieved demonstrations, one shot already boosts performance and a few shots reach a high level; combining three utilities performs best.Source: §6.6.1 · p. 11, Figure 7 · p. 12, Figure 8 · p. 12

6.6.3 Hybrid Integration Strategy.; 6.6.4 Semantic Coherence Score.; 6.6.5 Manual Evaluation on Structured-F1.

  • The Hybrid Integration Strategy removes 11% of Qwen proposals with comparable recall-AUC; Schema Coherence Score improves early top-k recall, and structured-F1 changes align with human preferences at 90%, 95%, and 92.5% accuracy.The respective human-evaluation accuracies are for sim.=EM, chrf, and BS; coherence gains are especially evident on Weather at Levels 1 and 2. Coherence scoring samples 10 schema-prefix permutations to avoid underperforming orders.Source: §6.6.3 · p. 13, §6.6.4 · p. 13, §6.6.5 · p. 13

7 Conclusion

TEAR combines adaptive table extraction with text-driven attribute recommendation and establishes benchmarks for both tasks on naturally occurring texts. Experiments with three open-source LLMs show state-of-the-art performance, while recommended attributes improve extraction in exploratory scenarios.

7 Conclusion

  • TEAR combines per-input demonstration adaptation with workflows that discover, consolidate, and rank new attributes for table extraction.The framework targets naturally occurring texts and exploratory schema design.Source: §7 · p. 13
  • Experiments with three open-source LLMs show state-of-the-art performance on both table extraction and attribute recommendation.The evaluation benchmarks cover naturally occurring texts.Source: §7 · p. 13
  • Recommended attributes effectively enhance table extraction in exploratory scenarios.Source: §7 · p. 13

8 Discussion

The discussion limits attribute recommendation to single-round quality and describes the framework’s contribution as task-specific data-driven components rather than scripted prompts alone. It also identifies figures on ranking-method recall and LLM–human agreement.

8 Discussion

  • The study focuses on single-round attribute recommendation quality; iterative refinement would require cross-round attribute relationships and a more complex benchmark.The authors note that open-ended candidates provide no inherent convergence point for continuous prompting.Source: §8 · p. 13
  • Figure 11 presents recommendation recall for different ranking methods with Qwen.Source: Figure 11 · p. 14
  • Figure 12 presents agreements between LLMs and humans.Source: Figure 12 · p. 14
  • The authors locate TEAR’s contribution in closing the gap between LLM general capability and task-specific competence through data-driven components and the framework.They state that stronger LLMs or better prompts alone cannot close that gap.Source: §8 · p. 14

A.1 LLM instructions

The appendix gives instructions for attribute discovery, integration, and extraction, including criteria for qualifying new attributes and dataset-specific output requirements. It also specifies which attributes are hidden during recommendation validation and testing.

A.1 LLM instructions

  • For attribute recommendation, validation and test attributes are excluded from the schema and hidden from the LLM.Source: §A.1 · p. 16
  • Extraction instructions request schema-formatted values as exact text substrings and omit attributes that are absent or cannot be determined.They also provide hints about likely attributes and values in the input text.Source: §A.1 · p. 16
  • The dataset instructions cover incident entities, weather conditions, and booking-dialogue domains, with conversation extraction limited to the final agreed-upon state after each turn.Conversation examples include Attraction, Hotel, Restaurant, Taxi, and Train; the instructions specify at most five entities.Source: §A.1 · pp. 16–17
  • The Discovery Instruction requests entity-organized JSON whose attribute values are lists of exact input-text substrings.It encourages discovery for each entity, including multiple attributes where applicable.Source: §A.1 · pp. 16–17
  • New attributes must be relevant, unique, non-overlapping, and at a suitable granularity, with a meaningful name and actual text values.The instruction asks for as many qualified attributes as possible.Source: §A.1 · p. 17
  • The Hybrid Integration Strategy merges concepts judged equivalent, semantically contained, or finer-grained than existing attributes; otherwise, it keeps them distinct.The judge instruction returns “One” for merging and “Two” for distinct attributes.Source: §A.1 · p. 17

A.2 Vendi Score

The appendix describes Vendi Score as the effective rank of a normalized similarity matrix. Its extremes correspond to fully dissimilar elements or an all-ones similarity matrix, and the experiment uses symmetric chrFβ pattern similarity.

A.2 Vendi Score

  • Vendi Score is the effective rank of the normalized similarity matrix.Source: §A.2 · p. 17
  • The score reaches |q| when all off-diagonal similarities are zero and falls to 1 when the similarity matrix contains all ones.Source: §A.2 · p. 17
  • The experiment defines pairwise similarity as the average of chrFβ(a,b) and chrFβ(b,a) to reflect pattern similarity.The matrix is constructed for candidate set q from a precomputed matrix over B.Source: §A.2 · p. 17

A.3 Supplementary Discussion for Hybrid Integration Strategy

The supplementary analysis describes a bottom-up search that prunes candidate attribute sets using monotonicity and bounds its search and LLM-call costs. It also notes that duplicate resolution remains uncertain because semantic equivalence is subjective and context-dependent.

A.3 Supplementary Discussion for Hybrid Integration Strategy

  • Monotonicity lets the search build low-diversity sets by size and prune extensions when their proper subsets are not qualified.For each extension, the algorithm first checks whether replacing an element yields a known qualified set, then computes its Vendi Score.Source: §A.3.1 · p. 17
  • The number of examined sets scales as O(N|P*|), linear in the output size |P*| and candidate count N.Each extension incurs O(i) dictionary lookups, with only a small fraction proceeding to a Vendi Score computation.Source: §A.3.2 · p. 18
  • Pruning makes the complexity factor smaller than the candidate count in practice, while the largest low-diversity set remains modest.The analysis attributes this to most candidate attributes being semantically distinct and not forming duplicate pairs.Source: §A.3.2 · p. 18
  • Greedy set cover bounds contextualized-analysis calls by (ln |B| + 1)(|B| − N′), though the worst case still incurs |P*| calls without deduplication.Larger duplicate sets can help remove confirmed duplicates with fewer queries and shrink later search space.Source: §A.3.2 · p. 18
  • Duplicate resolution has no ground truth because semantic equivalence is subjective and context-dependent, so inferred merges may conflict with user preferences.For example, a user may want “Medical center name” and “Hospital name” kept separate.Source: §A.3.4 · p. 18

A.3.4 Discussion of Hybrid Integration Strategy.

The appendix discusses pragmatic duplicate consolidation, standardized definitions for attribute proposals, annotation procedures, dataset variability, and evaluation cost. It also identifies the Llama-3-8B results and the datasets and schemas covered by its tables and figures.

A.3.4 Discussion of Hybrid Integration Strategy.

  • The Hybrid Integration Strategy takes a pragmatic rather than exhaustive approach, consolidating obvious duplicates without seeking a perfect solution.The paper grounds this choice in the subjectivity of semantic equivalence and reports that pruning conspicuous redundancies yields substantial gains.Source: §A.3.4 · p. 18

A.4 Generating Attribute Proposal Definitions

  • Proposal contexts are normalized into concise definitions to reduce variation in length and style that could bias LLM reasoning.The definitions are intended to capture an attribute’s meaning and characteristics for reliable extraction from other texts.Source: §A.4 · pp. 18–19

A.5 Discussion of existing datasets

  • Table 5 compares datasets used by prior text-extraction methods with this work, while Table 6 presents the dataset schema.The supplied passages identify the tables’ scope but do not provide their contents.Source: §A.5 · p. 19, Table 5 · p. 20, Table 6 · p. 20

A.6 Data Annotation

  • Three annotators agreed on schemas, independently annotated 100 texts per dataset, and reached 88% pairwise agreement for A–B and 82% for A–C.Disagreements are grouped into entity-, attribute-, and value-level types; Figures 13 and 14 show distributions for Weather and Incidents.Source: §A.6 · p. 19, Figure 13 · p. 19, Figure 14 · p. 19

A.7 Variability Comparison

  • Weather is the hardest dataset, followed by Incidents and Conversation; the new benchmarks also have more rare attributes than Conversation.The variability comparison uses unique value ratio and value ambiguity; the ranked coverage curve shows the stronger long-tail pattern in Incidents and Weather.Source: §A.7 · pp. 20–21, Figure 16 · p. 21

A.8 Discussion on Structured-F1

  • Structured-F1 is no more complex than triple F1 in practice, and is more efficient when n^2 > 1 and m > 3.When the table sizes are similar, the stated complexities are C1 = O(m^2c + 2n^2mc) and C2 = O(m^2n^2c).Source: §A.8 · p. 21

A.9 Other Results

  • Figures 17 and 18 report Llama-3-8B results for attribute recommendation recall and table extraction with text-driven attributes.The supplied passages do not state the figures’ outcome values.Source: §A.9 · p. 21, Figure 17 · p. 22, Figure 18 · p. 22

Related papers

Loading 2609.15205v1…