ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models

1 Introduction

• Diverse domains and schemata make triples ambiguous, while end-to-end neural methods typically need massive training sets that data-scarce tasks may lack.For example, “time” denotes an album’s release time in one domain and a world record in another.Source: §1 · p. 1
• ASDOT is designed to use any available examples, including none, and to outperform prior methods tailored to specific scenarios.The approach is motivated by practical settings ranging from few or no examples to examples from another domain.Source: §1 · pp. 1–2
• Prompted GPT-3 converts triples into less ambiguous sentences, which a pretrained LM such as T5 fuses into a coherent paragraph.The fusion stage can be finetuned using available in- or out-of-domain examples and weakly supervised data.Source: §1 · p. 2
• ASDOT consistently achieves significant gains over methods designed for zero-, few-, and full-shot, unseen-predicate, and out-of-domain settings.The introduction reports evaluation across these scenarios but supplies no metric values here.Source: §1 · p. 2

2 Related Work

• Many fully supervised neural methods require large amounts of in-domain training data, limiting their applicability to low-data scenarios.Earlier approaches include encoder-decoder models and later additions such as content selection, entity modeling, and style imitation.Source: §2 · p. 2
• Prior zero- and few-shot methods often target specific settings, whereas this work studies varying example counts and examples from other domains.The discussed prior work includes approaches based on pretrained models, content matching, prototype memory, and template rewriting.Source: §2 · p. 2
• Kasner and Dusek (2022) rely on human-written templates and a multi-component fusion approach, and study only zero-shot generation.The passage contrasts this scope with ASDOT’s comprehensive study of diverse any-shot problems.Source: §2 · p. 2

3 Any-Shot Data-to-Text Generation

• ASDOT’s two core steps are data disambiguation and sentence fusion, with a prompted GPT-3 model followed by a pretrained language model.The fusion model may be finetuned using public weakly supervised data or available examples.Source: §3 · pp. 2–3, Figure 1 · p. 3

3.1 The Any-Shot Data-to-Text Problems

• The any-shot problem covers varying quantities of labeled examples, including none, as well as examples from other domains, schemata, or predicates.These settings arise amid diverse data domains and structures that can leave insufficient examples for learning to understand and describe target data.Source: §3.1 · p. 3

3.2 Method Overview

• ASDOT separates triple understanding from paragraph generation, using GPT-3’s implicit knowledge to disambiguate triples before a pretrained LM fuses their sentences.The fusion stage can use available training examples to improve performance.Source: §3.2 · p. 3

3.3 Data Disambiguation

• GPT-3 uses demonstrations to turn ambiguous triples into sentences; predicate-specific templates reuse each generated sentence across triples sharing that predicate.The authors report that the same four demonstrations work across domains; unlike approaches with human-crafted templates, ASDOT automates the process without training examples.Source: §3.3 · p. 4

3.4 Sentence Fusion

• Sentence fusion concatenates the short sentences with a “summarize:” prefix and feeds them to pretrained T5 as a sequence-to-sequence task.The stage can be finetuned on public weak supervision or labeled data-to-text examples, including examples from other domains.Source: §3.4 · p. 4

4.1 Datasets

• WebNLG, E2E, and DART cover distinct data-to-text settings; DART has 4,299 predicates, while E2E has seven and is described as relatively easy.WebNLG includes a test-unseen split for experiments on unseen predicates.Source: §4.1 · pp. 4–5

4.2 Experimental Setup

• The setup uses GPT-3 Davinci to create predicate templates and T5 for sentence fusion, with WikiFluent finetuning in zero-/few-shot settings.When labeled examples are available, T5 is further finetuned on them; evaluation reports BLEU, METEOR, and PARENT-F1, with additional metrics and human evaluation also described.Source: §4.2 · p. 5

4.3 Zero-, Few-, to Full-Shot Learning

• 16.06 BLEU higher than T5-large on 10-shot WebNLG and 10.53 higher on 10-shot DART; ASDOT also consistently outperforms baselines across both datasets.The reported advantage is larger with fewer training examples; Figure 2 plots BLEU against the number of training examples.Source: Figure 2 · p. 5, Model BLEU METEOR P-F1 · p. 6

Model BLEU METEOR P-F1

• ASDOT remains consistently stronger than diverse baselines in full-shot settings, although its gains tend to be less significant than in zero-/few-shot settings.The authors attribute the smaller gap to all methods having many training examples.Source: Table 1 · p. 6, Model BLEU METEOR P-F1 · p. 7
• In the 50-shot WebNLG human evaluation, ASDOT is substantially better than baselines on faithfulness, contradiction, and fluency.The authors report p < 0.05 in Tukey’s HSD test for all measures.Source: Model BLEU METEOR P-F1 · p. 7, Table 2 · p. 7
• Removing weakly supervised finetuning lowers zero-shot WebNLG BLEU by 8.86 points, yet ASDOT without it still beats baselines in most cases.Replacing GPT-3-generated templates with human-written ones produces similar or slightly lower performance.Source: Model BLEU METEOR P-F1 · p. 7, Table 3 · p. 7

4.4 Generating for Unseen Predicates

• ASDOT consistently improves over all baselines on unseen-predicate tests, with larger gains on the more difficult DART set than on WebNLG.The comparison includes end-to-end T5 models; the DART test-unseen results are reported in Table 5.Source: §4.4 · p. 8, Table 5 · p. 8

4.5 Learning with Out-of-Domain Examples

• ASDOT outperforms the end-to-end T5-large baseline on both out-of-domain test sets.The evaluation trains on WebNLG and tests on DART and E2E after removing in-domain test examples and overlap with E2E evaluation.Source: §4.5 · p. 8, Table 6 · p. 8

4.6 Case Study

• Case studies show ASDOT disambiguating a Zolder fastest-lap triple that T5 misrenders and verbalizing “associated Band/associated Musical Artist” as “is associated with”.Table 7 presents examples from the out-of-domain and unseen-predicates settings.Source: §4.6 · pp. 8–9, Table 7 · p. 8

• ASDOT uses prompted GPT-3 to turn input triples into short sentences, then pretrained language models to fuse them into paragraphs.The approach covers zero-, few-, and full-shot settings, unseen predicates, and out-of-domain examples.Source: §5 · p. 9
• The two-stage design integrates pretrained language models’ implicit knowledge to support broad applicability across any-shot scenarios.The stated scope includes zero-/few-/full-shot, unseen-predicate, and out-of-domain settings.Source: §5 · p. 9
• Extensive experiments show significant improvements over diverse baselines.Source: §5 · p. 9

• GPT-3 disambiguates each triple locally, without access to full-table information that may be needed in difficult cases.Source: Limitations · p. 9
• Using GPT-3’s output directly as the final disambiguation can yield incorrect templates, especially in highly specialized domains.Source: Limitations · p. 9
• The current approach applies only to languages with access to large language models.Source: Limitations · p. 9

• The authors state that they follow the ACL and ACM codes of ethics.Source: Ethics Statement · p. 9
• The work introduces an algorithm rather than a new dataset, and its intended use may benefit people who have difficulty reading graphs or tables.The authors do not anticipate direct harm from the intended usage.Source: Ethics Statement · p. 9
• Harmful or adversarial inputs may lead to unintended outputs, and the authors advise against use outside controlled research until risks are mitigated.They also warn that naive deployment may enable malicious exploitation of backbone language models and say filtering precautions are needed.Source: Ethics Statement · p. 9
• Large-language-model commonsense reasoning may reinforce social stereotypes, warranting care with materials concerning marginalized populations.Source: Ethics Statement · p. 9
• The finetuning procedure takes around 1836 GPU-hours on NVIDIA GeForce RTX 3090 Ti GPUs, and prompting makes about 4600 GPT-3 API calls.Source: Ethics Statement · p. 9

A GPT-3 Prompt

• The Michael example pairs “birth Place | USA” with “Michael was born in the USA.”Source: A GPT-3 Prompt · p. 13
• The First Clearing example pairs its location field with “First Clearing is located at On NYS 52 1 Mi.”Source: A GPT-3 Prompt · p. 13
• The Abilene Regional Airport example pairs a city-served field with “Abilene Regional Airport serves Abilene.”Source: A GPT-3 Prompt · p. 13
• The Alfred Moore Scales example pairs an active-years field with “Alfred Moore Scales started to be active on.”Source: A GPT-3 Prompt · p. 13

B Experimental Details

• Zero- and few-shot settings use batch size 5 and beam-search size 5.Source: B Experimental Details · p. 13
• For other settings, model selection uses validation-set performance, choosing batch size from {2, 4, 8} and a second, unspecified setting from {1, 3, 5}, with sacreBLEU (Post, 2018).Source: B Experimental Details · p. 13
• The metrics and corpora used in the experiments are listed by URL in Tables 8 and 9, respectively.Source: B Experimental Details · p. 13

C Zero-/Few-shot Experimental Results

• BLEU, METEOR, and PARENT-F1 scores for zero- and few-shot WebNLG and DART experiments are presented in Tables 10 and 12.Source: C Zero-/Few-shot Experimental Results · p. 13, Table 10 · p. 14, Table 12 · p. 14
• BERTScore and BLEURT results for zero- and few-shot WebNLG and DART experiments are presented in Tables 11 and 13.Source: C Zero-/Few-shot Experimental Results · p. 13, Table 11 · p. 14, Table 13 · p. 14
• Tables 8 and 9 list the URLs for the experiment metrics and corpora, respectively.Source: Table 8 · p. 13, Table 9 · p. 13