DART: Open-Domain Structured Data Record to Text Generation

• Existing datasets use flat data structures that cannot encode rich semantic relationships, cover limited domains or predicates, and sometimes have loose input-sentence alignments.Source: §1 · p. 1
• DART is a large, open-domain corpus designed to harvest diverse predicates from Wikipedia tables, beyond those in domain-specific E2E and WebNLG ontologies.Source: §1 · p. 1
• Its tree ontology converts flat table schemas into semantic frames reflecting core and auxiliary relations, with sentence annotations from WikiSQL, WikiTableQuestions, E2E, and WebNLG.Source: §1 · p. 1
• State-of-the-art models that perform impressively on domain-specific datasets suffer on DART, revealing challenges from its open-domain scope and richer semantic structures.Source: §1 · p. 2
• Using DART for data augmentation improves existing models on WebNLG 2017, with the authors expecting results to generalize to other data-to-text datasets.Source: §1 · p. 2

2.1 Tree Ontology and Sentence Annotation on Tables

• Annotators build a tree ontology by assigning parent columns and placing the title according to its role, then review ontologies for structural errors and unify similar tables.For malformed tables, they may rename or add column names; the ontology root is [TABLECONTEXT].Source: §2.1 · p. 2
• Connected ontology components yield coherent cell selections, which annotators describe and reviewers correct before the selected cells are converted into triples.Component size is sampled from 2 to 5, with expansion parameter p sampled from 0.5 to 0.7 to vary shape; triple extraction connects highlighted nodes through their lowest common ancestor.Source: §2.1 · pp. 3–4

2.2 Automatically Converting Questions to Declarative Sentences

• Rule-based QA2D converts WikiSQL questions and SQL-retrieved answers into declarative sentences, with manual screening and cell alignment yielding 4,204 sentences.Only queries without aggregate commands are executed, restricting answers to those obtainable from single rows.Source: §2.2 · pp. 4–5

2.3 Incorporating Existing Datasets

• DART incorporates WebNLG 2017 triple sets and Cleaned E2E meaning representations, converting E2E slots into triples with NAME as subject and dropping records without NAME.WebNLG contributes 27,731 pairs, with up to 7 RDF triples per set across 15 domains.Source: §2.3 · p. 5

2.4 Predicate Unification

• A manually constructed mapping unifies predicates with the same meaning, such as “Hometown,” “Home Town,” and “Home Town/City” as “HOMETOWN.”The authors then assess diversity by counting unique predicates in DART’s partitions.Source: §2.4 · p. 5
• Wikipedia contributes more unique predicates than WebNLG and Cleaned E2E combined despite fewer pairs, and DART also shows topical variety in unique triples and vocabulary.The authors attribute this predicate diversity to DART’s domain diversity.Source: Table 2 · p. 3, §2.4 · p. 5

2.5 Dataset Split

• To make evaluation more challenging, DART groups similar tables into test and development sets using title-and-header Jaccard similarity above 0.5; the resulting train/dev/test sizes are 62,659/6,980/12,552 sentences.WebNLG 2017 and Cleaned E2E retain their original splits.Source: §2.5 · p. 5

3.1 Models

• The experiments compare a bidirectional LSTM, Transformer, BART-base and -large, and T5-small, -base, and -large on DART.BART and T5 use implementations and hyperparameters from Ribeiro et al. (2020); T5 receives the prefix “translate Graph to English:”.Source: §3.1 · p. 6

3.2 Evaluation Metrics

• Evaluation combines BLEU, METEOR, and TER with semantic metrics MoverScore, BERTScore, BLEURT, and PARENT.PARENT explicitly aligns reference and generated-text n-grams to the data contents.Source: §3.2 · p. 6

3.3 Results

• 50.66 BLEU gives T5-large the highest performance among tested models on DART, where pretrained models generally outperform others and larger models perform better.The authors attribute T5’s result to its multi-task pretraining, but note that pretrained models can hallucinate and do not fully capture the triple sets’ hierarchical ontology.Source: §3.3 · p. 6, Table 3 · p. 7
• T5-large fine-tuned on DART achieves new state-of-the-art results on all WebNLG 2017 test splits, and DART augmentation improves every tested model, especially on unseen data.The comparison includes the baseline Transformer and pretrained BART and T5 variants; the authors attribute the larger unseen-split gains to DART’s open-domain nature.Source: §3.3 · p. 6, Table 4 · p. 7

3.4 Ablation Study

• Human-written DART sentences bring the most WebNLG improvement, especially on unseen data, while adding E2E instances improves seen-split scores but hurts unseen-split performance.This trend holds across models; human-written sentences generally contribute more than auto-generated declarative sentences.Source: §3.4 · pp. 6–7

4 Human Evaluation

• The evaluation rated fluency and semantic faithfulness at three levels, using five annotators and 200 triplesets sampled across two DART partitions.The criteria were naturalness and grammaticality for fluency, and support from the input triples for faithfulness; each tripleset was paired with its reference and BART-base and T5-base outputs.Source: §4 · pp. 7–8
• Human-written references achieved the highest fluency and faithfulness, while the faithfulness gap from BART-base and T5-base outputs indicates room for improvement.Five annotators assessed 100 triplesets from each of two DART partitions, comparing each tripleset’s reference with outputs from both models.Source: §4 · pp. 7–8
• Rule-generated declarative sentences were less fluent and faithful than model outputs, but the partition was retained for economical data collection and topic diversity that may benefit generalization.The authors describe the declarative partition as an economical way to obtain large numbers of instances.Source: §4 · p. 8

Data-to-Text

• Most existing data-to-text datasets target specific domains or applications, whereas a major source of DART is Wikipedia tables spanning varied topics.Prior applications include sports commentaries (Wiseman et al., 2017), weather forecasts, biographies (Lebret et al., 2016; Liu et al., 2018), knowledge-base descriptions, dialogue responses (Wen et al., 2015, 2016), and commonsense reasoning (Lin et al., 2020).Source: §5 · p. 8

Representation of Data

• Data-to-text inputs include slot-value pairs, AMR, MRS, RDF triples, logic forms, or rows and entire tables in open-domain datasets such as WikiTableText (Bao et al., 2018), LogicNLP (Chen et al., 2020a), and ToTTo (Parikh et al., 2020).ToTTo found that highlighted cells with flat row and column headers outperformed using the entire table.Source: §5 · p. 8
• DART’s tree-structured table ontology encodes semantic dependencies among headers and can incorporate contexts such as titles, enabling logically consistent triples without information loss.An automatic procedure extracts connected components from the ontology tree as instance inputs.Source: §5 · p. 8

Model

• Traditional data-to-text systems use staged generation, while neural encoder-decoder models and pretrained models have shown promise, including fine-tuning on data-to-text tasks.Related work includes attention and copy mechanisms (Gehrmann et al., 2018; Puduppully et al., 2018, 2019; Castro Ferreira et al., 2019), pretrained GPT-2, BART, and T5 (Raffel et al., 2020), and fine-tuning by Chen et al. (2020c); Peng et al. (2020); Kale (2020).Source: §5 · p. 8

6 Conclusion

• DART introduces an open-domain structured data-to-text corpus with an ontology-preserving input representation that differentiates it from other open-domain corpora.Its inputs are semantic triples whose ontology structure contributes to the challenges reported for existing models.Source: §6 · p. 8
• Using DART for data augmentation improves performance on other data-to-text tasks.The authors also report that DART’s open-domain nature and semantic-triple ontology structure pose new challenges to several state-of-the-art data-to-text models.Source: §6 · p. 8
• Future work will explore controlled, high-fidelity generation that better incorporates the data ontology hierarchy.Source: §6 · p. 8

• DART combines public Wikipedia-table, E2E, and WebNLG resources with new table-structure and human-description annotations, but source data and descriptions may carry biases.Source: §7 · p. 9
• Authors and MTurk workers completed table-ontology annotation, faithful description writing, and sentence evaluation; MTurk pay was about $15 per hour across 125 hours, with possible annotator and record bias.Source: §7 · p. 9
• BART and T5 experiments may reproduce biases because the pretrained models learned from large news, books, and web-text corpora.Source: §7 · p. 9

Appendix

• The Appendix includes a WebNLG 2017 test-set ablation study comparing DART partitions, including ontology, E2E, and full-data combinations.The table caption defines dart_full as the combination of dart_ontology and dart_e2e; no ablation scores are provided in the supplied passage.Source: Table 6 · p. 12, Appendix · p. 12
• Ontology statistics cover WikiTableQuestions and WikiSQL samples, with a figure showing column-ontology depth distributions.The table caption defines branching factor as the average number of children across non-leaf ontology nodes.Source: Appendix · p. 12, Table 7 · p. 13, Figure 3 · p. 13
• A cleaning example shows annotators adding a missing column name, linking columns to it, and disambiguating repeated column names.The example also shows a table title that was not specific to the data.Source: Appendix · p. 12, Figure 5 · p. 14
• Manual WikiTableQuestions examples show table ontologies and sentences capturing row information, with title references highlighted or represented using [TITLE].Sentences referencing the table title are highlighted green.Source: Appendix · p. 12, Figure 6 · p. 14, Figure 7 · p. 14
• Examples of collected and generated sentences show annotators checking coherence and title use, marking sentences y, n, or x.The labels indicate title reference, no title reference, or nonsensical output, respectively.Source: Figure 9 · p. 15, Figure 10 · p. 15
• Examples include BART-base’s claim that 250k people were killed in the 2007 Philippine earthquake despite input triples describing 250k homeless in Peru, and T5-base’s stated activity span ending in 1616.Source: Appendix · p. 15