Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning

1 Introduction

• Earlier methods used form-specific encoders, such as GNNs for knowledge graphs and table-specific encoders, limiting transfer across structured forms.Examples include Koncel-Kedziorski et al. (2019), Ribeiro et al. (2020), Guo et al. (2020), Li et al. (2021), Liu et al. (2017), Bao et al. (2018), Nema et al. (2018), Jain et al. (2018), and Wang et al. (2022).Source: §1 · p. 1
• Although PLMs linearized as text-to-text frameworks achieve state-of-the-art results across tasks, cross-form adaptation and the effects of diverse linearizations remain unresolved.Xie et al. (2022) reported that a multi-task model may underperform a single-task model on tables; KG triple linearizations also differ between Nan et al. (2021) and Xie et al. (2022).Source: §1 · p. 1
• The method converts knowledge graph triples and meaning representations into virtual tables that can be linearized consistently with tables.Source: §1 · p. 1
• A unified representation improves zero-shot BLEU by 66% relatively when transferring from ToTTo to DART, and also improves multi-task performance over varied linearizations.The method is competitive with data-specific linearizations on individual tasks and was evaluated across five representative data-to-text tasks.Source: §1 · p. 2

Data-Type Specific Knowledge Encoding

• Knowledge-graph and meaning-representation encoders, table-specific models, and table-tailored pretrained models cannot directly transfer across structured-data types.Prior approaches include graph neural networks, recurrent and Transformer encoders, table-specific architectures, and pretrained models with specialized table structures or attention.Source: §2 · p. 2

Structured Data Linearization

• Linearizing structured inputs for pretrained text-to-text models is a simple, effective strategy reported to outperform architectures tailored to particular data types.Studies cover table and knowledge-graph inputs, multi-task training across grounding tasks, and pretraining and fine-tuning strategies.Source: §2 · p. 2

Zero/Few-Shot Data-to-Text Generation

• Zero- and few-shot studies explore KG-text joint pretraining, prompt tuning, table content selection, data augmentation, retrieval augmentation, and modular pipelines for domain independence (Chen et al., 2020b; Ke et al., 2021; Keymanesh et al., 2022; Chen et al., 2020d; Chang et al., 2021; Su et al., 2021; Kasner and Dusek, 2022).Joint pretraining and prompt tuning were studied for knowledge-graph generation; other approaches target table generation or few-shot generation more broadly.Source: §2 · p. 2

3 Unified Representation

• The unified representation converts knowledge-graph triples and meaning representations to virtual tables before applying the same linearization used for tables.Figure 1 illustrates the conversion and shared linearization across tables, KG triples, and meaning representations.Source: §3 · pp. 2–3, Figure 1 · p. 3

3.1 Virtual Table

• KG triples and attribute-value meaning representations become virtual tables, enabling a shared table-based input format.For triples, tail nodes become cell values and relations become column headers; each connected component becomes a separate table. For meaning representations, each attribute is a column header and its value is a cell value.Source: §3.1 · p. 3

3.2 Linearization of Tables

• Table inputs use a ToTTo-motivated linearization, with highlighted cells ordered left-to-right and top-to-bottom and whole tables linearized row-wise or column-wise.The basic units are delimited by start and end symbols; indentation in the example is for clarity and is not present in the actual input.Source: §3.2 · pp. 3–4

4.1.1 Zero-Shot Performance

• 39.82 BLEU on WebNLG unseen categories in zero-shot exceeded few-shot results from Ke et al. (2021) (37.18) and Aghajanyan et al. (2022) (18.5); unified transfer also surpassed varied transfer.On DART, the unified model’s zero-shot BLEU was 43% higher than HTLM’s; its zero-shot performance also exceeded the baseline models’ few-shot results.Source: §4.1.1 · p. 5, Table 4 · p. 6

4.1.2 Few-Shot Results

• For ToTTo-to-KG transfer, both source-to-target models beat target-only training, while unified transfer outperformed varied transfer, especially on WebNLG unseen categories.The comparison covers DART and WebNLG; the reported WebNLG advantage on unseen categories reinforces the zero-shot finding.Source: §4.1.2 · p. 5, Figure 2 · p. 6
• For KG-to-table transfer, the unified model’s PARENT advantage over varied transfer remained above 2 points, and it was more sample-efficient than target-only training.With 128 samples, varied and target-only performance converged; with only 8 samples, varied was only slightly better than unified.Source: Figure 2 · p. 6, §4.1.2 · p. 6
• On E2E meaning representations, unified transfer improved CIDEr particularly with fewer than 64 samples, producing more informative text than varied and target-only models.Figure 2 reports few-shot results for sample sizes 8, 16, 32, 64, and 128, averaged across five runs with 95% confidence intervals.Source: §4.1.2 · p. 5, Figure 2 · p. 6
• On LogicNLG, varied transfer performed better with 8 samples, but unified transfer surpassed it at k=128.The target requires table insights rather than ToTTo-style descriptions; the authors suggest that with few examples, unified transfer can favor descriptions, then improves as sample size grows.Source: §4.1.2 · p. 5, Figure 2 · p. 6

4.2 Full-Set Finetuning Results

• Multi-task unified models beat single-task models except for base WebNLG and outperformed varied models; varied-linearization BLEU fell on ToTTo (49.2 to 48.5), DART (49.0 to 48.1), and WebNLG-seen (65.9 to 64.1).These results indicate that multi-task training can hurt when datasets use different input formats, while a unified representation supports cross-task learning without generally reducing performance.Source: Table 5 · p. 7, §4.2 · p. 7
• In single-task full-set training, unified representation performed comparably to other linearizations on DART, WebNLG, and E2E after conversion to virtual tables.The reported similarity suggests that the conversion did not cause significant information loss.Source: Table 5 · p. 7, §4.2 · p. 7

4.3 Qualitative Study

• For a five-triple WebNLG input, at k=8 the varied model covered one fact while unified included many more nodes and relations; at k=128, unified covered all facts accurately.At k=128, the varied model still missed the “funk and disco” origin of pop music.Source: §4.3 · p. 7

5 Conclusion and Future Work

• The method converts tables, knowledge graph triples, and meaning representations into a single representation for data-to-text tasks.The conclusion describes this as the paper’s introduced approach.Source: §5 · p. 8
• The unified representation improves generalization across structured forms, especially in zero-shot or few-shot settings, and multi-task models consistently outperform single-task models.The authors note that the approach is particularly beneficial when data is scarce.Source: §5 · p. 8
• Future work could apply the method to knowledge-base question answering and test whether pre-training on large knowledge graphs improves knowledge transfer to table QA.The proposed transfer direction concerns learned commonsense knowledge, using the unified representation.Source: §5 · p. 8

Limitations

• The unified representation is only one option; alternatives such as Jupyter-notebook code generation (Yin et al., 2022) or hyper-text prompting (Aghajanyan et al., 2022) may perform better.Future work could convert all structured forms to markdown or hyper-texts as alternative unified representations.Source: Limitations · p. 9

• The authors agree with the ACL Ethics Policy.Source: Ethics Statement · p. 9
• Biases or fairness issues in knowledge bases may be reflected in generated text, so the authors urge caution in practice.Source: Ethics Statement · p. 9
• The study explores whether unified representations improve cross-task performance across structured forms and may contribute where data is scarce.Source: Ethics Statement · p. 9

A Data Statistics

• Dataset statistics summarize each dataset’s input type and number of examples, as presented in Table 7.Source: A Data Statistics · p. 13, Table 7 · p. 13

B Column-wise Linearization of (Sub)Table

• The column-wise table example marks the title, subtitle, and year column with structural tags, followed by an ellipsis for other columns.The example begins with <title> Alma Jodorowsky </title> and <sub_title> Filmography </sub_title>.Source: B Column-wise Linearization of… · p. 13

C Other Linearizations Used in Previous Papers

• The unified representation resembles ToTTo linearization, differing in its use of <title> and <sub_title> instead of ToTTo’s <page_title> and <section_title>.Source: C Other Linearizations Used in… · p. 13
• UnifiedSKG (Xie et al., 2022) linearizes the example KG triples as subject-field-value strings separated by vertical bars.The example includes William Wasmund’s field goals and extra points, each with value 0.Source: C Other Linearizations Used in… · p. 13
• LogicNLG (Chen et al., 2020a) represents an entire table as prose describing its title and row-by-row cell values.The example begins “Given the table title of Alma Jodorowsky, Filmograph” and describes the year, title, role, and notes in rows.Source: C Other Linearizations Used in… · p. 13
• The meaning-representation example is linearized by concatenating all meaning representations, including name, eat type, food, price range, and family friendliness.Source: C Other Linearizations Used in… · p. 13

D Implementation Details

• Zero- and few-shot experiments use T5-base for 30 epochs, with source-task learning rate 5e-5 and batch size 32, and target-task learning rate 2e-5 and batch size 8.Source: D Implementation Details · p. 13

• The section reports more detailed multi-task results for each dataset, with summaries in Tables 8–11.Source: E More Multi-Task Results · p. 13

F More Few-shot Results

• Additional few-shot results across more metrics are presented in Figures 3, 4, and 5.The supplied passages do not provide the figures’ results or metric values.Source: F More Few-shot Results · p. 13

G Human Evaluation

• The evaluation compared 8-shot source-to-target and unified source-to-target models on 50 randomly selected unseen-schema WebNLG examples, judged for factuality, triple coverage, and fluency.Three annotators assessed each sample, but two samples received only two annotations because one annotator did not respond.Source: G Human Evaluation · p. 13
• 75% of majority-vote results favored unified; among 21 consensus cases, it received unanimous preference in 15 (71.43%), with Fleiss Kappa around 0.23 (fair agreement).Source: G Human Evaluation · p. 13

H More Qualitative Study

• Additional few-shot predictions for ToTTo-to-WebNLG and ToTTo-to-LogicNLG transfer appear in case-study Tables 12 and 13, with error analysis under each example.Source: H More Qualitative Study · p. 13, Table 12 · p. 16, Table 13 · p. 17
• Supplementary benchmark tables report test-set performance on E2E clean, DART and WebNLG, and LogicNLG, plus development-set performance on ToTTo.Source: Table 8 · p. 14, Table 9 · p. 14, Table 10 · p. 14
• Figure 3 presents few-shot format transfer from ToTTo tables to WebNLG knowledge-graph triples.Source: Figure 3 · p. 15
• Figure 4 presents few-shot task transfer from ToTTo tables to E2E meaning representations.Source: Figure 4 · p. 15
• Figure 5 presents few-shot transfer to LogicNLG from ToTTo table-highlight descriptions or WebNLG knowledge-graph triples.Source: Figure 5 · p. 15