CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training

1 Introduction

• G2T verbalizes graph data, while T2G extracts graph structures from text for downstream tasks such as information retrieval and reasoning.Source: §1 · p. 1
• WebNLG 2017 has only 18K training pairs, far fewer than millions of NMT examples, while annotation is expensive and existing G2T and T2G models show limited performance.Source: §1 · pp. 1–2
• CycleGT trains G2T and T2G from fully non-parallel datasets through two cycle losses, addressing the challenge of learning across text and graph modalities.Source: §1 · p. 2
• CycleGT is comparable to many supervised models on WebNLG and best among unsupervised baselines on GenWiki, improving +11.47 BLEU over the best baseline on GenWikiFINE and +6.26 BLEU on GenWikiFULL.Source: §1 · p. 2
• The authors conclude that CycleGT's performance indicates an effective approach to data scarcity in both tasks and may help pave the way for scalable unsupervised learning.Source: §1 · p. 2

2 Formulation and Notations

• The formulation constrains text and graphs to share a latent-content distribution despite being different surface realizations, with an approximately one-to-one mapping between them.Source: §2 · p. 2
• Training assumes separate text and graph datasets with no observed alignment, while the ideal objective is defined over an unseen distribution of paired examples.Source: §2 · p. 2
• CycleGT approximates the ideal paired-data objective using the separately observed non-parallel text and graph data.Source: §2 · p. 2

3.1 G2T Component

• G2T linearizes each graph in the WebNLG triplet order and fine-tunes T5 to encode the sequence and generate a descriptive text.The framework can substitute another G2T architecture, including GNN-LSTM.Source: §3.1 · pp. 2–3

3.2 T2G Component

• T2G extracts or uses given entities, then predicts relations between every entity pair to form the graph.Its relation classifier includes a “norelation” type.Source: §3.2 · p. 3
• T2G represents each entity by average-pooling its contextualized word embeddings and trains its graph predictions with a maximum-likelihood objective.The number of entities is K = |NER(t)|.Source: §3.2 · p. 3

3.3 Cycle Training with Iterative Back Translation

• CycleGT alternates optimization of its two cycle losses to reduce the discrepancy between synthetic and unavailable paired-data distributions.The synthetic pairs are (t, T2Gϕ(t)) and (G2Tθ(g), g), formed from each text and graph in the non-parallel datasets.Source: §3.3 · pp. 3–4

3.4 Challenges of Cycle Training for G2T and T2G

• Discrete intermediate text or graph outputs prevent gradients from passing through both models, so CycleGT fixes one model while optimizing the other.It first fixes ϕ to optimize θ for the text cycle, then fixes θ to optimize ϕ for the graph cycle.Source: §3.4 · p. 4
• Unlike unsupervised NMT approaches that use other regularizers, CycleGT relies solely on cycle training and the inductive biases of its G2T and T2G modules.The cited NMT approaches use autoencoder losses, adversarial losses, and warm-start strategies.Source: §3.4 · p. 4

4.1 Datasets

• WebNLG 2017 has 13,036 training samples, WebNLG 2020 has 35,426, and GenWiki supplies unpaired Wikipedia text and DBpedia graphs in FULL and FINE variants.WebNLG 2020 extends the earlier dataset with broader categories; GenWiki text has more vocabulary and variety than crowdsourced WebNLG text.Source: §4.1 · p. 4

4.2 Evaluation Metrics

• G2T evaluation uses text-generation metrics including BLEU, METEOR, ROUGEL, and CIDEr, while WebNLG 2017 T2G uses micro and macro F1 for relation types.WebNLG 2020 G2T evaluation also reports chrF++, TER, BERTF1, and BLUERT.Source: §4.2 · p. 5

4.3 Comparison Systems for WebNLG 2017

• WebNLG 2017 comparisons include RuleBased and GT-BT unsupervised baselines, alongside supervised G2T systems and OnePass and the study’s T2G implementation.The supervised G2T comparators include encoder-decoder, entity-ordering, graph-attention, segmentation-and-alignment, planning, and T5 systems.Source: §4.3 · p. 5

4.4 Comparison Systems for GenWiki

• For GenWiki, comparisons include RuleBased, WebNLG-trained DirectTransfer, and NoisySupervised models built from entity-overlap distant pairs.GraphWriter is used as the architecture for all G2T models, including CycleGT, to keep the comparison fair.Source: §4.4 · p. 5

4.5 Implementation Details

• The WebNLG training and validation text and graphs are separated and shuffled to create non-parallel data; CycleGT uses pretrained T5 for G2T and a two-layer, 512-unit BiLSTM for T2G.The model is trained for 30 epochs.Source: §4.5 · p. 6

4.6 Main Results

• On WebNLG 2017, CycleGT reaches 55.5 BLEU for G2T versus 56.4 BLEU for supervised T5-Base, and its T2G scores are on par with its supervised counterpart.The authors also report that CycleGT beats the RuleBased and GT-BT unsupervised baselines for G2T.Source: §4.6 · p. 6, Table 1 · p. 6
• CycleGT’s unsupervised G2T performance is competitive with many methods on the WebNLG 2020 challenge leaderboard.The authors note that most, and possibly all, competing systems are supervised; Table 3 ranks systems by METEOR.Source: §4.6 · p. 6, Table 3 · p. 7
• CycleGT outperforms NoisySupervised by around 10 BLEU points on GenWiki, with results reported for GenWikiFINE and GenWikiFULL.Table 2 covers CycleGT and three unsupervised baselines on both dataset variants.Source: Table 2 · p. 6, §4.6 · p. 7

Data-to-Text Generation

• Data-to-text generation has moved from pipelined content and sentence planning to end-to-end neural encoder-decoder systems.Earlier datasets cited include 5K air-travel, 22K WeatherGov, 7K Robocup, and 5K RotoWire examples.Source: §5 · p. 7

Relation Extraction

• Relation extraction classifies relations between entities in shared textual context, with neural CNN, RNN, and BERT approaches following earlier feature-engineered methods.The cited datasets often contain only hundreds or thousands of examples, prompting research to shift toward distant supervision.Source: §5 · p. 7

Cycle Training

• Cycle training uses consistency between inverse functions, with prior applications in computer vision, back translation, dual learning, and text-graph conversion.The paper compares its approach with prior text-graph cycle training.Source: §5 · p. 7

• CycleGT is a cycle-learning framework for unsupervised graph-to-text and text-to-graph generation.Source: §6 · p. 8
• With the same amount of unsupervised data, CycleGT achieves results comparable to many supervised models.Source: §6 · p. 8
• CycleGT displays a clear advantage over other unsupervised models in the reported experiments.Source: §6 · p. 8