Split and Rephrase

1 Introduction

• Split-and-Rephrase produces shorter sentences that preserve meaning, requiring syntactic changes but no deletion or lexical or phrasal simplification.Examples include turning a relative clause into a main clause.Source: §1 · p. 1
• The task could help NLP systems process shorter inputs and provide shorter sentences for people with reading disabilities.The paper names parsers, semantic role labelers, and statistical machine translation systems as potential beneficiaries.Source: §1 · pp. 1–2
• WEBSPLIT is a benchmark for training and testing Split-and-Rephrase systems, with many splits and rephrasings, a compact vocabulary, and meaning representations linked to examples.The corpus’s vocabulary is 3,311 word forms for a little over 1 million training items, which the authors say reduces sparse-data issues and facilitates learning.Source: §1 · p. 2
• The paper proposes five model families, from a sentence-only encoder-decoder to semantic-input and partition-and-generate approaches.The semantic inputs include DRSs and RDF triples; the two partition-and-generate models differ in whether generation also uses the complex sentence.Source: §1 · p. 2

Sentence Splitting

• Existing sentence simplification corpora were considered ill suited for split-and-rephrase training: Wikipedia simplification pairs averaged 1.06 simple sentences per complex sentence, and only 6.1% were split.The cited evaluation also reports incorrect automatic alignments and poor generalization to other genres.Source: §2 · pp. 2–3
• WEBSPLIT averages 4.99 simple sentences per complex sentence, unlike the Wikipedia simplification and Newsela datasets, which have a low proportion of splits.The paper presents the high number of splits as a difference from both datasets.Source: §2 · p. 3

Rephrasing

• Prior paraphrasing research used template bootstrapping, corpus-extracted patterns, SMT, and neural machine translation approaches.The paper surveys these approaches as methods for sentence paraphrasing.Source: §2 · p. 3
• Sentence compression often deletes words and performs limited paraphrasing, while sentence fusion datasets were small; neither provided a general split-and-rephrase resource.The cited fusion dataset contained 1,858 instances, which the authors argue is not enough for learning a general model.Source: §2 · p. 3

Related Work

• Earlier work on sentence rewriting had not extensively studied the interaction between splitting and rephrasing, and no corpus supported models that perform both.The paper frames its benchmark and baseline models as addressing this gap.Source: §2 · p. 3

3.1 The WEBNLG Dataset

• WEBNLG supplies texts paired with RDF meaning representations; its 13,308 pairs cover 7,049 distinct representations and RDF sets of 1–7 triples.The corpus contains 60 distinct RDF tree shapes.Source: §3.1 · p. 4

3.2 Creating the WEBSPLIT Dataset

• The benchmark construction searches WEBNLG for texts whose RDF meanings partition the complex sentence’s meaning while preserving its content.The paired target texts must contain more than one sentence.Source: §3.2 · p. 4
• The construction segments WEBNLG verbalisations into sentences, manually correcting segmentation errors, then orders output sentences by RDF-triple traversal as a heuristic.The ordering heuristic reflects the observed tendency for WEBNLG texts to follow RDF-triple order, though this is not always true.Source: §3.2 · pp. 4–5, §3.3 · p. 5
• Across-entry pairing matches a complex sentence’s meaning representation with a disjoint partition whose texts form multiple sentences.Within-entry pairing also uses alternative verbalisations of the same meaning representation, one single-sentence and one multi-sentence.Source: §3.2 · pp. 4–5

3.3 Results

• 1,066,115 distinct complex-sentence/rephrasing pairs were created; rephrasings contain 2–7 sentences, averaging 4.99.The pairs include 5,546 distinct complex sentences.Source: §3.3 · p. 5

• The output is a sequence of at least two texts that preserves the meaning of the complex input sentence.Source: §4 · p. 6
• The supervised formulation aims to maximize the likelihood of output text T given input C and parameters θ.Source: §4 · p. 6
• M_C denotes C’s meaning representation, while M_1−n denotes a set of meaning representations partitioning M_C.Source: §4 · p. 6

5 Split-and-Rephrase Models

• Five models are proposed to explore Split-and-Rephrase by exploiting different levels of information available in WEBSPLIT.Source: §5 · p. 6

5.1 A Probabilistic, Semantic-Based Approach

• HYBRIDSIMPL adapts a semantic-based simplification approach by retaining splitting and SMT rephrasing, omitting deletion, and training on WEBSPLIT.Source: §5.1 · p. 6

5.2 A Basic Sequence-to-Sequence Approach

• SEQ2SEQ uses a three-layer LSTM encoder-decoder with local attention to predict shorter sentences from the complex input alone.It does not use the benchmark’s semantic information.Source: §5.2 · p. 6

5.3 A Multi-Source Sequence-to-Sequence Approach

• MULTISEQ2SEQ conditions generation on both the complex sentence and its associated RDF triples using separate encoders.The model uses distinct context vectors for the two sources.Source: §5.3 · pp. 6–7

5.4 Partitioning and Generating

• Partition-and-generate models first divide the input’s RDF triples into subsets, then generate rephrasings from each subset.The two variants either condition generation on both the sentence and subset, or on the subset alone; the pipeline assumes generation is independent of the full meaning representation given the sentence and partition.Source: §5.4 · p. 7

6.1 Training, Validation and Test sets

• The 5,546 distinct complex sentences are split into 4,438 training, 554 validation, and 554 test sentences, keeping validation and test inputs out of training.Source: §6.1 · pp. 7–8
• Direct-mapping systems train on 886,857 sentence-text pairs; pipeline partitioning and generation use 13,051 and 53,470 pairs, respectively.Source: §6.1 · p. 8

6.2 Implementation Details

• Neural models use three-layer LSTMs with 500 hidden states, train for 20 epochs, and retain the best validation-set model for testing.Batch sizes vary by model: 64 for SEQ2SEQ and MULTISEQ2SEQ, and 32 for the two partition-and-generate systems.Source: §6.2 · p. 8

6.3 Results

• The two partition-and-generate models outperform all others by a wide margin, supporting explicit semantic partitioning as key to effective rephrasing.The authors suggest that partitioning reduces complex generation to generating short sentences from smaller semantic chunks.Source: Table 3 · p. 8, §6.3 · pp. 8–9
• BLEU alone is insufficient because unsplit source sentences can score highly; the evaluation also measures output sentences per input and words per output sentence.Source: §6.3 · p. 8
• SEQ2SEQ and MULTISEQ2SEQ often produce non-meaning-preserving outputs on longer inputs, whereas partition-and-generate examples preserve meaning and split the input.The authors conjecture that HYBRIDSIMPL's poor performance may reflect reduced semantic parsing quality for complex named entities.Source: §6.3 · pp. 8–9

7 Conclusion

• The authors propose Split-and-Rephrase and construct a corpus from readily available NLG evaluation data.Source: §7 · p. 9
• Initial experiments indicate that splitting helps generate fluent, meaning-preserving rephrasings by reducing complex generation to a series of short-sentence generation tasks.Source: §7 · p. 9
• Future work includes learning to split without explicit semantic input and extending WEBSPLIT using the larger WebNLG corpus, as well as exploring sentence fusion.The current WebNLG corpus has 43,056 MR-Text pairs, 16,138 distinct MRs, and 15 DBpedia categories.Source: §7 · p. 9