PaperlayerView exact revision on arXiv

Source-linked AI summary

Controllable Sentence Simplification

Louis Martin, Benoît Sagot, Éric de la Clergerie, Antoine Bordes

arXiv:1910.02677v3cs.CL

TL;DR

Text simplification often produces one generic output despite different audience needs. ACCESS conditions Seq2Seq models on discrete controls for simplification attributes, enabling user-set outputs and achieving 41.87 SARI on WikiLarge, +1.42 over previous scores.

  • Problem

    Text simplification research mostly generated one generic output despite different audiences requiring different vocabulary and syntactic complexity.

  • Method

    ACCESS conditions Seq2Seq models on explicit control tokens for length, paraphrasing, lexical complexity, and syntactic complexity.

  • Results

    41.87 SARI on WikiLarge establishes a state-of-the-art result, a +1.42 improvement over previous scores.

  • Takeaways & Limitations

    The method can adapt simplification to audiences with different needs and extend to other simplification attributes.

  • Takeaways & Limitations

    Evaluation is bounded by WikiLarge's limited structural simplification, while Newsela could not be used because of legal and access restrictions.

Abstract

from arXiv · show

Text simplification aims at making a text easier to read and understand by simplifying grammar and structure while keeping the underlying information identical. It is often considered an all-purpose generic task where the same simplification is suitable for all; however multiple audiences can benefit from simplified text in different ways. We adapt a discrete parametrization mechanism that provides explicit control on simplification systems based on Sequence-to-Sequence models. As a result, users can condition the simplifications returned by a model on attributes such as length, amount of paraphrasing, lexical complexity and syntactic complexity. We also show that carefully chosen values of these attributes allow out-of-the-box Sequence-to-Sequence models to outperform their standard counterparts on simplification benchmarks. Our model, which we call ACCESS (as shorthand for AudienCe-CEntric Sentence Simplification), establishes the state of the art at 41.87 SARI on the WikiLarge test set, a +1.42 improvement over the best previously reported score.

1. Introduction

Text simplification supports audiences with different needs, but prior systems generally produced one generic output. ACCESS adds controllable simplification and achieves 41.87 SARI on WikiLarge, a +1.42 gain over previous scores.

  • Different audiences need different simplifications, such as reduced syntactic complexity for some aphasic readers and simpler vocabulary for second-language learners.
  • Prior text simplification research mostly generated a single generic simplification without adapting outputs to target populations.
  • ACCESS provides explicit control over simplification outputs so users can adapt them to audience needs.
  • 41.87 SARI on WikiLarge sets a new state-of-the-art score, improving over previous scores by +1.42.

2. Related Work

Sentence simplification has evolved from machine-translation-based methods toward Seq2Seq systems, while controllable generation can modify decoding or condition models during training. This work uses train-time control tokens with Seq2Seq models and extends prior controllable simplification approaches to grammatical attributes.

  • 2.1. Sentence Simplification: Sentence simplification has commonly treated simplification as monolingual machine translation using complex-simple sentence pairs.
  • 2.1. Sentence Simplification: Recent simplification systems increasingly use Seq2Seq machine-translation models, either directly or with reinforcement learning and external resources.
  • 2.2. Controllable Text Generation: This work prepends plain-text control tokens to source sentences during Seq2Seq training and requires neither external data nor a modified training objective.
  • 2.2. Controllable Text Generation: Controllable generation methods either modify decoding or condition the model on attributes during training.
  • 2.2. Controllable Text Generation: The approach extends earlier controllable simplification work by representing grammatical attributes and analyzing their influence on generated simplifications.

3. Adding Control Tokens to Seq2Seq

ACCESS controls sentence simplification by conditioning Seq2Seq models on discrete proxies for compression, paraphrasing, lexical complexity, and syntactic complexity. Users set target ratios through prepended tokens, while the paper excludes some attributes and leaves per-sentence ratio prediction for future work.

  • 3.1. Controlled Attributes: ACCESS conditions a Seq2Seq model on explicit control tokens representing important attributes of sentence simplification.
  • 3.1. Controlled Attributes: The controlled attributes are compression, paraphrasing, lexical complexity, and syntactic complexity.
  • 3.1. Controlled Attributes: Paraphrasing lets users choose between safer outputs close to the source and more aggressive simplification with potentially more mistakes.
  • 3.1. Controlled Attributes: The method does not control every possible attribute, excluding factors such as tense and voice while leaving per-sentence ratio prediction for future work.
  • 3.2. Explicit Control Tokens: NbChars represents compression, LevSim measures source-target modification, WordRank proxies lexical complexity, and DepTreeDepth approximates syntactic complexity.
  • 3.2. Explicit Control Tokens: Control tokens encode target-to-source attribute ratios, discretized into bins and prepended to the source sentence at inference time.
  • 3.2. Explicit Control Tokens: For example, 71 source characters and 22 target characters produce a compression ratio of 0.3 and the <NbChars 0.3> token.

4. Experiments

Experiments train and evaluate Transformer-based Seq2Seq models on WikiLarge using FKGL and SARI, then compare ACCESS with prior simplification systems. ACCESS achieves the strongest reported SARI score while using explicit control-token parametrization.

  • Evaluation: The experiments evaluate simplicity with FKGL and overall simplification quality with SARI.SARI averages F1 scores for additions, keeps, and deletions across 1- to 4-gram orders.
  • Evaluation: BLEU is excluded because it favors systems that leave the source unchanged; copying the source reaches 99.37 BLEU on WikiLarge.The reported SARI implementation includes deletion recall to match previous work.
  • ACCESS Configuration: ACCESS uses three control tokens: NbChars0.95, LevSim0.75, and WordRank0.75.The ratios are selected through validation-set performance and written as subscripts in the reported parametrization.
  • Control-Token Analysis: ACCESS’s control tokens improve out-of-the-box Seq2Seq performance, with WordRank giving +2.28 SARI and LevSim +1.23 over models without parametrization.The analysis links the gains to WikiLarge validation examples containing mostly small paraphrases and to hypotheses more dissimilar from the source.

5. Ablation Studies

The ablation study evaluates control-token combinations on WikiLarge using greedy forward selection. Lexical and paraphrase controls provide the strongest gains, while adding three or more controls yields no significant improvement over their two-token combination.

  • The ablation study evaluates control-token combinations on the WikiLarge validation set using greedy forward selection.Scores are reported as means over 10 runs with 95% confidence intervals, with significance tested against the Transformer baseline.
  • +2.28 SARI is the gain from WordRank alone over models without parametrization.WordRank performs best among single control tokens, consistent with the validation set containing mostly small paraphrases.
  • +1.23 SARI is the gain from LevSim alone, making it the second-best single control token.The result supports the reported association between greater source-hypothesis dissimilarity and better simplification performance.
  • +0.88 and +0.66 SARI are the gains from NbChars and DepTreeDepth, respectively.The reported gains are more modest because the WikiLarge validation set contains little content reduction.
  • +4.04 SARI is the gain from combining WordRank and LevSim, while adding three or more controls produces no statistically significant improvement over this setup.The performance boost is nearly additive initially but saturates quickly; the reported significance threshold is p-value < 0.01.

6. Analysis of the Influence of Control Tokens

The analysis tests whether four control tokens steer their intended simplification attributes. Because single-token models also shortened sentences, the study adds a NbChars1.00 constraint to isolate the other controls' effects.

  • ACCESS targets four attributes—length, paraphrasing, lexical complexity and syntactic complexity—using NbChars, LevSim, WordRank and DepTreeDepth control tokens.
  • Figure 2 varies each control-token target ratio from 0.25 to 1.00 and compares output attributes with ground truth across 2,000 validation sentences.Rows represent control-token models, columns represent output attributes, and colors represent target ratios or ground truth.
  • Table 4 illustrates the token effects through example simplifications, using varying target ratios and a NbChars1.00 constraint for LevSim, WordRank and DepTreeDepth.
  • Single-token models for LevSim, WordRank and DepTreeDepth mainly reduced sentence length, confounding their apparent attribute effects.Shortening decreases Levenshtein similarity, WordRank when complex words are deleted, and dependency-tree depth.
  • A NbChars1.00 training constraint is added at inference time to keep length unchanged while testing the other control tokens.This constraint is used to study cross-influence more clearly.

7. Conclusion

The paper concludes that explicit control tokens improve Seq2Seq sentence simplification and produce their intended effects. The approach is also presented as extensible to additional attributes and audiences with different needs.

  • Explicitly conditioning Seq2Seq models on length, paraphrasing, lexical complexity or syntactic complexity significantly improves sentence-simplification performance.
  • Analysis confirms that each control token has its desired effect on generated simplifications.
  • The method is easy to extend to other text-simplification attributes and can adapt simplification to audiences with different needs.

9. Language Resource References

The cited references cover prior work on text simplification data, statistical machine translation, and deep reinforcement learning for sentence simplification.

  • Xu, Wei and Callison-Burch, Chris and Napoles, Courtney are listed among the language-resource references.
  • Problems in current text simplification research: New data can help is listed as a 2015 reference.
  • Optimizing statistical machine translation for text simplification and Sentence simplification with deep reinforcement learning are listed as references.
Loading 1910.02677v3…