Source-linked AI summary
EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing
Yue Dong, Zichao Li, Mehdi Rezagholizadeh, Jackie Chi Kit Cheung
TL;DR
Existing simplification systems generally learn transformations implicitly through complex-simple sentence pairs, motivating a more explicit editing framework. EditNTS uses an end-to-end neural programmer-interpreter to predict and execute edit operations, and it outperforms prior MT-based systems on benchmark and human evaluations. Its explicit operation traces also make simplification more interpretable and controllable, though some comparisons lack complete evaluation outputs.
Problem
MT-based simplification systems learn relatively infrequent editing operations implicitly from complex-simple sentence pairs rather than predicting them explicitly.
Method
EditNTS uses an end-to-end neural programmer-interpreter that predicts edit operations and executes them to produce simplified sentences.
Results
+1.89 SARI on Newsela and +1.41 SARI on WikiSmall versus state-of-the-art MT-based simplification models; human judges also rated EditNTS better on fluency, simplicity, and average overall quality.
Takeaways & Limitations
Explicit edit operations provide interpretable transformation traces and let users control the relative emphasis on deletion, copying, and adding words.
Takeaways & Limitations
EditNTS cannot be compared with NSELSTM on edit F1 scores and FKGL because its full outputs were unavailable.
Abstract
from arXiv · showhide
We present the first sentence simplification model that learns explicit edit operations (ADD, DELETE, and KEEP) via a neural programmer-interpreter approach. Most current neural sentence simplification systems are variants of sequence-to-sequence models adopted from machine translation. These methods learn to simplify sentences as a byproduct of the fact that they are trained on complex-simple sentence pairs. By contrast, our neural programmer-interpreter is directly trained to predict explicit edit operations on targeted parts of the input sentence, resembling the way that humans might perform simplification and revision. Our model outperforms previous state-of-the-art neural sentence simplification models (without external knowledge) by large margins on three benchmark text simplification corpora in terms of SARI (+0.95 WikiLarge, +1.89 WikiSmall, +1.41 Newsela), and is judged by humans to produce overall better and simpler output sentences.
1 Introduction
EditNTS addresses the limitations of machine-translation-based simplification by explicitly predicting edit operations with an end-to-end neural programmer-interpreter. The approach aims to make simplification more interpretable and controllable while improving automatic and human evaluations.
- Motivation: MT-based simplification systems learn editing operations implicitly from complex-simple sentence pairs, even though most source words usually remain unchanged.This makes the relatively infrequent simplifying operations harder to learn directly.
- Approach: KEEP operations let the model skip unchanged words and focus on sentence parts requiring modification.This design resembles a residual path that directly copies information when it is not the focus of editing.
- Properties: Generated edit programs make the simplification process more interpretable than black-box sequence-to-sequence systems.The operations provide traces of how complex sentences are transformed into simpler ones.
- Properties: EditNTS can prioritize different simplification operations by changing their loss weights, supporting task-specific control such as compression or lexical replacement.The model therefore exposes control over the relative behavior of its edit operations.
- Approach: EditNTS explicitly models simplification with a programmer and interpreter that predict and execute edit operations end to end.The interpreter executes programs and summarizes the partial output before the next edit decision, while also regularizing outputs toward grammatical sequences.
2 Related Work
Earlier simplification systems mainly adapted statistical or neural machine translation, while prior edit-based work used externally generated labels and downstream tools. EditNTS instead frames simplification as an end-to-end neural programmer-interpreter process.
- MT-based Sentence Simplification: SMT-based systems integrate human-curated features, whereas NMT-based systems extract features automatically in an end-to-end fashion.Both approaches treat simplification as a translation problem over complex-simple sentence pairs.
- MT-based Sentence Simplification: MT-based models learn simplification rewrites implicitly from parallel complex-simple sentences.The related-work discussion identifies this implicit learning as the shared basis of the main MT-based approaches.
- Edit-based Sentence Simplification: Prior edit-based work predicted KEEP, REPLACE, and DELETE labels using silver alignments and required a downstream module to apply replacements.Its classifier was non-autoregressive and depended on MASSAlign-derived labels and external replacement handling.
- Edit-based Sentence Simplification: EditNTS replaces those external alignment and simplification components with an end-to-end neural programmer-interpreter model.The model executes predicted edit operations directly on the input sentence.
- Neural Programmer-Interpreter Models: Neural programmer-interpreters learn to execute programs from execution traces, and EditNTS adapts this framework to monolingual sentence simplification.The paper positions EditNTS alongside earlier NPI applications in arithmetic, sorting, algebra, and machine-translation post-editing.
3 Model
EditNTS models sentence simplification as a deterministic sequence of explicit edit operations predicted by a programmer and executed by an interpreter. The model constructs expert programs from complex-simple sentence pairs, uses contextual partial outputs for sequential decisions, and addresses label imbalance during training.
- EditNTS Model: EditNTS learns explicit edit operations rather than directly mapping complex sentences to simplified sentences with a conventional sequence-to-sequence model.The model represents simplification as a program over the input sentence.
- EditNTS Model: The edit-operation set is ADD(W), KEEP, DELETE, and STOP, with OOV words copied by predicting KEEP on the corresponding complex tokens.The programmer has V + 3 prediction candidates, including the fixed vocabulary and three non-word operations.
- EditNTS Model: At each step, the programmer selects an operation for the current input word using the partial output, previous edit labels, and a context vector over the complex sentence.The edit pointer advances through the input according to previously predicted KEEP and DELETE operations.
- EditNTS Model: The interpreter applies each operation, updates the partial simplified sequence, and feeds its context back to the programmer for the next decision.KEEP and DELETE advance the input pointer, ADD(W) leaves it in place, and STOP terminates editing.
- Edit Label Construction: Expert edit sequences are deterministic shortest paths from complex to simple sentences, with ties resolved by preferring ADD before DELETE.The paths are generated with dynamic programming similar to Levenshtein distance without substitutions.
- Edit Label Construction: Because DELETE labels are especially imbalanced, training uses inverse-frequency loss weights and conservatively pads KEEP operations if STOP occurs early.The padding ensures outputs remain conservative with respect to the complex input sequence.
4 Experiments
The experiments evaluate EditNTS on three benchmark text simplification datasets against SMT-, NMT-, and edit-label baselines using automatic and human measures.
- Datasets: Three benchmark datasets—WikiSmall, WikiLarge, and Newsela—provide the experimental testbeds for sentence simplification.WikiSmall and WikiLarge contain aligned Wikipedia pairs, while Newsela contains professionally rewritten news articles across grade levels.
- Baselines: The comparison includes three SMT-based systems, four NMT-based systems, and a BiLSTM sequence-labeling model trained on the authors’ edit labels.The SMT baselines include PBMT-R, Hybrid, and SBMT-SARI; the NMT baselines include NTS, NSELSTM, DRESS, DRESS-LS, and DMASS+DCSS.
- Evaluation metrics: SARI evaluates add, delete, and keep rewriting through n-gram F1 scores, while FKGL measures readability, with lower FKGL indicating simpler output.The study also reports the percentage of unchanged sentences and treats SARI as its most important measurement.
- Evaluation caveat: The authors cannot directly compare their sequence-labeling reimplementation with the prior model because that model is unavailable and used nonstandard splits.Their reported comparison is based on a good-faith reimplementation trained with the authors’ edit labels.
- Evaluation metrics: Human judges rate system outputs for fluency, adequacy, and simplicity using a five-point Likert scale.The human evaluation compares EditNTS with strong MT-based, external-knowledge-based, and sequence-labeling systems.
5 Results
EditNTS achieves strong automatic and human-evaluated simplification results across three benchmarks, while its explicit edit operations support controllable generation and interpretable editing behavior.
- Automatic evaluation: EditNTS reports corpus-level FKGL, SARI, add/keep/delete F1, and unchanged-sentence percentages across three benchmark datasets.These measures assess readability, simplification quality, edit accuracy, and how often outputs copy sources unchanged.
- Automatic evaluation: EditNTS produces easier-to-understand outputs and fewer unchanged sentences than MT-based systems.Lower FKGL indicates easier readability, while MT-based systems more often copy the source unchanged.
- Automatic evaluation: +1.89 and +1.41 SARI over state-of-the-art TS models on Newsela and WikiSmall, respectively.On WikiLarge, EditNTS exceeds DRESS-LS by +0.95 SARI and is comparable to PBMT-R.
- Human evaluation: Human judges rate EditNTS higher than MT-based systems on fluency, simplicity, and average overall quality, while judging it adequate despite more deletions.EditNTS also performs significantly better than Seq-Label on fluency.
- Controllable generation: Changing loss weights on ADD, KEEP, and DELETE controls output length, copied content, and novel content.This gives EditNTS control over the relative prevalence of different edit operations during generation.
- Ablation studies: Removing the interpreter reduces performance, while POS tags and attention provide additional gains.The ablation compares the full model with variants removing POS tags, the interpreter, or context.
6 Conclusion
The NPI-based model explicitly predicts and executes edit labels for sentence simplification, outperforming prior machine-translation-based systems while producing interpretable and controllable operation traces.
- The model uses a programmer to predict edit labels and an interpreter to execute them into simplified tokens.
- It outperforms previous state-of-the-art machine-translation-based simplification models on most automatic evaluation metrics and human ratings.
- Generated edit-operation traces make the simplification process more interpretable than black-box machine-translation systems.
- The model can prioritize different simplification operations, providing control over the simplification process.