Source-linked AI summary
A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification
Mounica Maddela, Wei Xu
TL;DR
Length- and frequency-based heuristics for lexical simplification do not always align with human judgments. The paper combines a human-rated word-complexity lexicon with a neural readability ranking model, reporting state-of-the-art results across several lexical simplification tasks and producing SimplePPDB++.
Problem
Common length- and frequency-based heuristics do not always match human judgments of word simplicity, creating a challenge for lexical simplification.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
SimplePPDB++ contains over 10 million paraphrase rules with readability categories, allowing a flexible trade-off between high-quality and high-coverage paraphrases.
Source: §4.2 · p. 6
Abstract
from arXiv · showhide
Current lexical simplification approaches rely heavily on heuristics and corpus level features that do not always align with human judgment. We create a human-rated word-complexity lexicon of 15,000 English words and propose a novel neural readability ranking model with a Gaussian-based feature vectorization layer that utilizes these human ratings to measure the complexity of any given word or phrase. Our model performs better than the state-of-the-art systems for different lexical simplification tasks and evaluation datasets. Additionally, we also produce SimplePPDB++, a lexical resource of over 10 million simplifying paraphrase rules, by applying our model to the Paraphrase Database (PPDB).
1 Introduction
Lexical simplification ranks candidate paraphrases by simplicity, but common length- and frequency-based heuristics can conflict with human judgments. The paper introduces a human-rated word-complexity lexicon and neural ranking model, reports improvements across lexical simplification tasks, and creates SimplePPDB++.
1 Introduction
- Lexical simplification comprises identifying complex words, generating alternatives, and ranking substitutions by simplicity.The paper notes uses such as presenting alternative expressions as reading assistance.Source: §1 · p. 1
- Corpus-based length and frequency assumptions often misidentify simpler words; in sampled PPDB pairs, the simpler word was longer in 21% and less frequent in 14%.For example, foolishness is simpler than folly despite being longer and less frequent in Google 1T Ngram.Source: §1 · p. 1
- The authors create a human-rated complexity lexicon of 15,000 English words and a neural readability ranking model with Gaussian-based feature vectorization.The model uses human ratings and other numerical features to measure word or phrase complexity.Source: §1 · p. 1
- The model reports state-of-the-art Substitution Ranking Pearson correlations of 0.714 with the lexicon and 0.702 without it, plus improvements on Substitution Generation and Complex Word Identification.The authors also release SimplePPDB++, a resource of over 10 million paraphrase rules with readability scores.Source: §1 · p. 2
2 Constructing A Word-Complexity Lexicon with Human Judgments
The lexicon assigns human-rated complexity scores to the 15,000 most frequent English words. Eleven fluent non-native English speakers rated words, and the authors averaged ratings after filtering outliers.
2 Constructing A Word-Complexity Lexicon with Human Judgments
- The lexicon covers the 15,000 most frequent English words, which the authors found sufficient for simplification because the ranking model can estimate complexity outside the lexicon.Each word’s final score is based on averaged human ratings.Source: §2 · p. 2
- Eleven fluent non-native English speakers rated words on a six-point scale, with 5–7 ratings collected for each word.Annotators were shown the most frequent capitalization form when a word had multiple capitalized versions.Source: §2 · p. 2
- Table 1 presents words with their averaged complexity scores alongside five annotators’ ratings on a scale where 1 is simplest and 6 most complex.The authors note that different native languages among annotators may have contributed to judgment variance.Source: Table 1 · p. 2, §2 · p. 2
- Discarding about 3% of ratings that differed by at least 2 from the remaining mean raised overall inter-annotator agreement from 0.55 to 0.64.The final score for each word averages the retained ratings.Source: §2 · p. 2
3 Neural Readability Ranking Model for Words and Phrases
The Neural Readability Ranking model estimates the relative complexity of word or phrase pairs, using the lexicon and optional context. It applies Gaussian-based feature vectorization and is reported to outperform state-of-the-art approaches across multiple lexical simplification tasks.
3 Neural Readability Ranking Model for Words and Phrases
- The NRR model can use the word-complexity lexicon and available context to predict the complexity of words or phrases, outperforming state-of-the-art approaches across multiple simplification tasks.Its task-specific configurations are described in the experiments.Source: §3 · p. 3
3.1 Neural Readability Ranker (NRR)
- For an input pair, NRR predicts a relative complexity score; its architecture processes each item’s features and then feeds them jointly to a task-specific feedforward network.A negative output means the first item is simpler than the second.Source: §3.1 · p. 3
3.2 Features
- Features combine lexicon ratings and presence indicators with lexical, corpus, collocational, and pairwise embedding information.For a pair, the model includes each item’s features, their differences, and embedding similarity and difference.Source: §3.2 · p. 3
3.3 Vectorizing Numerical Features via Gaussian Binning
- Gaussian radial basis functions map numerical features into vectors, representing their values relative to evenly spaced bins and allowing feature ranges to be represented smoothly.The model normalizes the bin distances into a k-dimensional vector and vectorizes features other than word2vec vectors.Source: §3.3 · pp. 3–4
3.4 Training and Implementation Details
- The NRR model is trained to minimize mean squared error, with three hidden layers of eight nodes and dropout of 0.2.The reported setup uses 10 bins and γ = 0.2 without extensive parameter tuning, and reports results after 100 epochs.Source: §3.4 · p. 4
4 Lexical Simplification Applications
The paper evaluates its lexicon and neural readability ranking (NRR) model across substitution ranking, SimplePPDB++, substitution generation, and complex word identification.
4.1 Substitution Ranking
- On SemEval 2012 substitution ranking, full NRR significantly improves on the state-of-the-art neural baseline in P@1 and Pearson correlation.It learns pairwise candidate rank differences, then aggregates pairwise complexity scores to rank candidates.Source: §4.1 · p. 5, Table 2 · p. 5
4.2 SimplePPDB++
- NRRall+binning+WC significantly beats SimplePPDB: accuracy rises 3.2 points, simplifying precision 7.4 points, and complicating precision 4.0 points.SimplePPDB++ assigns readability categories to all 13.1 million lexical and phrasal rules in PPDB 2.0 XL.Source: §4.2 · pp. 6–7
4.3 Substitution Generation
- Using the same PPDB paraphrase candidates, SimplePPDB++ enables improved selection and ranking of simplifying substitutions over the previous SimplePPDB version.The evaluation considers mean average precision, precision@1, and the number of paraphrases generated per target.Source: §4.3 · p. 7, Table 5 · p. 7
4.4 Complex Word Identification
- Adding the lexicon improves SV000gg and nearest-centroid systems across F-score, accuracy, and G-score (p < 0.01).WC-only is a simple, efficient lookup; on SemEval 2016, its best accuracy and F-score may reflect that only 5% of test instances are complex.Source: §4.4 · p. 8, Table 6 · p. 8
5 Related Work
Earlier lexical simplification work largely used lexical and corpus features, while manually constructed lexica and neural feature-vectorization offered related approaches. The paper positions its human-rated lexicon and ranking model against these lines of work and notes an open question about using its resource in neural machine translation.
5 Related Work
- Earlier complex-word identification used frequency thresholds or classifiers combining lexical and language-model features, while substitution ranking relied on corpus statistics and similarity features.Examples include threshold-based CWI, CWI classifier ensembles (Shardlow, 2013a; Paetzold and Specia, 2016a; Yimam et al., 2017; Kriz et al., 2018), and substitution ranking methods using word length, frequencies, semantic similarity, or language models (Biran et al., 2011; Bott et al., 2012; Kajiwara et al., 2013; Glavaš and Štajner, 2015; Horn et al., 2014).Source: §5 · pp. 8–9
- The authors describe their model as the first pairwise neural ranking model to vectorize numeric features and embed human judgments from a 15,000-word complexity lexicon.They contrast it with earlier neural lexical simplification work using a feedforward network with language-model probability features (Paetzold and Specia, 2017).Source: §5 · p. 9
- Sentence simplification has also used statistical or neural machine translation, but integrating PPDB paraphrase rules into neural machine translation remains an open question.The passage cites PPDB integration into statistical machine translation for sentence simplification and bilingual translation (Mehdizadeh Seraj et al., 2015).Source: §5 · p. 9
- Prior simplification lexica included medical terminology, vocabulary familiarity ratings, school-textbook word lists, common-word lists, and vocabulary lists ordered by complexity.The paper states that no previous study had manually built a large human-judged word-complexity lexicon and shown substantial improvements on automatic simplification systems.Source: §5 · p. 9
- The paper relates its numeric-feature vectorization to feature quantization for neural entity linking and feature embeddings for neural dependency parsing.The cited examples are Sil et al. (2017), Liu et al. (2016), and Chen and Manning (2014).Source: §5 · p. 9
6 Conclusion
The paper concludes that its neural readability ranking model improves performance across lexical simplification tasks and releases both a human-rated word-complexity lexicon and a large paraphrase resource. It identifies domain-, user-, and language-specific lexicon expansion as future work.
6 Conclusion
- The paper reports significant performance improvement over state-of-the-art systems across various lexical simplification tasks.The conclusion characterizes this as a result of the proposed neural readability ranking model.Source: §6 · p. 9
- The authors release a manually constructed lexicon of 15,000 English words and SimplePPDB++, an automatically constructed resource of over 10 million paraphrase rules with quality and simplicity ratings.Source: §6 · p. 9
- Future work would extend the lexicon to specific domains, different target users, and other languages.Source: §6 · p. 9