Classification of Natural Language Processing Techniques for Requirements Engineering

I. INTRODUCTION

• NLP technologies’ industrial uptake remains limited and fragmented despite progress in NLP and NLP4RE research.The introduction describes a continuing gap between NLP4RE research and practical application.Source: §I · p. 1
• The paper synthesizes 57 commonly used NLP techniques in RE for research and practice.Source: §I · p. 1
• The techniques are classified by typical NLP pipeline tasks and by linguistic analysis capability.The two classification approaches are presented as complementary ways to organize the techniques.Source: §I · p. 1

II. BACKGROUND: 40 YEARS OF NLP4RE

• NLP support for RE includes both traditional NLP contributions and more recent applications of ML and DL.Source: §II · p. 1
• The paper provides a brief history of NLP4RE as background and motivation.Source: §II · p. 1
• Its focus is on NLP techniques, not ML and DL techniques.Source: §II · p. 1

A. Traditional NLP for RE

• Early approaches extracted entities using syntactic rules and assumed requirements followed constrained, predictable formats, which are rarely found in practice –.Source: A. Traditional NLP for RE · p. 1
• In the 1990s, findphrases and OICSI used lexical affinity and semantic cases for extraction,.Source: A. Traditional NLP for RE · p. 1
• Later 1990s and early 2000s tools used rule-based techniques for term extraction and model generation, while QuARS and ARM targeted defect detection with dictionary-based techniques –.Source: A. Traditional NLP for RE · pp. 1–2
• Early-2000s work applied information retrieval to requirements tracing and statistical NLP to identify shallow knowledge and trace relationships,,.Source: A. Traditional NLP for RE · p. 2
• Since the late 2000s, tools have addressed ambiguity, model generation, defects, smells, and equivalent requirements –.NLP applications also extended to legal documents, privacy policies, and user stories –.Source: A. Traditional NLP for RE · p. 2

B. Machine Learning and Deep Learning for RE

• ML-based NLP approaches are described as effective, less reliant on expert manpower, and straightforward to port across domains compared with traditional rule-based approaches.Source: B. Machine Learning and Deep Learning… · p. 2
• A 2007 approach classified non-functional requirements using weighted indicator terms and became a frequently used baseline for assessing later techniques, –.Source: B. Machine Learning and Deep Learning… · p. 2
• Recent ML work increasingly analyzes stakeholder feedback from sources such as app reviews and issue trackers to support data-driven and crowd-based RE –.The passage identifies feedback analysis as a main recent trend in NLP4RE research.Source: B. Machine Learning and Deep Learning… · p. 2
• ML and DL have also been applied to glossary extraction, requirements classification, and requirements tracing, –.Source: B. Machine Learning and Deep Learning… · p. 2
• Initial DL experiments in RE have addressed requirement classification, discovery from issue reports, and extraction and classification from contracts –[51].The authors predict that DL-based RE research will grow rapidly and overtake ML-based work.Source: B. Machine Learning and Deep Learning… · p. 2

III. METHOD

• The literature base began with 404 NLP4RE studies from a systematic review covering work through 2019.The authors supplemented this source with a targeted review for more recent publications.Source: §III · p. 2
• A targeted review sought techniques emerging in the preceding two years, focusing on major RE and software engineering conferences and journals.The venues included RE, REFSQ, ICSE, REJ, JSS, ASE, DKE, IST, and TSE.Source: §III · p. 2
• The authors defined an NLP technique by its role in performing a particular NLP task, then synthesized varied terms into 57 standardized techniques.They consulted NLP expert books during standardization; Tables I and II present the resulting techniques.Source: §III · p. 2

IV. CLASSIFYING NLP TECHNIQUES BY TASKS

• The task-based classification relates NLP techniques to tasks, with NLP tools and resources supporting task performance.The paper defines a task as text-processing work performed through one or more techniques.Source: §IV · p. 4, Figure 1 · p. 5
• POS tagging assigns grammatical tags to words, while semantic tagging extracts useful information and syntactic and semantic analysis examine structure, meaning, and relations.POS tagging uses sentence context, including neighboring words, to infer a word's part of speech.Source: §IV · p. 4
• Frequency analysis examines word or term frequencies, similarity analysis estimates similarity between text elements, and rule-based analysis uses grammatical or semantic rules or patterns.The similarity task includes identifying semantic relatedness or synonyms.Source: §IV · pp. 4–5
• Text segmentation breaks text into sentences or words, while normalization standardizes word forms and removes stop words or unnecessary characters.The paper presents the task-based classification results in Table III.Source: §IV · p. 5

V. CLASSIFYING NLP TECHNIQUES BY LINGUISTIC ANALYSIS LEVELS

• The classification uses seven linguistic levels: phonology, morphology, lexical, syntactic, semantic, discourse, and pragmatic analysis.The levels are attributed to Liddy.Source: §V · p. 5
• Morphological analysis examines meaning-bearing parts of words, while lexical analysis interprets individual word meanings.Morphological parts include prefixes, roots, and suffixes; lexical analysis may use a lexicon or dictionary.Source: §V · p. 5
• Syntactic analysis uses sentence grammar, while semantic analysis addresses word or compositional meanings and discourse analysis connects meaning across sentences.Discourse examples include anaphora and coreference resolution.Source: §V · p. 5
• The authors found no techniques for phonetic or pragmatic analysis; they attribute this to NLP4RE's focus on document text processing rather than natural language understanding.The paper treats supported levels of analysis, especially higher levels, as indicators of an NLP tool's competence.Source: §V · p. 5

• The knowledge base organizes 57 commonly used RE techniques by NLP task and analysis level.Source: §VI · p. 5
• Progressive queries let users compare techniques for a text-processing task and judge their relevance to the task at hand.Source: §VI · pp. 5–6
• Planned extensions will connect techniques—including their combinations, order, and case fit—and identify tools that support each technique.Source: §VI · p. 6