Source-linked AI summary
Advancing Requirements Engineering through Generative AI: Assessing the Role of LLMs
Chetan Arora, John Grundy, Mohamed Abdelrazek
TL;DR
Requirements Engineering remains difficult because communication, ambiguity, resource constraints, and limited automation complicate requirements work. This chapter examines LLMs across RE stages using a process vision, SWOT analyses, and a preliminary evaluation, concluding that they can augment RE but are not a silver bullet and require human expertise and contextual safeguards.
Problem
Requirements Engineering is critical but challenged by communication complexity, early-stage uncertainty, inconsistent requirements, and inadequate automation support.
Method
The chapter analyzes LLM use across elicitation, specification, analysis, and validation through an LLM-driven RE process, SWOT analyses, and a preliminary evaluation.
Results
The chapter concludes that LLMs can enhance RE tasks by automating, streamlining, and augmenting human capabilities, while presenting their own threats.
Takeaways & Limitations
LLMs should complement human RE expertise because domain nuance, context, biases, inaccuracies, and limited context length constrain unchecked use.
Takeaways & Limitations
LLM outputs require human validation because over-automation can overlook crucial requirements or constraints, and domain expertise remains necessary for prioritization and compliance.
Abstract
from arXiv · showhide
Requirements Engineering (RE) is a critical phase in software development including the elicitation, analysis, specification, and validation of software requirements. Despite the importance of RE, it remains a challenging process due to the complexities of communication, uncertainty in the early stages and inadequate automation support. In recent years, large-language models (LLMs) have shown significant promise in diverse domains, including natural language processing, code generation, and program understanding. This chapter explores the potential of LLMs in driving RE processes, aiming to improve the efficiency and accuracy of requirements-related tasks. We propose key directions and SWOT analysis for research and development in using LLMs for RE, focusing on the potential for requirements elicitation, analysis, specification, and validation. We further present the results from a preliminary evaluation, in this context.
1 Introduction
Requirements Engineering is critical but difficult because teams face resource constraints, ambiguity, inadequate skills and tools, producing costly downstream requirement problems. The chapter examines how LLMs could support RE tasks and presents a SWOT analysis and preliminary evaluation.
- RE identifies, analyzes, and documents system needs and constraints to create a well-defined set of requirements.
- Resource constraints, inadequate training, early-stage uncertainty, weak automation, and implementation-first practices can cause teams to cut corners in RE.
- Inconsistent, incomplete, and incorrect requirements can increase development costs, cause delays, and create later-stage difficulties.
- The chapter investigates LLMs for requirements elicitation, analysis, specification, and validation through research directions, SWOT analysis, and preliminary evaluation.
2 LLMs-driven RE Process
The proposed LLM-driven RE process organizes RE into elicitation, specification, analysis, and validation, with LLM agents interacting with stakeholders through prompting. Its implementation depends on the domain and project context.
- LLMs-driven RE Process: The RE process comprises elicitation, specification, analysis, and validation, covering domain understanding, documentation, evaluation, negotiation, and quality assurance.
- LLMs-driven RE Process: The exact LLM instantiation and contextualization depend on the problem domain and project, such as differences between ActApp and safety-critical systems.
- LLMs-driven RE Process: LLMs can interact with stakeholders and with one another to elicit, specify, negotiate, analyze, and validate requirements and generate quality-assurance artefacts.
- LLMs-driven RE Process: Prompt engineering designs and tests prompts using task, language, and model knowledge to improve output quality.
3 Requirements Elicitation
LLMs can support elicitation by processing domain information, identifying unknowns, translating terminology, simulating stakeholder interactions, and generating candidate requirements. The chapter emphasizes human oversight because domain nuance, privacy, bias, and model updates constrain these uses.
- Requirements Elicitation: Requirements elicitation identifies and documents project information, system needs, expectations, and constraints through activities including domain, stakeholder, and feasibility analysis.
- Role of LLMs: LLMs can process domain literature, identify gaps and ambiguities, support as-is and regulatory analysis, and help structure domain knowledge.
- Role of LLMs: LLMs can translate technical jargon, support multilingual stakeholders, analyze feedback, and simulate persona-based user journeys when suitable resources are available.
- Role of LLMs: Human analysts remain necessary because LLMs lack empathy and nuanced domain expertise, while outputs may contain bias, privacy risks, contextual misinterpretations, and update-related inconsistencies.
- Role of LLMs: Persona-based prompts can ask an LLM to act as an ActApp user and generate uniquely identified requirements with rationales.
- Role of LLMs: LLMs may conduct virtual stakeholder interviews and synthesize information from forums, social media, and reviews to identify needs and concerns.
4 Requirements Specification
LLMs can transform unstructured requirements into structured specifications, classify and prioritize them, and cross-reference standards. However, domain experts must validate outputs because over-automation can overlook critical requirements and constraints.
- Requirements Specification: Requirements specification converts elicited information into structured documentation that serves as a blueprint for system design and implementation.
- Role of LLMs: LLMs can format requirements into EARS or user-story templates, classify functional and non-functional requirements, and generate supporting artefacts.
- Role of LLMs: LLMs can suggest prioritization using technical dependencies, project goals, and historical data.
- Role of LLMs: Prioritization, critical non-functional requirements, and compliance decisions require expert validation because LLM outputs may be inaccurate or over-automated.
- Role of LLMs: An EARS prompt converts an unformatted ActApp requirement into a state- or event-oriented requirement such as a condition under which notifications are withheld.
- Role of LLMs: LLMs can generate refined user stories and cross-reference non-functional requirements, including patient-data privacy requirements, with standards and guidelines.
5 Requirements Analysis
Requirements analysis evaluates and refines requirements for quality before design and implementation, while LLMs can automate quality checks, risk identification, and countermeasure proposals. However, their limited domain understanding, ambiguity handling, and context window constrain reliability.
- Requirements analysis evaluates and refines requirements for coherence, comprehensiveness, and attainability before design and implementation.
- LLMs can automate quality evaluation by flagging ambiguity, vague terms, inconsistencies, incompleteness, gaps, and overlaps.
- LLMs equipped with domain knowledge can identify risks and propose new requirements as countermeasures.
- In ActApp, LLM analysis identified missing notification timing and GDPR-related data-retention and deletion information.
- LLMs offer consistent, automated analysis and can use historical data, but may miss nuanced domain context, misinterpret ambiguity, and lose context across large documents.
6 Requirements Validation
Requirements validation checks whether documented requirements represent stakeholder needs and are ready for later development. LLMs can support review, stakeholder-perspective simulation, traceability, acceptance criteria, and test scenarios, but struggle with tacit knowledge and overall project context.
- Validation reviews requirements for defects, stakeholder alignment, traceability, acceptance criteria, and test scenarios.
- LLMs can flag ambiguities, inconsistencies, and violations during manual review using predefined validation heuristics.
- LLMs can simulate stakeholder perspectives to anticipate misinterpretations and identify clarification needs.
- LLMs can support traceability to design documents and regulatory codes, formulate acceptance criteria, and propose test scenarios.
- Validation remains constrained because LLMs struggle to process tacit knowledge and the overall project, domain, and stakeholder context.
- In ActApp, LLMs generated acceptance criteria and uncovered a need to comply with Australia’s Therapeutic Goods Act.
7 Preliminary Evaluation
The preliminary ActApp evaluation examined whether ChatGPT could support requirements elicitation through participant interactions and expert comparison. Results were categorized by match quality, with the evaluation emphasizing feasibility rather than comprehensive assessment.
- The evaluation focused on the feasibility of integrating LLMs into requirements elicitation rather than comprehensively assessing LLMs across RE.
- Four participants used ChatGPT during a 45-minute session to elicit user-story-style requirements from a project brief.
- An expert identified 20 key ActApp user requirements and mapped participant outputs as full, partial, or no matches.
- Full matches counted as 1 true positive and partial matches as 0.5 true positives when calculating precision and recall.
- The evaluation results highlighted the significance of participant experience when using ChatGPT for elicitation.
8 Lessons Learned
The lessons learned emphasize that LLM output depends strongly on prompts, context, and RE experience, while LLMs may surface previously unrecognized requirements. The evaluation also reports inaccuracies, context limitations, domain-knowledge gaps, and automation-bias risks.
- Comprehensive prompts and contextual information are important because slightly different prompts can produce very different outputs.
- Experienced requirements engineers obtained better results by formulating prompts, interpreting responses, and eliciting quality output.
- The evaluation found four potentially relevant requirements not included in ActApp’s original set, although false positives may overload engineers.
- Participants reported context-maintenance problems and inaccuracies, while limited domain knowledge can hinder correct and complete requirements.
- Automation bias may lead users to place unfounded confidence in LLM-generated requirements.
9 Conclusion
LLMs could enhance RE by automating and augmenting tasks across its stages, but their use remains constrained by domain, context, and human-centered challenges.
- LLMs could automate, streamline, and augment RE tasks across elicitation, specification, analysis, and validation.The chapter highlights stakeholder simulation, alternative-requirement generation, quality improvement, standards cross-referencing, and structured documentation.
- LLMs are not a universal solution because they may struggle with deep domain nuances and overall context.
- Unchecked automation and over-specification can undermine the human-centric view of requirements.
- The chapter also reports lessons learned from applying LLMs to RE for the ActApp app for T2D patients.