Source-linked AI summary
Naming the Pain in Requirements Engineering: Contemporary Problems, Causes, and Effects in Practice
D. Méndez Fernández, S. Wagner, M. Kalinowski, M. Felderer, P. Mafra, A. Vetrò, T. Conte, M. -T. Christiansson, D. Greer, C. Lassenius, T. Männistö, M. Nayabi, M. Oivo, B. Penzenstadler, D. Pfahl, R. Prikladnicki, G. Ruhe, A. Schekelmann, S. Sen, R. Spinola, A. Tuzcu, J. L. de la Vara, R. Wieringa
TL;DR
RE is important but difficult to study empirically because it is uncertain, interdisciplinary, and context-dependent. The paper uses the NaPiRE family of replicated practitioner surveys to analyze RE problems, their criticality, and their causes and effects. Across 228 companies in 10 countries, it identifies contemporary problem patterns and provides an empirical basis for problem-driven RE research, while acknowledging limits in respondent coverage and survey-based perceptions.
Problem
RE lacks comprehensive empirical evidence about which practical problems exist, how critical they are, and how they relate to context, causes, and effects.
Method
The study analyzes NaPiRE survey data using predefined RE problems, structured relevance and criticality questions, and qualitative open and axial coding of causes and effects.
Results
The analysis of 228 companies across 10 countries identifies critical RE problems, context-dependent problem patterns, and reported causes and effects surrounding the most critical problems.
Takeaways & Limitations
The findings provide an empirical foundation for aligning RE research and organisational prevention efforts with practically recurring problems and their reported causes.
Takeaways & Limitations
The results are based on a limited and unevenly distributed respondent sample, and surveys reveal stakeholder perceptions rather than empirically backed-up knowledge of practices.
Abstract
from arXiv · showhide
Requirements Engineering (RE) has received much attention in research and practice due to its importance to software project success. Its interdisciplinary nature, the dependency to the customer, and its inherent uncertainty still render the discipline difficult to investigate. This results in a lack of empirical data. These are necessary, however, to demonstrate which practically relevant RE problems exist and to what extent they matter. Motivated by this situation, we initiated the Naming the Pain in Requirements Engineering (NaPiRE) initiative which constitutes a globally distributed, bi-yearly replicated family of surveys on the status quo and problems in practical RE. In this article, we report on the qualitative analysis of data obtained from 228 companies working in 10 countries in various domains and we reveal which contemporary problems practitioners encounter. To this end, we analyse 21 problems derived from the literature with respect to their relevance and criticality in dependency to their context, and we complement this picture with a cause-effect analysis showing the causes and effects surrounding the most critical problems. Our results give us a better understanding of which problems exist and how they manifest themselves in practical environments. Thus, we provide a first step to ground contributions to RE on empirical observations which, until now, were dominated by conventional wisdom only.
1 Introduction
Requirements engineering is important but difficult to investigate because it is uncertain, interdisciplinary, and context-dependent. The NaPiRE initiative addresses weak empirical grounding by analyzing practitioners’ problems and their causes and effects.
- 1 Introduction: RE remains difficult to standardise because uncertainty, interdisciplinary influences, and socio-economic and process factors vary across industrial settings.These factors also make RE difficult to investigate and improve.
- 1 Introduction: Empirical evidence in RE is weak, with isolated and small-scale studies offering limited generalisability and uncertain practical impact.Consequently, contributions are often not demonstrably aligned with practitioners’ problems.
- 1 Introduction: NaPiRE is a globally distributed, bi-yearly family of practitioner surveys examining RE status quo, problems, causes, and effects.Its objective is to establish an open knowledge base for understanding practical RE and supporting a holistic theory.
- 1 Introduction: The article contributes empirical analyses of practitioners’ RE problems, their criticality for project failures, and the causes and effects of the most critical problems.It also considers context factors such as whether development is agile or plan-driven.
- 1 Introduction: The study analyzes contemporary RE problems and their criticality, including causes and effects, using data from 228 companies across 10 countries.The analysis is intended to provide a basis for problem-driven RE research.
2 Related Work and Background
Prior RE research examines techniques, practices, and problems, but lacks a large, replicated, worldwide analysis linking practical problems with their causes. NaPiRE extends earlier survey work through international replication and comprehensive analysis.
- 2 Related Work and Background: Existing RE survey research covers techniques, methods, general practices, and contemporary issues, but these areas have rarely been combined comprehensively.The literature includes studies of technique effects and investigations of problems and causes.
- 2 Related Work and Background: Earlier industry surveys addressed project-failure causes, organisational practices, or selected RE problems, but did not support a broad analysis of contemporary RE phenomena.Several studies were geographically or substantively limited.
- 2 Related Work and Background: NaPiRE addresses the gap of large, replicated, worldwide analysis of RE problems together with their causes in practice.Previous work had not provided this combination at the required scale.
- 2 Related Work and Background: NaPiRE uses bi-yearly international surveys to build a broader empirical account of industrial RE and observe trends over time.The initiative expanded from an initial Germany-Netherlands run to a second run in 10 countries.
- 2 Related Work and Background: The article extends prior NaPiRE work by analyzing the complete international dataset, including problems, causes, and effects.Earlier publications focused on one or two countries or tested cause-effect analysis on subsets of the data.
- 2 Related Work and Background: Earlier Brazil-Germany analysis found human interactions among dominant factors across countries, project types, and company types.It also observed more standardised development models in Brazil and more plan-driven RE in Germany.
3 Study Design
NaPiRE uses replicated survey research to study RE problems, their contexts, and their causes and effects. The design combines structured problem assessment with qualitative coding of open responses, while recognising survey and coding limitations.
- 3 Study Design: NaPiRE conducts bi-yearly, globally distributed surveys to investigate industrial RE status quo and problems across context factors.The questionnaire is continuously adapted in response to earlier data.
- 3 Study Design: The study asks which RE problems practitioners experience, how critical they are, how they relate to context, and what causes and effects they have.The research questions guide the study design.
- 3.2 Instrument: The instrument combines open and closed questions, five-point ordinal Likert scales, multiple-choice responses, free-text options, and conditional question paths.Conditional logic omits irrelevant follow-up questions based on prior responses.
- 3.2 Instrument: The problem analysis presents 21 literature-derived RE problems and asks respondents to assess their relevance and select the five most critical problems.Follow-up questions address causes, effects, and mitigation strategies for selected problems.
- 3.3 Data Collection: Data collection used invitation-only, anonymous surveys coordinated independently by country representatives across participating companies and business units.Collection phases varied among countries and sometimes occurred in multiple tranches.
- 3.4 Data Analysis: Qualitative analysis used open coding to develop saturated categories and axial coding to identify relationships such as causes and consequences.The analysis combines predefined RE-problem categories with codes emerging from free-text answers.
4 Study Results
The survey identifies recurring requirements-engineering problems, with their frequency and criticality varying across organisational and process contexts. Cause-effect analyses connect prominent problems to organisational, project, customer, and product consequences.
- Problems and patterns: More than 30% of respondents cited each of the first five critical RE problems, although occurrence frequency did not always match perceived contribution to project failure.Communication flaws between the project team and customer were associated with project failure more often than incomplete and/or hidden requirements despite occurring less frequently.
- Problems and patterns: Incomplete and/or hidden requirements was the most cited problem overall and consistently ranked among the top three across organisational and process clusters.Communication flaws between the project team and customer were also among the three most cited problems in nearly all clusters.
- Contextual patterns: Moving targets affected all plan-driven and large agile organisations, with reported causes including weak customer-side change management, unclear business vision, and poorly written requirements.For large agile organisations, the free-text responses did not provide a clear explanation, although larger and more complex projects were suggested as a possible context.
- Contextual patterns: Time boxing appeared mainly in agile and small organisations, where bad estimations, unrealistic release promises, and frequent scope changes were reported as related reasons.Sales and marketing were specifically blamed for promising unrealistic dates or releasing early to customers.
- Causes: The main reported causes were lack of time, lack of RE-team experience, and weak RE-team qualification, while specific causes were associated with particular problems.Lack of time was mainly linked to time boxing; lack of experience to incomplete or hidden and underspecified requirements; and excessive team distribution to team communication flaws.
- Causes and effects: Cause-effect analyses linked incomplete or hidden requirements and customer-team communication flaws to time overruns, poor product quality, customer dissatisfaction, and post-implementation rework.For incomplete or hidden requirements, project or organisational effects represented approximately 43% of reported effects; for customer-team communication flaws, they represented approximately 47%.
5 Conclusions
The study uses NaPiRE survey data to identify critical requirements-engineering problems, their context-dependent patterns, and their perceived causes and effects. Its findings support problem-driven research and practice while remaining bounded by sampling, survey-perception, coding, and instrument-design limitations.
- Problems in RE (RQ 1): 228 companies across 10 countries identified incomplete or hidden requirements, communication flaws with customers, and moving targets as the most cited problems and those most related to project failure.The analysis also considered organisational size, process model, causes, effects, and manifestations beyond simplified project-failure views.
- Problem Patterns (RQ 2): Problem patterns varied by process model and organisational size: incomplete or hidden requirements appeared across clusters, while moving targets occurred mainly in plan-driven and large organisations.Communication flaws with the customer were not a major problem for small plan-driven organisations.
- Cause-Effect Analysis (RQ 3): Reported causes differed across context blocks and influenced specific problems unevenly, indicating that problem prevention should account for organisational and process context.The analysis identified the most reported causes and examined their influence on the most critical requirements-engineering problems.
- Cause-Effect Analysis (RQ 3): For incomplete or hidden requirements, reported causes included weak qualification or experience, time pressure, limited business vision, poor elicitation, abstract specifications, and missing completeness checks.Suggested countermeasures included training, improving elicitation, adjusting portfolio management, and strengthening completeness checks through inspections or Definition of Ready practices.
- Impact and Implications: The findings provide empirically grounded problem patterns and possible success factors for steering research, maturity models, improvement efforts, and practitioners’ assessment of their RE situation.The analysis extends prior evidence with causes, effects, context dependencies, and effects beyond project failure.
- Limitations: Generalisability is limited by the respondent population, uneven domain and system-family distribution, reliance on stakeholder perceptions, subjective qualitative coding, and potentially oversimplified instrument categories.The authors call for continued replications and complementary methods such as interviews and case studies.
A List of Contributors and Roles
The paper identifies contributor roles spanning project coordination, methodology, data work, visualization, writing, and review, with roles distributed across the initiative and manuscript.
- NaPiRE distinguishes roles for conceptualisation, project administration, methodology, instrument design, and data collection.
- Data analysis and data curation cover interpreting survey data and managing annotated research data for initial and later reuse.
- Data visualisation concerns presenting results, while original-draft writing covers preparing the initial manuscript draft.
- Review and editing covers critical review, commentary, or revision, including pre- and post-publication stages.
- The first two authors initiated and coordinated NaPiRE, while the first six authors led data analysis and/or writing for this manuscript.
- Roles marked with an asterisk follow the exact classification introduced in the cited prior work.