When Stakeholder-centric Requirements Engineering is Not Enough: An Action Research Study on Legacy System Modernisation

I. INTRODUCTION

• Legacy modernisation requires decisions about future system needs and which existing functions to retain, adapt, or replace, making it an RE as well as a technical problem.The introduction frames this as a socio-technical challenge shaped by stakeholder perspectives.Source: §I · p. 1
• The study tested whether elicited and consolidated stakeholder knowledge could yield target-system requirements suitable for comparison with the system-as-is.The case involved a multinational energy company renewing an emissions-reporting application across related systems in different countries.Source: §I · pp. 1–2
• Stakeholder-centric RE produced requirements stakeholders generally considered understandable and correct, but substantial disagreement remained about which requirements the legacy systems implemented.The disagreement pointed to fragmented understanding of the existing system.Source: §I · p. 2
• The intervention used semi-structured elicitation interviews, Delphi consensus-building, and MoSCoW prioritisation.These practices were applied as stakeholder-centric RE techniques in the modernisation effort.Source: §I · p. 2
• The findings indicate that stakeholder-centric RE alone may not establish a shared, verifiable account of legacy functionality, motivating technical evidence to complement stakeholder perspectives.The paper points to software analytics that triangulate stakeholder accounts with evidence from the system, where documentation is often scarce.Source: §I · p. 2
• The paper contributes case-based evidence on requirements quality and the limits of RE practices for reliable reuse assessment, alongside lessons learned and reflections.It also calls for closer collaboration between RE, mining software repositories, and software architecture communities on the socio-technical problem.Source: §I · p. 2

a) Reverse engineering and restoration

• Early reverse-engineering and restoration work offered foundational renewal ideas, but the reviewed contributions provided limited process guidance for contemporary large-scale renewals.Examples include iterative renewal decisions and stepwise replacement patterns that sustain operations.Source: §II · p. 2

b) Cloud Migration and Microservices

• Cloud migration and microservices research offers technical frameworks, roadmaps, and artifact-driven analyses, often emphasizing restructuring rather than stakeholder-centred RE.The review describes cloudification as a common modernisation approach and notes context-dependent refactoring choices.Source: §II · p. 2

c) Industrial Case Studies

• Industrial case studies address re-implementation, modernisation cost assessment, and model-based reconstruction of legacy knowledge from artifacts.One reviewed reverse-engineering approach reported average savings of 75% in effort to understand system construction and technological debt analysis, and 99% cost improvement in documentation production versus traditional approaches.Source: §II · p. 3

d) Surveys

• Surveys report that published modernisation work emphasizes operational costs and tool support, while practitioner accounts also foreground fragmented expertise, legacy-business-logic understanding, and communication issues.The review contrasts literature-review findings with socio-technical challenges identified in practitioner-oriented work.Source: §II · p. 3

e) Research Gap

• Previous work largely treated modernisation as architecture or refactoring, with relatively little explicit RE focus; the cited Khadka et al. case gave stakeholders a prominent role but lacked detailed reproducible steps and structured RE practices.The authors characterize studies explicitly addressing modernisation from an RE perspective as scarce.Source: §II · p. 3
• This study examines whether stakeholder-centric RE can support modernisation and finds that its applied practices produced high-quality requirements but not a reliable legacy-to-target comparison.The contribution connects requirements quality with the separate challenge of determining legacy-system support.Source: §II · p. 3

III. RESEARCH DESIGN

• The intervention diagnosed a need to elicit and specify target-system requirements and compare them with legacy functionality to determine reuse.The study defined gap analysis as assessing whether target requirements were fully, partially, or not supported by legacy features.Source: §III · pp. 3–4
• Action planning selected stakeholder-centric RE practices, formulated research questions, and defined procedures and metrics for evaluating their effectiveness.The plan aimed to produce requirements specifications usable for gap analysis.Source: §III · p. 4
• The practices were applied over six months with company stakeholders, producing a requirements specification that was evaluated through stakeholder questionnaires.The evaluation assessed requirement quality and performed gap analysis; lessons learned were discussed in a final plenary workshop.Source: §III · p. 4

IV. DIAGNOSIS: MODERNISATION AS A RE PROBLEM

• The reporting applications validated time-series data and derived reportable quantities, but were end-of-life and increasingly inadequate for current business needs.The company sought renewal and possible harmonisation across countries after migrating the relevant reporting data to cloud infrastructure.Source: §IV · p. 4
• The organisation lacked a systematic basis for deciding which legacy functions to retain, change, or introduce because the target situation and its differences from existing functionality were unclear.The problem was framed as eliciting and formalising target needs, then using the specification to support gap analysis.Source: §IV · p. 4

V. ACTION PLANNING

• The intervention aimed to produce requirements that support gap analysis between the system-as-is and system-to-be in a multinational modernisation context.Stakeholder-centric RE was chosen to centre stakeholder goals and knowledge rather than simply reproduce existing functionality.Source: §V · p. 5
• The selected sequence covered stakeholder identification, semi-structured interviews and thematic analysis, Delphi-based consensus, user-story specification, and MoSCoW prioritisation.The practices were opportunistically selected given the study's time constraints and were not intended as a prescriptive RE lifecycle.Source: §V · p. 5
• RQ1 asked whether stakeholders judged the resulting requirements understandable and correct, while RQ2 asked whether they judged legacy support consistently.The evaluation strategy operationalised the questions through stakeholder assessments of requirement quality and legacy-system coverage.Source: §V · p. 5
• Requirement quality was assessed through stakeholder questionnaire judgements of understandability and whether each requirement addressed a need.Understandability and correctness were selected as the quality dimensions considered most relevant to the company's renewal objective.Source: §V · p. 5
• Stakeholders classified each requirement as fully, partially, or not covered by current systems, and the analysis used Krippendorff’s Alpha plus descriptive proportions to assess outcomes.The analysis combined agreement measures with proportions of requirements judged understandable, correct, or supported by legacy systems.Source: §V · p. 6

• The selected RE practices were applied in sequence, from stakeholder identification through requirements specification and prioritisation.Source: §VI · p. 6

A. Identification of Stakeholders

• Fourteen stakeholders were identified across functional-beneficiary, technical, and commercial roles, including external technical actors with legacy-system knowledge.Management discussions added a solution designer, solution architect, legacy system manager, and developer representative.Source: A. Identification of Stakeholders · p. 6

B. Requirements Elicitation Interviews and Workshop

• Semi-structured interviews elicited role-specific perspectives on current work, business goals, technical data flow, and expectations for the renewed system.Interview guides varied by stakeholder role while covering expectations for renewal and characteristics of the current system.Source: B. Requirements Elicitation Interviews… · pp. 6–7
• Thematic analysis organised interview material into seven functionality groups and produced 40 themes representing recurring needs and concerns.The groups covered data ingestion, validation, calculation, export, plant characteristics, data analytics, and general topics.Source: B. Requirements Elicitation Interviews… · p. 7
• Fourteen stakeholders discussed each theme through three Delphi-based SWOT rounds to validate interpretations, harmonise understanding, and refine subsequent work.Themes were not formalised as user stories during the workshop, though its interactive format supported shared understanding.Source: B. Requirements Elicitation Interviews… · p. 7

• The final dataset contained 105 requirements organised into 16 topics, or Agile epics.Requirements were formalised as user stories following a role, action, goal, and how format.Source: C. Requirements Specification · p. 7

D. Requirements Prioritisation

• Each stakeholder rated every requirement as Must, Should, Could, or Won’t to identify aspects for early elaboration toward the envisioned MVP.Source: D. Requirements Prioritisation · p. 7
• Developers and architects did not rate requirements, while Swedish users did not complete prioritisation and only two of three management representatives responded.The reported response rate was around 58%.Source: D. Requirements Prioritisation · pp. 7–8
• Ratings were weighted by stakeholder group; one German user’s responses were counted four times to balance representation against four Dutch users.Source: D. Requirements Prioritisation · p. 7
• At least 80% of stakeholders agreed on the precise priority for fewer than 20% of requirements, and filtering out those with fewer than half Must-or-Should votes left 56 MVP requirements.Source: D. Requirements Prioritisation · p. 8

A. RQ1: Requirements Quality

• Under the lenient interpretation, a majority judged all requirements understandable and correct; under the tight interpretation, 80% were understandable and 66% correct.Agreement alpha was 0.94 for understandability and 0.90 for correctness.Source: A. RQ1: Requirements Quality · p. 8

B. RQ2: Gap-analysis consistency

• Agreement on functionality mapping was low (alpha 0.45); no requirement had unanimous mapping agreement, and only 50% were mapped consistently enough for majority interpretation under the lenient threshold.The requirements supported requirement-by-requirement assessment, but the assessment was not sufficiently consistent to support strong conclusions about reuse potential.Source: B. RQ2: Gap-analysis consistency · pp. 8–9
• Stakeholders differed on legacy coverage, while prioritisation agreement was reached for only 20% of requirements despite a Delphi workshop intended to build shared understanding.The authors identify fragmented views of existing functionality as the main limitation, rather than requirements understandability.Source: B. RQ2: Gap-analysis consistency · p. 8
• The legacy system was generally not considered to include most envisioned functionality, suggesting unmet needs but possibly reflecting elicitation focused on novel requirements.The authors note that this focus may have led stakeholders to emphasize what was missing rather than envisioning the complete system.Source: B. RQ2: Gap-analysis consistency · pp. 8–9

Stakeholder Needs and the System-as-is

• The intervention produced requirements stakeholders largely considered understandable and correct, but agreement on legacy functionality mapping was weaker, revealing fragmented knowledge of what the systems already provided.The discussion argues that modernisation first requires a shared understanding of existing systems, while noting that this study began with stakeholder needs rather than the status quo.Source: §VIII · p. 9
• Legacy elicitation should reconstruct existing functionality and the needs it already satisfies, including why behaviours exist, so obsolete features can be recognized.The proposed two-step interview process uses one script for the system-as-is, including strengths and weaknesses, and another for novel system-to-be requirements.Source: §VIII · p. 9

From Stakeholder-centric RE to Evidence-based RE

• When stakeholder views diverge and legacy knowledge has eroded, reports, configurations, process traces, documentation, and implementation artefacts can complement reconciliation.The discussion presents triangulation as a way to move toward a more evidence-based, collectively validated understanding for modernisation decisions.Source: §VIII · p. 9

The Role of Automation and Large Language Models (LLMs)

• The discussion identifies feature location, specification mining, process mining, and architecture recovery as system-understanding techniques that can complement stakeholder input.It also points to automation and LLMs as potential supports for analysing large, layered legacy systems.Source: §VIII · pp. 9–10
• The authors recommend closer collaboration between RE, software-repository mining, and software-architecture communities to develop evidence-based modernisation strategies.They also suggest exploiting automation, including architecture recovery, specification mining, and process mining.Source: §VIII · p. 10

Stakeholder Unavailability and Insufficient Evidence

• Stakeholder availability and system evidence were limited in the case, and disagreement persisted despite repeated interaction, motivating explicit treatment of knowledge uncertainty.The authors suggest recording both stakeholder agreement and evidence-support scores for elicited information.Source: §VIII · p. 10

The Need for Agile Renewal

• The discussion proposes iterative, incremental renewal with representative participation, progressive inclusion, small functionality slices, and reprioritization as understanding changes.A living backlog could track requirements, ambiguities, evidence needs, and candidate migration slices; conflicting accounts should be checked against system evidence when available.Source: §VIII · p. 10

Internal Validity

• Because the industrial action-research intervention combined several RE practices, the effect of any individual practice cannot be isolated; activity order and varying stakeholder participation may also have influenced outcomes.The practices included stakeholder identification, interviews, thematic analysis, prioritisation, and questionnaire-based evaluation.Source: §IX · p. 10

External Validity

• The study covers one modernisation effort in one multinational energy company and should not be assumed to generalise directly beyond similar contexts.The authors characterize its contribution as analytical rather than statistical generalisation.Source: §IX · p. 10

Construct Validity

• Effectiveness was assessed through perceived requirements quality and requirements-based gap analysis, while quality focused on understandability and correctness rather than all possible attributes.Coverage judgments capture whether requirements were fully, partially, or not supported, but not broader process, organisational, or architectural gaps; agreement scores reflect consistency in this setting rather than a theory-neutral measure of truth.Source: §IX · pp. 10–11

X. CONCLUSION

• Stakeholder-centric RE produced requirements stakeholders largely considered understandable and correct, but using them to assess existing legacy coverage exposed substantial disagreement and fragmented system understanding.The conclusion calls for automated techniques combined with human judgement to understand the system-as-is.Source: §X · p. 11
• Participants considered stakeholder-centric RE a non-negotiable starting point because it made ideas explicit and revealed an agreement and knowledge problem.The conclusion says RE should combine stakeholder and system knowledge, possibly through automated methods.Source: §X · p. 11
• Legacy-system RE should progressively reduce uncertainty and build a sufficiently evidence-based basis for modernisation decisions rather than resolve renewal in one pass.The study evaluated requirements by asking stakeholders to assess requirements quality and perform gap analysis.Source: §X · p. 11