Source-linked AI summary
Requirements Engineering for Machine Learning: Perspectives from Data Scientists
Andreas Vogelsang, Markus Borg
TL;DR
The paper examines how requirements engineering should adapt to machine-learning systems, using interviews with four data scientists to explore requirements practices. It finds that ML's shift from coding to training introduces new requirements challenges, including performance measures, explainability, discrimination, and legal constraints.
Problem
The paper addresses the open question of whether machine learning's development paradigm requires requirements engineering to adapt beyond conventional software practices.
Method
The authors interviewed four data scientists about how ML experts elicit, specify, and assure requirements and expectations.
Results
The study indicates that requirements engineering for ML systems must address performance measures, explainability, freedom from discrimination, and specific legal requirements.
Takeaways & Limitations
Requirements engineers should connect ML techniques and performance concepts to stakeholder needs and domain context.
Takeaways & Limitations
Interviewee behavior may have been affected by the presence of the interviewer, and this reactivity could influence the observations.
Abstract
from arXiv · showhide
Machine learning (ML) is used increasingly in real-world applications. In this paper, we describe our ongoing endeavor to define characteristics and challenges unique to Requirements Engineering (RE) for ML-based systems. As a first step, we interviewed four data scientists to understand how ML experts approach elicitation, specification, and assurance of requirements and expectations. The results show that changes in the development paradigm, i.e., from coding to training, also demands changes in RE. We conclude that development of ML systems demands requirements engineers to: (1) understand ML performance measures to state good functional requirements, (2) be aware of new quality requirements such as explainability, freedom from discrimination, or specific legal requirements, and (3) integrate ML specifics in the RE process. Our study provides a first contribution towards an RE methodology for ML systems.
I. INTRODUCTION
The paper examines whether ML’s shift from manually coded rules to training data requires requirements engineering to evolve. Interviews with four data scientists identify performance measures and new quality requirements as central RE concerns.
- ML engineering shifts behavior specification from manually coded rules toward rules generated from training data and a fitness function.
- The paper argues that RE must evolve because data scientists make key decisions about fitness functions, data, and quality assurance.
- Requirements engineers should understand quantitative ML measures to specify functional requirements for predictive performance.
- New ML-related requirements include explainability and freedom from discrimination.
A. Machine Learning and Software Engineering
ML systems learn features from data rather than implementing them entirely in source code, creating interpretation and assurance challenges. Existing RE and data-mining approaches provide related foundations but leave RE for ML systems insufficiently developed.
- ML is organized into supervised, unsupervised, and reinforcement learning, with supervised learning requiring high-quality labelled data.
- ML learns system features from training data, while conventional software implements features manually in source code.
- ML models can be opaque and intrinsically difficult to interpret, limiting the applicability of source-code reviews and exhaustive coverage testing.
- Prior work addresses ML for RE tasks and data-analytics modeling, but little work details RE for ML systems.
- Unlike self-adaptive systems, supervised ML systems are trained on data but may not continuously monitor or retrain after deployment.
III. STUDY DESIGN
The study investigates how data scientists handle requirements for ML systems and which requirements-related challenges they encounter. It uses three research questions covering practices, processes, and challenges.
- RQ1 asks how data scientists elicit, document, and analyze requirements for ML systems.
- RQ2 asks which processes data scientists follow and which process parts relate to requirements.
- RQ3 asks what requirements-related challenges data scientists face.
A. Research Method
The research uses an exploratory qualitative design based on semi-structured interviews with four data scientists. Participants discussed their projects, ML roles, and how expectations are elicited, communicated, documented, and tested.
- The researchers used semi-structured interviews whose guide allowed question order to follow the conversation’s natural flow.
- Interview questions explored participants’ backgrounds, typical projects, ML’s role, and practices for handling expectations.
- Four anonymous data scientists from research and industry were interviewed for approximately one hour each.
- The authors collaboratively coded interviews, with each author reviewing and validating the other’s coding.
D. Threats to Validity
The qualitative interview study identifies several validity threats and describes measures used to reduce researcher, interpretation, and reactivity concerns.
- The study design faces five qualitative-research validity threats identified by Maxwell.
- Recording interviews and linking transcript annotations to recordings supported traceability of the collected data.
- Explaining the study goal beforehand and reviewing the interview guide helped reduce interpretation errors and improve reliability.
- The researchers limited researcher bias by remaining open to interview outcomes and cross-validating annotated codes.
- Reactivity could not be eliminated, although the researchers acknowledged its possible influence on observations.
B. Explainability
Explainability is an important ML quality requirement involving both what a model learned and why it made individual predictions. Requirements work should also identify when users need explanations.
- Explainability covers explaining the model’s learned behavior and explaining individual predictions.
- Simpler techniques such as decision trees and Naive Bayes are easier to comprehend than neural networks.
- Customers may favor explainable models even when constraining models to produce explanations decreases predictive power.
- Reducing the number of features can improve model explainability when performance remains similar.
- Requirements engineers should elicit which situations require explanations from the user’s perspective.
C. Freedom from Discrimination
Freedom from discrimination is presented as a new ML quality requirement constrained by social and legal acceptability. Requirements engineers must identify protected characteristics and connect data preparation, model analysis, explainability, and legal compliance.
- Freedom from discrimination means using only forms of discriminatory logic that are societally and legally accepted.
- ML discrimination can be implicit and can amplify bias present in training data, making it harder to identify than explicit rules.
- Requirements engineers must identify protected characteristics that the ML algorithm must not use to discriminate between samples.
- Teams can address discrimination by excluding protected characteristics during data preparation or analyzing important model features afterward.
- Feature analysis connects discrimination assessment with explainability by revealing attributes that strongly influence decisions.
- GDPR and other legal constraints require anticipated model needs, consent-aware data use, retraining after data removal, and predefined regulated processes.
- Requirements engineers must track legal requirements and ensure data lineage shows that illegal features did not influence the final training dataset.
E. Data Requirements
ML systems require explicit attention to training data as part of the system, including its quantity, quality, provenance, collection, formats, and ranges.
- Data Requirements: Training data needs specified and validated requirements, not merely algorithm selection, because it is integral to ML systems.The paper compares training data with source code in compilation and argues that data requires testing and validation practices.
- Data Requirements: Data quantity requirements should account for class representation and may include identifying additional sources to augment small or homogeneous datasets.More data is not automatically helpful; additional sources can help explain phenomena in the original data.
- Data Requirements: Data quality depends on dimensions such as completeness, consistency, and correctness, while collection practices can influence correctness.Interviewees emphasized cleaning and augmenting data before training and warned that public datasets may be less trustworthy when maintenance is weak.
- Data Requirements: Human labeling and incentive-driven collection can introduce bias or produce training data that does not reflect reality, undermining deployment usefulness.A model may perform well during development yet yield useless deployed results when data was entered to satisfy organizational incentives.
- Data Requirements: Requirements engineers should specify data collection, formats, ranges, and provenance based on the problem domain for use by data scientists.Data provenance requires critically questioning data sources.
V. RE PROCESS FOR ML SYSTEMS
The RE process for ML systems must incorporate data sources, ML performance measures, and exploratory data analysis while coordinating domain, data-science, and legal perspectives.
- Elicitation: Stakeholder analysis should identify potentially relevant additional data sources, with data scientists and legal experts involved from the beginning.Additional data may improve the quantity and quality of core data.
- Analysis: Requirements analysis should define how the ML system will be assessed using performance measures appropriate to stakeholder needs and the problem domain.Requirements engineers must translate stakeholder demands into suitable measures and examine whether false positives and false negatives have equal costs.
- Elicitation: During exploratory data analysis, requirements engineers should facilitate discussions between customers and data scientists about data exclusion, processing, representation, and augmentation.The phase also examines data quality and how data must be enriched.
C. Specification
ML requirements specifications must strengthen data requirements and distinguish expected predictive performance on training data from desired performance during operations.
- Data Requirements: Specifications should identify data collection, formats, ranges, and the necessary quantity and quality of data.These data requirements are elicited from the problem domain and provide input to data scientists.
- Predictive Performance: Predictive power should be expressed using the performance measures established during requirements elicitation and analysis.Training-data performance can be checked after training, whereas runtime performance is assessed during operations.
- Quality Requirements: Specifications should state whether discrimination is critical and which characteristics must be protected from use in classification, alongside explainability requirements.Explainability requirements identify which situations and decisions need explanation.
D. Verification & Validation
Verification and validation for ML systems must account for changing data and continue during operation, while RE connects data-science techniques to customer needs and context.
- Verification & Validation: Because model behavior depends on training data, validation must ensure that training data corresponds to real data and continue throughout system operation.Runtime-data monitoring and analysis are considered essential for maintaining ML-system performance.
- Conclusions: The study presents RE for ML as a first step toward a methodology that includes ML-specific requirements and contextualizes technical decisions for stakeholders.The conclusion identifies explainability, freedom from discrimination, and specific legal requirements as examples of new requirement types.
- Conclusions: Requirements engineers should relate data-science methods for balancing, cleaning, validating, and explaining data to customers’ needs and context.Generic data-science guidelines address pitfalls and advice but do not replace this contextual relation.
- Future Work: The study’s future work includes broader interviews and examining ML-specific requirements in safety-critical and hybrid conventional-software contexts.Planned extensions include requirements engineers, cyber-physical-system engineers, and integration with RE for surrounding software.