Source-linked AI summary
Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systems
Hiroshi Kuwajima, Fuyuki Ishikawa
TL;DR
Traditional software-quality approaches do not directly fit AI systems whose behavior is obtained through ML training, while ethical requirements such as fairness and explainability are emerging. The paper analyzes how SQuaRE should be adapted to ML and the Ethics guidelines, identifying quality concepts and measurement gaps relevant to that adaptation.
Problem
Traditional quality approaches do not work effectively for behavior obtained through ML training, and AI systems bring requirements such as fairness and explainability.
Source: §I · p. 1Method
The paper examines SQuaRE concepts for ML-specific changes and maps the Ethics guidelines to identify quality concepts that should be added.
Source: §I · p. 1Results
The analysis identifies adaptations including domain-specific partitioning of fuzzy functions, fairness additions, and quality measures whose techniques are missing or still immature for ML systems.
Source: 1) Metrics with Large Fuzzy Evaluation… · p. 3, 2) Metrics with Missing Techniques · p. 4, 2) Fairness · p. 5Takeaways & Limitations
The paper offers holistic conceptual guidance for assessing AI quality by incorporating ML-specific concerns and AI ethics into SQuaRE.
Source: §I · p. 1Assessment items focused on internal implementations of data management are outside the scope of this paper.
Source: 6) Other Requirements · p. 6
Abstract
from arXiv · showhide
More and more software practitioners are tackling towards industrial applications of artificial intelligence (AI) systems, especially those based on machine learning (ML). However, many of existing principles and approaches to traditional systems do not work effectively for the system behavior obtained by training not by logical design. In addition, unique kinds of requirements are emerging such as fairness and explainability. To provide clear guidance to understand and tackle these difficulties, we present an analysis on what quality concepts we should evaluate for AI systems. We base our discussion on ISO/IEC 25000 series, known as SQuaRE, and identify how it should be adapted for the unique nature of ML and $\textit{Ethics guidelines for trustworthy AI}$ from European Commission. We thus provide holistic insights for quality of AI systems by incorporating the ML nature and AI ethics to the traditional software quality concepts.
I. INTRODUCTION
The paper argues that traditional software-quality approaches do not transfer directly to ML-based AI systems, whose trained, often black-box components can constitute core system functions. It proposes adapting SQuaRE both for ML-specific quality concerns and for ethical requirements such as fairness and explainability.
I. INTRODUCTION
- Evaluating only accuracy across a whole dataset is insufficient; assessment should be more granular, requirement-specific, and reflected in data design.Source: §I · p. 1
- SQuaRE cannot be applied to ML-based AI systems as-is because trained, parameter-rich components can be black-box, unexplainable core functions.A survey cited in the paper found that more than 40% of participants said existing approaches do not work for quality assurance of ML-based AI systems.Source: §I · p. 1
- AI systems raise distinctive ethical requirements, including human rights under AI autonomy, fairness, and explainability.Source: §I · p. 1
- The paper identifies SQuaRE updates by examining which concepts ML invalidates and which concepts the Ethics guidelines imply should be added.Its aim is to provide a holistic view incorporating ML-specific concerns and AI ethics at the level of quality characteristics and sub-characteristics.Source: §I · p. 1
A. SQuaRE
SQuaRE organizes software-quality concepts into hierarchical models and defines measures for evaluating sub-characteristics. The Ethics guidelines provide ethical principles, top-level requirements, and question-based assessment items that can be mapped to those concepts.
A. SQuaRE
- SQuaRE hierarchically organizes quality models, characteristics, and sub-characteristics, while quality measures specify mathematical evaluations using quality measure elements.For example, Mean time between failure uses operation time and the number of system/software failures.Source: A. SQuaRE · pp. 1–2
- The paper considers Product quality for development-time evaluation and Quality in use for run-time evaluation.Source: A. SQuaRE · p. 2
B. Ethics Guidelines for Trustworthy AI
- The Ethics guidelines define four ethical principles and seven requirements, with sub-requirements and question-based assessment items.The seven top-level requirements include human agency and oversight, technical robustness and safety, privacy and data governance, transparency, fairness, societal and environmental well-being, and accountability.Source: B. Ethics Guidelines for Trustworthy AI · p. 2
C. Related Work
Prior work has addressed AI quality through focused guidance and techniques, while the paper seeks to complement these efforts with holistic guidance. Its analysis checks existing SQuaRE measures for ML applicability and maps ethical assessment items to identify missing quality concepts.
C. Related Work
- Industry guidance and research have addressed testing, maintenance, data management, security, explainability, adversarial examples, and fairness, while software-engineering work has largely focused on testing and verification.Source: C. Related Work · p. 2
- The paper aims to complement these focused studies with holistic guidance for industry planning, decision-making, and management.Source: C. Related Work · p. 2
III. ANALYSIS METHODOLOGY
- For RQ1, the analysis exhaustively checks existing SQuaRE quality measure elements to find metrics that are not useful or applicable to ML-based AI systems.The concrete metric review is intended to expose gaps that broad, abstract quality characteristics may not reveal.Source: §III · p. 2
- For RQ2, the analysis exhaustively checks Ethics guidelines assessment items and maps their sub-requirements to SQuaRE characteristics to identify missing concepts.Source: §III · p. 2
A. Modifying Existing (Sub-)Characteristics in SQuaRE
The analysis identifies two reasons to adapt SQuaRE for ML-based AI: large fuzzy evaluation targets complicate successful-count measures, and some sub-characteristics lack effective measurement techniques. It proposes application-specific decomposition, data-set-aware measures, and explicit treatment of measurement uncertainty.
1) Metrics with Large Fuzzy Evaluation Targets:
- Large fuzzy ML functions make successful-count measures difficult to apply directly; Functional completeness, for example, counts missing versus specified functions.The paper interprets missing functions as those not successfully trained despite inclusion in the relevant data sets, and specified functions as those included in those data sets.Source: 1) Metrics with Large Fuzzy Evaluation… · pp. 2–3
- Application-specific decomposition of large fuzzy tasks, functions, or contexts is proposed for affected SQuaRE characteristics and sub-characteristics.The paper describes splitting data sets into partitions for fine-grained evaluation, incorporating domain experts and their knowledge.Source: 1) Metrics with Large Fuzzy Evaluation… · p. 3
- Measures of successful tasks, functions, or contexts should account for training, test, and operational data because they play different roles.The paper links training data to design specifications, test data to requirements specifications, and operational data to usage.Source: 1) Metrics with Large Fuzzy Evaluation… · pp. 3–4
- Because ML functions can succeed or fail on the same fine-grained function, each quality measure should consider uncertainty indices such as bias and variance.The paper notes that DNN behavior can change unstably because of high nonlinearity.Source: 1) Metrics with Large Fuzzy Evaluation… · p. 4
A. Modifying Existing (Sub-)Characteristics in SQuaRE
- A second reason for adapting SQuaRE is that effective measurement techniques have not been established for some ML-based sub-characteristics, especially with DNN implementations.Source: 2) Metrics with Missing Techniques · p. 4
2) Metrics with Missing Techniques:
The paper identifies SQuaRE measures that do not fit ML systems or lack established techniques, and proposes extensions covering monitoring, complexity, data integrity, modularity, and modifiability.
2) Metrics with Missing Techniques:
- Operability’s Monitoring capacity should incorporate recommended XAI techniques, although the paper says such techniques have yet to be developed.XAI techniques can explain or interpret ML components, but the field is described as being at an early stage.Source: 2) Metrics with Missing Techniques · p. 4
- Integrity’s Internal data corruption prevention should include adversarial examples, for which the paper reports no established or recommended prevention techniques.The paper describes adversarial examples as small, human-unrecognizable noise that can cause image-recognition models to infer classes or objects with high confidence.Source: 2) Metrics with Missing Techniques · p. 4
- Modularity should assess whether ML components are independent of surrounding components and systems, a property current technologies cannot measure.The paper notes that components trained with specific surrounding systems may be tightly fitted to them.Source: 2) Metrics with Missing Techniques · p. 4
- Modifiability should account for expected training time when estimating the work time for a specific modification, even though that time is generally unknown before training.The proposed extension evaluates expected training time without actually conducting training.Source: 2) Metrics with Missing Techniques · p. 4
- Maintainability should cover both structural complexity and behavioral complexity, including robustness, because cyclomatic complexity is constant for ML components.Suggested structural measures include parameters, layers, numerical precision, and multiply-accumulate operations.Source: 2) Metrics with Missing Techniques · pp. 4–5
B. Adding New (Sub-)Characteristics in SQuaRE
The paper maps the Ethics guidelines’ sub-requirements to SQuaRE and proposes extensions for human-autonomy risk. It identifies autonomy, oversight, explainability, and communication as relevant human–AI interaction concerns.
B. Adding New (Sub-)Characteristics in SQuaRE
- The analysis exhaustively maps Ethics guidelines sub-requirements to SQuaRE quality characteristics and identifies extensions for AI systems.The authors describe this as their second analysis, focused on the Ethics guidelines.Source: B. Adding New (Sub-)Characteristics in… · p. 5
- The guidelines address fundamental-rights impacts and human–AI interaction through agency, oversight, explainability, and communication requirements.Communication also emphasizes audience-appropriate clarification, feedback cycles, and psychological aspects.Source: B. Adding New (Sub-)Characteristics in… · p. 5
- The paper proposes adding Mitigation of risk on human autonomy as a sub-characteristic under Freedom from risk in Quality in use.The proposal follows the guidelines’ focus on respect for human autonomy.Source: B. Adding New (Sub-)Characteristics in… · p. 5
1) Autonomy and Human:
The paper proposes reframing usability for autonomous AI as collaboratability, encompassing control, understanding, and collaboration effectiveness. It also treats unfair bias as a distinct risk to address in SQuaRE.
1) Autonomy and Human:
- Usability should extend to Collaboratability, with Controllability, Explainability, and Collaboration Effectiveness as additional sub-characteristics.This reframing reflects autonomous AI roles while retaining human control and understanding.Source: 1) Autonomy and Human · p. 5
- The paper treats unfair bias and user diversity as fairness concerns and proposes addressing them as a distinct kind of risk.The cited guideline describes fairness in terms of distribution of benefits and costs and freedom from bias, discrimination, and stigmatisation.Source: 2) Fairness · p. 5
2) Fairness:
The paper proposes adding unfair-bias risk mitigation to SQuaRE’s Freedom from risk characteristic. It distinguishes this proposal from technical robustness and safety, whose requirements are largely common to dependable systems.
2) Fairness:
- Freedom from risk in Quality in use should add the sub-characteristic Mitigation of risk by unfair bias.The paper presents this as an extension addressing fairness.Source: 2) Fairness · p. 5
- Technical robustness and safety are described as largely common to dependable systems, while AI-specific emphasis includes accuracy.The paper connects accuracy evaluation to earlier discussion of functionality, including completeness and correctness.Source: 3) Accuracy · p. 5
3) Accuracy:
The analysis identifies privacy as a missing SQuaRE concern and proposes an explicit privacy sub-characteristic. It also notes reproducibility as a reliability concern in AI research.
3) Accuracy:
- SQuaRE lacks specific privacy items despite partial coverage through Security, and the paper recommends extending SQuaRE to address privacy.The proposed extension is not limited to AI systems.Source: 4) Privacy · p. 5
4) Privacy:
- The Security characteristic should become Security and Privacy, with Privacy added as a product-quality sub-characteristic.Source: 4) Privacy · p. 5
- Reproducibility concerns whether the same behavior appears in experiments under the same conditions, with outcomes potentially affected by randomness and configuration parameters.Source: 5) Accountability · p. 5
5) Accountability:
The paper treats accountability as a meta-level consideration for documenting evaluation evidence, rather than a SQuaRE characteristic. It also identifies areas already covered, partly covered, or excluded when mapping other Ethics guidelines requirements.
5) Accountability:
- Ethics-guideline traceability concerns documenting system construction for responsibility, unlike SQuaRE’s traditional focus on internal deliverables for maintainability.Auditability also calls for traceability and logging of processes and outcomes, with separate auditability for each aspect.Source: 5) Accountability · pp. 5–6
- Evaluation planning and conduct should account for evidence justifying results for quality characteristics and sub-characteristics; accountability is treated at SQuaRE’s meta-level.Source: 5) Accountability · p. 6
- Fallback and safety requirements, along with reliability assessment, are covered by existing Reliability, Security, and Freedom from risk characteristics.Source: 6) Other Requirements · p. 6
6) Other Requirements:
- Data management, monitoring, and access control are partly covered by Security, while SQuaRE’s Data quality may cover them; most assessment items concern out-of-scope internal implementations.Source: 6) Other Requirements · p. 6
- Accessibility and universal-design requirements concerning disability and diverse backgrounds are included in the product Accessibility sub-characteristic.Source: 6) Other Requirements · p. 6
- The analysis excludes stakeholder participation and social-impact requirements, as well as organizational activities, negative-impact reporting, trade-off documentation, and redress.Source: 6) Other Requirements · p. 6
V. CONCLUDING REMARKS
The paper analyzes how SQuaRE could be adapted for ML-based AI systems and finds that most Ethics guidelines content is not covered. Its proposed guidance addresses external system quality, while internal quality remains for future investigation.
V. CONCLUDING REMARKS
- The analysis finds that most of the Ethics guidelines are not covered by SQuaRE and presents preliminary guidance for industrial practitioners.The paper notes that SQuaRE did not originally consider ML-based implementations.Source: §V · p. 6
- The paper focuses on external system quality, leaving internal development and operation activities and deliverables outside its scope.Future investigation is intended to address internal quality and guidance for training data, specifications, test design, runtime monitoring, and maintenance.Source: §V · p. 6