Source-linked AI summary
A Review of Software Quality Models for the Evaluation of Software Products
Jose P. Miguel, David Mauricio, Glen Rodriguez
TL;DR
Software quality evaluation has become important as software products and component-based construction have expanded, creating a need for models suited to different products and domains. This paper reviews basic, tailored, and Free/Open Source models, compares them using ISO 25010, and identifies their scope and deficiencies. It concludes that general models can be difficult to apply specifically, tailored models address narrower domains, and communication is absent as a quality factor despite its importance in complex Internet-connected systems.
Problem
The growth of software products and component-based construction creates challenges for evaluating quality across general, tailored, and Free/Open Source products.
Method
The paper reviews and classifies software product quality models, compares them using ISO 25010, and examines models for Free/Open Source software.
Results
The review finds that general models are difficult to apply to specific cases, tailored models cover smaller domains, and Free/Open Source models emphasize community participation.
Takeaways & Limitations
Communication should be considered in software quality, particularly for quality components in complex systems where the Internet makes communication critical.
Takeaways & Limitations
No ideal quality model capturing all aspects of Free Software quality has yet been defined.
Abstract
from arXiv · showhide
Actually, software products are increasing in a fast way and are used in almost all activities of human life. Consequently measuring and evaluating the quality of a software product has become a critical task for many companies. Several models have been proposed to help diverse types of users with quality issues. The development of techniques for building software has influenced the creation of models to assess the quality. Since 2000 the construction of software started to depend on generated or manufactured components and gave rise to new challenges for assessing quality. These components introduce new concepts such as configurability, reusability, availability, better quality and lower cost. Consequently the models are classified in basic models which were developed until 2000, and those based on components called tailored quality models. The purpose of this article is to describe the main models with their strengths and point out some deficiencies. In this work, we conclude that in the present age, aspects of communications play an important factor in the quality of software
1. Introduction
The paper reviews software product quality models, distinguishing general basic models from domain-specific tailored models and emphasizing the growing importance of component and communication quality.
- Scope and concepts: Software quality concerns whether products meet specified requirements and user needs, while this paper focuses on finished products rather than construction processes.Quality models organize characteristics, relationships, and metrics to support product evaluation.
- Basic models: Basic models are hierarchical, adaptable to different software products, and oriented toward evaluation and improvement.The paper identifies Mc Call, Boehm, FURPS, Dromey, ISO 9126, and ISO 25010 among the main basic models.
- Tailored models: Tailored quality models emerged from component-based software development and target particular application domains with variable feature importance.Component reuse makes product success strongly dependent on component quality.
- Industry and component models: Organizations and companies developed specialized models and component-selection practices to address domain-specific evaluation needs.Reported examples include FURPS, Siemens’ MIDAS, and Alvaro’s component-certification framework.
- Review scope: The review covers basic, tailored, and Free/Open Source quality models, extending earlier reviews that focused mainly on basic models.The comparison uses ISO 25010, and the review also considers product-oriented models for Free/Open Source software.
2. Methodology
The methodology combines a literature search and classification with standardized terminology for comparing software product quality models and excluding process-focused studies.
- Literature search: The review searched Google Scholar, Science Direct, Ebsco, Trove, and NDTLD for literature on software quality models, metrics, COTS components, and Free/Open Source software.The search terms covered general, tailored, component, and open-source quality models.
- Inclusion and exclusion criteria: Articles were classified by relevance and model type, while studies evaluating software construction processes were excluded.The retained scope targets quality aspects of finished software products.
- Terminology: Table 1 establishes common terminology using concepts from international quality standards and the literature.The terminology draws mainly on American Society for Quality and ISO sources.
- Quality characteristics: The terminology organizes quality into characteristics and subcharacteristics, including functionality, efficiency, usability, reliability, maintainability, portability, security, and compatibility.Definitions specify aspects such as performance relative to resource use, fault tolerance, modifiability, and interoperability.
- Component attributes: Additional component-oriented attributes include configurability, self-containment, testability, reusability, replaceability, scalability, and supportability.These attributes describe component configuration, independent operation, validation after modification, reuse, substitution, expansion, and serviceability.
3. Basic quality models
Basic quality models provide global, hierarchical frameworks for evaluating software products, evolving from early models to ISO 9126 and ISO 25010. They organize quality through characteristics, metrics, lifecycle views, and quality-in-use attributes, but differ in coverage and measurement limitations.
- Basic models make global assessments of software products and can be adjusted to different software types.
- McCall Model: McCall organizes quality into Product Review, Product Operation, and Product Transition perspectives, linking characteristics with metrics.
- McCall Model: McCall’s yes-or-no measurement approach limits accuracy, and its omission of functionality diminishes the user’s perspective.
- Boehm Model: Boehm extends McCall with factors at different levels, including utility, maintainability, and portability.
- Dromey Model: Dromey evaluates quality from the product perspective and supports dynamic product-specific evaluation, but does not explain how to implement its approach in practice.
- FURPS Model: FURPS separates functional requirements from non-functional characteristics covering usability, reliability, performance, and support, but omits portability.
- ISO Models: ISO 9126 distinguishes internal and external quality from quality in use, while ISO 25010 updates the framework with security and compatibility and extends portability through transferability.
4. Tailored Quality Models
Tailored quality models emerged with component-based software development to evaluate components in domain-specific, user-sensitive ways. The section presents models that adapt basic models for COTS, CASE tools, certification, and user needs.
- Tailored Quality Models: Component-based software development shifted non-basic quality models toward evaluating components, including COTS available in the market.The section illustrates COTS-based product construction through associated development activities.
- Bertoa Model: Bertoa adapts ISO 9126 by defining quality attributes for effective COTS evaluation and distinguishing features relevant to individual components.
- GEQUAMO Model: GEQUAMO decomposes characteristics into sublayers so end users, developers, and managers can build models with customized attribute weights.The model is presented through a CASE-tool decomposition.
- Alvaro Model: Alvaro proposes a component-certification framework covering quality modeling, technical certification, certification processes, and controlled component metrics.The article focuses on the quality-components model within this broader framework.
- Rawashdeh Model: Rawashdeh focuses on COTS and user needs, constructing quality models through attribute decomposition, metric distinction, user identification, and ISO 9126/Dromey ideas.Its four-step process separates internal metrics from external metrics during evaluation.
5. Open Source Models
Open-source quality models adapt established models to account for open-source characteristics, but no ideal model yet captures every aspect of quality. The section describes maturity-based, business-readiness, automated source-code, and context-sensitive approaches.
- Open Source Models: Open-source quality models adapt ISO 9126 by adding open-source-specific aspects, yet an ideal model covering all quality aspects remains undefined.
- Cap Gemini Model: The Cap Gemini model evaluates product maturity through product and application indicators, assigning each subindicator a value from 1 to 5 for a total score.
- OpenBRR Model: The Business Readiness Rating framework uses seven categories to accelerate open-software evaluation and support better choices with a small assessment set.Its categories can be refined for greater granularity.
- SQO-OSS Model: SQO-OSS hierarchically evaluates source code and community processes, emphasizing automation, continuous monitoring, and automatic metric collection.
- SQO-OSS Model: SQO-OSS does not evaluate functionality and considers only community factors that can be measured automatically.
- QualOSS Model: QualOSS treats product, community, and software-process characteristics as equally important, while making most assessment highly automated and context-dependent.Its quality assessment depends on the usage context and the purposes pursued by the user or organization.
6. Model Comparison
The paper compares basic and tailored quality models while noting that tailored-model comparisons are context-sensitive. ISO 25010 is judged the most complete basic model, and reliability appears across all basic models.
- Basic Model Comparison: The review compares basic models using ISO 25010 because it provides the latest standardized terminology.
- Basic Model Comparison: ISO 25010 is the most complete basic model, covering 26 of the 28 assessed features.
- Basic Model Comparison: Reliability is common to all basic models and is linked in the review to users’ opinions and product success.
- Basic Model Comparison: Feature presence can imply related features because characteristics contain broader groupings; transferability, for example, relates to portability and adaptability.
- Tailored Model Comparison: Tailored models are harder to compare because they serve particular contexts, including product-oriented, domain-specific, and user-oriented evaluations.The absence of a feature does not invalidate a tailored model.
7. Conclusions
The review finds that general quality models are difficult to apply to specific cases, while tailored models address narrower domains but remain limited. It also identifies user communities and communication as important quality considerations, and notes that feature importance varies by application domain.
- Conclusions: General quality models are difficult to apply to specific cases, whereas tailored models target smaller domains using ISO 9126 as a starting point.Tailored models select domain-specific features and subfeatures, but the paper notes that this specialization creates limitations.
- Conclusions: ISO 25010 updated ISO 9126 in 2007 by increasing the fundamental characteristics from six to eight.The paper identifies ISO 25010 as a reference for future model development.
- Conclusions: For Free Software, user communities should be treated as a high-level feature because they influence construction and product acceptance.The review also reports that Free/Open Source models emphasize community participation.
- Conclusions: None of the studied models incorporates communication as a quality factor, although communication components are needed at all levels, especially in Internet-based complex systems.The paper presents communication as a critical factor for contemporary complex systems.
- Conclusions: Assigning equal value to factors and criteria is relative because their importance depends on the application domain.The paper gives transferability as an example that can be crucial when software is installed on different machines.