PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Factors in Quantum Software Quality

Jianjun Zhao

arXiv:2609.29705v1cs.SE

TL;DR

Quantum software’s probabilistic behavior, backend dependence, and hybrid workflows are not explicitly represented by classical quality models. The paper builds a base model using McCall’s structure, extending its factors and linking them to criteria and candidate metrics. The model is intended as a context-tailorable starting point for quality assessment across the quantum software lifecycle.

  • Problem

    Classical quality models do not explicitly capture quantum concerns such as probabilistic assessment and backend-dependent behavior, while quantum software quality also depends on execution conditions and hybrid workflows.

    Source: §1 · p. 1, §2.3 · p. 4

  • Method

    The paper uses McCall’s factor-criterion-metric structure, consolidates classical quality factors, adds quantum-specific factors where needed, and defines criteria, mappings, and candidate metrics with conditions for interpretation.

    Source: §1 · pp. 1–2

  • Results

    The paper presents a base quantum software quality model comprising a consolidated factor set, quantum-specific extensions, quality criteria, factor-to-criterion mappings, and representative candidate metrics.

    Source: §9 · p. 28

  • Takeaways & Limitations

    The model provides a common starting point for structuring and assessing quantum software quality while allowing adaptation to different application, platform, and organizational contexts.

    Source: §9 · p. 28

    Assessment remains context-dependent: correctness and reliability often require statistical evidence under stated execution conditions, and program behavior may vary across backends and calibration periods.

    Source: §2.3 · p. 4

Abstract

from arXiv · show

As quantum computing evolves, the demand for high-quality quantum software is increasing. Classical software quality models, such as McCall's model, provide a useful foundation, but they do not explicitly capture distinctive properties of quantum software, including probabilistic measurement outcomes, backend-dependent behavior, and hybrid quantum-classical workflows. This paper proposes a base quality model for quantum software using McCall's factor-criterion-metric structure as the organizing backbone. It broadens the set of quality factors by drawing on representative classical software quality models and standards and introduces quantum-specific extensions only where needed. We define the resulting factor set and quality criteria, provide a many-to-many mapping of core factor-to-criterion relationships, discuss representative candidate metrics together with the execution, reference, and interpretation conditions needed to use them, and illustrate how the resulting model can be applied across the quantum software lifecycle. The model is intended as a common starting point for structuring, assessing, and communicating quality concerns in quantum software while supporting context-specific adaptation. Further expert feedback, case studies, and empirical studies can be used to evaluate and refine the model and its application across different quantum software contexts.

1 INTRODUCTION

Quantum software’s probabilistic outcomes, backend dependence, and hybrid workflows complicate quality assessment using classical assumptions alone. The paper proposes a context-tailorable base model that extends McCall’s structure and connects quality factors to criteria and candidate metrics.

1 INTRODUCTION

  • Probabilistic measurement, noise and device conditions, backend calibration, and hybrid workflows make quantum software quality difficult to assess and communicate using classical assumptions alone.Quantum software must also account for physical properties and constraints, including entanglement and no-cloning.Source: §1 · p. 1
  • Classical quality models provide established structures, but do not explicitly address quantum concerns such as probabilistic outcomes, backend dependence, and hybrid workflows.The paper identifies a lack of a widely used base model for structuring, assessing, and communicating quantum software quality.Source: §1 · p. 1
  • The proposed base model retains McCall’s factor-criterion-metric structure, broadens the classical factor set, and adds quantum-specific extensions where retained concepts are insufficient.It defines criteria, a many-to-many factor-to-criterion mapping, and representative candidate metrics with execution, reference, and interpretation conditions.Source: §1 · p. 1
  • The model can be tailored to application domains, platforms, organizational contexts, risk profiles, and evidence needs, with lifecycle use illustrated through selected factors, criteria, and metrics.Further expert feedback, case studies, and empirical studies can evaluate and refine the model and its applications.Source: §1 · pp. 1–2

2 BACKGROUND

The background introduces quantum-computing concepts relevant to software quality, reviews classical quality models, and explains why their assumptions do not fully match quantum software. These gaps motivate retaining useful classical structures while adding quantum-specific concerns where needed.

2 BACKGROUND

  • Quantum software quality is shaped by quantum-computing concepts and execution context, so the background focuses on concepts that directly affect quality rather than providing a full quantum-computing introduction.Measurement produces probabilistic outcomes, and noise, backend conditions, compilation choices, and hybrid orchestration influence behavior and assessment.Source: §2.1 · pp. 2–3

2.1 Basic Concepts of Quantum Computing

  • Quantum computation involves qubits and states, probabilistic measurement, superposition, and entanglement, whose preservation may matter to intended computation.Repeated shots estimate outcome frequencies or expectation values, with finite-sampling uncertainty; noise and decoherence add errors and variability.Source: §2.1 · p. 2
  • Backend constraints and changing calibration, compilation choices, and hybrid workflows can affect circuit behavior and quality evidence.Compilation can change depth, gate counts, and error sensitivity; hybrid systems also rely on classical orchestration of execution, parameter updates, and results.Source: §2.1 · pp. 2–3

2.2 Overview of Classical Software Quality Models

  • Classical quality models distinguish high-level concerns from assessable properties and measures, while McCall’s model organizes these through factors, criteria, and metrics.Later models and standards broaden the concerns represented, including compatibility, security, supportability, interaction capability, and safety.Source: §2.2 · p. 3

2.3 Limitations of Classical Models in the Quantum Context

  • Classical concerns remain relevant, but their usual assumptions do not fully match quantum behavior; classical models do not make statistical evidence and execution conditions explicit.Assessment often relies on probabilistic outcomes, while entanglement, measurement placement, noise, calibration drift, and compilation choices affect behavior.Source: §2.3 · pp. 3–4
  • Quality assessment must account more explicitly for backend, calibration, and toolchain context, as the same logical program may behave differently across devices, settings, and time.Hybrid execution boundaries and physical validity—including intended unitary and measurement semantics and physical constraints—also require explicit treatment; these gaps motivate retaining useful classical concepts and adding quantum-specific factors and criteria where needed.Source: §2.3 · pp. 4–5

3 QUALITY FACTORS FOR QUANTUM SOFTWARE

The section defines a tailorable quantum-software quality-factor model rooted in McCall, broadened with classical sources, and extended by four quantum-specific factors where classical concepts are insufficient.

3.1 Terminology and the Factor-Criterion-Metric View

  • Factors are high-level quality objectives, criteria are assessable properties, and metrics measure criteria under stated assumptions and execution conditions.Quantum metrics may depend on backend, transpilation options, shot count, random seeds, mitigation choices, and applicable reference conditions.Source: §3.1 · p. 5

3.2 Design Principles for Constructing the Base Model

  • The model preserves McCall’s factor-criterion-metric structure, extends it conservatively, distinguishes factors from criteria, and is designed for context-specific adaptation.A quantum-specific concern becomes a factor when it is a stable, distinct high-level objective; otherwise, it is treated as a criterion.Source: §3.2 · p. 5

3.3 Construction Procedure for the Base Model

  • Construction proceeds from preserving classical terminology and consolidating concepts to adding quantum concerns, assigning them to layers, and defining criteria, mappings, and candidate metrics.The model does not impose a universal metric set because measurements depend on evaluation context.Source: §3.3 · p. 7

3.4 Classical Backbone of the Proposed Model

  • The classical backbone retains McCall’s operation, revision, and transition groups, adapts their assessment to quantum execution, and adds recurring concerns from later models and standards.Table 1 preserves source terminology, while Table 2 gives quantum-adapted factor definitions and cross-references without implying strict equivalence.Source: §3.4 · p. 7
  • Related factors remain separate when they represent distinct assessment targets, even where their criteria or evidence overlap.Examples distinguish functionality from correctness, integrity from security, maintainability from modifiability and flexibility, and portability from hardware adaptability.Source: §3.4 · pp. 7–8

3.5 Quantum-Specific Factors

  • Four quantum-specific factors address distinct objectives: Entanglement Robustness, Quantum Hardware Adaptability, Hybrid Quantum-Classical Interoperability, and Physical Validity.Entanglement Robustness applies when entanglement is essential; the other factors concern device variation, quantum-classical coordination, and physical or semantic constraints.Source: §3.5 · p. 8
  • Measurement bias, statistical repeatability, and mitigation sensitivity remain criteria that can inform correctness, reliability, and verifiability rather than separate factors.Their assessments provide evidence across multiple quality concerns.Source: §3.5 · p. 8

3.6 Tailoring the Base Model

  • Tailoring begins with a context profile, then selects relevant factors, criteria, mappings, metrics, and assessment conditions, documenting assumptions and justified changes for traceability.Profiles may capture domains, backends, execution modes, hardware and toolchain assumptions, risks, workflow, stakeholders, and assurance or reproducibility needs.Source: §3.6 · p. 8

4 QUALITY CRITERIA FOR QUANTUM SOFTWARE

This section defines assessable criteria for evaluating quantum software quality factors, combining McCall-rooted criteria with classical extensions and quantum-specific concerns. It maps factors to criteria and explains their interpretation in quantum execution contexts, including how to assess no-cloning compliance, assumption preservation, and measurement-related properties.

4.1 Overview of Quality Criteria

  • Table 3’s criteria combine McCall-rooted criteria, representative classical extensions where the original set is insufficient, and quantum-specific additions; the proposed definitions are not an exhaustive list.Additional classical criteria are retained when an extended classical factor cannot be assessed adequately using McCall’s criteria alone and the concern is general software quality rather than quantum mechanics.Source: §4.1 · pp. 8–9

4.2 Factor-to-Criterion Mapping

  • The core factor-to-criterion mapping is many-to-many: several criteria can assess a factor, and one criterion can support multiple factors, but one favorable criterion does not establish a factor.Profiles may tailor mappings to context, provided additions, omissions, or reinterpretations are documented; context-dependent secondary relationships can be justified where needed.Source: §4.2 · p. 9

4.3.1 Classical Criteria Rooted in McCall’s Model.

  • Classical criteria retain their terminology but require quantum-aware interpretation: consistency concerns uniform evidence and conventions, while accuracy is statistical and needs explicit references, acceptance conditions, and procedures.Machine independence concerns classical host systems, while data and communications commonality address standardized representations and protocols; conciseness must preserve assumptions and evidence interpretability.Source: §4.3.1 · pp. 9, 13

4.3.2 Additional Classical Criteria Beyond McCall’s Model.

  • Additional classical criteria distinguish resource use from time or storage efficiency, and cover reproducibility, recovery, security and safety, interface and version compatibility, and transparent result reporting.Reproducibility requires recorded artifacts, dependencies, configurations, and procedures and differs from statistical repeatability; fail-safe behavior calls for clear invalid-configuration reporting and controlled states rather than misleading conclusions.Source: §4.3.2 · p. 13

4.4 Quantum-Specific Criteria

  • Quantum-specific criteria support four quantum-specific factors, while measurement- and mitigation-related criteria remain at the criterion level because they also assess classical factors such as correctness, reliability, and verifiability.The quantum-specific criteria address entanglement, hardware adaptation, hybrid coordination, and quantum-computation constraints.Source: §4.4 · pp. 13–14, §4.4.1 · p. 14

4.4.1 Criteria Supporting the Quantum-Specific Factors.

  • Entanglement stability and robustness, backend portability, calibration robustness, and constraint tolerance assess whether required behavior persists across stated execution settings, backends, transformations, or device conditions.Portability includes adaptation effort and post-transfer behavior, without implying independence from backend-specific constraints.Source: §4.4.1 · p. 14
  • Hybrid-workflow criteria assess orchestration, synchronization, and disruption handling, while unitarity preservation applies to intended unitary regions and measurement semantics soundness to preserved measurement behavior.The criteria cover workflow control and data flow, timing and dependencies, recovery or controlled degradation, and preservation of measurement basis, placement, outcome interpretation, state evolution, and measurement-conditioned control.Source: §4.4.1 · p. 14

4.4.1 Criteria Supporting the Quantum-Specific Factors

  • No-cloning compliance requires abstractions, interfaces, and transformations to avoid perfect duplication of arbitrary unknown quantum states, while permitting copying measured classical information or preparing known states.The criterion is especially relevant to high-level abstractions, compiler transformations, and distributed workflows.Source: §4.4.1 · p. 14
  • Assumption preservation asks whether declared physical and semantic assumptions remain valid after toolchain transformations and during execution under stated conditions.Source: §4.4.1 · p. 14

4.4.2 Measurement-Related Criteria Supporting Classical Factors

  • Measurement-related criteria provide evidence for assessing classical quality factors, particularly correctness, reliability, and verifiability.Source: §4.4.2 · p. 14
  • Measurement bias is persistent deviation from an explicit reference beyond expected sampling variation, assessed using a stated statistical estimation procedure.Statistical repeatability does not rule out bias, and run-to-run variation alone does not establish it; the criterion primarily supports correctness and secondarily reliability.Source: §4.4.2 · pp. 14–15
  • Statistical repeatability concerns consistency across nominally identical executions within expected sampling variation, supporting reliability rather than the artifact-and-condition re-execution captured by reproducibility.Source: §4.4.2 · p. 15
  • Mitigation sensitivity measures how results or conclusions change across mitigation configurations and post-processing choices with other conditions fixed, supporting reliability and verifiability.Excessive dependence on a particular configuration can weaken conclusion stability and independent assessment.Source: §4.4.2 · p. 15

5 CANDIDATE METRICS FOR QUANTUM SOFTWARE QUALITY

The section presents candidate metrics as criterion-level evidence for quantum software quality, adapted to quantum artifacts and execution behavior and interpreted in context. It leaves scoring and aggregation to project-specific quality profiles rather than prescribing a universal scheme.

5.1 Role of Metrics in the Factor-Criterion-Metric Structure

  • Metrics provide measurable evidence at the criterion layer, not factor-level scores, and quantum assessment adds numerical, distributional, uncertainty, resource, robustness, and cross-execution measures.Metric evidence may come from classical artifacts as well as circuits, transpilation, measurement outcomes, backend and calibration data, mitigation, and hybrid optimization traces.Source: §5.1 · p. 15

5.2 Measurement Objects and Evidence Sources

  • Quantum metric evidence spans specifications and programs, circuits and backend records, outputs and optimizer traces, and reproducibility artifacts; measures may be static, execution-based, or cross-context.Examples include static circuit statistics, controlled-run success probability or distributional distance, and comparisons across backends, calibration windows, or shot budgets.Source: §5.2 · p. 15

5.3 Required Elements of a Candidate Metric

  • Operational candidate metrics specify the criterion, measurement object and inputs, measurement rule, evaluation context, and interpretation rule.Reference or acceptance conditions and aggregation rules may also be needed; these choices, along with thresholds and weights, belong to a tailored quality profile rather than a universal standard.Source: §5.3 · p. 16
  • The model’s candidate metrics are computable or checkable once contextualized, but fixed thresholds, weights, aggregation rules, and acceptance bounds are left to tailored quality profiles.This profile-dependent approach keeps the model a base for assessment rather than a standardized measure with universal rules.Source: §5.3 · p. 16

5.4 Types of Metrics in the Quantum Software Context

  • Quantum quality measurement extends ratios and checklists with structural, distributional, repeatability, robustness, portability, and lifecycle-evidence metrics.Examples include circuit depth, output-distribution distances, between-run variation, changes across backends or calibration windows, and evidence re-evaluated as software evolves.Source: §5.4 · pp. 17–18

5.5 Representative Candidate Metrics

  • Table 5 presents representative criterion-linked candidate metrics with their measurement objects, rules, interpretation contexts, and reference needs, while allowing project-specific refinement.Criteria may require multiple metrics, and a metric such as circuit depth can inform different criteria with different interpretations; the table does not fix thresholds, weights, or factor-level scores.Source: §5.5 · p. 18, Table 5 · p. 19

5.6 Metric Interpretation and Factor-Level Use

  • Metric values acquire meaning through the criteria and quality profile they support, reflecting a many-to-many relationship between metrics, criteria, and factors.Some directions are usually clear, such as smaller error or circuit depth, while success probabilities and cross-backend agreement require explicit acceptance bounds or tolerances; reference-based judgments also depend on the chosen reference.Source: §5.6 · p. 18
  • Metrics may not be comparable across backend, shot count, transpilation, mitigation, or calibration settings, so interpretation should record the evaluation context.For example, noisy-backend results may be less stable than ideal-simulator results, while mitigation may reduce value error but increase variance or calibration sensitivity.Source: §5.6 · p. 23
  • Factor-level assessment requires a tailored quality profile specifying selected factors, criteria, metrics, evidence, acceptance conditions, and aggregation rules.Profiles may set different priorities, thresholds, weights, confidence requirements, and reassessment triggers; the model prescribes no universal aggregation formula.Source: §5.6 · p. 23
  • Projects may use candidate metrics as defined, refine them, add domain-specific metrics, or exclude irrelevant ones, but must define each selected metric and relate it to a criterion.Table 5 identifies measurable or checkable evidence for Table 3 criteria while leaving thresholds, weights, acceptance conditions, and aggregation rules to the tailored profile.Source: §5.6 · p. 23

6 USING THE PROPOSED MODEL ACROSS THE QUANTUM SOFTWARE LIFECYCLE

The model is presented as a descriptive structure for organizing quality reasoning and evidence across the quantum software lifecycle, with links among classical and quantum-specific concerns. A VQA example shows how to tailor a quality profile and when its evidence needs reassessment.

6 USING THE PROPOSED MODEL ACROSS THE QUANTUM SOFTWARE LIFECYCLE

  • The model organizes factors, criteria, and evidence across lifecycle stages; it is descriptive, not a prescribed lifecycle process or fixed checklist.Assessment begins with requirements and continues through design, implementation, testing, maintenance, and post-deployment operation.Source: §6 · p. 23, §6.1 · p. 23

6.1 Quality Concerns Across Lifecycle Stages

  • Requirements and design should make quality expectations, execution assumptions, evidence needs, and quantum-classical workflow boundaries explicit before testing.Requirements can specify backend class, noise conditions, shot budget, mitigation policy, and acceptable uncertainty; design also addresses circuit assumptions and orchestration.Source: §6.1 · pp. 23–24
  • Implementation and testing preserve context for interpreting results, while maintenance reassesses earlier evidence when backends, toolchains, or execution conditions change.Relevant records include run metadata, compiled circuits, seeds, shot counts, mitigation settings, and backend configurations; testing uses controlled repeated executions and treats evidence as context-sensitive.Source: §6.1 · p. 24

6.2 Interactions Between Classical and Quantum-Specific Quality Concerns

  • Classical criteria such as traceability and instrumentation help make quantum-specific evidence interpretable, while quantum-specific criteria can reveal weaknesses hidden by clean software structure.Backend connectivity, calibration drift, mitigation choices, measurement semantics, and hybrid orchestration can qualify quality claims beyond what modularity or static structure indicates.Source: §6.2 · pp. 24–25

6.3 Illustrative Quality Profile for a VQA Project

  • A VQE-style molecular-energy profile selects factors and criteria around energy accuracy, repeatability, measurement bias, mitigation sensitivity, portability, efficiency, and preserved assumptions.Candidate metrics depend on stated references, controlled execution conditions, and linked artifacts and settings; changes to the backend, transpiler, ansatz, or other profile choices require reassessment.Source: §6.3 · p. 25

7 DISCUSSION AND OPEN CHALLENGES

The discussion frames the model as a starting point whose consistent use requires shared measurement conventions, empirical validation, practical tool support, and evidence that remains meaningful as environments evolve.

7 DISCUSSION AND OPEN CHALLENGES

  • The model is a base quality model rather than a fixed standard, and its consistent application across projects, platforms, and toolchains remains an open challenge.The challenges include shared metric interpretation, empirical validation, practical assessment tools, and maintaining assessment under changing environments.Source: §7 · p. 26

7.1 From Candidate Metrics to Common Measurement Conventions

  • Comparable metric results require conventions specifying the measured behavior, execution conditions, aggregation, reference, uncertainty treatment, and reporting format.Values can depend strongly on backend, transpilation, shot count, random seeds, mitigation settings, and execution or calibration period.Source: §7.1 · p. 26

7.2 Empirical Validation of the Model

  • The lifecycle discussion and illustrative VQA profile demonstrate possible application, but do not validate the model or its candidate metrics.The paper calls for expert feedback, representative case studies, and empirical studies across backends, workloads, execution periods, and toolchain versions.Source: §7.2 · p. 26

7.3 Tool Support for Quality Assessment

  • Practical use needs integrated tools that link evidence from quantum software activities to quality factors, criteria, requirements, and execution assumptions.Such support would track provenance and help identify supported claims, applicable assumptions, and changes that require reassessment.Source: §7.3 · p. 26

7.4 Maintaining Quality Evidence Under Hardware and Toolchain Evolution

  • Evidence from one backend, calibration period, toolchain, mitigation procedure, or platform policy may no longer support the same quality claim after conditions change.Assessment should record provenance and context, identify changes that trigger reassessment, and distinguish evidence that remains valid from evidence needing reproduction or reinterpretation.Source: §7.4 · p. 26

8 RELATED WORK

Prior work addresses quantum software requirements, quality attributes, metrics, defects, and testing or verification techniques, but these contributions do not provide the broad factor-criterion-metric model proposed here. The paper positions its model as a structure that complements those separate lines of research.

8 RELATED WORK

  • Requirements research identifies quantum-specific constraints and quality concerns, but does not provide a general model organizing them across factors, criteria, and metrics.The reviewed work covers device constraints, quantum, classical and hybrid requirements, and challenges including technology evolution, security, testing, and verification.Source: §8 · p. 27
  • Verdugo et al. argue for a quantum software quality environment and discuss quantum-specific concerns, but do not construct a detailed base quality model.Their discussion includes accuracy and precision under noisy execution, platform dependence, and concepts such as entanglement, superposition, and no-cloning.Source: §8 · p. 27
  • Quantum software metrics have been proposed for internal attributes and specific concerns, including code structure and circuit understandability, rather than a general quality model.Cruz-Lemus et al. propose circuit metrics; later work reports an empirical study and the QMetrics prototype.Source: §8 · p. 27
  • A repository study of 32,296 issues from 123 open-source projects reports frequent defects affecting usability, maintainability, and interoperability, among other quality attributes.It characterizes observed defects and their impacts but does not define a general quality model.Source: §8 · p. 27
  • Testing, debugging, and formal verification provide evidence for particular criteria, while their reviewed techniques do not themselves constitute a general software quality model.The review covers approaches including coverage, mutation, metamorphic, and fuzz testing, and notes quantum-specific difficulties such as probabilistic behavior and noise.Source: §8 · p. 27
  • Existing studies address distinct activities and quality concerns, whereas this work organizes broad classical and quantum-specific factors, criteria, mappings, and candidate metrics in one model.The model also supports context-specific tailoring and lifecycle use.Source: §8 · p. 27

9 CONCLUSION

The paper concludes with a base quality model for quantum software, organized around factors, criteria, and candidate metrics. It is intended to adapt to context and to support further evaluation and refinement.

9 CONCLUSION

  • The model consolidates classical quality concepts, adds quantum-specific extensions where needed, and maps core factors to criteria in a many-to-many structure.It also discusses candidate metrics and the conditions needed to interpret them, with lifecycle use illustrated.Source: §9 · p. 28
  • The model allows adaptation across application domains, platforms, and organizational contexts rather than prescribing one fixed quality profile.Its factors, criteria, mappings, and candidate metrics are presented as a basis for evaluation and refinement through expert feedback, case studies, and empirical studies.Source: §9 · p. 28
  • Further work is needed on model evaluation, tool support, comparable measurement conventions, and tailored quality profiles across quantum software classes.Candidate metrics also need study across backends, toolchains, execution settings, and application domains.Source: §9 · p. 28

A REVERSE CRITERION-TO-FACTOR INDEX

Table 6 provides a reverse criterion-to-factor index derived directly from Table 4, intended as a lookup aid; the core factor-to-criterion mappings remain in Table 4.

A REVERSE CRITERION-TO-FACTOR INDEX

Related papers

This paper cites

Loading 2609.29705v1…