PaperlayerView exact revision on arXiv

Source-linked AI summary

Software Engineering for AI-Based Systems: A Survey

Silverio Martínez-Fernández, Justus Bogner, Xavier Franch, Marc Oriol, Julien Siebert, Adam Trendowicz, Anna Maria Vollmer, Stefan Wagner

arXiv:2105.01984v2cs.SEcs.AIcs.LG

TL;DR

AI-based systems require distinct Software Engineering practices, but synthesized knowledge about developing, maintaining, and evolving them remains limited. This systematic mapping study examined 248 studies, finding concentrated attention on testing and quality, with data-related issues the most recurrent challenge.

  • Problem

    Synthesized knowledge about Software Engineering practices for developing, operating, maintaining, and evolving AI-based systems is limited.

  • Method

    The study conducted a systematic mapping study to structure and quantify published research on Software Engineering for AI-based systems.

  • Results

    Software testing accounted for 115 / 248 studies and software quality for 59 / 248, while data-related issues were the most recurrent challenge.

  • Takeaways & Limitations

    The findings identify testing and quality as especially studied SWEBOK areas and data-related issues as a prominent challenge for Software Engineering of AI-based systems.

  • Takeaways & Limitations

    The study reports threats to validity and cautions that some conclusions rely on a low number of papers.

Abstract

from arXiv · show

AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic mapping study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.

1 INTRODUCTION

AI-based systems differ from traditional software because behavior is inferred from data, creating distinct engineering and quality challenges. The study characterizes SE4AI practices and synthesizes its research landscape, approaches, and challenges.

  • AI-based systems infer rules and behavior from training data rather than specifying them entirely as program code.
  • Building and evolving these systems requires collaboration between data scientists and software engineers, alongside attention to changing datasets, infrastructure, ethics, and equity.
  • Ignoring these differences can produce poor AI-based systems with technical debt.
  • The study characterizes Software Engineering for Artificial Intelligence by synthesizing current SE knowledge for researchers, practitioners, and educators.
  • The systematic mapping study covers literature published from January 2010 to March 2020 and synthesizes bibliometrics, system characteristics, SWEBOK-aligned approaches, and challenges.

2 BACKGROUND

AI and software engineering have developed in both directions: AI methods support SE tasks, while SE methods support AI software. Existing reviews cover specialized topics, but the paper identifies a need for a broader synthesis of SE4AI.

  • AI4SE applies AI techniques to support software engineering tasks, while SE4AI applies software engineering methods to develop AI software.
  • As AI-based systems became larger and more complex, their dependence on learning data made traditional SE methods insufficient.
  • Prior reviews examined topics including ML deployment, development, testing, quality, architecture, and design patterns.
  • A review of 906 studies found only 15 covering the SE4ML direction, and its authors concluded that applying SE practices to ML/DL systems remained difficult.
  • The authors argue that no systematic mapping or general review had comparable breadth and depth, motivating a holistic overview of SE4AI literature.

3 RESEARCH METHODOLOGY

The study uses established systematic-mapping and systematic-review guidelines to identify and quantify SE research for AI-based systems. Its research questions address research characterization and system characteristics.

  • The systematic mapping study follows Petersen et al.’s SMS guidelines and also uses Kitchenham and Charters’ systematic-literature-review guidelines.
  • Its aim is to systematically select and review published literature while structuring the SE4AI field in a broader, quantified manner.
  • Using GQM, the study analyzes SE approaches to structure and classify proposed solutions and existing challenges for AI-based systems.
  • The research questions cover how SE4AI research is characterized and which terms, scopes, and quality goals describe AI-based systems.

RQ3. Which SE approaches for AI-based systems have been reported in the scientific literature?

The study searches broadly for SE4AI research using Scopus and complementary snowballing, then classifies extracted evidence using SWEBOK-related areas and newly proposed topics. Screening produced 248 primary studies, with substantial inter-rater agreement.

  • The search combines a Scopus query with backward and forward snowballing to capture SE4AI literature and mitigate selection bias.
  • The query combines SE terms, AI- and ML-based-system terms, and SWEBOK Knowledge Areas.
  • The study limits primary studies to the new wave of AI from January 2010 through March 2020.
  • Two researchers independently screened papers, with a third resolving disagreements; Cohen’s kappa was 0.751, indicating substantial agreement.
  • The process identified 21 seed papers, 56 papers after the first snowballing iteration, and 171 after the second, yielding 248 primary studies.
  • The classification adds Software Runtime Behaviour and topics covering cost and efficiency, quality, robustness and operability, ML/AI methods, activities, data, process, and adoption.

4 RQ1: HOW IS SOFTWARE ENGINEERING RESEARCH FOR AI-BASED SYSTEMS CHARACTERIZED?

The bibliometric analysis portrays SE4AI as a rapidly growing field with substantial industry participation and uneven geographic distribution. Empirical studies frequently use case studies and benchmarks, but often provide limited realism or discussion of validity threats.

  • Annual trend: Since 2015, publications approximately doubled annually, rising from 4 in 2015 to 102 in 2019.The study period ends in March 2020, so 2020 publications are not shown.
  • Geographic distribution: 40% of studies had first authors affiliated with the USA; North America had twice Asia’s absolute publication count.The USA’s share was four times China’s and five times Germany’s and Japan’s.
  • Research methods: 37.8% of empirical studies used case studies, while 22.4% used benchmarks.Benchmarks, case studies, and experiments often used industry data, but realistic-scale evaluations remained uncommon.
  • Evaluation realism: 60.1% of empirical studies used real-world subjects, objects, or datasets, whereas 39.9% used students or simulated data settings.Industry-authored papers used realistic-size applications more often, while academic papers used more toy examples.
  • Research rigor: 65.4% of empirical studies did not discuss threats to validity, and only 17.6% discussed evaluation validity in detail.Threats were especially often omitted in case studies, experiments, and benchmarks.
  • Author affiliations: 40.7% of the 248 studies involved industry or collaboration authors, indicating substantial industry participation.The study identifies 101 industry and collaboration papers.

5 RQ2: WHAT ARE THE CHARACTERISTICS OF AI-BASED SYSTEMS?

The mapped studies reveal diverse AI-based-system scopes, technologies, and application domains, with terminology often left undefined. Research concentrates on learning-based systems, safety and dependability, and system-level applications, while several domains and quality attributes remain underexplored.

  • AI technology: AI is consistently used to denote learning components rather than rule-based expert systems.This usage corresponds to the newer wave of AI associated with learning from data.
  • Scope of analysis: Almost half of the studies investigate complete AI-based systems, while one quarter focuses directly on AI components.Remaining studies address specific algorithms or infrastructure without considering specific applications.
  • Application domain: More than a quarter of studies concerns automotive applications, while almost half examines AI generically without an application domain.Many other domains account for only 0.8% to 3.3% of studies.
  • Quality attributes: Dependability and safety form the dominant quality cluster with 179 mentions, followed by functional suitability and accuracy with 68.Safety, robustness, reliability, and security are the four most frequently mentioned individual quality goals.
  • AI is commonly associated with deep learning and treated as part of a complex software system.
  • Terminology: Terminology varies substantially, and most primary studies do not explicitly define the terms they use.The most commonly defined terms include “deep learning system” and “ML system,” with five definitions each.

6 RQ3: WHICH SE APPROACHES FOR AI-BASED SYSTEMS HAVE BEEN REPORTED IN THE SCIENTIFIC LITERATURE?

The review organizes reported SE contributions into 11 SWEBOK Knowledge Areas and finds approaches spanning requirements, design, construction, testing, and verification. Testing, safety-oriented design, and infrastructure receive substantial attention, whereas system-level patterns and holistic requirements approaches are less developed.

  • The 248 primary studies were classified into 11 SWEBOK Knowledge Areas according to their SE contributions.Eight areas were further divided into subcategories.
  • Software design: Software design studies propose quality-focused strategies, infrastructure platforms, design challenges, and patterns for AI-based systems.Safety dominates the 11 studies in the design-for-quality category, while system-level patterns and reference architectures remain limited.
  • Infrastructure and platforms: Reported approaches include platforms for end-to-end ML development, model management, deployment, reusable model microservices, and cyber-physical systems.Several platforms follow model-driven engineering principles or target specialized domains such as safety-critical systems.

6.3 Software construction (23 studies)

The review identifies 23 software-construction studies and a much larger body of testing research. Construction contributions provide tools, platforms, practices, and guidelines, while testing emphasizes generated and selected cases, quality attributes, and metamorphic methods.

  • Software construction: 23 studies addressed software construction, including specialized tools, holistic platforms, construction practices, and educational material.Reported tools and platforms support activities such as deployment, serving, model development, version control, and workflow management.
  • Software construction: Several construction studies report practices and guidelines derived from industrial experience, surveys, interviews, case studies, and analyses of development challenges.Examples include large-scale ML experience reports, framework comparisons, and tactics for iterative model development.
  • Software construction: The maturity, selection rationales, and adoption levels of proposed construction tools and platforms remain unclear.
  • Software testing: 115 studies focused on software testing, with 55 unique studies addressing test cases, including 40 on generation and 12 on selection.
  • Software testing: Metamorphic testing accounts for 16 of 36 testing-method studies, while coverage criteria appear in 14 of 20 testing-metric studies.Fuzzing and mutation testing follow metamorphic testing with six and five studies, respectively.

6.5 Software maintenance (6 studies)

Software maintenance is sparsely represented in the mapping, with six studies focused mainly on bugs, debugging, model management, and reported maintenance challenges. This limited coverage marks maintenance as an underdeveloped area in the reviewed literature.

  • Only six studies addressed software maintenance, grouped into bug analysis and prediction, debugging support, and maintenance challenges.
  • Maintenance approaches: Maintenance research includes empirical analyses of bugs and their impact, explanatory debugging for interactive ML, and a TensorFlow debugger based on dataflow graphs.
  • Research coverage: The review characterizes software maintenance as a topic with hardly any studies and several open challenges.The identified maintenance work also includes model-management experience reports and surveys of DL practitioners.

6.8 Software quality (59 studies)

Software quality is a major SE4AI focus, spanning ML-specific attributes, assurance, certification, standards adaptation, monitoring, defects, and attacks. Yet only 17 publications address defining and assessing AI-system quality, while most work emphasizes testing and formal verification.

  • Scope: Software quality research covers requirements, measurement, assessment, and assurance, but quality management remains unevenly developed.The literature spans multiple perspectives, including prediction performance, training mechanisms, and lifecycle support.
  • Evidence gaps and risks: 17 publications address defining and assessing AI-system quality, while other studies examine assurance cases, ML defects, adversarial attacks, and model theft.Assurance cases are applied to automated driving and CNN-based perception functions.
  • Transparency and roadmaps: FactSheets are proposed to document purpose, performance, safety, security, and provenance information for customers, supporting transparency and trust.Quality roadmaps also propose ML-specific extensions to ISO 25000 and address processes, verification, validation, and formal analysis.
  • Quality characteristics: Safety, robustness, and explainability are among the most commonly discussed AI-system quality characteristics.The literature also discusses provability and monitorability as prerequisites for quality assessment and certification.
  • ML-specific quality: ML-specific quality trends include safety, ethics, explainability, ISO 25000 adaptations, and quantitative measurement of AI quality aspects.Several studies examine how individual AI quality aspects relate within an established SE quality model.
  • Standards and certification: Six studies address updating ISO 26262 through deficiency analyses, adaptation proposals, and a methodology for identifying functional deficiencies.These contributions respond to differences between AI-based and traditional software systems.
  • Assurance: 11 primary studies propose ML quality-assurance frameworks covering attributes such as safety, security, robustness, data validation, auditing, and continuous assurance.Approaches include simulation and runtime monitoring for managing identified risks.
  • Key takeaways: Software quality studies predominantly focus on ML quality attributes, frameworks, assurance, and certification, prompting updates to ISO 25000 and ISO 26262.Technical debt discussions warn that data-driven systems can create ongoing maintenance costs when earlier quality deficits accumulate.

6.9 Remaining SWEBOK areas (4 studies)

Research outside the dominant testing and quality areas remains limited and fragmented. Software maintenance is especially neglected, while process research is recent and context-specific, with few holistic approaches connecting SE areas.

  • SE management: SE management studies emphasize active sensemaking because AI software projects create dynamic, complex settings requiring coherence across environments, model ecosystems, and business contexts.Another study introduces end-to-end algorithmic auditing for the internal AI-system development lifecycle.
  • Configuration management: Only one identified paper addresses software configuration management, reporting Amazon experiences with ML model management and outlining its challenges.One additional paper concerns professional practice through documentation for AI systems.
  • Software maintenance: 6 / 248 studies address software maintenance, leaving maintenance, maintainability, evolvability, and technical debt comparatively underrepresented.The paper identifies this as an important research gap for long-lived production systems.
  • Software maintenance: The maintenance gap may reflect researchers’ and practitioners’ focus on initial creation, while many newer AI systems have not yet required sophisticated maintenance approaches.The passage presents these as possible reasons rather than established causes.
  • SE process: SE-process contributions are recent but context-specific; no standard process or manifesto integrates data collection and AI modeling into the SE lifecycle.Some processes evolved from data mining, such as CRISP-DM, while lacking an SE perspective.
  • Research maturity: Researchers are still discovering challenges and practices, indicating that SE4AI remains in a formative stage and needs further studies, especially in less prevalent SWEBOK areas.State-of-practice studies use surveys, interviews, and StackOverflow mining to identify concrete practices and challenges.
  • Holistic integration: 189 / 248 studies focus on a single SWEBOK area, while only a few holistic studies examine the interplay needed across areas for successful AI-based systems.The paper identifies potential for approaches incorporating multiple SE facets in their entirety.

7 RQ4: WHAT ARE THE EXISTING CHALLENGES ASSOCIATED WITH SE FOR AI-BASED SYSTEMS?

The challenges of engineering AI-based systems span requirements, design, construction, testing, quality assurance, and professional practice, with data-related issues recurring across areas. Existing SWEBOK topics do not fully capture these challenges, prompting extensions for AI/ML-specific concerns and applicability in practice.

  • Three proposed SWEBOK extension topics rank among the five most frequently referenced challenge topics.
  • Software requirements: Requirements engineering is challenged by unrealistic accuracy demands, difficult-to-measure requirements, ML-specific trade-offs, uncertainty, and the need to identify discriminatory features.
  • Software design: AI-based system design must address entangled ML models, conflicting quality goals, ethical behavior, undeclared customers, orchestration, metadata, and deployment complexity.
  • Software construction and testing: Construction and testing are complicated by data dependence, complex tools, third-party components, underspecified data and models, and incomplete execution-context coverage.
  • Software Engineering Professional Practice: Professional practice challenges include fragmented toolchains, insufficient expertise, unrealistic customer expectations, continuous-improvement demands, and difficulty explaining black-box failures.
  • Software quality: Quality assurance and verification remain difficult because AI outputs cannot be completely predicted and safety principles, standards, and scalable criteria are not clearly established.

8 THREATS TO VALIDITY

The study addresses selection, data, and research-validity threats through systematic-review standards, snowballing, researcher checks, and documented extraction procedures. Its sample supports general tendencies, but the survey does not claim completeness and includes single-researcher extractions.

  • The threats are organized into study selection validity, data validity, and research validity.
  • Study selection validity: The SMS cannot guarantee inclusion of all relevant works, and snowballing supports valid general tendencies but not completeness.
  • Study selection validity: Snowballing, multiple indexes, external bibliography checks, and iterative search-string pilots were used to mitigate sampling and publication bias.
  • Study selection validity: 248 primary studies were reached through iterative backward and forward snowballing, although the emerging field and limited resources led the authors to stop there.
  • Study selection validity: Each paper was screened by two researchers, with disagreements discussed and unresolved cases referred to a third researcher.
  • Data validity: Most papers were extracted by one researcher, leaving some classifications open to interpretation despite piloting, meetings, and documented decisions.
  • Data validity: The study used SWEBOK for qualitative analysis, iterated classifications among researchers, and inductively grouped extracted quality-attribute terms into codes.

9 CONCLUSIONS

The survey maps SE4AI research and finds a rapidly growing, formative field concentrated in testing and quality, with data-related challenges and underrepresented maintenance. It also reports limited terminology consistency, widespread validity-reporting gaps, and a need for continued survey updates.

  • The survey maps software engineering for artificial intelligence across the SE4AI literature.
  • Research landscape: 102 papers were published in 2019 after almost no papers up to 2015, indicating strong growth in the field.
  • Study quality: Practitioner involvement improves case-study realism and, more significantly, scale, while authors often ignore threats to validity.
  • Terminology: The terminology of primary studies is not homogeneous, making contribution scope difficult to judge; the authors propose taxonomies and the term AI component.
  • Research landscape: Automotive is the most mentioned application domain, while almost half of contributions address no specific domain.
  • SE areas: 115 studies address software testing and 59 address software quality, making them the most prevalent SE areas in the sample.
  • SE areas: Software construction, requirements, and especially maintenance are less represented and offer substantial potential for further research.
  • Challenges: 25% of identified challenges are strongly tied to system domains, while most remaining challenges are technical and data-related rather than economic.

A DATA AVAILABILITY

The replication package makes the study’s search, extraction, and classification materials available for reproducibility.

  • The replication package contains the search string, data extraction form, and detailed classifications from the data analysis.

B CREDIT AUTHOR STATEMENT

The credit statement assigns conceptualization, analysis, investigation, data curation, writing, review, visualization, and project administration across the listed authors.

  • The authors contributed across conceptualization, formal analysis, investigation, data curation, writing, review, and visualization.
  • Project administration is explicitly listed among the contributions.
Loading 2105.01984v2…