PaperlayerView exact revision on arXiv

Source-linked AI summary

Challenges and solutions when adopting DevSecOps: A systematic review

Roshan N. Rajapakse, Mansooreh Zahedi, M. Ali Babar, Haifeng Shen

arXiv:2103.08266v2cs.SE

TL;DR

DevSecOps adoption must reconcile rapid DevOps delivery with secure software delivery, while existing evidence on practitioner challenges and solutions remained fragmented. This study conducts a systematic literature review of 54 peer-reviewed studies and identifies 21 challenges, 31 solutions, their mappings, and research gaps. Tool automation, shift-left security, continuous security assessment, and balancing delivery speed with security emerge as central findings.

  • Problem

    Rapid DevOps delivery creates difficulty maintaining software security because established security practices can require substantial manual effort or time.

  • Method

    The study conducts a systematic literature review using searches, screening, quality assessment, snowballing, data extraction, and thematic analysis of peer-reviewed studies.

  • Results

    21 challenges, 31 specific solutions, their mappings, and key research gaps were identified across People, Practices, Tools, and Infrastructure.

  • Takeaways & Limitations

    The findings highlight developer-centered security testing for continuous practices, automation of traditionally manual security work, and the need to balance delivery speed with security.

  • Takeaways & Limitations

    The results may be affected by imperfect study collection, selection and extraction bias, and publication bias.

Abstract

from arXiv · show

Context: DevOps has become one of the fastest-growing software development paradigms in the industry. However, this trend has presented the challenge of ensuring secure software delivery while maintaining the agility of DevOps. The efforts to integrate security in DevOps have resulted in the DevSecOps paradigm, which is gaining significant interest from both industry and academia. However, the adoption of DevSecOps in practice is proving to be a challenge. Objective: This study aims to systemize the knowledge about the challenges faced by practitioners when adopting DevSecOps and the proposed solutions reported in the literature. We also aim to identify the areas that need further research in the future. Method: We conducted a Systematic Literature Review of 54 peer-reviewed studies. The thematic analysis method was applied to analyze the extracted data. Results: We identified 21 challenges related to adopting DevSecOps, 31 specific solutions, and the mapping between these findings. We also determined key gap areas in this domain by holistically evaluating the available solutions against the challenges. The results of the study were classified into four themes: People, Practices, Tools, and Infrastructure. Our findings demonstrate that tool-related challenges and solutions were the most frequently reported, driven by the need for automation in this paradigm. Shift-left security and continuous security assessment were two key practices recommended for DevSecOps. Conclusions: We highlight the need for developer-centered application security testing tools that target the continuous practices in DevSecOps. More research is needed on how the traditionally manual security practices can be automated to suit rapid software deployment cycles. Finally, achieving a suitable balance between the speed of delivery and security is a significant issue practitioners face in the DevSecOps paradigm.

1. Introduction

DevOps accelerates software delivery but complicates secure delivery because conventional security activities can be manual or time-consuming. This study therefore systematically examines DevSecOps adoption challenges, solutions, and research gaps.

  • DevOps reduces the time from a system change to production, but rapid delivery makes maintaining software security difficult.
  • Traditional security activities often require substantial manual effort or time, so applying them unchanged can hinder deployment speed.
  • DevSecOps integrates security controls and practices into the DevOps cycle while keeping security a priority.
  • The study addresses the reported difficulty of transitioning to DevSecOps by identifying adoption challenges and available solutions for practitioners.
  • The authors systematically select, thematically analyze, and present DevSecOps challenges, solutions, and gaps for further research.

2. Background and related work

DevSecOps extends DevOps by integrating security throughout continuous software practices and emphasizing collaboration among development, operations, and security. Earlier reviews addressed related topics but generally differed in scope, evidence base, or coverage of continuous-practice security.

  • Continuous Software Engineering organizes software activities as an ongoing flow, including practices such as Continuous Integration.
  • Continuous Integration combines frequent code integration with automated builds and tests to detect integration failures early.
  • Continuous Delivery automatically deploys outputs to production when quality gates pass, whereas Continuous Deployment and related practices define continuous delivery relationships.
  • DevOps is treated as a combination of values, principles, methods, practices, and tools that enables continuous and rapid delivery.
  • DevSecOps integrates security principles and practices through communication, collaboration, and integration among development, operations, and security teams.
  • This review combines adoption challenges, proposed solutions, their mapping, and continuous-practice security, extending coverage beyond earlier reviews.

3. Methodology

The study uses a Kitchenham-and-Charters-guided systematic literature review, combining targeted searches, eligibility screening, quality assessment, snowballing, collaborative extraction, and thematic synthesis.

  • The authors selected a systematic literature review because it can support informed decisions about technology selection and adoption.
  • The review followed Kitchenham and Charters’ guidance and organized its design around research questions, search, selection, extraction, and synthesis.
  • The search covered DevSecOps-related terms and security aspects of continuous integration, delivery, deployment, and software engineering.
  • Scopus, IEEE Xplore, and ACM Digital Library were searched in June 2020, retrieving 460 papers and leaving 283 after duplicate removal.
  • Eligibility required peer-reviewed English full-text studies addressing security in DevOps or selected continuous practices, excluding reviews and papers of five pages or fewer.
  • Quality assessment, supervised snowballing, and screening of secondary-study references produced a final in-scope set of 54 papers.
  • The authors extracted and reviewed data collaboratively, then used Braun and Clarke’s thematic analysis with NVivo open coding to synthesize challenges and solutions.

4. Results and Discussion

The results section first presents an overview of the primary studies and then addresses the study’s research questions.

  • The results begin by characterizing the primary studies before discussing answers to the research questions.
  • The section is organized around an overview followed by the study’s research-question findings.
  • The passage establishes the results section’s two-part sequence without reporting substantive findings in this excerpt.

4.1. Overview of the primary studies

The review analyzed studies published from 2011 to June 2020, with research interest emerging around 2013–2014 and growing thereafter.

  • 54 primary studies published from 2011 to June 2020 formed the review’s evidence base.
  • Research interest in DevOps and continuous software engineering security emerged around 2013–2014 and increased gradually.
  • 33 of 54 studies, or over 60%, appeared across a broad range of conferences.

4.2. Overview of the results

The review organized its findings into four themes and identified 21 adoption challenges alongside 31 proposed solutions. These findings were mapped by theme and by the number of primary studies addressing each challenge or solution.

  • The findings were classified into People, Practices, Tools, and Infrastructure.
  • 21 challenges and 31 proposed solutions were identified and assigned to the most relevant themes.
  • The results were presented according to the number of primary studies associated with each challenge or solution.Figure 5 represents this frequency through bubble size.

4.3. RQ1: What are the specific DevSecOps adoption challenges?

DevSecOps adoption challenges span tool selection, automation, configuration, analysis, containers, CI/CD pipelines, infrastructure, and collaboration. Across these areas, security activities and tooling must operate within rapid, frequently changing delivery environments.

  • Tools: Different team preferences create tool-selection problems, while the lack of standards complicates security automation across many available tools.
  • Tools: SAST tools generate false positives, require expert review, and may scan too slowly or consume too many resources for frequent incremental commits.
  • Tools: DAST tools require software to be built, installed, and configured, involve manual setup, and typically take longer to run, limiting frequent release use.
  • Tools: Container reuse and third-party elements can introduce vulnerabilities, expand the attack surface, and expose systems to insecure configurations or access controls.
  • Tools: CI systems, IaC scripts, CaC tools, and CD pipelines present security risks through exposed attack vectors, security smells, weak prioritization, and insufficiently prominent security requirements.
  • Tools: Shared pipeline access can expose infrastructure and applications to risks when team members lack sufficient security knowledge or awareness.

4.4. RQ2: What are the solutions proposed to address the DevSecOps adoption challenges?

The literature proposes solutions spanning tools, practices, infrastructure, and people to address DevSecOps adoption challenges. Tool-oriented measures emphasize standardization, contextual configuration, automation, continuous assessment, and pipeline protection.

  • Tools: Tool standards can reduce challenges caused by difficult tool selection and usage decisions.Studies recommend community convergence on commonly accepted selection and usage guidelines.
  • Tools: Security-supported documentation and centralized repositories improve access to tool configurations and security settings.Clear documentation can address configuration-management and security-setting challenges.
  • Tools: Hybrid testing, continuous vulnerability assessment, cloud services, orchestration, virtualization, and infrastructure-as-code address tool and pipeline security limitations.Proposals include IAST, combined static and dynamic analysis, cloud-based analysis, container orchestration, build encapsulation, and IaC security checks.
  • Practices: Practice-oriented solutions include shifting security left, continuous assessment, rapid patching, threat analysis, machine-readable requirements, compliance automation, metrics, documentation, and behavioral analytics.These proposals target earlier detection, continuous feedback, automated compliance, measurable security activity, audit evidence, and user-behavior insight.
  • Tools: Context-specific tool configuration, targeted checks, and optimized pipelines can improve scanning practices and reduce unnecessary scanning time.Recommended practices include avoiding defaults, excluding non-production code, preventing duplicate checks, and running static analyses in parallel.

4.4.2. Summary of the tools related solutions •

Tools-related solutions focus on adapting security tooling to rapid DevSecOps delivery through standards, contextual configuration, automation, continuous assessment, and cloud or hybrid services.

  • Tools: Tools targeting DevSecOps, including hybrid tools and continuous vulnerability assessment tools, are recommended for rapid delivery environments.Cloud solutions are also recommended when they reduce drawbacks of in-house tools.
  • Practices: Machine-readable standards, automated compliance testing, and requirement-analysis toolchains support automation of traditionally manual security practices.These approaches translate requirements into analyzable models and adapt standards into testable criteria.
  • Practices: Security metrics, process documentation, logging, and behavioral analytics provide measurement, audit evidence, and feedback for continuous practices.Proposed metrics include developer activity, commits, discovered mistakes, correction time, and penetration-testing results.
  • Practices: Shifting security left places security activities early, while continuous assessment treats security as a concern throughout development and after deployment.Continuous monitoring is cited for regulated environments and for feedback from operations to development.
  • Practices: Rapid patch delivery and threat analysis use DevOps practices to address production vulnerabilities and identify threats across development outputs.STRIDE threat modeling is specifically proposed for detecting vulnerabilities in continuous delivery pipelines.

4.4.4. Summary of the solutions related to practices •

Practice and infrastructure solutions extend security across the DevSecOps lifecycle while supporting regulated, complex, and heterogeneous environments. People-focused measures emphasize collaboration, controlled communication, champions, and training.

  • Practices: Security should remain a key concern from development start through the entire lifecycle, supported by automation, metrics, and related strategies.The synthesis combines shifting-left and continuous-security recommendations with machine-readable SLAs and other automated practices.
  • Infrastructure: Strict access management, infrastructure as code, simulation environments, and model-driven engineering address security needs in complex infrastructures.IaC supports versioning, testing, building, and deployment, while simulation enables internal testing in regulated environments.
  • Infrastructure: Systematic vulnerability evaluation, data-security-focused hybrid life cycles, and frameworks target regulated, multi-cloud, and other challenging infrastructures.Frameworks provide holistic support for DevSecOps adoption in several infrastructure settings.
  • People: Multidisciplinary teams, standardized communication, separation of duties, and automated information sharing support collaboration without removing controls.Continuous communication of risks and evaluation results is especially required in regulated or restricted environments.
  • People: Security champions bridge development and security teams and can reduce developer resistance because they are typically members of the development team.Champions are security-minded developers who usually receive the most security training within a team.
  • People: HRM programs, security knowledge sharing, and training address organizational concerns and help developers use security tools and interpret their outputs.Training supports recognition of genuine security issues and false positives in static analysis.

4.5. DevSecOps support in practice

DevSecOps support should integrate suitable security controls into the workflow from project inception without undermining rapid delivery. The proposed combination includes hybrid testing tools and security-testing services to balance deployment speed with secure outputs.

  • DevSecOps support in practice: Integrating security controls without negatively affecting rapid delivery is a central practical DevSecOps challenge.The range of available assessment methods can also overwhelm developers in rapid-deployment environments.
  • DevSecOps support in practice: Development teams should establish the workflow early and integrate tools designed to facilitate rapid software deployment.IAST is proposed as a hybrid tool combining advantages of traditional application-security testing tools.
  • DevSecOps support in practice: Application security testing as a service can reduce result times and provide higher testing speed and accuracy when in-house expertise is unavailable.Static analysis as a service is given as an example for specific testing activities.
  • DevSecOps support in practice: Combining the proposed solutions can help practitioners balance deployment speed with the security of delivered outputs.The conclusion concerns the combined use of workflow, tooling, and service-based support.

4.6. RQ3: What are opportunities for future research or gap areas for technological development (e.g., tool support) or framework support in this domain?

The study maps DevSecOps challenges and proposed solutions to identify gaps requiring future research, tool development, and framework support. Key opportunities concern developer-centered tools, automation of security practices, role clarity, empirical validation, and balancing delivery speed with security.

  • The need for security tools that complement rapid deployment cycles: Application security testing tools need to complement rapid DevSecOps deployment cycles, including through hybrid tools and service-based delivery models.The review identifies hybrid SAST, DAST, and IAST approaches and application security testing as a service as areas needing further research.
  • The need for security tools that target developers: Developer-centered security tools should support developers’ workflows rather than security experts’ workflows, addressing tool selection, configuration, integration, and documentation challenges.The study identifies empirically evaluated developer-centered security solutions as a gap area.
  • Continuous vulnerability discovery and management: Practical guidance is needed to balance DevOps release speed and software security while improving continuous vulnerability discovery and management.Engineers face substantial effort in triaging and rectifying vulnerabilities, creating a speed-security trade-off.
  • Consensus on shift left and continuous security: More research is needed to automate traditionally manual security practices and establish consensus on implementing and validating shift-left security and continuous security assessment.Threat modeling, penetration testing, and code review were among the practices not addressed by the limited solutions captured.
  • People, roles, and metrics: Future studies should clarify security roles, investigate people-related effects across DevSecOps components, and develop empirically validated security metrics.The review links unclear decision-making roles and limited people-focused research to tool, practice, and measurement gaps.
  • Empirically validated frameworks: Empirically validated frameworks are needed for adopting DevSecOps in complex, resource-constrained, or highly regulated environments.Existing frameworks were often presented as contextual case studies or were not adequately evaluated, limiting evidence for broader applicability.

5. Threats to validity

The review identifies threats from incomplete study collection, selection and extraction bias, and publication bias, while describing mitigation steps and their expected effects.

  • Overall threats: The review results may also be affected by imperfect primary-study collection, selection and extraction bias, and publication bias.These threats are reported as common concerns in systematic reviews.
  • Missing primary studies: The review may miss relevant primary studies because of limitations in search methods and non-comprehensive venues or databases.Pilot searches, iterative search-string refinement, multiple sources, and backward and forward snowballing were used to reduce this threat.
  • Selection and extraction bias: Study-selection and data-extraction bias may affect the review, despite a protocol defining research questions, procedures, eligibility criteria, and extraction strategy.The protocol was intended to improve consistency in selecting studies and extracting data.
  • Publication bias: Publication bias may favor reporting positive results, but the review included many studies describing negative effects such as practitioner challenges.The authors therefore judge that unreported negative results would have only a moderate effect on the identified gaps.

6. Conclusion

The conclusion identifies automation and tool usage as the main research focus, while emphasizing shift-left security, continuous assessment, and unresolved automation and speed–security challenges.

  • Tools and automation: Automation and tool usage are the main DevSecOps research focus, because older SAST and DAST technologies have drawbacks for rapid deployment cycles.The conclusion identifies hybrid tools and IAST as current development needs.
  • Practices: Shift-left security and continuous security assessment are key recommended practices, but tool support for continuous assessment remains lacking.The conclusion specifically calls for tools supporting continuous security assessment throughout the deployment cycle.
  • Tool-related risks: Tool-related security issues, including container vulnerabilities, require developers to reduce their effects when integrating tools into deployment pipelines.The conclusion connects tool adoption with the need for appropriate developer action.
  • Automation gaps: Inability to automate traditionally manual security practices remains a critical problem in DevSecOps.The supplied conclusion identifies automation of manual practices as an unresolved field challenge.

9. Appendix: List of selected papers

The appendix lists selected papers covering security in DevOps and DevSecOps across continuous delivery, cloud, containers, frameworks, assessment, and related practices.

  • Cloud and containers: The selected papers include work on container platforms, secure registries, cloud applications, and systems-of-systems security.These entries include OKD and Harbor, secure cloud registries, cloud applications, and systems-of-systems security.
  • Continuous practices: Several papers address continuous integration, deployment, security assessment, patch delivery, and continuous software engineering.The list includes studies on continuous integration, continuous deployment, continuous security assessment, patch delivery, and continuous software engineering.
  • Security practices and tools: The appendix includes studies on DevOps security practices, secure environments, compliance, and security-focused continuous integration tools.Examples cover regulated environments, DevOps security programs, compliance at velocity, and security-focused CI.
  • Assurance and infrastructure: Other selected papers examine trustworthy systems, vulnerability analysis, cloud security, microservices, containers, and security assurance.The entries span IoT systems, Docker vulnerabilities, microservice infrastructures, container orchestration, and DevOps assurance.
  • Additional topics: The list also contains research on static analysis, DevOps implementation, self-healing systems, cybersecurity monitoring, GDPR compliance, and container deployment frameworks.These papers address CI static analysis, regulated adoption, cybersecurity monitoring, cloud compliance, and continuous container evolution.
Loading 2103.08266v2…