PaperlayerView exact revision on arXiv

Source-linked AI summary

Software Security Patch Management -- A Systematic Literature Review of Challenges, Approaches, Tools and Practices

Nesara Dissanayake, Asangi Jayatilaka, Mansooreh Zahedi, M. Ali Babar

arXiv:2012.00544v3cs.SE

TL;DR

Known vulnerabilities continue to be exploited despite available patches, and organisational patching has complex socio-technical challenges. This systematic review synthesises 72 studies on challenges, solutions, and their evaluation. It finds that half of common challenges lack directly associated solutions, while rigorous industrial evaluation is limited.

  • Problem

    Known vulnerabilities remain a source of cyberattacks despite available patches, and the socio-technical challenges of organisational patching have received limited research attention.

    Source: §1 · p. 2, §4.1 · p. 15, §4.5 · p. 17

  • Method

    The authors systematically reviewed 72 primary studies and organised reported challenges, solutions, and evaluation evidence.

    Source: §3.1 · p. 8, §10 · p. 32

  • Results

    50% of common challenges are not directly associated with solutions, and only 20.8% of solutions have been rigorously evaluated in industry.

    Source: §10 · p. 33

  • Takeaways & Limitations

    Patch information retrieval, patch testing, and post-deployment verification are underresearched, with five studies (6.9%) each, indicating a need for further study.

    Source: §10 · p. 33

    The findings may not generalize to grey literature or studies outside the review period, and the authors cannot guarantee that all relevant studies were captured.

    Source: §9.2 · p. 31, §9.3 · p. 31

Abstract

from arXiv · show

Context: Software security patch management purports to support the process of patching known software security vulnerabilities. Given the increasing recognition of the importance of software security patch management, it is important and timely to systematically review and synthesise the relevant literature on this topic. Objective: This paper aims at systematically reviewing the state of the art of software security patch management to identify the socio-technical challenges in this regard, reported solutions (i.e., approaches, tools, and practices), the rigour of the evaluation and the industrial relevance of the reported solutions, and to identify the gaps for future research. Method: We conducted a systematic literature review of 72 studies published from 2002 to March 2020, with extended coverage until September 2020 through forward snowballing. Results: We identify 14 socio-technical challenges, 18 solution approaches, tools and practices mapped onto the software security patch management process. We provide a mapping between the solutions and challenges to enable a reader to obtain a holistic overview of the gap areas. The findings also reveal that only 20.8% of the reported solutions have been rigorously evaluated in industrial settings. Conclusion: Our results reveal that 50% of the common challenges have not been directly addressed in the solutions and that most of them (38.9%) address the challenges in one phase of the process, namely vulnerability scanning, assessment and prioritisation. Based on the results that highlight the important concerns in software security patch management and the lack of solutions, we recommend a list of future research directions. This study also provides useful insights about different opportunities for practitioners to adopt new solutions and understand the variations of their practical utility.

1. Introduction

Known vulnerabilities remain a major source of cyberattacks despite available patches, while organisational patching is complex and socio-technical. This review synthesises research on that process to identify challenges, solutions, evaluation rigour, and research gaps.

1. Introduction

  • Many cyberattacks exploit known vulnerabilities for which patches already existed, while patching in organisational IT environments involves complex issues.The paper links such attacks to severe financial, reputational, and human consequences.Source: §1 · p. 2
  • More than 50% of organisations miss the recommended 72-hour window for patching critical vulnerabilities, and around 15% remain unpatched after 30 days.The paper describes patch management as balancing early patching against the volume of enterprise applications.Source: §1 · p. 2
  • Socio-technical aspects have received relatively limited attention, and the authors found no review organising this literature's challenges and solutions.These aspects include organisational processes, policies, skills, resources, and interactions between people and technical solutions.Source: §1 · pp. 2–3
  • The review analyses 72 primary studies to consolidate knowledge, identify socio-technical challenges, classify solutions, assess evaluation rigour and industrial relevance, and highlight future research gaps.The authors also frame the review as guidance for researchers and practitioners seeking to understand the process and transfer research outcomes to industry.Source: §1 · p. 3
  • The paper examines whether reported solutions have been evaluated rigorously and are relevant to industry, and identifies concerns requiring further research.Its contributions include analysis intended to inform transferability to industrial environments.Source: §1 · p. 3

2. Background and Related Work

The background defines software security patch management as a multi-phase process for applying security patches and situates this review among work on related topics. Earlier reviews focus on standards or dynamic software updating, whereas this review examines challenges and solutions across the process.

2.1. Overview of the software security patch management process

  • Software security patch management covers identifying, acquiring, testing, installing, and verifying patches for software products and systems.The process focuses on organisations applying patches to deployed third-party software after vendors release them.Source: §2.1 · p. 4
  • The five phases are patch information retrieval; vulnerability scanning, assessment and prioritisation; patch testing; deployment; and post-deployment verification.Testing includes preparing patches for installation, while verification includes monitoring for interruptions and handling post-deployment issues.Source: §2.1 · pp. 4–5

2.2. Other reviews related to software security patch management

  • The authors found no existing systematic review or mapping study focused on the software security patch management process.Table 1 is identified as a comparison of this study's contributions with related reviews and surveys.Source: §2.2 · p. 5, Table 1 · p. 6
  • Gentile and Serio surveyed patch-management standards and practices for complex infrastructure, particularly industrial control systems, and proposed a general-purpose workflow for that context.The present review also covers practices, but focuses on challenges and solutions beyond the ICS context.Source: §2.2 · p. 5
  • Reviews of dynamic software updating focus on live patching to avoid restarts or downtime and therefore address the deployment phase.This review distinguishes itself by identifying and analysing challenges and solutions across software security patch management.Source: §2.2 · pp. 5–6

3. Research Methodology

The authors followed systematic-review guidance and used an agreed protocol and collaborative review process. They framed research questions about challenges, solutions, and evaluation, then developed and refined a Scopus search strategy through pilot searches.

3. Research Methodology

  • The review followed Kitchenham and Charters guidelines, with a protocol, jointly developed search string, jointly selected studies, and supervised extraction, quality assessment, and synthesis.The team discussed and verified the synthesis in weekly meetings over eight months.Source: §3 · p. 6

3.1. Research Questions

  • The research questions address socio-technical challenges, available solutions, and how solutions have been evaluated.The review classifies technical approaches and tools separately from practices such as recommendations, guidelines, and shared experience.Source: §3.1 · pp. 7–8

3.2. Search Strategy

  • The authors constructed and refined search terms through pilot searches, aiming to include known studies despite inconsistent terminology in this emerging topic.They used Scopus and searched titles, abstracts, and keywords; they kept the search generic and did not restrict it to a time range.Source: §3.2 · pp. 8–9

TITLE-ABS-KEY ((‘software’ OR ‘system’) AND (‘patch*’) AND (‘security’ OR ‘vulnerabilit*’))

The review selected 72 primary studies using explicit inclusion criteria and a quality threshold, then extracted and synthesised evidence through descriptive and thematic analysis mapped to the patch-management process.

3.3. Study Selection

  • 72 primary studies remained after applying inclusion and exclusion criteria and excluding studies below the publication-quality threshold.The search retrieved 2,434 studies from Scopus; backward and forward snowballing extended coverage through September 2020.Source: §3.3 · p. 9

3.4. Assessing the publication quality

  • Studies were graded Yes, Partially, or No, scored 2, 1, or 0, and included only if their average quality score was at least 0.5.Two studies were excluded on quality; a second author independently checked a randomly selected subset, with disagreements resolved through discussion.Source: §3.4 · pp. 9–10

3.5. Data Extraction

  • The researchers used a predefined extraction form, piloted on five randomly selected studies, to collect demographic information and critical summaries for answering the research questions.The first author conducted the pilot under the other authors’ supervision and refined the form through discussion.Source: §3.5 · p. 10

3.6. Data Analysis and Synthesis

  • Descriptive statistics summarised demographic data, while thematic analysis classified challenges and solutions, iteratively grouped codes into themes, and mapped findings to process phases.All authors reviewed the synthesis results in weekly meetings.Source: §3.6 · pp. 10–11

3.7. Overview of Selected Primary Studies

The 72 reviewed studies span 2002–2020, but their distribution is uneven across process phases and research types. Most concentrate on vulnerability scanning, assessment, and prioritisation, and validation research predominates.

3.7.1. Demographic data

  • The 72 primary studies were published from 2002 to 2020; 43 (60%) appeared in conferences and 40% in scientific journals.The authors found no relevant studies published before 2002.Source: §3.7.1 · p. 11, Figure 3 · p. 12

3.7.2. Studies distribution in the software security patch management process

  • 38.9% of studies focused on vulnerability scanning, assessment, and prioritisation, while only 6.9% focused on patch information retrieval, testing, or post-deployment verification.Twenty studies (27.8%) addressed more than one process phase and were classified as overall-process studies.Source: §3.7.2 · p. 11, Figure 4 · p. 12

3.7.3. Research Type

  • 44 studies (61.1%) reported validation research, compared with 10 (13.9%) reporting evaluation research using stronger empirical methods.The review also classified 13 studies (18.1%) as solution proposals and 9 (12.5%) as experience papers; it found no philosophical or opinion papers.Source: §3.7.3 · p. 12

4. Socio-technical challenges in software security patch management

The review identifies 14 socio-technical challenges, spanning cross-cutting organisational and technical barriers as well as problems at each patch-management phase. These include coordination, resource and tool limitations, context-sensitive assessment, testing, deployment, and verification difficulties.

4.1. Common challenges

  • The review identified 14 challenges, distinguishing those common across process phases from challenges specific to individual phases.Common barriers include stakeholder coordination and conflicting organisational policies, while technical limitations and resource constraints also recur.Source: §4 · p. 12, §4.1 · pp. 15–16
  • Cross-cutting challenges include stakeholder coordination, conflicting organisational policies, increasing patch complexity, and limited human expertise, process guidance, and automation.Existing tools also lack integration, accuracy, security, and scalability.Source: §4.1 · pp. 15–16

4.2. Patch information retrieval related challenges

  • Patch information retrieval is hindered by fragmented sources and a lack of automated validation, filtering, and classification tailored to organisational needs.Practitioners may spend hours monitoring multiple sources; delayed application can increase zero-day attack risk.Source: §4.2 · p. 16

4.3. Vulnerability scanning, assessment and prioritisation related challenges

  • Scanning can miss vulnerabilities and misconfigurations, while assessment tools may fail to account for dynamic organisational contexts and technical or business priorities.Existing approaches are often “one size fits all,” and public vulnerability data may lack metrics needed for accurate prioritisation.Source: §4.3 · pp. 16–17

4.4. Patch testing related challenges

  • Patch testing lacks adequate automation, in part because dependencies and production-like test environments are difficult to manage; manual testing can be delayed and error-prone.Manual testing is also used to avoid unexpected breakdowns from faulty or malicious patches.Source: §4.4 · p. 17

4.5. Patch deployment related challenges

  • Faulty patches and missing configuration or dependency changes can cause deployment errors and downtime, while organisational downtime constraints can delay installation.These constraints are particularly challenging in critical infrastructure contexts such as healthcare.Source: §4.5 · p. 17

4.6. Post-deployment patch verification related challenges

  • Many solutions lack automated post-deployment verification that gives an overview of patch state, making it difficult to locate problems after installation.Manual auditing is time-consuming and cannot guarantee that every intrusion or attacker change is found and reversed.Source: §4.6 · pp. 17–18

5. Approaches and tools proposed to facilitate software security patch management

The review maps proposed approaches and tools to the patch-management process, finding that approaches greatly outnumber tools. Reported solutions span retrieval, assessment, testing, deployment, and post-deployment verification, with varying capabilities across phases.

5. Approaches and tools proposed to facilitate software security patch management

  • 75% of proposed solutions were approaches, while 5.6% were tools, and the review mapped their capabilities to process phases.Reported capabilities include retrieval and filtering, vulnerability and risk assessment, testing, deployment, and post-deployment verification.Source: §5 · p. 18, Table 6 · p. 18

5.1. Patch information retrieval related solutions

  • Patch-information solutions combine retrieval from multiple sources with filtering, classification, validation, downloading, and distribution.One reported approach uses a trust and reputation system to verify patch information from varied sources.Source: §5.1 · p. 19

5.2. Vulnerability scanning, assessment and prioritisation related solutions

  • Scanning solutions aggregate results to show system patch state, while assessment approaches adapt vulnerability-risk analysis to context and supplement CVSS with additional metrics.Proposed assessment methods include context-specific risks and real-time feedback on remediation delays and patch applicability.Source: §5.2 · pp. 19–20

5.3. Patch testing related solutions

  • Testing approaches include automated detection and recovery for faulty or malicious patches, but the review found only one proposed solution for surviving crashes from faulty patches.Other reported approaches include tandem execution for exploit detection and blockchain for patch integrity during distribution.Source: §5 · p. 19, §5.3 · p. 20

5.4. Patch deployment related solutions

  • Deployment solutions target heterogeneous environments, lower deployment time and overhead, and reduce downtime or reboots.Examples include Shadow Reboot and checkpoint/restart; one reported checkpoint/restart approach reduces downtime to three seconds.Source: §5 · p. 19, §5.4 · pp. 20–21

5.5. Post-deployment patch verification related solutions

  • Post-deployment solutions automate verification and repair past exploits, including systems that show patch status or restore exploited changes while preserving legitimate user changes.The review also lists automated repair of past exploits as a reported capability.Source: §5 · p. 19, §5.5 · p. 21

6. Practices proposed to successfully implement software security patch management

The review classifies reported practices as common across the process or specific to individual phases, spanning planning and coordination through post-deployment verification.

6.1. Common Practices

  • Formal policies, procedures, defined roles, management support, and stakeholder communication are recommended to coordinate the process and reduce administrative overhead and approval delays.The review also recommends measuring policy effectiveness and updating policies and procedures regularly.Source: §6.1 · p. 23

6.2. Patch information retrieval related practices

  • Formal responsibilities for retrieving, downloading, and disseminating patch information, including maintaining source lists, are reported to reduce information-retrieval latency.The review also recommends upstream and downstream infrastructure for patch distribution to limit latency.Source: §6.2 · p. 24

6.3. Vulnerability scanning, assessment and prioritisation related practices

  • Organisations should monitor active and inactive applications, retain scanning histories, maintain system inventories, and assess vulnerability risk in their own context.This avoids relying solely on vendor vulnerability scores; monitoring inactive applications matters because they may still be exploited.Source: §6.3 · p. 24

6.4. Patch testing related practices

  • Testing practices include preparing test environments in advance, testing all security patches, and developing and testing back-out procedures.These recommendations address testing time and the risks of faulty or malicious patches.Source: §6.4 · p. 24

6.5. Patch deployment related practices

  • Patch installation should be timely but balance testing time, attack risk, and service availability through risk-focused management and severity-based scheduling.Scheduling can account for both patch severity and its impact on managed systems.Source: §6.5 · p. 24

6.6. Post-deployment patch verification related practices

  • Regularly monitoring patch status and collecting client feedback after each job can verify deployment and detect post-deployment problems early.The review connects tracking with identifying potential exploits during deployment and adverse effects on service continuity.Source: §6.6 · p. 24

7. Evaluation of the reported solutions in software security patch management

The review classifies how approaches, tools, and practices were evaluated, then assesses evaluation rigour and industrial relevance. Most solutions lacked industrially relevant evaluation, and replication was uncommon.

7. Evaluation of the reported solutions in software security patch management

  • The review adapted an existing evaluation-type classification by adding simulation with real data and no evaluation.Evaluation types were examined by solution type and process phase, with the distribution shown in Figure 6.Source: §7 · p. 25, §7.1 · p. 25, Table 8 · p. 25

7.1. Types of evaluation used to assess the proposed solutions

  • For approaches, laboratory experiments with software subjects (30.36%) and simulations with artificial data (28.57%) were the most frequent evaluation types.Four studies reported no evaluation; case studies (3.57%), laboratory experiments with human subjects (1.79%), and rigorous analysis (1.79%) were rare.Source: §7.1 · p. 26, Figure 6 · p. 26
  • For tools, simulation with real data was used most often (50%), while practice evaluations most often used experience (56.25%) or case studies (31.25%).The reported practice case studies included industrial pilots, practitioner surveys, and interviews; one recommendation set was based on rigorous analysis of 1593 client-side vulnerabilities.Source: §7.1 · p. 26
  • Eleven studies (15.3%) used two evaluation types, including combinations of simulation and discussion to assess robustness and practical utility.Some studies proposed an approach and a tool addressing multiple process phases.Source: §7.1 · p. 26

7.2. The level of rigour and industrial relevance of the reported solutions

  • Only 15 solutions (20.8%) had industrially relevant evaluation, while 65 studies (90.3%) evaluated their solutions in only one study.The industrially relevant evaluations comprised seven case studies, three field experiments, and five experience reports; the authors identify a general lack of replication.Source: §7.2 · p. 27

8. Discussion

The discussion maps challenges to solutions and identifies underexplored process phases and socio-technical concerns. It recommends work on human-AI collaboration, tool standardisation, industrial evaluation, and contextual reporting.

8. Discussion

  • The challenge–solution mapping identifies dependencies among challenges and among practices, including how weak test strategy can exacerbate test-quality problems and faulty deployments.The authors contrast negative challenge dependencies with practices that positively support one another.Source: §8 · p. 27, Figure 7 · p. 28

8.1. Need for more investigation on the less explored software security patch management phases

  • Vulnerability scanning, assessment, and prioritisation drew 28 studies (38.9%), while patch information retrieval, testing, and post-deployment verification each drew only five (6.9%).The authors call for more research on these less-explored phases in a process whose stages are tightly coupled.Source: §8.1 · pp. 28–29

8.2. Need for focus on socio-technical aspects in software software security patch management

  • Limited attention to collaboration, coordination, and communication is a concern because these socio-technical factors can affect timely remediation amid complex patch interdependencies.The authors propose further research on these factors and tools that support coordination across tasks and stakeholders.Source: §8.2 · p. 29

8.3. Human-AI collaboration for securing software systems

  • The authors recommend balancing automation with human decision-making where patch complexity or technology limits prevent full automation.They identify human-AI collaboration as a research opportunity and envision systems that provide timely information and guide practitioner decisions.Source: §8.3 · pp. 29–30

8.4. Standardisation of heterogeneous tools

  • Patch heterogeneity, organisational software diversity, and limited tool scalability motivate research on adaptable architectures to standardise heterogeneous patch-management tools.The authors observe that most reported solutions are compatible only with Linux.Source: §8.4 · p. 30

8.5. Real-world, rigorous evaluations

  • Because few studies used mature evaluation methods such as field experiments and case studies, the authors recommend more rigorous evaluation with practitioners.They state that robust evaluation can improve research-outcome quality and transferability to industrial adoption.Source: §8.5 · p. 30

8.6. Contextual factor

  • Only 20.8% of reviewed studies provided industry-related evidence, prompting a recommendation to report contextual factors to support credibility and quality.The paper says such evidence helps researchers assess practical utility and practitioners consider adoption.Source: §8.6 · p. 30

9. Threats to Validity

The review describes measures to address validity threats in study selection, extraction, synthesis, and search, while acknowledging limits to coverage and generalisability.

9.1. Internal validity

  • To address internal validity threats, the authors used a review protocol, piloted extraction, cross-checked study selection and extracted data, and resolved disagreements through team discussion.They also used snowballing and reported negative findings to mitigate publication bias.Source: §9.1 · p. 31

9.2. External validity

  • The authors acknowledge that findings may not generalise to grey literature or studies outside the review period.They used broad searches and Scopus to increase coverage, but note this boundary remains.Source: §9.2 · p. 31

9.3. Construct validity

  • The authors cannot guarantee that all relevant primary studies were captured, given limits in search-string construction and digital-library coverage.They used pilot searches, a broad search string, and snowballing to mitigate the risk of missing studies.Source: §9.3 · pp. 31–32

9.4. Conclusion validity

  • To reduce researcher bias in synthesis, the authors shared codebooks, reviewed emerging analyses together, and discussed disagreements until reaching agreement.The first author led analysis and synthesis, with other authors examining codes, themes, and results.Source: §9.4 · p. 32

10. Conclusion

The review synthesises 72 primary studies on software security patch management, identifying socio-technical challenges, solutions, and gaps in evaluation and coverage. Its mapping finds that half of common challenges lack directly associated solutions, while solutions cluster in vulnerability scanning, assessment, and prioritisation.

10. Conclusion

  • 50% of common challenges are not directly associated with solutions, revealing open research challenges.The review identifies 14 socio-technical challenges and classifies solutions into six themes of approaches and tools and 12 practices.Source: §10 · pp. 32–33
  • 37.5% of reviewed studies address vulnerability scanning, assessment, and prioritisation, while patch information retrieval, testing, and post-deployment verification each have only 5 studies (6.9%).The latter phases are identified as underresearched.Source: §10 · p. 33
  • Only 20.8% of solutions were rigorously evaluated in industry using representative approaches such as field experiments and case studies.The review calls for robust evaluation with more representative and diverse cases.Source: §10 · p. 33
  • Reported solution types comprise 75% approaches, 19.4% practices, and 5.6% tools.The review also reports limitations in current tools, including accuracy, security, and scalability.Source: §10 · p. 33
  • Socio-technical factors affect all process phases, but their role and influence on timely vulnerability remediation remain empirically underexamined.The authors recommend further research and practice attention to these factors.Source: §10 · p. 33
  • The review identifies human-AI collaboration as a future research opportunity because human involvement remains necessary amid patch management’s complexity and dynamic nature.Its challenge-solution mapping and practice classification are also presented as guidance for practitioners assessing and improving organisational patching.Source: §10 · pp. 33–34

Appendix A. Selected primary studies in the review.

This appendix identifies selected primary studies on software security patch management, covering vulnerability assessment, risk management, patch validation and deployment, and system-administrator practices.

Appendix A. Selected primary studies in the review.

  • P1, P2, P20, and P19 address security patch management, including cross-site management for large-scale heterogeneous environments.The listed titles include operating-system security patch management, internet-based intrusion reduction, and a cross-site patch-management model.Source: Appendix A · pp. 35–36
  • P3, P9, P12, and P16 cover software updates and patching through multi-version updates, persistent-state automation, shadow patching, and staged deployment.P16 is described as integrated software upgrade testing and distribution.Source: Appendix A · pp. 35–36
  • Among the listed works, one title concerns identifying vulnerabilities; the supplied passage names “A race for security: Identifying vulnerabilities on” but does not specify the target, scale, or any outcome.Source: Appendix A · p. 36
  • P6, P7, P13, and P14 focus on vulnerability prioritization, context-aware scoring, or vulnerability assessment.Their titles mention analytic hierarchy process, context information, VULCAN, and VRank.Source: Appendix A · p. 35

Enterprise patching and risk management

  • Other listed works focus on enterprise patching and security risk, including P35, P50, P51, and P52.Their titles concern security patch dilemmas, risk assessment and mitigation in IT companies, proactive vulnerability assessments in cloud deployments, and bridging operational and strategic security realms.Source: Appendix A · pp. 37–38

Integrity, auditing, and validation

  • Several studies examine patch integrity, auditing, and validation, including P46, P47, P57, P60, and P68.Their titles describe patch integrity and verification methods, patch-based auditing for web application vulnerabilities, online validation with delta execution, and replay for debugging and patch validation.Source: Appendix A · pp. 37–39

Appendix A. Selected primary studies in the review.

  • P71 is titled “SLA-driven Applicability Analysis for Patch.”Source: Appendix A · p. 39
  • P72 is titled “A Design for a Hyperledger Fabric Blockchain-Based Patch-Management System.”Source: Appendix A · p. 39

Appendix B. Data Extraction Form.

The data extraction form records study demographics, solution characteristics, reported challenges, evaluation quality, limitations, and future work.

Appendix B. Data Extraction Form.

  • The form captures study identity and context, including title, authors, venue, year, publication type, area of focus, target users, and application domain.It also records solution type and research type.Source: Appendix B · pp. 39–40
  • Challenge and solution fields record socio-technical challenges, proposed approaches and tools with their capabilities, and practices for successful security patch implementation.Source: Appendix B · p. 40
  • Evaluation fields capture evaluation type, rigour, and industrial relevance, alongside solution limitations, threats to validity, and reported future work.Source: Appendix B · p. 40
Loading 2012.00544v3…