Source-linked AI summary
Barriers to Using Static Application Security Testing (SAST) Tools: A Literature Review
Zachary Wadhams, Clemente Izurieta, Ann Marie Reinhold
TL;DR
Software vulnerabilities and breaches make application security important, yet many teams have not adopted SAST fully. This paper surveys recent literature to identify developers’ apprehension and usage problems, finding usability challenges that require both developer investment and tool improvements. It concludes that simplifying SAST usage can support more consistent integration into development workflows.
Problem
Many development teams have not adopted SAST fully, and the study asks why developers hesitate to initiate or continue using it.
Method
The authors conducted a structured literature review, manually screening papers and cataloging developer-facing SAST issues.
Results
The review found usability challenges that can make SAST cumbersome and lead developers to reduce usage until tools no longer serve their original purpose.
Takeaways & Limitations
SAST tool creators should simplify interfaces, output presentation, false-positive reporting, setup, and automated fixing to address developer needs.
Takeaways & Limitations
The stringent search strings may have excluded some publications, although the authors consider the resulting sample of 89 papers sufficient for viewing the current SAST field.
Abstract
from arXiv · showhide
Developers face a challenging problem with no clear solution. Modern software breaches can wreak havoc on businesses and individuals alike. With code vulnerabilities being a leading cause, securing applications must be a priority for developers. Static Application Security Testing (SAST) has the potential to harden applications by assisting in the identification and resolution of security vulnerabilities. Despite this, many development teams have not adopted SAST tools into their environment. In this paper, we survey the recent literature to uncover why some developers are apprehensive towards SAST and identify what specific problems they encounter when using it. We found a variety of usability problems developers face when using SAST. Some are inherent of the tool and ultimately require some level of developer investment while others are tool shortcomings that SAST tool creators must address. Ultimately, we argue that in order to drive widespread adoption and consistent SAST usage, developers will need to embrace that some investment is required. Simultaneously, developers will be more likely to integrate SAST tools into their workflows if the creators of SAST tools simplify many aspects related to tool usage. Surmounting the primary obstacles preventing the adoption of SAST requires full consideration of both the technical and human factors.
1 Introduction
Software vulnerabilities and security incidents underscore the need for secure coding practices. Although SAST can identify and help resolve vulnerabilities throughout the development lifecycle, teams may avoid full adoption because of developer-facing challenges.
- Software security incidents can affect millions of people and cause billions of dollars in damage.The paper cites the 2024 CrowdStrike outage and 2020 SolarWinds attack as examples of far-reaching consequences.
- SAST can be implemented at any point in the development lifecycle to identify and help resolve vulnerabilities.
- Despite SAST’s benefits and relative implementation ease, some teams limit analysis, target only security-critical components, or rely mainly on manual code reviews.
- The study investigates why developers hesitate to adopt SAST fully and which issues lead them to initiate less use or abandon it.
2 Related Work
Earlier work examined particular aspects of SAST usability and effectiveness. This study instead seeks a broad understanding of SAST usability through a comprehensive literature review.
- Previous research included interviews with 20 software developers about SAST usability issues.
- Other studies examined SAST tools’ effectiveness in security code reviews.
- The present study uses a comprehensive literature review to understand the current state of SAST usability broadly.
- The review process examined papers from the ACM Digital Library and IEEE Xplore, cataloged developer issues, and analyzed those issues systematically.
3 Methodology
The authors conducted a structured literature search across ACM Digital Library and IEEE Xplore, refined the search strings, manually screened the results, and analyzed developer-related SAST challenges in 89 relevant papers.
- The study gathered papers from ACM Digital Library and IEEE Xplore because they contain extensive peer-reviewed software-security and development research.
- Initial searches returned thousands of papers from each database, so the authors repeatedly refined the search strings to reach a manageable set.
- The final searches returned 240 ACM papers and 108 IEEE Xplore papers; removing 118 irrelevant conference papers reduced the ACM set to 122.
- The authors manually reviewed 230 papers for actual SAST implementation relevance and identified 89 relevant papers.
- They cataloged developer-usage challenges in the selected papers and recorded how many papers reported each problem.
4 Results and Discussion
Research on SAST implementation increased from 2019 to 2023, while recurring usability problems—including false positives, setup effort, poor output, and workflow disruption—made tools cumbersome and less attractive. The review indicates that adoption depends on both developer investment and tool improvements such as customization, trustworthy results, auto-fixes, and rapid feedback.
- Research publications increased annually from 2019 to 2023, except for a slight drop in 2022.
- False Positives: Over two-thirds of reviewed papers identified false positives as a recurring developer pain point.False positives waste time, reduce trust, increase frustration, and lower productivity.
- False Positives: False positives are commonly handled through manual review and ignore lists, but effective mitigation remains an open research area.
- Poor Output: SAST outputs in XML or JSON can overwhelm developers because analyses may contain thousands of lines of sparse, minimally formatted text.
- Time-Consuming Setup: Time-consuming setup discourages adoption when developers spend hours configuring tools or conclude that the investment is not worthwhile.The review encourages clearer, more comprehensive setup instructions.
- Manual Effort to Fix: Missing automatic fixes and guidance increases the manual effort required to resolve SAST alerts and can frustrate developers.Auto-fix capabilities improved usability and workflow integration in the cited studies.
- Workflow Disruption: Workflow disruption can decrease productivity, increase frustration, cause context switching, and create resistance to security practices.Rapid deployment and timely feedback are presented as important for implementation, because delays may produce skepticism about effectiveness.
- Other Problems: Limited customization can hinder environment integration, language support, scalability, and rule customization.
5 Threats to Validity
The review used a structured search and manually examined 89 papers, but its stringent search strings may have excluded relevant publications and its manual process may limit scalability. The authors frame these methodological choices as efforts to understand the literature deeply while reducing bias.
- Validity goals: The study’s validity discussion emphasizes reliability and reducing bias as goals of identifying potential threats.The paper presents these goals alongside its methodological choices.
- Search and selection: The review identified 89 relevant papers through a structured search and selection process.The authors used tailored search strings and manually assessed papers for relevance.
- Search and selection: Stringent search strings may have excluded some publications, limiting the review’s coverage of the SAST field.The authors nevertheless consider the 89-paper sample a sufficient window into the field’s current state.
- Review procedure: The study manually reviewed all 89 papers rather than using automated approaches such as NLP.This choice supported deeper engagement with authors’ ideas and discussion of potential solutions, but may limit scalability.
6 Conclusion and Future Work
The paper contributes a broad account of SAST usability challenges for both tool creators and development teams. It identifies concrete design and planning responses, while calling for future research on the depth and severity of reported problems.
- Contribution: The review offers insights into SAST usability challenges relevant to tool creators and development teams.Its contribution addresses teams currently using SAST as well as those considering adoption.
- Implications for tool creators: Tool creators can use the findings to design more user-friendly interfaces, clearer outputs, better false-positive reporting, simpler setup, and automatic fixing.These features target developer needs and challenges identified by the review.
- Implications for development teams: Development teams can use the findings to plan for false positives, improve tool output, and account for workflow disruption during implementation.The paper presents this planning knowledge as a way to support more effective SAST use.
- Future work: Future work should examine how deeply individual papers discuss each usability problem and potentially infer its severity or overall impact on developers.The authors suggest NLP could help identify textual signals of severity.