Source-linked AI summary
Large Language Models for Fuzz Testing in Microservices: A Systematic Literature Review
Ying Song, Ke Ping, Yuqing Wang, Xiaozhou Li
TL;DR
Microservice fuzzing faces large, stateful API spaces, while existing reviews lack a consolidated account of LLM-assisted approaches. This SLR analyzes 20 studies from 2024–2026 and finds semantic input generation, growing agentic and retrieval-augmented designs, and modest effectiveness gains alongside heterogeneous evaluation and limited real-world coverage.
Problem
Existing API fuzzers struggle with invalid inputs, complex constraints, deeper states, and cross-service dependencies, while prior reviews do not consolidate LLM-based fuzzing in microservice systems.
Method
The paper conducts a protocol-driven systematic literature review using structured search, screening, quality assessment, data extraction, and synthesis.
Results
Across 20 studies, LLMs mainly serve as semantic input generators in black-box fuzzing, with increasing closed-loop, multi-agent, and RAG-augmented architectures and modest improvements in coverage and vulnerability detection.
Takeaways & Limitations
The review provides a taxonomy, consolidated evaluation view, and research directions centered on microservice-aware, cost-aware, and correctness-aware fuzzing.
Takeaways & Limitations
Generalizable conclusions are constrained by non-standardized benchmarks and metrics, limited cost reporting, and a corpus biased toward single-service experiments.
Abstract
from arXiv · showhide
Microservice systems (MSS) increasingly rely on heterogeneous APIs whose combinatorial input space and stateful dependencies challenge traditional fuzz testing. Meanwhile, Large Language Models (LLMs) have recently been introduced to enhance fuzzing with semantic reasoning over specifications, inputs, and runtime feedback. This paper presents a systematic literature review (SLR) of LLM-assisted fuzz testing for microservices to synthesise how LLMs are applied, evaluated, and what challenges remain. Following established SLR guidelines, we analyze 20 primary studies published between 2024 and 2026. Results show LLMs are mainly used as semantic input generators in black-box fuzzing, with a growing shift towards agent-based and retrieval-augmented architectures, improving valid input generation and modestly increasing coverage and vulnerability detection. However, evaluation remains heterogeneous, with limited benchmark standardization, scarce cost reporting, and a bias toward single-service experiments, highlighting a gap with real-world multi-service systems. This review provides a taxonomy of LLM roles and integrations, a consolidated view of evaluation practices, and a mapping of open challenges to research directions, supporting the design and deployment of LLM-driven fuzzing in microservices.
1 Introduction
Microservice APIs create large, stateful testing spaces that expose limitations in traditional fuzzers. This review examines how LLMs address these challenges and synthesizes their roles, evaluation, and open gaps.
- Microservice APIs challenge systematic testing through large parameter spaces, inter-parameter dependencies, and multi-step cross-service workflows.
- Existing API fuzzers often generate invalid inputs, miss complex constraints and deeper program states, and cannot use documentation, errors, or business-logic dependencies effectively.
- LLMs support fuzzing by processing API specifications, source code, logs, error messages, and request–response traces for generation, mutation, response analysis, and workflow coordination.
- The review applies a protocol-driven SLR with structured search, dual-reviewer screening, quality assessment, and data synthesis.
- The study contributes a taxonomy, consolidated evaluation practices, and research gaps including realistic multi-service benchmarks and cost-aware evaluation.
2 Related Work
Existing reviews separately examine LLM-based fuzzing or microservice testing, leaving their intersection insufficiently covered. This SLR addresses that intersection directly.
- Prior LLM-fuzzing reviews organize work across domains such as binary, compiler, protocol, and deep-learning-library targets rather than microservice-specific testing.
- Microservice-testing reviews cover testing techniques, levels, and challenges but do not focus on fuzzing or LLM-based techniques.
- The reviewed secondary literature therefore leaves the intersection of LLM-based fuzzing and microservice systems uncovered.
3 Methodology
The study follows established SLR guidance to identify, screen, assess, extract, and synthesize evidence on LLM-based fuzzing in microservice systems. Its research questions cover applications, evaluation, and future challenges.
- The review follows Kitchenham’s SLR guidelines and Wohlin’s snowballing procedure to identify, evaluate, and synthesize relevant research.
- The methodology comprises research-question definition, search and source selection, inclusion and exclusion screening, quality assessment, and data extraction and synthesis.
- The research questions characterize LLM applications, assess evaluation practices and effectiveness, and identify open challenges and future directions.
- The search combines LLM, fuzzing, testing, and system-context terms across major digital libraries, arXiv, and backward and forward snowballing.
- Screening used title-and-abstract and full-text phases, with two researchers reviewing each article and a third resolving disagreements.
- Quality assessment used four DARE-derived dimensions scored on a three-point scale, with inclusion requiring at least 6 out of 12.
- Data extraction covered bibliographic, model, role, fuzzing, target, evaluation, and replication information, followed by qualitative and descriptive synthesis.
4 Results
The corpus contains 20 recent studies and is concentrated on REST APIs, while model choices remain diverse. The results map publication trends, study characteristics, and LLM usage across the selected literature.
- 4.1 Corpus Overview: 20 studies span 2024 (2 papers), 2025 (13 papers), and 2026 (5 papers), indicating an emergent research topic.
- 4.1 Corpus Overview: The corpus is dominated by REST-based APIs, with limited representation of GraphQL, CGI/IoT services, and hybrid architectures.
- 4.1 Corpus Overview: Only a small number of studies report industrial-scale cases, while most rely on benchmarks or single-service setups.
- 4.1 Corpus Overview: The selected studies are presented through yearly publication trends and document-type distributions.
- RQ1: RQ1 analyzes LLMs by model choice, pipeline role, fuzzing visibility, and integration.
- RQ1: GPT-family models are most common, but Llama, Mistral, Qwen, DeepSeek, CodeBERT, and fine-tuned or quantised Llama variants are also used.
LLM Role in the Fuzzing Pipeline.
LLMs are used primarily as semantic input generators in black-box fuzzing, while mutation, oracle, and orchestration roles remain less common. Integration is increasingly iterative, multi-agent, and augmented with program or system data.
- 16 studies use LLMs as input generators in black-box fuzzing, compared with 1 grey-box and 1 white-box study.
- LLM-based oracles appear in 6 black-box and 2 grey-box studies, while mutation-operator use remains limited to 1 grey-box and 1 white-box study.
- Five black-box studies use agent-based LLM orchestrators, with little adoption of orchestration in other visibility settings.
- Prompt-based integration is most common, with feedback-driven iteration more prevalent than one-shot generation; several recent studies use multi-agent architectures.
- Program analysis supports 4 studies and logs or traffic support 3, whereas retrieval-based approaches support 2 and learning-based integration remains limited.
4.3 RQ2: How are LLM-based microservice fuzzers evaluated?
Evaluation covers diverse targets, metrics, and baselines, but studies usually test individual REST services rather than cooperating microservice clusters. Coverage and effectiveness enable the strongest cross-study comparisons, while cost reporting and benchmark convergence remain limited.
- Target systems: Most studies evaluate individual REST services rather than cooperating microservice clusters, leaving cross-service workflows underexplored.
- Metrics: Evaluation metrics span coverage, effectiveness, input quality, and cost, but only coverage and effectiveness are reported comparably across most studies.
- Metrics: Fewer than half of the studies report cost-related metrics, and the field has not converged on a benchmark suite.
- Baselines: Traditional fuzzers are the dominant baseline, while comparisons only against other LLMs weaken effectiveness claims.
- Baselines: Three studies compare against human-written test suites or expert ground truth, providing a rarely used sanity check.
- Baselines: 11 out of 20 studies (55%) release replication packages, facilitating reproducibility and further empirical validation.
4.4 RQ3: Open challenges and future research directions.
The review identifies barriers to deployable LLM-assisted microservice fuzzing, including weak generation guarantees, limited evaluation infrastructure, and engineering costs. It proposes microservice-aware, cost-aware, and correctness-aware designs as future directions.
- Open challenges: Current approaches struggle with guarded branches, hallucinated or invalid tests, and complex inter-parameter or OpenAPI dependencies.
- Open challenges: Evaluation lacks unified microservice benchmarks and often relies on weak 5xx-response oracles that cannot fully capture functional or security failures.
- Engineering constraints: Token cost, latency, compute overhead, non-determinism, and IP or compliance concerns hinder CI/CD integration.
- Correctness constraints: Existing systems provide limited guarantees that generated tests satisfy API grammars, protocol constraints, or distributed authorisation logic.
- Future directions: Future research should combine LLMs with static or symbolic analysis, dependency graphs, runtime feedback, and cross-API reasoning.
- Future directions: Cost-aware deployment should use smaller, quantised, or domain-specific models with deterministic decoding, alongside stronger functional or security oracles.
5 Discussion
The review identifies convergence toward black-box, feedback-driven, agentic fuzzing, while highlighting realism, cost, and correctness constraints that limit deployment.
- Architectural convergence: LLM-assisted fuzzing is converging on black-box, feedback-driven, agentic pipelines combining input generation, response or log analysis, and lightweight orchestration.
- Implications: Table 8 organizes open challenges and corresponding future-work directions.
- Microservice realism gap: Most evaluations target isolated REST services rather than cooperating microservice clusters, leaving cross-service workflows, queues, authorization, meshes, and sidecars rarely tested.
- Cost and deployability: Token cost, latency, energy use, non-determinism, and compliance concerns remain barriers to CI/CD adoption, increasing the appeal of smaller or locally deployable models.
- Implications: The review recommends combining LLM fuzzing with program analysis or dependency information and prioritizing cluster benchmarks, cost-aware evaluation, and correctness-aware generation.
6 Threats to Validity
The review reports threats from corpus selection, heterogeneous reporting, subjective synthesis, and limited benchmark coverage, while describing mitigation through reviewer checks and recurring-pattern analysis.
- Study-selection validity: Relevant studies may have been missed despite searches across major digital libraries and snowballing, while arXiv inclusion may introduce variability in study quality and maturity.
- Data validity: Hybrid configurations and heterogeneous reporting require reviewer interpretation and limit direct comparability across metrics, models, and experimental setups.
- Research validity: Taxonomy design and qualitative aggregation remain subject to researcher bias, and nonstandardized benchmarks and metrics constrain generalizable conclusions.
- Research validity: The synthesis focuses on recurring patterns across multiple studies and avoids overgeneralizing isolated results.
7 Conclusion
The paper reviews 20 studies of LLM-assisted fuzz testing for microservice systems and finds a dominant semantic-generation pattern alongside emerging agentic architectures and persistent evaluation gaps.
- 20 primary studies published between 2024 and 2026 were synthesized in the systematic literature review.
- LLMs are mainly used as semantic input generators in black-box fuzzing, with increasing adoption of closed-loop, multi-agent, and retrieval-augmented architectures.
- LLMs improve valid input generation and modestly enhance coverage and vulnerability detection, but evaluation remains inconsistent and cost reporting is limited.
- The field's stated directions are smaller or fine-tuned models, hybrid LLM–program analysis, and standardized, cost-aware multi-service benchmarks.
Appendix - The Selected Papers (SPs)
The selected-paper appendix lists studies on LLM-assisted testing and fuzzing across REST, GraphQL, web CGI, service composition, automotive, and related API-testing settings.
- The corpus also includes work on GraphQL testing, gray-box IoT CGI fuzzing, LLM-assisted service composition, industrial REST API evaluation, and automotive test automation.
- The appendix identifies SemFuzz as a semantic-guided framework for detecting unauthorized API access in mobile applications.
- Additional listed studies address API documentation and request mutations, white-box API testing, log coverage, reusable REST workflows, API dependencies, and small language models.
- The selected studies include REST API testing and fuzzing systems using LLMs, including TestGPT-Server, Dyner, Restless, AutoRestTest, and LlamaRestTest.
- The appendix lists multi-agent and semantic-graph approaches for REST API testing, including a multi-agent semantic-graph method and MASTEST.