Source-linked AI summary
Database-Augmented RAG for Automated Repair of REST API Misuses
Shoei Inoue, Norihiro Yoshida, Erina Makihara, Shiyu Yang, Katsuro Inoue
TL;DR
REST API misuse repair is difficult because error feedback is insufficient and it remains unclear how specifications should be organized for RAG. The paper evaluates 11 database configurations for Database-Augmented RAG, finding that a four-database configuration reached an 88.6% repair rate versus 54.3% for the baseline. The results support separating specifications by version and content type in the studied datasets.
Problem
REST API specification changes and insufficient error responses complicate misuse diagnosis, while the effective organization of specifications in RAG databases remains unclear.
Method
The study proposes Database-Augmented RAG and evaluates 11 configurations that organize REST API specifications by storage strategy, version, and content type.
Results
88.6% repair rate was achieved by the four-database configuration, compared with 54.3% for the baseline.
Takeaways & Limitations
Separating retrieval evidence by API version and content type can improve REST API misuse repair in the studied datasets.
Takeaways & Limitations
The comparison cannot fully isolate whether gains arise from database structure, retrieved-context quantity, or their interaction.
Abstract
from arXiv · showhide
Many Internet of Things (IoT) services provide Representational State Transfer (REST) APIs, which require client developers to implement applications that conform to the corresponding API specifications. When client programs contain API misuse, developers debug them based on error responses. However, such responses are often insufficient for identifying the root cause, requiring developers to repeatedly communicate with the server. Retrieval-Augmented Generation (RAG) is a promising approach for providing large language models (LLMs) with external knowledge. However, in automated repair of REST API misuses, it remains unclear how specifications should be stored in a RAG database. This study evaluates how different configurations for organizing API specifications affect RAG-based repair of REST API misuse. We constructed 11 RAG configurations with different database structures and compared their repair rates with a baseline method. For evaluation, we used REST API misuse cases collected from real-world repositories. The results show that, in the studied datasets, the baseline method achieved a repair rate of 54.3%, whereas a RAG-based method using four databases achieved a maximum repair rate of 88.6%. These results indicate that organizing specifications according to version and content type can be an effective design choice for RAG-based REST API misuse repair.
1 Introduction
IoT REST API clients are vulnerable to specification changes and ambiguous error feedback, making misuse difficult to diagnose and repair. This study evaluates Database-Augmented RAG configurations for automated REST API misuse repair.
- IoT client development is error-prone because applications must integrate heterogeneous devices, unstable networks, and changing vendor specifications.
- HTTP error messages and status codes often do not identify misuse root causes, forcing repeated request-and-response debugging cycles.This trial-and-error process increases development time and risks incomplete fixes.
- REST API specification updates require corresponding client-code changes, but neglected updates can introduce defects and vulnerabilities.
- Database-Augmented RAG adds domain-specific information and repair details to LLM prompts, then compares its performance with a baseline method.
- The study evaluates whether separating retrieval contexts by API version and content type improves repair performance in the studied dataset.
2 Background
REST APIs expose concrete misuse risks when clients lag behind evolving specifications, while existing LLM repair methods generally lack external API knowledge. The paper motivates partitioned RAG databases to provide more precise, controllable specification context.
- SwitchBot and Fitbit illustrate IoT REST APIs whose specifications changed through version updates, deprecations, and authentication transitions.SwitchBot introduced sign, t, and nonce headers in version 1.1; Fitbit transitioned from OAuth 1.0 to OAuth 2.0.
- A SwitchBot misuse can involve both a deprecated endpoint and missing request headers required by the latest specification.The correction updates the endpoint and adds sign, t, and nonce alongside Authorization.
- Existing LLM repair studies report general repair capabilities but do not cover repairs requiring external REST API specifications.
- Standalone LLMs lack up-to-date external knowledge, whereas fine-tuning improves domain knowledge but incurs substantial maintenance costs.
- Conventional single-database RAG can mix deprecated and latest specifications or intermix code with natural-language text, producing temporally inconsistent context.
- Database-Augmented RAG partitions specification content into multiple databases by version and information type to improve retrieval precision and prompt controllability.The proposed study evaluates database configurations and identifies an effective configuration for REST API misuse repair.
3 Proposed Method
The proposed method evaluates RAG architectures that vary specification type and storage strategy for automated REST API misuse repair. It extends standard RAG by partitioning specifications according to version and content type, while using tailored prompts and retrieval procedures.
- Configuration Design: 11 RAG database configurations were evaluated to identify an effective architecture for automatically correcting REST API misuse.The configurations vary specification type and storage strategy.
- Configuration Design: D and L denote deprecated and latest specifications, while m, s, c, and t denote alternative storage strategies.The strategies include mixed, separated, code-only, and text-only storage.
- Baseline Method: The baseline uses basic instructions, the target program, and specification links, with GPT-4o accessing linked specifications through built-in web search.Full specifications could not fit within GPT-4o’s context window.
- Standard RAG Method: Standard RAG chunks specification text, embeds target functions and specification chunks, retrieves up to 60 relevant segments, and inserts them into a GPT-4o repair prompt.The prompt includes background information, repair instructions, database information, and the target program.
- Database-Augmented RAG Method: The database-augmented DsLs configuration partitions deprecated and latest specifications into separate code-snippet and natural-language databases.It searches each of four databases and retrieves the top 60 segments from each, using a three-stage prompt.
- Dataset and Evaluation: Misuse cases were collected from SwitchBot and Fitbit repositories, filtered for specification-based repair, and evaluated using successful patch generation across five executions.Success additionally required matching repository code, compiling successfully, and fixing all misuse instances in the target function.
4 Results
Across the studied misuse cases, database organization substantially affected repair performance: separating specification versions and content types generally improved retrieval and repair, with the four-database configuration reaching the highest repair rate. The evaluation also identifies retrieval, prompt, and dataset-design limitations that constrain interpretation.
- Failure analysis: Repair failures arose from partial correction of multiple errors, missed specification inconsistencies, and insufficient vector-retrieved context.Unsupported repairs included references to a nonexistent v2.0 API and misuse of Bearer authentication.
- Failure analysis: Deprecated-only prompts produced a 0% repair rate for mixed misuse cases involving request headers and endpoints.The deprecated context encouraged outdated API usage, especially when multiple corrections were required.
- Content type: Latest-specification code snippets helped DLs and DsLs repair multiple misuse instances, while gains were limited for single-endpoint errors.DsL achieved the highest single-endpoint repair rate, but exceeded the baseline by only 1.4 cases.
- Version separation: A 3.8% higher repair rate and 1.0 more mixed-misuse repairs favored DL over DLm.Both methods used deprecated and latest specifications, but DL stored them in separate databases.
- Content type: DcLc repaired 0.6 more cases than DtLt, indicating that code snippets were more effective retrieval targets than textual descriptions in this setting.Natural-language text was recommended as secondary support rather than the primary retrieval target.
- Overall results: 88.6% was the highest repair rate, achieved by the four-database configuration.The repair rate increased as more databases were added.
- Threats to validity: The evaluation covered only two API services and 21 misuse cases, concentrated on request headers and endpoints.The authors caution that the 88.6% repair rate should not be treated as a direct estimate for broader REST API settings.
- Threats to validity: The comparison cannot fully isolate whether improvements came from database structure, retrieved-context quantity, or their interaction.The baseline also used web search and official-specification links, making it unlike a controlled non-RAG comparison.
5 Related Work
Prior work addresses API misuse detection, automated program repair, and RAG, but REST API misuse repair requires external, current specifications and remains insufficiently covered.
- Java API misuse studies detect structural misuse but do not address REST-specific misuse or automated repair.
- REST API research has detected specification violations through fuzzing and program analysis, but has not addressed automated repair.
- Conventional template-based repair relies on predefined patterns, limiting fixes for novel errors or complex repairs.
- LLM-based repair has surpassed conventional tools, but generated patches may be hallucinated or incorrect.
- Static analysis cannot guarantee compliance with external REST specifications, motivating RAG-based access to specification information.
- Unlike prior RAG studies using repair histories or past code, this study retrieves REST specifications because they are reliable yet frequently changing knowledge sources.
6 Conclusion
The study compares baseline repair with 11 RAG database configurations and finds that current, well-organized specifications improve REST API misuse repair. It also identifies dataset scale, retrieval segmentation, and evaluator dependence as future-work boundaries.
- 54.3% was the baseline repair rate, compared with 88.6% for the best-performing RAG configuration.
- Configurations using only deprecated specifications performed worse than the baseline, while those incorporating the latest specifications generally improved accuracy.
- Separating evidence by version and content type further improved repair of multiple misuses.
- Latest code snippets were especially important in the dataset because they helped fix multiple errors within one function.
- Future work will test generalization with more services and misuse types, improve paired retrieval, and reduce subjective evaluator variation.