Source-linked AI summary
DomAgent: Leveraging Knowledge Graphs and Case-Based Reasoning for Domain-Specific Code Generation
Shuai Wang, Dhasarathy Parthasarathy, Robert Feldt, Yinan Yu
TL;DR
Domain-specific code generation is difficult because generic LLMs lack specialized knowledge and workflow context. DomAgent combines knowledge-graph reasoning with case-based retrieval through DomRetriever, and the authors report improved benchmark and real-world performance, including practical deployment in truck software. The approach aims to provide broad coverage with compact case sets and flexible integration with LLMs.
Problem
Generic LLMs often lack the specialized knowledge and workflow-specific adaptation required for domain-specific programming tasks.
Source: §1 · p. 1Method
DomAgent uses DomRetriever to combine top-down knowledge-graph retrieval with bottom-up case-based reasoning and iteratively synthesize relevant knowledge and examples.
Source: §1 · pp. 2–3Results
DomAgent outperforms similarly sized LLMs on benchmark and real-world truck tasks, with 30% of cases achieving comparable performance to random sampling with 80%.
Source: §6 · p. 9Takeaways & Limitations
The system provides a flexible domain-knowledge and case retriever that can integrate with external LLMs and support deployed truck CAN-signal code generation.
Source: §1 · p. 3, §6 · p. 9The authors identify broader industrial evaluation and finer-grained retrieval similarity as future work for validating generalizability and improving case retrieval.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Large language models (LLMs) have shown impressive capabilities in code generation. However, because most LLMs are trained on public domain corpora, directly applying them to real-world software development often yields low success rates, as these scenarios frequently require domain-specific knowledge. In particular, domain-specific tasks usually demand highly specialized solutions, which are often underrepresented or entirely absent in the training data of generic LLMs. To address this challenge, we propose DomAgent, an autonomous coding agent that bridges this gap by enabling LLMs to generate domain-adapted code through structured reasoning and targeted retrieval. A core component of DomAgent is DomRetriever, a novel retrieval module that emulates how humans learn domain-specific knowledge, by combining conceptual understanding with experiential examples. It dynamically integrates top-down knowledge-graph reasoning with bottom-up case-based reasoning, enabling iterative retrieval and synthesis of structured knowledge and representative cases to ensure contextual relevance and broad task coverage. DomRetriever can operate as part of DomAgent or independently with any LLM for flexible domain adaptation. We evaluate DomAgent on an open benchmark dataset in the data science domain (DS-1000) and further apply it to real-world truck software development tasks. Experimental results show that DomAgent significantly enhances domain-specific code generation, enabling small open-source models to close much of the performance gap with large proprietary LLMs in complex, real-world applications. The code is available at: https://github.com/Wangshuaiia/DomAgent.
1 INTRODUCTION
Domain-specific code generation requires both specialized knowledge and workflow-aware adaptation, which generic LLMs often lack. DomAgent addresses this gap by combining knowledge-graph retrieval with case-based reasoning for targeted domain adaptation.
Motivation
- Generic LLMs degrade on specialized programming tasks because they lack domain knowledge, third-party library familiarity, and context-specific workflow adaptation.The introduction illustrates this challenge through truck software that reads CAN signals and applies domain-specific mappings.Source: §1 · p. 2
- Fine-tuning can provide domain knowledge but imposes high development and maintenance costs and adapts poorly to constantly updated libraries.Retrieval-augmented generation offers a more flexible alternative by adding external knowledge without modifying model weights.Source: §1 · p. 2
Related challenges
- Existing retrieval methods improve access to external information, but textual similarity alone may miss explicit package and function relationships needed for complex domain-specific code generation.Long retrieved contexts can also obscure key information, motivating structured and concise retrieval.Source: §1 · p. 2
Bottom-up retrieval
- Case-based reasoning supplies task examples, yet fixed or small case sets may fail to cover diverse scenarios while constructing comprehensive libraries is labor-intensive.The paper identifies broad coverage with few examples as a practical unresolved challenge.Source: §1 · p. 2
Proposed approach
- DomAgent integrates top-down knowledge graphs with bottom-up case-based reasoning, while DomRetriever dynamically combines both to refine relevant knowledge and representative cases.Knowledge-graph relations guide compact case selection by covering functions and packages broadly.Source: §1 · p. 2
Validation
- The system is validated on a data-science benchmark and real-world truck software tasks, where the authors report effectiveness and practical robustness.DomRetriever is also designed for integration with other LLM pipelines.Source: §1 · pp. 2–3
2 RELATED WORK
Related work improves domain-specific code generation through structured knowledge retrieval and reusable examples. However, textual retrieval can miss code relationships, while maintaining broad and current case libraries remains costly.
2.1 Knowledge Retrieval for Code Generation
- Knowledge-graph retrieval represents packages, functions, and classes with explicit relationships, making structured domain knowledge suitable for code repair and repository-level generation.This structure complements conventional text and semantic retrieval methods.Source: §2.1 · p. 3
- Retrieval systems improve relevance through BM25, embeddings, chunking, AST-based segmentation, and filtering or compression of long contexts.These techniques address retrieval precision and the risk that excessive context obscures key information.Source: §2.1 · p. 3
2.2 In-Context Learning and Case-Based Reasoning
- In-context learning and case-based reasoning guide LLMs with representative examples, while dynamic semantic retrieval and AST re-ranking improve selection over fixed few-shot prompts.Segmented prompting can combine content from multiple sources for broader contextual coverage.Source: §2.2 · p. 3
- Large, optimized case bases can improve example quality, but constructing and maintaining them is labor-intensive and costly in fast-evolving industrial environments.This motivates methods that achieve broad coverage without requiring comprehensive case libraries.Source: §2.2 · p. 3
3 PROBLEM DEFINATION
The paper formulates domain-specific code generation as retrieving structured knowledge and relevant examples for a task, then synthesizing target code. It seeks a small, diverse case base with broad coverage and limited manual effort.
Problem formulation
- Given a task q, knowledge graph G, and case base B, the objective is to retrieve domain knowledge and relevant cases before synthesizing target code ŷ.The formulation treats structured knowledge and reusable examples as complementary generation resources.Source: §3 · p. 3
Knowledge graph
- The knowledge graph models entities such as packages, functions, descriptions, and parameters, with edges encoding relations such as function-package membership.These relations provide the structured representation used for domain knowledge retrieval.Source: §3 · p. 3
Case base
- The case base is a collection of reference code cases that should be small, representative, diverse, and coverage-rich while minimizing costly manual construction.The central case-base objective is broad coverage with limited effort.Source: §3 · p. 3
4 METHODS
DomAgent combines KG-guided case selection, top-down domain-knowledge retrieval, bottom-up case retrieval, and reasoning-based refinement for domain-specific code generation. Its agent training teaches retrieval-tool use and filters retrieved information before code generation.
4 METHODS
- DomAgent integrates representative-case construction, knowledge and case retrieval, reasoning-based refinement, code generation, and reinforcement-learning-based tool use.The three core components are case-base construction, retrieval-augmented code generation, and agent training.Source: §4 · p. 3, §4.2.1 · p. 5, §4.3 · p. 5
4.1 Hierarchical Case Selection with Knowledge Graph Guidance
- Hierarchical case selection uses KG package structure and function-level clusters to cover packages while preserving diverse functional usage patterns.Candidate cases are retained when they increase package or cluster coverage, and construction stops after both coverage ratios exceed thresholds.Source: §4.1 · pp. 3–4
- The case base stores executable code cases in a vector database indexed by natural-language task descriptions for similarity-based retrieval.Cases are executed before storage to verify that they run correctly.Source: §4.1 · p. 4
4.2.1 DomRetriever.
- DomRetriever selects relevant KG knowledge top-down and candidate cases bottom-up, then chooses the case with greatest package overlap with the retrieved knowledge.Knowledge nodes are selected by task-to-node similarity, while cases are initially ranked by task similarity and refined using package overlap.Source: §4.2.1 · pp. 4–5
- A reasoning LLM compares the selected case with retrieved knowledge, removes functionally irrelevant items, and can retrieve supplementary knowledge through search tools.The refined knowledge and specialized solution are combined with the original task before code generation.Source: §4.2.1 · p. 5, §4.2.2 · p. 5
4.2.2 Code Generation.
- The final generator receives the task, refined knowledge, and specialized solution, and may be the same reasoning model or a more powerful LLM.This design separates retrieval reasoning from target-code generation while allowing either shared or distinct models.Source: §4.2.2 · p. 5
4.3 Agent Training
- Agent training fine-tunes retrieval-triggering tokens from synthesized reasoning traces and applies reward-guided GRPO to teach relevance filtering.A reward model evaluates the query against retrieved knowledge and cases, providing feedback for agent optimization.Source: §4.3 · p. 5
5 EXPERIMENTS
Experiments on DS-1000 and truck CAN signal tasks show that DomAgent improves domain-specific code generation through knowledge retrieval, case-based reasoning, filtering, and reinforcement learning. The method raises performance for small open-source models and approaches or exceeds stronger baselines across benchmark and industrial settings.
5.1 Experimental Setup.
- The experiments use DS-1000’s 1,000 tasks over seven data-science packages and 776 truck CAN signals spanning six functional domains, with pass@1 as the evaluation metric.The case base uses 300 DS-1000 examples, while the truck setting uses manufacturer documentation and engineer-authored tests.Source: §5.1 · pp. 5–6
5.2.1 DS-1000.
- 58.6% is achieved by LLaMA3.1-8B DomRetriever paired with GPT-4o, a +7.6% gain over GPT-4o alone, while DomAgent reaches 39.2% and 40.5% with Qwen2.5-7B and LLaMA3.1-8B.The small-model agents outperform previous coding agents by approximately +2.0 to +3.0 percentage points.Source: §5.2.1 · p. 6
5.3 Ablation Study.
- Knowledge grounding and case-based reasoning each improve performance, their combination produces roughly additive gains, and DomAgent’s retrieval mechanism further improves the standard RAG pipeline.The combined approach is especially competitive on specialized Pandas and TensorFlow tasks requiring multistep reasoning.Source: §5.3.1 · p. 6, Table 3 · p. 8
5.3.1 DS-1000.
- KG-guided case selection consistently beats random sampling across sampling proportions, reaching near-full performance with only 30% of the data.Performance improves as the sampling proportion increases for both strategies, but knowledge-guided selection is most advantageous in the low-sampling regime.Source: §5.3.1 · p. 6, Figure 3 · p. 7
5.2.2 Truck CAN Signal.
- 96.64 pass@1 for Qwen2.5-7B-based DomAgent and 98.04 for GPT-4o-based retrieval on six truck CAN signal domains, with both reaching 100% on Driver Productivity, Energy, and Visibility.These results substantially exceed vanilla Qwen2.5-7B at 39.62 and GPT-4o at 71.22 pass@1.Source: §5.3.2 · p. 7, Table 2 · p. 7
5.3.2 Truck CAN Signal.
- On truck CAN tasks, adding knowledge grounding improves Qwen2.5-7B by 31.27 points and GPT-4o by 16.58 points, while adding case-based reasoning yields further gains of 44.95 and 19.88 points.The gains are attributed to domain documentation and retrieved examples that support library invocation and CAN-signal transformation.Source: §5.3.2 · p. 7, Table 4 · p. 8
6 CONCLUSION AND FUTURE WORK
DomAgent combines structured knowledge retrieval, case selection, and LLM reasoning for domain-specific code generation. Experiments and factory deployment support its effectiveness, while future work targets finer retrieval, lighter adaptation, and broader industrial validation.
6 CONCLUSION AND FUTURE WORK
- DomAgent combines KG-based domain-knowledge retrieval, package-overlap case reranking, and LLM reasoning that filters irrelevant retrieved information.The agent is trained to invoke retrieval tools during reasoning, using supervised fine-tuning and reinforcement learning to improve retrieval relevance.Source: §6 · p. 8
- Selecting 30% of cases with KG guidance achieved performance comparable to random sampling with 80%, while DomAgent outperformed similarly sized LLMs on benchmark and truck-domain datasets.DomAgent also serves as a retriever that can be integrated with external LLMs.Source: §6 · p. 9
- Future work will add AST-based structural similarity, explore soft prompts for lightweight domain adaptation, and evaluate DomAgent across more industrial scenarios.These directions target retrieval quality, adaptation efficiency, and broader validation of practicality and generalizability.Source: §6 · p. 9