Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks

1 Introduction

• Repository-level issue resolution spans software maintenance and evolution, unlike function-level code generation or work on self-contained files.The paper frames this as a gap between LLM success on function-level benchmarks and real-world repository tasks.Source: §1 · p. 2
• Existing leading approaches typically use proprietary-model agents, whose decision-making can be unpredictable and whose closed models limit access, customization, and code privacy.The paper asks whether open-source LLMs can handle repository-level tasks without agent-based approaches.Source: §1 · p. 2
• CGMs combine repository code graphs and open-source LLMs through semantic node encoding and adapter mapping, graph-aware attention, and fine-tuning; Graph RAG retrieves relevant subgraphs.The framework comprises a Rewriter, Retriever, Reranker, and CGM Reader.Source: §1 · pp. 2–3
• 43.00% resolution rate on SWE-bench Lite with open-source Qwen2.5-72B and agentless Graph RAG, ranking first among open-source-model methods, second among open-source-code systems, and eighth overall.The result surpasses the previous best open-source-model method, Moatless+DeepSeek-V3, by 12.33%.Source: §1 · pp. 2–3

2.1 Large Language Models for Code

• Open-source models struggle with repository-level coding; topologically sorting repositories can exceed context limits and obscure explicit dependencies, motivating text-rich graphs aligned with LLMs.The paper notes that closed-source models have limited accessibility, raise data-privacy concerns, and cannot be fine-tuned on task-specific data.Source: §2.1 · p. 3

2.2 Graphs in Code Language Models

• Prior graph integrations use attention masks, graph-to-text conversion, or positional encodings, but mainly target encoders and file- or function-level tasks; CGM applies graph-aware attention to decoder-only LLMs at repository scale.CGM encodes each text-rich node with a pretrained text encoder and projects it into the LLM input space via an adapter.Source: §2.2 · pp. 3–4

2.3 Agent-drive Methods for Software Engineering

• Agent-based approaches can accumulate errors, require 30–40 interaction turns per issue, and complicate data-driven improvement; CGM instead uses open-source LLMs with agentless Graph RAG.The paper also notes that SWE-Gym may bias training toward trajectories that lead an agent to correct answers.Source: §2.3 · p. 4

2.4 Agentless Methods for Software Engineering

• Agentless methods use fixed localization and editing stages; Agentless has a ten-step pipeline and a 40.67% SWE-bench Lite resolution rate using closed-source Claude-3.5 Sonnet, while CGM aligns graph and text modalities rather than flattening retrieved snippets.Figure 2 distinguishes hierarchical containment edges from reference edges such as calls, imports, and extends.Source: §2.4 · pp. 4–5, Figure 2 · p. 5

3 Code Graph Construction

• Each repository is represented as a directed graph whose nodes are code entities and whose edges encode hierarchical or reference dependencies.The schema includes up to seven node types and five edge types, ranging from repository-level nodes to fine-grained attributes.Source: §3 · p. 5
• AST parsing builds a top-down hierarchy from the REPO root through children such as packages and text files to functions, adding directed parent-to-child edges.This hierarchy mirrors code organization from file systems and programming-language syntax rules.Source: §3 · pp. 5–6
• Lightweight semantic analysis adds reference edges for calls, inheritance, and imports, connecting entities horizontally and potentially introducing cycles.Unlike hierarchical edges, reference edges are typically not part of an AST.Source: §3 · p. 6
• Nodes retain their original content and line ranges; post-processing removes child text from parent nodes so each node encapsulates its corresponding code snippet.These text-rich attributes support graph traversal, retrieval, and semantic model training.Source: §3 · p. 6

4.1 Model Architecture

• CGM converts node text into decoder-ready node tokens and replaces causal attention between those tokens with a graph-adjacency-based mask.The encoder and adapter map node content into the LLM input space; graph-aware attention restricts attention to neighboring nodes.Source: Figure 3 · p. 6, §4.1 · p. 7

4.2 Training Strategies

• Training first reconstructs code from sampled subgraphs, then fine-tunes on issue-patch pairs using prompts that may omit or add files.Subgraph outputs are kept below 8,000 tokens and ordered to preserve graph dependencies; fine-tuning prompts intentionally include irrelevant files in 10% of cases and omit an oracle file in another 10%.Source: §4.2 · pp. 7–8

4.3 The Graph RAG Framework

• The four-module Graph RAG workflow rewrites an issue, retrieves a connected subgraph, reranks candidate files, and uses CGM to generate a patch.Rewriter extracts code elements and enriches the query; Retriever combines string matching and semantic search, then expands anchor nodes with neighbors and upstream connections.Source: Figure 3 · p. 6, §4.3 · p. 8
• Reranker selects files in two stages, narrowing ten filename-ranked candidates to five scored by file-skeleton relevance.Reader combines retrieved graph node tokens with the selected files’ full text to generate the response.Source: Figure 3 · p. 6, §4.3 · pp. 8–9

5.1 Repository-Level Issue Fixing

• 43% resolution rate on SWE-bench Lite places CGM-SWE-PY first among open-source-model methods, second among open-source systems, and eighth overall.It exceeds Moatless+DeepSeek-V3 by 12.33%; CGM-SWE-PY also slightly surpasses Agentless+Claude-3.5-Sonnet by 2.33% and outperforms Agentless+GPT-4o by 11.00%.Source: §5.1 · pp. 9–10
• On SWE-bench-java Verified, CGM-Multi records a 14.29% resolution rate and significantly outclasses SWE-Agent.Source: §5.1 · p. 10

5.2 Repository-Level Code Completion

• CGM-Multi performs on par with or better than similarly sized models on CrossCodeEval and outperforms baseline models on ComplexCodeEval.The evaluation uses one-hop target-file neighbors for context; CGM-Multi processes the subgraph, while baselines receive node text.Source: §5.2 · pp. 10–11, Table 3 · p. 11
• Across base models, CGM typically outperforms other RAG methods on CrossCodeEval, including RepoFuse, which flattens graph context into text.The authors attribute CGM’s advantage over RepoFuse to explicit integration of structural information in the subgraph.Source: §5.2 · p. 11, Table 4 · p. 11

5.3 Ablation Studies

• Ablations identify Rewriter, Retriever, and Reranker, joint encoder-adapter-decoder fine-tuning, graph attention masking, and subgraph reconstruction as contributors to optimal performance.The authors also report that CGM generalizes across backbone sizes, with potential for resource-constrained scenarios.Source: §5.3 · p. 12

6 Conclusion

• With Graph RAG, CGM reaches a 43.00% resolution rate on SWE-bench Lite using open-source models, demonstrating that closed-source models and agent-based approaches are not necessarily required for these tasks.The conclusion presents CGM as a graph-enhanced architecture for repository-level code understanding.Source: §6 · p. 12

A Case in Issue Fix Scenario

• Figure 4 contrasts a linear diff-formatted patch with a code graph that makes containment, inheritance, and function-call relationships explicit.The example issue concerns disabling automatic checkpoint loading in pytorch-lightning.Source: Figure 4 · p. 17

B.1 Node and Edge Types in Code Graph

• The code graph defines node and edge categories, with current support for Python and Java.Tables 5 and 6 list the node and edge types, respectively.Source: §B.1 · p. 17, Table 5 · p. 17, Table 6 · p. 17

B.2 Handling of Complex Dependiences

• Dynamic-call resolution includes possible subclass overrides, while Class Hierarchy Analysis handles inheritance from multiple parent classes.The dynamic-call approach follows over-approximation; the inheritance procedure uses CHA.Source: §B.2 · p. 18

B.3 Search on Code Graph

• Graph search starts from source nodes found through methods such as indexing or keyword and embedding matching, then uses a scenario-dependent strategy to retrieve a relevant repository subgraph.Possible strategies include one-hop neighborhoods, DFS, BFS, and random walks.Source: §B.3 · p. 18

C.1.1 Training Data

• Training covers subgraph reconstruction, issue fixing, and code completion in Python and Java, with test-set repositories filtered out to avoid data leakage.The appendix reports 500k Python and 360k Java reconstruction subgraphs, 200k issue-patch pairs, and 250k completion samples per language.Source: §C.1.1 · p. 18

C.1.2 CGM-Multi

• CGM-Multi starts from Qwen2.5-72B-Instruct and trains on reconstruction and fine-tuning data in Python and Java using a CodeT5+ encoder and an MLP adapter.Training uses 4-bit QLoRA; the adapter maps the encoder output to the LLM hidden dimension.Source: §C.1.2 · pp. 18–19

C.1.3 CGM-SWE-PY; C.1.4 CGM 7B Series

• CGM-SWE-PY uses Python reconstruction data and 14k SWE-bench training issue-fixing pairs, while the 7B series trains language-specific models for code completion.The 7B models use rank 32 and alpha 16 for LoRA; other settings follow CGM-Multi.Source: §C.1.3 · p. 19, §C.1.4 · p. 19

C.2 Recall Results for the Graph RAG Framework

• Graph RAG component recall is lower on SWE-bench-java Verified than on SWE-bench Lite; the authors suggest that Java issues may more often require modifying multiple files.Table 7 reports recall for the Retriever and two Reranker stages on both benchmarks.Source: §C.2 · p. 19

C.3 Hyperparameters for Inference

• Inference uses topk = 20, topp = 0.8, temperature = 0.7, and repetition penalty = 1.1 for the LLMs in the Graph RAG framework.Source: §C.3 · p. 19

C.4.1 Code Graph Construction

• Code graph construction usually takes 3 minutes or more, but it can run offline and therefore does not affect real-time inference workflows.Incremental updates and parallel processing are noted as possible latency optimizations.Source: §C.4.1 · p. 20

C.4.2 Cost of Each Module

• CGM inference latency increases by 0.5–0.7 seconds per 1k tokens, with the reported 1k-to-8k-token comparison rising from 3.9 seconds to 8.6 seconds.Table 8 reports inference time and memory cost for CGM (Qwen2.5-72B).Source: §C.4.2 · p. 20, Table 8 · p. 20

C.5.2 Datasets

• The issue-fixing evaluation covers SWE-bench Lite, SWE-bench Verified, and SWE-bench-java Verified, using Docker execution environments.The datasets cover Python and Java repositories and contain 300, 500, and 91 issues, respectively.Source: §C.5.2 · pp. 20–21

C.5.3 Evaluation Metrics

• Resolve Rate (% R) measures whether generated patches correctly address an issue and include the ground-truth edits as a subset.Source: §C.5.3 · p. 21

C.6 Experimental Setup of Code Completion

• For code completion, the incomplete file is the center of a one-hop ego graph, and nodes needing completion are excluded from the retrieved subgraph.The graph substructure and incomplete file are supplied as graph and text inputs to CGM.Source: §C.6.1 · p. 21

C.6.2 Datasets •

• CrossCodeEval evaluates cross-file completion, while ComplexCodeEval samples are evaluated by completing a function’s middle line from its context.The study evaluates Java and Python, selecting 100 samples per language from ComplexCodeEval.Source: §C.6.2 · p. 21

C.6.3 Evaluation Matrics

• Exact Match (EM) checks exact equality with the reference, while Edit Similarity (ES) measures similarity using Levenshtein distance normalized by string length.The ES formula uses the maximum length of the prediction and reference strings as its denominator.Source: §C.6.3 · pp. 21–22

C.6.4 Baselines: Base Model

• The base-model comparisons include Mistral-Large-Instruct-2411, DeepSeek-V2.5, Qwen2.5, and Qwen2.5-Coder; text inputs above 8k are truncated for the stated baselines.Mistral-Large-Instruct-2411 is the selected Mistral-Large version.Source: §C.6.4 · p. 22

C.6.5 Baselines: RAG Method

• Retrieval baselines include BM25, R2C2-Coder, RepoFuse, and RLCoder, which use different combinations of text matching, structural context, graph retrieval, and learned retrieval.BM25 results are taken directly from CrossCodeEval.Source: §C.6.5 · pp. 22–23

• The section evaluates Graph RAG components on SWE-bench Lite and CGM design on CrossCodeEval in separate ablation studies.The stated aim is to assess the contribution of each RAG component and the effectiveness of the model design.Source: §C.7 · p. 23
• The CGM ablation uses CrossCodeEval to evaluate the model design, alongside the separate Graph RAG framework ablation.Source: §C.7 · p. 23
• Table 10 reports SWE-bench Lite resolve rates for the impact of the rewriter, retriever, and reranker modules.Source: Table 10 · p. 23

C.7.1 Variants of Graph RAG Framework

• Removing Retriever causes an 11.33% performance degradation, larger than the 8.33% drop from removing Rewriter.Without Retriever, all files enter Reranker Stage 1; without Rewriter, retrieval uses the original issue description.Source: §C.7.1 · pp. 23–24
• Removing Reranker produces the largest single-component drop, decreased by -24.67%, highlighting its role in selecting relevant files for the Reader.The variant instead supplies the five files most similar to the query in embedding space.Source: §C.7.1 · p. 24
• Removing the first three Graph RAG modules reduces performance by 33.33%, with the authors attributing the drop possibly to noise and context-window truncation.The ablation feeds the entire repository graph, truncated when necessary, directly to the Reader.Source: §C.7.1 · p. 24
• Without CGM Reader, the flattened-graph baseline achieves 5.33% on SWE-bench Lite, 37.67% below CGM.This baseline flattens code snippets according to topological structure.Source: §C.7.1 · p. 24

C.7.2 Variants of CGM

• Training the adapter alone increases EM by 22.26% for Java and 21.43% for Python versus GGM-Freeze; adding decoder and encoder training yields further reported gains.Adding decoder training improves EM by 5.33% for Java and 4.80% for Python; training encoder, adapter, and decoder together adds 5.71% for Java and 6.12% for Python.Source: §C.7.2 · pp. 24–25
• Replacing CGM’s graph-aware attention mask with a causal mask drops EM by 8.61% for Java and 5.56% for Python.The comparison tests structural integration.Source: §C.7.2 · p. 25
• Removing subgraph reconstruction pre-training reduces overall EM improvements by 7.65%.The ablation tests the contribution of the reconstruction task.Source: §C.7.2 · p. 25

C.8 Generalization of CGM on Different Backbones

• CGM’s performance positively correlates with decoder coding and instruction-following abilities; Llama3.1-70B-Instruct trails Qwen2.5-72B by 17.67% but exceeds Lingma-SWEGPT by 18.33%.The reported results compare CGM across the listed backbones.Source: §C.8 · p. 25, Table 12 · p. 25

D CGM for Issue Fixing: A Case Study

• The case-study figures show issue and intermediate Rewriter, Retriever, and Reranker outputs alongside generated patches with and without code graphs and a gold patch.Green boxes mark successful patches and the red box marks an unsuccessful patch.Source: Figure 5 · p. 30, Figure 6 · p. 31

E Limitations

• The current code graph schema is untested beyond Python and Java, and extending it to other paradigms requires re-examining how graphs capture paradigm-specific structures.The paper names multiparadigm Rust and functional Haskell as examples.Source: E Limitations · p. 31

F Prompt Template Example

• The prompt-template examples cover Rewriter’s Extractor and Inferer and Reranker’s Stage 1 and Stage 2 prompts.The section identifies Figures 7–10 as the examples.Source: F Prompt Template Example · p. 31, Figure 7 · p. 32, Figure 8 · p. 33