PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Vector Similarity: Hierarchical Context-Aware Graph RAG vs Standard RAG in Enterprise Code Migration

Nilesh Jaiswal, Aniket Agrawal, Arjit Shukla, Divya Malhotra, Saurabh Garg, Suchit Puri, Suddhasatwa Bhaumik

arXiv:2609.12464v1cs.AI

TL;DR

Standard RAG can miss topological relationships in legacy-code migration, leaving child translations without inherited context. The paper presents HCRG, which uses a graph and cached architectural context for parent-first translation. It reports improved functional and dependency-related outcomes alongside trade-offs in code complexity and docstring preservation.

  • Problem

    Standard RAG’s isolated text chunks can omit inherited parents and distant dependencies, limiting its ability to preserve codebase topology during migration.

    Source: §1 · pp. 1–2

  • Method

    HCRG parses code into ASTs, maps architectural dependencies into a property graph, and caches repository structure to support parent-first translation.

    Source: §1 · p. 2

  • Results

    Graph RAG scored 83.2% aggregate Migration Quality versus Standard RAG’s 65.5%, while API hallucinations fell from 56.4% to 16.2%.

    Source: §7 · p. 11

  • Takeaways & Limitations

    Graph-based topological context provides a more viable route to functional enterprise code migration than relying on flat vector retrieval alone.

    Source: §7 · pp. 10–11

    Graph RAG’s Cyclomatic Complexity Consistency fell to 46.7% from Standard RAG’s 71.6%, and Docstring Preservation fell to 61.0% from 67.0%.

    Source: §6.2 · p. 9

Abstract

from arXiv · show

As enterprises modernize legacy monolithic systems to microservices, Large Language Models (LLMs) are heavily utilized for automated code translation. However, traditional vector-based Retrieval-Augmented Generation (Standard RAG) struggles to capture topological relationships. It fetches isolated chunks that sever inheritance chains, leading to high compilation failure rates. This paper introduces a Hierarchical Context-Resident Graph (HCRG) methodology to resolve these limitations. Our pipeline uses tree-sitter for Abstract Syntax Tree (AST) extraction, maps architectural edges into a Google Cloud Spanner Property Graph, and serializes this structure into a Gemini Context Cache for topological, parent-first code translation. We shift evaluation from naive text-overlap to a custom 7-metric Software Engineering framework. Traditional metrics like CodeBLEU (which scored 91% for both methods) effectively masked Standard RAG's structural failures behind syntactically plausible but broken code. Empirically, Graph RAG decisively mitigates dependency loss: API hallucination rates dropped from 56.4% to 16.2%, Dependency Resolution Quality improved from 34.8% to 65.9%, and Parent-Child Consistency rose from 26.7% to 45.5%. However, Graph RAG introduces specific trade-offs. The dense global context causes defensive over-engineering by the LLM, reducing Cyclomatic Complexity Consistency from 71.6% to 46.7%, and slightly degrades Docstring Preservation (67.0% to 61.0%). Ultimately, while trading code complexity for reduced hallucinations, Graph RAG provides a substantially more viable, architecturally sound path for automated enterprise codebase modernization.

1 Introduction

Legacy-code migration requires architectural understanding across files, but Standard RAG’s chunking can omit inheritance and dependency context. The paper presents HCRG for parent-first translation and reports improved structural outcomes, while noting trade-offs in complexity and docstring preservation.

1 Introduction

  • Standard RAG can miss inherited parents and distant interfaces when translating a child class, severing architectural relationships and risking compilation failures, missing imports, and API hallucinations.Enterprise operations may span multiple files, interfaces, and base classes, beyond line-by-line syntax translation.Source: §1 · pp. 1–2
  • HCRG uses tree-sitter ASTs and a Spanner Property Graph to cache the repository architecture and translate parent classes before children.The cached architecture skeleton grounds child translations in newly generated Python parent code; graph edges include INHERITS, CALLS, and IMPORTS.Source: §1 · p. 2
  • Both methods achieved an identical 91% average CodeBLEU score on spring-petclinic-genai, masking structural differences behind lexical overlap.The evaluation therefore adds seven AST-based Software Engineering metrics, including Parent-Child Consistency and Dependency Resolution Quality.Source: §1 · p. 2
  • 56.4% to 16.2% API and dependency hallucination rates accompanied Graph RAG’s higher Dependency Resolution Quality and Parent-Child Consistency.Dependency Resolution Quality rose from 34.8% to 65.9%, and Parent-Child Consistency from 26.7% to 45.5%.Source: §1 · p. 2
  • Graph RAG reduced Cyclomatic Complexity Consistency and Docstring Preservation, with the paper attributing these trade-offs to defensive over-engineering and reduced attention to secondary instructions.The implementation uses Spanner and Gemini 2.5 Pro Vertex AI, while the authors describe HCRG’s principles as platform-agnostic.Source: §1 · p. 2

2 Glossary of Terms

The glossary distinguishes flat vector retrieval from graph-based retrieval and defines HCRG as a method that combines a serialized architectural view with graph access. It also identifies the AST, DAG, database, and evaluation metrics used in the pipeline.

Abstract Syntax Tree (AST)

  • An AST represents source-code syntax hierarchically, enabling deterministic extraction of classes, function definitions, and arguments across Java, Python, and Go.The pipeline uses tree-sitter for polyglot parsing.Source: §2 · p. 3

Standard RAG (Vector Search)

  • Standard RAG embeds arbitrary text chunks and retrieves by cosine similarity, which can miss a child class’s inherited interface when lexical or semantic similarity is low.The glossary contrasts this with graph retrieval based on explicit entity relationships.Source: §2 · p. 3

Hierarchical Context-Resident Graph (HCRG)

  • HCRG serializes the repository’s highest-level structure into the LLM’s active working memory and falls back to the graph database for deep AST chains.The serialized “Architecture Skeleton” provides a low-latency global view of classes, parent hierarchies, and method signatures.Source: §2 · p. 3

Directed Acyclic Graph (DAG)

  • A DAG orders inheritance-based migration so parent code is generated before dependent child code and can ground the child translation.The glossary illustrates the ordering with BaseEntity before Person, then Person before Vet.Source: §2 · p. 3

Google Cloud Spanner

  • Spanner unifies relational and graph data, storing logical AST nodes, vector embeddings, and edges queried through GQL.The listed edges include INHERITS, CALLS, and IMPORTS.Source: §2 · p. 3

Software Engineering (SE) Utility Metrics

  • The custom evaluation framework contains seven deterministic migration-viability metrics, from Code Migration Quality to Docstring Preservation.The metrics include Parent-Child Consistency, Type Hint Completion, Linter Status, Dependency Resolution Quality, and Cyclomatic Complexity Consistency.Source: §2 · p. 3

3 Architectural Paradigms & Diagrams

The baseline retrieves isolated chunks by semantic similarity, while the graph architecture represents code relationships and uses them to ground parent-first migration. The supplied figures identify the two paradigms, and the text describes their contrasting retrieval mechanisms.

3.1 Baseline Methodology: Standard RAG (Vector Search)

  • Standard RAG splits source files into chunks and retrieves by semantic similarity, which can return sibling classes instead of required parents or distant interfaces.The baseline’s stated limitations include context loss, hidden dependencies, and single-hop reasoning boundaries.Source: §3.1 · p. 4, Figure 1 · p. 4

3.2 Graph RAG Architecture (Hierarchical Context-Resident Graph)

  • Graph RAG parses code into ASTs, maps architectural edges into a property graph, and caches the structural skeleton to translate parent classes before dependent children.The architecture explicitly injects translated parent hierarchies so child generation is grounded in structural dependencies rather than semantic guesses.Source: §3.2 · pp. 4–5, Figure 2 · p. 5

4 Pipeline Innovations and Experimental Setup

The pipeline combines graph-based topological ordering, cached architecture context, and semantic fallback, then reconstructs a compile-ready repository. Its experimental setup compares vector and graph ingestion and retrieval under shared generation settings.

4.1 Pipeline Optimizations

  • The topological migration queue builds a DAG from INHERITS and CALLS edges, resolves cyclic dependencies, and injects generated parent code into child prompts.This supports precise super() mapping, method overriding, and reduced hallucination of inherited attributes.Source: §4.1 · pp. 5–6
  • A one-hour Context Cache stores a global JSON architecture skeleton of classes, parent-child relationships, and method signatures to reduce repeated input-token costs and latency.The cache provides a persistent, low-latency architectural view during translation.Source: §4.1 · p. 6
  • Vector search supplements explicit topological edges by retrieving semantically relevant AST structures such as loosely coupled utilities and unstructured documentation.Embeddings are stored alongside graph nodes in Spanner.Source: §4.1 · p. 6
  • Repository reconstruction converts Java paths and filenames to Python conventions, creates package initializers, and preserves non-code assets.The pipeline targets a compile-ready repository rather than isolated snippets.Source: §4.1 · p. 6

4.2 Experimental Setup Parameters

  • Standard RAG used 800-character chunks with 150-character overlap and top-five cosine-distance retrieval, while Graph RAG used tree-sitter ASTs, mapped edges, and up to three graph hops.Graph RAG’s architecture skeleton was cached for one hour.Source: §4.2 · p. 6
  • Both pipelines used Gemini 2.5 Pro with generation temperature fixed at T = 0 for predictable code generation.Source: §4.2 · pp. 6–7

5 Evaluation Logic Methodology & Enhanced Context

The evaluation shifts from lexical overlap to structural validation and defines Migration Quality as a weighted measure of functional viability. It also specifies AST-based checks and a global lookup constraint for evaluating generated code.

5 Evaluation Logic Methodology & Enhanced Context

  • The evaluation uses Python AST topology checks and pyflakes syntax verification to assess structural faithfulness rather than text prediction.The framework includes seven software-engineering metrics anchored by an aggregate Migration Quality score.Source: §5 · p. 7

5.1 Redefining Migration Quality (MQ)

  • Migration Quality (MQ) is a weighted composite for functional viability that heavily penalizes failures to map cross-file logic.Its weights are dependency resolution 40%, inheritance accuracy 30%, type integrity 20%, and syntactic correctness 10%.Source: §5.1 · p. 7

5.2 Deep Dive: Extracting AST Evaluation Metrics

  • Dependency and API Resolution validates imports against standard-library names, an internal whitelist, and dynamic availability, penalizing unresolvable libraries.Parent-Child Consistency compares Java inheritance intent with Python class bases, flagging missing translated parents or invented ones.Source: §5.2 · p. 7
  • Type Hint Completion measures fully typed function arguments and return annotations, excluding self and cls from the argument check.The script traverses synchronous and asynchronous function definitions.Source: §5.2 · p. 8
  • Static Analysis / Linter Score uses pyflakes and deducts 10 points per localized syntax error or unused or undefined variable.The check runs directly against generated Python code stored in a temporary file.Source: §5.2 · p. 8
  • Cyclomatic Consistency compares Java and generated Python complexity, allowing a variance of 2 or 10% to count as consistent.The tolerance accounts for natural-language shifts such as Python list comprehensions being less complex than nested Java loops.Source: §5.2 · p. 8

5.3 The Hallucination “Global Lookup” Mechanism

  • File-level checks can mislabel valid calls to methods defined in other modules as hallucinations, so the evaluation implements a Global Lookup constraint.An ast.Call is deemed a hallucination only if it does not exist in the global lookup sources.Source: §5.3 · p. 8

3. A whitelist of common collection methods (e.g., append, keys, split)

The item identifies the Global Symbols Table as representing the entire Spanner codebase.

3. A whitelist of common collection methods (e.g., append, keys, split)

  • The Global Symbols Table represents the entire Spanner codebase.Source: §3 · p. 8

6 Empirical Results and Analysis

On the spring-petclinic-genai Java-to-Python benchmark, Graph RAG improved structural migration measures over Standard RAG, while also showing trade-offs in complexity consistency and docstring preservation.

6 Empirical Results and Analysis

  • 31.1% Dependency/API Resolution gain and 18.8% Parent-Child Consistency gain accompanied a 16.2% hallucination rate for Graph RAG.These aggregate results compare Graph RAG with Standard RAG in the benchmark.Source: Table 1 · p. 8, §6 · p. 8

6.1 Structural Integrity and Object-Oriented Mapping

  • Dependency & API Resolution Quality (DRQ) improved by 31.1% (from 34.8% to 65.9%), while Type Hint Completion (THC) rose by 29.5% (from 51.4% to 80.9%).The pipeline supplies deterministic dependency signatures and return types to support structural mapping.Source: §6.1 · p. 9

6.2 Analysis of Lexical Overlap Bias and Contextual Trade-offs

  • CodeBLEU scores were near-parity at 90.6% for Standard RAG and 91.1% for Graph RAG, despite divergent structural quality.The passage attributes this mismatch to CodeBLEU's bias toward localized string overlap and inability to penalize broken API contracts or hallucinated imports.Source: §6.2 · p. 9
  • Cyclomatic Complexity Consistency fell to 46.7% from Standard RAG's 71.6%, while Docstring Preservation fell to 61.0% from 67.0%.The paper links the complexity drop to defensive over-engineering and the docstring drop to attention dilution in the context-heavy prompt.Source: §6.2 · p. 9

6.3 Class-Level Variance Analysis

  • For high-variance controllers, Standard RAG scored 3% Migration Quality on OwnerController.java, versus Graph RAG's 77% and 80% on OwnerController.java and VisitController.java, respectively.Controllers require multi-hop resolution across Services, DTOs, and Repository interfaces that are usually outside the same file.Source: §6.3 · p. 9
  • Standard RAG scored 0% Dependency/API Resolution Quality on external endpoints, while Graph RAG reached DRQ = 100% and near-zero hallucinations.The comparison concerns high-variance controller and test files.Source: §6.3 · p. 10, Table 2 · p. 10
  • In OwnerController.java, Cyclomatic Complexity Consistency was 18% for Graph RAG versus 84% for Standard RAG.The paper characterizes this as context-informed over-engineering versus line-by-line logic copying.Source: §6.3 · p. 10
  • For self-contained files such as PetType.java, both methods performed similarly when the class context fit within one 800-character vector chunk.PetType.java had identical scores across the methods, including 0% hallucinations and 100% Docstring Preservation and Cyclomatic Consistency.Source: §6.3 · p. 10

7 Conclusion and Future Work

The conclusion presents HCRG as restoring multi-hop topological context for enterprise modernization and reports improved functional translation outcomes over Standard RAG, alongside complexity and docstring trade-offs. It proposes hybrid retrieval and richer AST/runtime context as future work.

7 Conclusion and Future Work

  • HCRG decomposes code into an AST and loads relational edges into a property graph to re-establish multi-hop topological context for enterprise modernization.The paper identifies Google Cloud Spanner as its property graph database.Source: §7 · p. 10
  • 83.2% vs 65.5% aggregate functional translation accuracy; parent-first sorting cut API hallucinations from 56.4% to 16.2% and raised DRQ by 31.1%.The comparison is between Graph RAG and the Standard RAG baseline.Source: §7 · p. 11
  • 91.1% vs 90.6% CodeBLEU scores create false equivalence, masking architectural failures in Standard RAG.The conclusion contrasts near-parity text-biased metrics with functional translation accuracy.Source: §7 · pp. 10–11
  • Global structural awareness degrades CCC by 24.9% and DP by 6.0%, with defensive over-engineering and attention dilution cited as explanations.The model attempts to compensate for distant edge cases and prioritizes structural adherence over secondary comment translation.Source: §7 · p. 11
  • Future work proposes hybrid Graph/Vector RAG, broader AST parsing for reflection and complex dependencies, and runtime traces added to the static graph.The proposed hybrid assigns core object-oriented logic to Graph RAG and unstructured documentation to Vector RAG.Source: §7 · p. 11
Loading 2609.12464v1…