Source-linked AI summary
Legacy System Modernization with Coding Agents: A Case Study
Iago da Silva Rodrigues Alves, Cristiano Politowski, João Eduardo Montandon
TL;DR
Legacy systems are difficult and costly to modernize, especially when manual rewriting risks behavioral differences. This paper studies Claude Code migrating 12 VB6 ERP features to C# .NET 10 and finds 70% overall equivalence, with substantially better results for low-complexity features; conclusions about high-complexity migration remain limited by the small sample.
Problem
Legacy systems support critical processes but impose maintenance and modernization burdens because obsolete technologies are costly to replace and manual rewrites can introduce behavioral differences.
Method
A real industrial case study used Claude Code to migrate 12 VB6 ERP features to C# .NET 10 under a single-generation strategy, measuring equivalence and agent efficiency.
Results
70% overall equivalence was achieved across 12 features, ranging from 92% for low-complexity features to 47% for high-complexity features.
Takeaways & Limitations
The strategy shows promising results especially for low and well-scoped features, while fragmented files and implicit business rules detracted from performance.
Takeaways & Limitations
Only two high-complexity features were evaluated, limiting the robustness of conclusions at that complexity level.
Abstract
from arXiv · showhide
Legacy systems built on discontinued platforms are a recurring technological liability in organizations that depend on these applications to sustain critical business processes. Although modernization is strategically necessary, it is costly and error-prone when performed exclusively through manual effort. In this paper, we report on a case study conducted in a real industrial setting, where we evaluate both the effectiveness and efficiency of AI coding agents in supporting the migration of legacy systems to modern platforms. By using one version of Claude Code agent, we migrated 12 features of distinct complexity levels from a corporate ERP system written in Visual Basic 6 to C# .NET 10 under a single-generation strategy. Once the migration sessions were completed, we measured the equivalence of the migrated features against the original ones, and collected data about the time spent and the number of tokens consumed by the agent during the process. In our experiment, the agent achieved an average equivalence of 70%, with a strong asymmetry across complexity levels; low-level features achieved 92%, and high-complexity ones scored 47%. Similar asymmetry was observed in the cost-based metrics; low-level features consumed 1.47M tokens ($1.66), whereas high-level ones used 9.09M ($10.28). We reveal the practical scenarios and circumstances where this migration strategy is more effective, as well as discuss the limitations and challenges of using AI agents for legacy system modernization.
1. Introduction
Legacy systems remain critical but costly liabilities, motivating evaluation of AI agents for modernization. This case study examines Claude Code’s migration of VB6 ERP features and reports effectiveness, efficiency, practical scope, and limitations.
- Motivation: Organizations retain obsolete systems because they support critical processes, yet maintaining them creates technological liability and modernization challenges.Challenges include high maintenance costs, fewer specialists, outdated documentation, and incompatibility with modern components.
- Motivation: LLMs and AI agents can assist migration by interpreting source-code syntax and semantics, inferring intent, and generating equivalents in other languages.AI agents combine language models with tools and environment access for complex software-engineering tasks.
- Study: Claude Code migrated 12 features from a production VB6 ERP system to C# .NET 10 under a single-generation strategy, with equivalence and efficiency measured.The case study covered two modules in a real industrial setting.
- Results: 70% overall equivalence was observed, with 92% for low-complexity, 81% for medium-complexity, and 47% for high-complexity features.Equivalence covered persistence and functional behavior.
- Results: Low-complexity features consumed 1.47M tokens on average ($1.66), whereas high-complexity features consumed 9.09M tokens on average ($10.28).The reported efficiency metrics therefore varied substantially with feature complexity.
- Contributions: The study contributes an evaluation methodology and identifies practical circumstances, limitations, and challenges affecting AI-assisted legacy modernization.The methodology measures effectiveness through persistence and functional instructions and efficiency through time, cost, and token consumption.
2. Study Design
The study migrates selected features from two actively used VB6 ERP modules into a planned C# .NET 10 architecture. It varies feature complexity, establishes rule and database-operation baselines, and executes isolated single-pass agent sessions.
- Legacy System: The selected ERP has operated for over two decades and uses VB6, a plugin architecture, and separate modules for mall-management functions.The core contains 819K LOC, while the selected secondary modules support measurements and purchases.
- Legacy System: MEASUREMENTS handles resource consumption and cost calculations, whereas PURCHASES manages suppliers, negotiations, orders, approvals, and cost allocation.The modules cover numerical calculations and transaction-heavy approval workflows, respectively.
- Legacy System: Both modules couple business logic to VB6 forms and classes, allowing the study to examine agent performance across different domains and conditions.The modules are actively used in daily operations by clients.
- Target Architecture: The target platform was C# .NET 10 with layered Controllers, Services, and Repositories, Dapper data access, and the existing SQL Server database.The target architecture reflects the owning company’s technology direction.
- Selected Agent: Claude Code 2.1.96 ran on the Opus 4.6 1M model, providing long context, filesystem and terminal access, and multilingual programming capabilities.These capabilities supported analyzing, creating, editing, compiling, and executing migration artifacts.
- Selected Features: The study selected 12 features—six low, four medium, and two high complexity—and classified complexity using lines of code, methods, and class dependencies.A feature met at least two of three criteria for assignment to a level; high-level features involved complex calculations, batch processing, or sophisticated workflows.
- Evaluation Baseline: The researchers manually counted rules and database operations to establish a legacy baseline for comparing migrated behavior and persistence.Rules included validations, calculations, and restrictions; database operations included SQL and persistence-layer manipulation.
- Migration Execution: Each feature ran in an isolated environment containing instructions, read-only VB6 sources, and a migration directory with the expected target structure.The target structure included Data, Endpoints, Models, and Services.
3. Research Questions
The study evaluates coding-agent migration through two research questions: whether migrated features preserve legacy behavior and how much time, cost, and token usage the process requires. It operationalizes these questions with equivalence and efficiency metrics covering persistence, functional behavior, time, cost, and tokens.
- RQ1: Equivalence: The evaluation asks whether migrated features preserve the original system’s persistence state and functional behavior.Persistence concerns database state, while functional behavior concerns business rules and validations.
- RQ2: Efficiency: The study measures migration efficiency through iteration time, monetary cost, and token consumption.These metrics address total session time, pricing-based cost, and input/output tokens extracted from session telemetry.
- RQ1: Equivalence: Persistence Parity compares records stored by the original and migrated features under the same usage scenario.A record matches when the same values are stored in the same tables and fields.
- RQ1: Equivalence: Functional Parity checks whether both versions apply identical business rules, validations, approval workflows, and calculations.A rule is equivalent when both versions behave exactly alike for the same inputs, including rejecting the same invalid inputs for the same reasons.
- Metric calculation: Equivalence percentages are calculated separately for persistence and functional parity by dividing equivalent instructions by the evaluated instructions.The study counts equivalent instructions for each dimension before computing the corresponding percentage.
4. Results
Across 12 migrations, Claude Code achieved 70% overall equivalence, but results varied sharply with feature complexity and exposed recurring omissions in broader workflows and integrations. Efficiency likewise worsened as complexity increased.
- Equivalence: 100% persistence parity was achieved for low-level features, falling to 77% for medium-level and 51% for high-level features.High-level failures included entire persistence workflows left unimplemented.
- Equivalence: 83% functional parity was achieved for low-level features and 85% for medium-level features, but high-level features reached only 42%.Missing high-level rules mostly involved sophisticated workflows and cross-module validations.
- Validation: 93% reviewer agreement supported the functional-equivalence assessment, with 110 of 160 rules classified as equivalent by the second reviewer.Disagreements concentrated on partially migrated rules that were implemented but not connected to execution flows.
- 70% overall equivalence was achieved across the 331 evaluated instructions, with 92% for low-, 81% for medium-, and 47% for high-complexity features.
- Failure analysis: 100 of 331 instructions were not migrated, most often because the agent overlooked adjacent modules, database operations, or individual rules.Adjacent-module omissions accounted for 42 cases, followed by missing database operations with 22 and missing rules with 20.
- Efficiency: 7 minutes 38 seconds and $10.23 were required on average for high-level features, compared with 2 minutes 48 seconds and $1.66 for low-level features.Migration time increased 2.7× and cost increased 6.2× from low- to high-level features.
5. Discussion
The agent’s migration performance depended strongly on code organization and rule explicitness, while migration costs were driven mainly by input volume. Fragmented code and implicit business rules reduced equivalence, especially as complexity increased.
- 85% average functional equivalence for medium-level features exceeded the 83% achieved by low-level features.The result was counterintuitive because complexity levels and instruction counts differed.
- 67% feature equivalence for low-level F9 contrasted with 92% for medium-level F10 despite F9 having fewer lines of code and rules.F9 had 926 LOC and six rules, whereas F10 had 4,158 LOC and 12 rules.
- Fragmented rules across forms, global objects, and domain classes appeared to determine performance more than feature size alone.The agent captured rules concentrated in well-scoped .cls domain classes but missed behaviors implemented in .frm events or referenced global objects.
- Explicit validations and database operations were migrated more successfully than implicit rules hidden in parameters, configuration values, or form conditions.The agent omitted restrictions such as NaoEnergia, cfg_codigotr, and implicit query thresholds.
- 1.46M input tokens versus 8.6K output tokens for low-level features, and 9.06M versus 30.3K for high-level features, show strong input-output asymmetry.Input-to-output ratios were 170x for low-level and 299x for high-level features.
- Migration cost was largely affected by received input volume rather than generated output, with F4, F6, and F12 standing out as input-token outliers.The outliers consumed 7.20M, 12.29M, and 5.84M input tokens, respectively.
6. Related Work
Related research has extensively studied LLMs for software engineering, but legacy system modernization remains underrepresented. Prior migration studies report complexity-related performance limits, while iterative human-in-the-loop cycles can improve equivalence.
- Legacy system modernization remains underrepresented among LLM applications to software engineering.A systematic review of 395 papers found code generation and comprehension to be the most explored tasks.
- This study addresses that gap by evaluating modernization of enterprise systems maintained for decades in outdated languages such as VB6.
- Previous code-translation studies found that model performance is constrained by the complexity of the migrated code.Those studies examined translations among languages including Python, Java, C++, and Rust using test suites for functional equivalence.
- Iterative human-in-the-loop cycles significantly increased migrated-code equivalence in prior work.This paper instead used a single-generation strategy to assess autonomous agent efficiency.
7. Threats to Validity
The study mitigated some validity risks through predefined criteria and independent review, but its complexity labels, single-generation design, configuration, and small high-complexity sample constrain interpretation and generalization.
- Construct Validity: 0.84 inter-rater agreement supported the reliability of manually collected functional-equivalence judgments.A second reviewer independently evaluated all 160 business rules, and both reviewers reached the same overall functional equivalence.
- Internal Validity: Complexity classifications may change for features near category boundaries when threshold choices differ.The study used lines of code, method count, and class dependencies, with at least two of three criteria required for classification.
- Internal Validity: The single-generation design limits extrapolation to iterative approaches, which tend to produce higher equivalence.
- External Validity: Findings are bounded by one VB6 ERP, one source-to-target language pair, one agent, and one model configuration.Replication is required before generalizing to other agents, models, domains, or language combinations.
- Conclusion Validity: Only two high-complexity features limit the robustness of conclusions about migrations at that level.The observed high-complexity trends are treated as indicative, and expanding the sample is proposed as a next step.
8. Conclusion
The case study evaluated an AI coding agent migrating 12 features from an industrial VB6 ERP to C# .NET 10 for behavioral equivalence and efficiency. It found promising results for low and well-scoped features, while identifying fragmented code and implicit rules as performance barriers.
- The study migrated 12 features from two industrial ERP modules from VB6 to C# .NET 10 and evaluated effectiveness and efficiency.Efficiency covered time, cost, and token consumption.
- 70% functional equivalence was achieved across all 12 features in approximately 59 minutes at a cost of $50.29.
- Low-level features reached 92% equivalence at an average cost of $1.66, whereas high-level features scored 47% at $10.23.
- Fragmented code and implicitly coded business rules detracted from agent performance, especially in complex migration cases.
- Future work will test iterative generation, multi-agent workflows, feature decomposition, and additional agents and language pairs.