PaperlayerView exact revision on arXiv

Source-linked AI summary

Identifying Latent Declarative Representations of Code for Assisting Repository Migration

Shraddha Surana, Ashwin Srinivasan, Michael Bain

arXiv:2608.23619v1cs.SEcs.AI

TL;DR

Legacy repositories embed undocumented domain knowledge, making repository-scale modernization difficult. ADFD-Migrate makes an inferred declarative representation explicit through ADFDs, uses static-analysis-guided refinement and dependency-aware chunking, and evaluates ports on behavioral agreement and migration completeness. On 50 Fortran repositories, it outperformed direct translation across these dimensions, while behavioral probes remained non-exhaustive and expert verification was unavailable.

  • Problem

    Undocumented legacy code embeds domain knowledge and complicates understanding and modernization, while direct LLM translation lacks cross-module structure, error correction, and an inspectable intermediate representation.

  • Method

    ADFD-Migrate infers an inspectable ADFD from bounded repository context, refines it with static-analysis coverage checks, chunks dependencies for target generation, and compares source and target ADFDs for regeneration.

  • Results

    The method passed 327 of 382 source-oracle probes and achieved a 17–59 percentage-point outcome-index advantage over direct translation on 47 repositories.

  • Takeaways & Limitations

    An inspectable semantic bottleneck can improve repository-scale migration coverage and integration while enabling lower-cost generation for many repositories.

  • Takeaways & Limitations

    The 382 behavioral probes are targeted checks rather than exhaustive system tests, and human expert verification was unavailable.

Abstract

from arXiv · show

Legacy software repositories embed decades of domain knowledge in undocumented code, making understanding and modernization difficult. We treat a program as the implementation of an unobserved, declarative description of its computation and investigate whether making this latent declarative representation explicit improves repository-scale porting. ADFD-Migrate approximates the latent representation with an annotated data-flow diagram (ADFD) of processes, data stores, external entities, flows, and behavioral contracts. An LLM infers the source ADFD from bounded repository context, guided by static-analysis coverage checks. Dependency-aware chunking orders bounded process groups for target-language generation. Differences between the source ADFD and a statically recovered target ADFD then guide regeneration. We evaluate ADFD-Migrate on f2x50, a new benchmark of 50~Fortran repositories spanning 1.5k--1.6M lines of code and three complexity tiers, and assess the resulting ports along two dimensions: porting soundness, measured by source-oracle behavioral agreement, and porting completeness, measured by a composite migration outcome index. Against 382 curated Fortran-oracle probes, the generated Python passes 327 (85.6\%), with 40 repositories passing every attempted probe. ADFD-Migrate exposes all 382 planned behaviors as runnable targets, compared with 99 and 98 for direct and repository-context translation and 69 and 30 for the static-profile and dependency-chunking ablations. It also achieves a 93.1\% mean migration outcome index and a 17--59 percentage-point outcome-index advantage over direct translation on 47 repositories. These results suggest that an inspectable semantic bottleneck can improve the coverage and integration of repository-scale migration while enabling lower-cost generation for many repositories.

Glossary

The paper defines the concepts underlying ADFD-Migrate and situates its repository-scale Fortran-to-Python migration framework. Its contributions include an inspectable ADFD intermediate representation, dependency-aware generation, and the f2x50 benchmark.

  • ADFD means annotated data-flow diagram, representing processes, data stores, external entities, flows, and process contracts.
  • A behavioral probe compares Fortran and generated Python behavior on the same input using a source-oracle test.
  • A latent declarative representation is an unobserved, language-independent specification of the computation implemented by a program.
  • ADFD-Migrate makes the latent representation explicit and inspectable, then uses it for cross-language repository porting.
  • Dependency-aware chunking partitions ADFD process dependency graphs into bounded, topologically ordered groups while preserving referential integrity.The strategy uses Tarjan’s SCC detection and Kahn’s topological sorting.
  • f2x50 is a curated benchmark of 50 open-source Fortran repositories across three complexity tiers with converted Python repositories, intermediate ADFDs, and evaluation artifacts.

2. Background and Related Work

The paper distinguishes ADFD-Migrate from existing migration and agentic coding approaches by using a semantic, inspectable intermediate representation and dependency-aware repository decomposition. It also describes iterative ADFD construction and graph-based chunking to address repository scale.

  • Background and Related Work: Existing Fortran tools differ in output and scope: f2py creates bindings, fable produces Fortran-like C++, and LFortran targets a subset of Fortran 2018.
  • Background and Related Work: Prior LLM translation methods commonly operate at file or function level, while repository-scale approaches use varied decomposition, intermediate-language, planning, or knowledge-graph strategies.
  • Background and Related Work: ADFD-Migrate translates through a semantic intermediate representation capturing computational intent and is designed for human inspectability, unlike call-graph decomposition based on syntactic structure.
  • Our Approach: The ADFD semantic bottleneck organizes generation around processes, data stores, entities, flows, and behavioral contracts, enabling checking, review, partitioning, and source–target comparison.
  • Our Approach: Source ADFD construction iteratively combines bounded repository context with static-analysis profiles, coverage scoring, and gap directives.
  • Our Approach: Dependency-aware chunking partitions large ADFD process graphs into bounded, topologically ordered groups to preserve referential integrity and generation order.

4. Empirical Evaluation

The evaluation tests whether ADFD-mediated migration improves repository-scale Fortran-to-Python porting over direct translation, using behavioral agreement and migration completeness across the f2x50 benchmark.

  • Evaluation dimensions: The study evaluates porting soundness through behavioral agreement and porting completeness through executable, maintainable target repositories.Behavioral agreement compares source and target behavior on identical inputs; completeness measures whether repositories are fully migrated.
  • Benchmark: f2x50 provides a repository-scale benchmark of 50 Fortran repositories spanning diverse scientific domains and three structural-complexity tiers.Repositories exceed 1,000 lines of code, are compilable, open source, and lack public Python translations.
  • Procedure: For each repository, researchers curate source probes, construct an ADFD, generate Python, recover and compare a target ADFD, then regenerate using detected differences.Repositories are processed by complexity class with fixed generator assignments and dependency-aware generation settings.
  • Measures: Behavioral agreement compares Fortran-oracle and generated-Python outputs on identical inputs, counting absent, unusable, or incorrect target behavior as probe failures.The study reports agreement over all planned probes and separately tallies missing functionality and runnable-only rates.
  • Measures: The migration outcome index combines semantic alignment, executability, code quality, and implementation rate, and is distinct from a correctness percentage.Correctness claims rely on behavioral agreement rather than the outcome index itself.

5. Results

ADFD-Migrate substantially outperforms direct translation on planned-probe behavioral agreement and repository-level migration outcomes, with its largest advantages appearing in runnable coverage and complex repositories.

  • 5.1. Porting Soundness: Behavioral Agreement: 85.6% of 382 planned probes agree with Fortran oracles for ADFD-Migrate, compared with 21.7% for each direct-translation baseline, 11.0% for static-profile ablation, and 3.7% for dependency-chunking ablation.ADFD-Migrate passes 327 probes, and 40 of 50 repositories pass every probe.
  • 5.1. Porting Soundness: Behavioral Agreement: All 382 planned probes are runnable for ADFD-Migrate, versus 99 and 98 for the two direct baselines and 69 and 30 for the ablations.The overall agreement gap is therefore driven primarily by whether methods produce runnable target behavior.
  • 5.1. Porting Soundness: Behavioral Agreement: 85.6%, 83.8%, and 84.7% are the runnable-only agreement rates for ADFD-Migrate and the two direct baselines, showing comparable correctness when those methods produce runnable code.The ablations have lower runnable-only rates of 60.9% and 46.7%.
  • 5.2. Porting Completeness: Migration Outcome Index: 93.1% is ADFD-Migrate’s mean migration outcome index across 50 repositories, exceeding direct translation by 16.9–58.5 percentage points across tiers.The outcome-index gap increases with repository complexity.
  • 5.2. Porting Completeness: Migration Outcome Index: The repository-context baseline raises the direct mean from 0.654 to 0.709 but remains 21.5 percentage points below ADFD-Migrate and loses on 46 of 47 pairs.Static-profile and dependency-chunking ablations also trail ADFD-Migrate on all 47 available pairs.
  • 5.3. Supporting Analysis: Mechanism and Practicality: Import success remains the main integration weakness at 72.5–79.8% by tier, while deterministic reconciliation raises mean import success from 79.7% to 82.3%.Syntax validity is near-perfect, indicating that bounded generation is more reliable than integration.

6. Discussion

The discussion clarifies how ADFD-Migrate handles data flow, control flow, testing, and validity boundaries, while identifying assumptions and unresolved evaluation limitations.

  • Additional Questions and Answers: ADFD-Migrate relies on Fortran’s non-overlapping argument convention and treats aliases as one data store, without running alias analysis.Pointer-heavy languages such as C would require language-specific points-to analysis in their adapters.
  • Additional Questions and Answers: Control flow remains inside process specifications and contracts, making control-flow-heavy repositories more challenging.quadpack recovered 1/12 probes and roots-fortran 0/7 because stopping logic was not correctly recovered.
  • Additional Questions and Answers: The 382 behavioral probes use source-executed expected values fixed before porting, avoiding dependence on uneven or potentially faulty Fortran test suites.Translating an existing test suite could confound port failures with test defects.
  • Limitations and Threats to Validity: Behavioral agreement is a lower-bound correctness signal because the probes are targeted case-level checks rather than exhaustive system tests.Exhaustive equivalence testing is infeasible across repositories requiring specialized inputs, runtimes, and output oracles.
  • Limitations and Threats to Validity: The outcome index supports comparative migration assessment but is not a correctness percentage.The study also lacks human expert verification, uses non-deterministic generation, and reports possible judge family bias without depending on it for correctness claims.

7. Conclusion

ADFD-Migrate makes an inferred declarative representation explicit and uses it to organize and refine repository migration. On f2x50, it combines strong behavioral agreement with broader runnable-behavior coverage and consistent outcome-index advantages, while acknowledging formula-level failures.

  • Conclusion: ADFD-Migrate treats a program as implementing an unobserved declarative description and makes an inspectable ADFD approximation explicit.Dependency-aware chunking then organizes target generation in bounded, ordered units.
  • Conclusion: 327 of 382 source-oracle probes pass, while 3.9× as many planned behaviors become runnable targets as with either direct baseline.The benchmark contains 50 repositories.
  • Conclusion: The outcome-index advantage over direct translation is 17–59 percentage points across repository tiers on 47 repositories.The conclusion reports this pattern alongside the behavioral results.
  • Conclusion: ADFD approximation, dependency-aware chunking, and semantic decomposition are presented as ways to structure migration beyond relying only on model scale.Future work includes failed-probe-driven refinement, stronger numerical contracts, additional language pairs, and domain-aware repair.

CRediT authorship contribution statement

The supplied material combines implementation details of the ADFD-Migrate architecture with repository-profile construction, dependency-aware generation, target alignment, evaluation boundaries, and benchmark documentation.

  • Architecture: Figure A.1 expands the workflow into source ADFD construction, an explicit semantic bottleneck, target generation, and source–target alignment and refinement.Static-analysis coverage, dependency-aware chunks, and ADFD differences support these stages.
  • Source ADFD Construction: Static analysis produces a repository profile containing definitions, module relationships, call edges, signatures, and input/output information for source-ADFD inference and coverage checking.Only a bounded projection and ranked excerpts condition the LLM, while the complete profile remains available to the coverage agent.
  • Target-Code Generation: Dependency-aware generation collapses strongly connected components, topologically orders processes, and packs bounded chunks while preserving component integrity.Each chunk receives its sub-ADFD, referenced contracts, and upstream interface summaries.
  • Alignment and Refinement: After generation, deterministic Python analysis recovers a target profile and ADFD whose differences from the source ADFD guide bounded regeneration.The evaluation-only LLM judge is outside this refinement loop.
  • Evaluation Boundary: Coverage and ADFD alignment diagnose representation completeness and structural preservation, whereas matched source-oracle probes provide the primary behavioral evaluation.Structural checks cannot establish behavioral correctness.
  • Artifacts and Configuration: The benchmark artifacts, inferred ADFDs, generated Python, metrics, manifests, and analysis outputs are publicly available through the project repository and Zenodo.Appendix A documents the implemented architecture used in the experiments.

C.1. Model Configurations

The study compares ADFD-Migrate with direct, repository-context, static-profile, and dependency-chunking alternatives using controlled model configurations and structural and semantic alignment checks.

  • Model configurations: Three generator configurations assigned models by repository completion capacity rather than complexity tier.Qwen-3-Coder 30B handled low-tier repositories and four medium-tier repositories; 480B handled the remaining medium-tier repositories; Claude Sonnet 4.5 handled complex-tier repositories.
  • Controlled comparisons: Each repository used the same assigned generator for ADFD-Migrate, baselines, and ablations.
  • Generation settings: Temperature 0.3, JSON output, and a 1200 s timeout were fixed across generation models rather than tuned empirically.The non-zero temperature permitted bounded refinement retries after repeated structural defects.
  • Baselines and ablations: The evaluated alternatives were independent file translation, repository-context translation, static-profile translation without an ADFD, and dependency-chunking translation without an ADFD.
  • Alignment: A deterministic structural comparison drives refinement, while an LLM semantic comparison is used only for evaluation.Structural differences include unmatched processes, flows, and data stores after name and flow normalization; composite alignment scores are diagnostic only.
  • Behavioral evaluation: Behavioral probes use source-side routines, executable Fortran or f2py oracles, identical inputs, and predefined comparison modes with numeric tolerances where applicable.

C.5. Prompts

The prompts expose bounded structural and behavioral context for ADFD-mediated generation while keeping baselines and ablations on comparable output and engineering requirements.

  • C.5. Prompts: ADFD inference uses two LLM passes per module family to produce structure first, then specifications, conditions, and typed I/O.
  • C.5. Prompts: Each ADFD prompt supplies a bounded profile projection, repository metadata, constraints, and later gap directives plus the previous best ADFD.The schema supports coverage scoring through source-file mappings and typed process inputs and outputs.
  • C.5. Prompts: ADFD target-generation prompts request complete Python implementations using suitable numerical libraries while preserving precision, contracts, interfaces, and documentation.
  • C.5. Prompts: Direct translation prompts provide each source file in full and request equivalent Python without repository-level intermediate context.
  • C.5. Prompts: Repository-context prompts add module names, interfaces, dependencies, shared constants, and naming context while retaining independent file generation and excluding declarative representations.
  • C.5. Prompts: The static-profile ablation provides repository-level static-analysis summaries, whereas dependency-chunking prompts provide call-graph chunks and upstream interface summaries.
  • C.5. Prompts: All generation variants share the same assembly schema, path conventions, and engineering requirements, differing only in the context supplied.
  • C.6. Why Not Reference-Based Metrics: Reference-based text metrics are excluded because f2x50 has no reference Python translations and cross-language similarity poorly represents functional correctness.

C.8. Generated Tests

Generated tests provide runtime and packaging diagnostics, while source-oracle probes and outcome-index comparisons supply the principal cross-method evaluation evidence.

  • C.8. Generated Tests: Generated pytest suites test syntax, imports, packaging, and runtime behavior, but their expected outcomes are not independent Fortran-oracle evidence.
  • C.8. Generated Tests: AST-level structural comparisons record function, class, identifier, and call-graph differences as diagnostics rather than outcome-index components or refinement triggers.
  • C.8. Generated Tests: Table D.1 decomposes the 382 source-oracle probes by repository tier.
  • C.8. Generated Tests: Table D.2 reports outcome-index distributions for ADFD-Migrate and four non-ADFD alternatives on a common 47-repository subset.
  • C.8. Generated Tests: Table D.3 presents representative repository-level migration outcome comparisons.
  • C.8. Generated Tests: 0.270 mean outcome-index points separate ADFD-Migrate from direct translation across all available pairs, with positive differences also against every other comparison.The corresponding mean improvements are 0.426 over static-profile, 0.389 over dependency-chunking, and 0.215 over repository-context direct; all Wilcoxon tests report p<0.001.

D.3. Generated Tests and Import Reconciliation

The appendix reports generated-test and import diagnostics, examines whether outcome scores track behavioral agreement, and tests sensitivity to aggregation and model-scale choices.

  • D.3. Generated Tests and Import Reconciliation: 99.9% of 1,206 non-skipped generated tests passed, but the tests are runtime sanity checks rather than independent evidence of Fortran equivalence.There were 1,232 executions: 1,205 passed, one failed, and 26 were skipped.
  • D.3. Generated Tests and Import Reconciliation: 770 of 972 generated modules imported successfully; missing dependencies and ImportError were the dominant failure categories.
  • D.3. Generated Tests and Import Reconciliation: 82.3% mean module-level import success followed reconciliation, up from 79.7%, after 37 files were modified across 12 repositories.The pass repairs verifiable paths, reexports, package initializers, and standard-library imports without synthesizing logic.
  • D.4. Outcome Index Versus Behavioral Agreement: Outcome index and behavioral agreement are not monotonic across repositories.The appendix identifies formula, numerical-kernel, and diagnostic-formatting defects among the worst-agreement cases.
  • D.4. Outcome Index Versus Behavioral Agreement: Contract-level defects included bisection termination, Padé coefficients, numerical kernels, and diagnostic formatting or side effects.The paper proposes richer contracts and source-oracle probes, while noting that human ratification was not evaluated.
  • D.5. Sensitivity of the Migration Outcome Index: 98.9% of 10,000 random Dirichlet weightings preserved a positive mean ADFD–baseline gap.The mean sampled gap was 0.143 and the median was 0.146.
  • D.5. Sensitivity of the Migration Outcome Index: 93.0% mean outcome index was achieved by the local 30B model on all 20 low-tier repositories, versus 76.1% for its direct-translation ports.Medium-tier model comparisons are descriptive because assignment depended on successful completion.
  • D.5. Sensitivity of the Migration Outcome Index: Source ADFDs contain up to 132 processes by tier and become 1–42 dependency-ordered chunks with c_max = 3; large SCCs remain unsplit.

D.7. Cost Analysis

The cost analysis reports zero metered generation charges for low- and medium-complexity repositories, while complex repositories incurred approximately $117. It also describes substantial savings and call reduction from dependency-aware chunking.

  • Zero metered generation charges were recorded for low- and medium-complexity repositories under local and cloud-hosted Ollama configurations.
  • Approximately $117 was incurred across the ten complex repositories.
  • ADFD-Migrate cost about 54% as much as the plain direct baseline for seven complex repositories: $86 versus $159.
  • lapack used 18 process chunks instead of one call per 3,587 files, reducing generation calls by approximately 200×.lapack was excluded from the evaluation analysis.
  • The explicit ADFD provides an inspectable artifact-level checkpoint before target-code generation.The same representation also supports RATIFY, REFUTE, REVISE, and REJECT interactions in the Dhaani protocol.

Semantic Beh. Outcome Time Model

The supplied material consists primarily of a partial metrics row and references to prior work on code translation, structured synthesis, and automated software engineering.

  • A partial metrics row lists 99.0%, 101/113, 95.7%, and 123 min.The supplied text does not identify the corresponding metric labels for these values.
  • Prior work cited here includes F2PY, a tool for connecting Fortran and Python programs.
  • The references include research on unsupervised programming-language translation and specification-driven code translation with large language models.
  • Additional cited work covers code translation with compiler representations, automated software engineering agents, and repository-level semantic code generation.
Loading 2608.23619v1…