Source-linked AI summary
Unsupervised Translation of Programming Languages
Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot, Guillaume Lample
TL;DR
Existing transcompilers depend on expert-built rewrite rules, while neural translation is constrained by scarce parallel programming-code data. TransCoder trains an unsupervised neural transcompiler from monolingual code and outperforms rule-based commercial baselines, though selecting the best beam hypothesis remains limited by compilation errors.
Problem
Neural machine translation for programming languages is limited by the scarcity of available parallel resources, despite the difficulty and expertise required by rule-based transcompilation.
Method
TransCoder uses unsupervised machine translation to translate functions among C++, Java, and Python using monolingual source code and a language-selection decoder token.
Results
TransCoder significantly outperforms rule-based baselines, reaching 74.8% computational accuracy for C++ →Java and 68.7% for Java →Python, versus 61% and 38.3%.
Takeaways & Limitations
A fully unsupervised method can translate complex language-specific patterns and outperform commercial systems without requiring source- or target-language expertise.
Takeaways & Limitations
Beam N - Top 1 selection is limited by compilation errors in Java and C++ outputs, and the authors leave improving it to future work.
Abstract
from arXiv · showhide
A transcompiler, also known as source-to-source translator, is a system that converts source code from a high-level programming language (such as C++ or Python) to another. Transcompilers are primarily used for interoperability, and to port codebases written in an obsolete or deprecated language (e.g. COBOL, Python 2) to a modern one. They typically rely on handcrafted rewrite rules, applied to the source code abstract syntax tree. Unfortunately, the resulting translations often lack readability, fail to respect the target language conventions, and require manual modifications in order to work properly. The overall translation process is timeconsuming and requires expertise in both the source and target languages, making code-translation projects expensive. Although neural models significantly outperform their rule-based counterparts in the context of natural language translation, their applications to transcompilation have been limited due to the scarcity of parallel data in this domain. In this paper, we propose to leverage recent approaches in unsupervised machine translation to train a fully unsupervised neural transcompiler. We train our model on source code from open source GitHub projects, and show that it can translate functions between C++, Java, and Python with high accuracy. Our method relies exclusively on monolingual source code, requires no expertise in the source or target languages, and can easily be generalized to other programming languages. We also build and release a test set composed of 852 parallel functions, along with unit tests to check the correctness of translations. We show that our model outperforms rule-based commercial baselines by a significant margin.
1 Introduction
Transcompilation is costly and difficult because languages differ in syntax, APIs, libraries, and typing, while existing tools rely largely on expert-built rewrite rules. TransCoder addresses limited parallel data by translating C++, Java, and Python from monolingual code and reports strong performance against commercial systems.
- Transcompilers translate between languages at similar abstraction levels, supporting platform migration and interoperability.
- $750 million and 5 years were spent converting the Commonwealth Bank of Australia’s platform from COBOL to Java.The example illustrates the cost and duration of migrating an existing codebase.
- Rule-based transcompilers require handcrafted AST rewrite rules, substantial development time, and expertise in both programming languages.Dynamic-to-static translation additionally requires variable-type inference, which is difficult and sometimes impossible.
- TransCoder trains on monolingual source code to translate functions among C++, Java, and Python without parallel training data.The approach uses open-source GitHub code and is intended to reduce required translation work and expertise.
- TransCoder grasps language-specific patterns and fully unsupervised translation outperforms commercial rule-based systems.The paper also releases 852 parallel functions in three languages with unit tests for correctness evaluation.
- The authors will make the code and pretrained models publicly available.
2 Related work
Prior source-code translation work generally depends on supervised parallel data, existing transcompilers, or programmer-created annotations. This paper’s related work also highlights limitations of BLEU and syntactic validity in evaluating generated code.
- Source-to-source translation: Earlier programming-language translation systems trained supervised models on parallel corpora mined from multilingual projects or created through existing transcompilers.
- Source-to-source translation: BLEU is not reliable for code translation because valid generations can differ substantially from reference translations.
- Translating from source code: Several studies translated source code into pseudo-code or docstrings using programmer-written annotations and GitHub repositories.
- Other applications: Neural code-generation systems commonly lack guarantees that generated functions compile or are syntactically correct.
- Unsupervised Machine Translation: Unsupervised machine translation methods address settings where parallel resources are rare or nonexistent by training from monolingual corpora.
3 Model
TransCoder is a shared seq2seq Transformer trained across programming languages with cross-lingual pretraining, denoising auto-encoding, and back-translation. These components align similar code, promote valid and noise-robust generation, and create training pairs from monolingual data.
- Model: TransCoder uses one attention-based seq2seq Transformer model shared across programming languages.Training follows initialization, language modeling, and back-translation principles from unsupervised machine translation.
- Cross Programming Language Model pretraining: Cross-lingual masked language-model pretraining maps sequences with similar meaning to shared latent representations across languages.Programming-language anchor points include common keywords, digits, mathematical operators, and English strings.
- Cross Programming Language Model pretraining: Masked language modeling randomly masks source-code tokens and predicts them from context while alternating batches from different languages.This produces cross-lingual sequence representations.
- Cross Programming Language Model pretraining: The model initializes the encoder and decoder from XLM, while decoder source-attention parameters are initialized randomly.
- Denoising auto-encoding: The decoder receives a special token indicating the desired output programming language.For example, a Python input can be decoded with the C++ start symbol to generate C++.
- Denoising auto-encoding: Denoising auto-encoding trains valid function generation from noisy encoder outputs and improves encoder robustness to noisy inputs.This robustness supports back-translation training.
- Back-translation: Back-translation couples opposite-direction models so each generates noisy source sequences that supervise reconstruction in the other direction.
4 Experiments
TransCoder is trained on monolingual C++, Java, and Python code and evaluated on parallel functions using unit tests and multiple translation metrics. Beam search and computational-accuracy evaluation reveal valid translations that reference-based metrics can miss, while TransCoder outperforms the rule-based baselines supplied here.
- Training data and setup: TransCoder uses a single multilingual model trained with denoising auto-encoding and back-translation on C++, Java, and Python source code.The model uses a shared encoder-decoder and alternates between the two objectives during training.
- Evaluation data and metrics: The evaluation uses 852 parallel functions from C++, Java, and Python, with unit tests checking whether generated functions produce correct outputs.The functions are extracted from multilingual solutions and selected to compute the same result with similar algorithms.
- Qualitative analysis: Examples show TransCoder inferring types, mapping containers and methods across languages, translating control flow, and aligning libraries, but it can mishandle type-dependent operators.One reported failure copies the Java NOT operator for an integer where the bitwise operator should be used.
- Decoding: Beam search improves computational accuracy by up to 33.7% in Java →Python, but highest-log-probability selection performs worse than considering valid hypotheses in the beam.The model often includes a correct translation among its hypotheses without assigning it the highest probability.
- Evaluation data and metrics: 3.1% of C++ →Java translations exactly match references, while 60.9% pass unit tests, showing that reference match and BLEU poorly reflect computational accuracy.The results motivate computational accuracy as an evaluation measure for translations that differ syntactically from the reference but remain valid.
- Comparison with baselines: TransCoder reaches 74.8% computational accuracy for C++ →Java and 68.7% for Java →Python, versus 61% and 38.3% for the respective rule-based baselines.The baselines use manually constructed rewrite rules and expert programming knowledge.
5 Conclusion
The paper demonstrates that unsupervised machine-translation methods can create a fully unsupervised source-code transcompiler. TransCoder generalizes across programming languages and outperforms commercial solutions, while several decoder and compiler-based improvements remain possible.
- TransCoder applies unsupervised machine translation to source code without parallel training data.
- The approach can be generalized to any programming language and does not require expert knowledge.
- TransCoder outperforms commercial solutions by a large margin.
- Simple decoder constraints, dedicated architectures, compiler output, or iterative error correction could improve performance further.
Broader Impact
Automatic transcompilation could help programmers integrate code across teams and projects and reduce the cost of modernizing obsolete codebases.
- Automatic transcompilation could make programmers more efficient when integrating code from other teams or open source projects.
- It could lower the cost of updating obsolete-language codebases to more recent languages.
- Many large banks, insurance companies, and utilities still run code written in COBOL.
A.1 Training dataset
The training dataset combines function-level code from multiple languages with comments retained after an empirical comparison, while dataset statistics cover the full GitHub corpus and extracted functions.
- The final training data retains comments because keeping them produced better overall results.
- Table 3 reports size in gigabytes, numbers of files and functions, and token counts for the full GitHub dataset and extracted functions.
- The Python tokenizer is robust to extra spaces and new lines except inside strings, while indentation is represented with INDENT and DEDENT tokens.
- The model is trained and evaluated on functions, with standalone functions used for reported results after yielding better results than training on all functions.
B Evaluation
Evaluation uses parallel functions and unit tests to assess semantic equivalence, with carefully generated inputs and manual checks addressing edge cases in the test scripts.
- The evaluation corpus gathers multilingual GeeksforGeeks solutions and generates unit tests to assess semantic correctness and Computational Accuracy.
- Each function has a script containing a reference function, a replacement marker, and a main routine comparing both functions across inputs.
- 10 random inputs are generated from Java parameter types, and scripts are retained only after perfect reference-function agreement in under 10 seconds.
- Equality checks vary by output type, while in-place functions are evaluated through side effects on mutable arguments.
- Random input generation alternates among character and integer sets to make tests discriminative for representations such as integers and binary values.
- Scripts are manually checked for constant outputs, in-place behavior, and printing, and functions mainly printing or writing files are removed.
- Figure 4 illustrates parallel C++, Java, and Python implementations of checkDivisibility, which tests divisibility by 13 for a string-represented long integer.
- Table 4 reports language-specific validation and test-set sizes, average tokens per function, and ten unit tests per function.
C.1 Detailed results
Greedy decoding failures commonly arise from compilation errors in Java and C++ targets, runtime or syntax errors in Python, and incorrect outputs.
- Compilation errors account for many greedy-decoding failures when the target language is Java or C++.The authors suggest constraining the decoder to generate compilable code.
- Runtime errors mainly occur when translating from Java or C++ into Python, including Python syntax errors because Python is interpreted.
- Most remaining errors result from programs returning the wrong output on evaluation tests.
C.2 Ablation study
The ablation and translation examples examine comments, cross-lingual representations, type and library conversion, decoding strategies, and failure modes across C++, Java, and Python.
- Training data ablation study: Keeping comments improves computational accuracy for C++→Python, Java→C++, and Java→Python, but has no significant impact in the other directions.The comparison covers different beam sizes and training sets with comments either retained or removed.
- Cross-lingual token embeddings: Cross-lingual token embeddings place keywords used in similar contexts close together across C++, Java, and Python.The visualization includes keywords such as exception-catching constructs.
- Correct translations: TransCoder translates primitive types, equivalent data structures, methods, and libraries, including C++ unordered_map into Java HashMap with get and put methods.
- Correct translations: For Python-related translations, TransCoder infers types such as list→std::vector and preserves calls to non-standard functions and global variables.
- Robustness to variable names: TransCoder remains correct after parameter renaming and uses inferred variable types to translate char* parameters into Java String with charAt.
- Decoding strategy: Beam search produces accurate C++→Python translations where greedy decoding fails, correcting issues such as integer-versus-double division.
- Baseline comparison: Rule-based baselines copy unsupported Java library functions and List methods, whereas TransCoder converts them properly.
- Failure cases: TransCoder can fail when translating C++ or Python into Java because it does not account for variable types in methods or operators.Examples include translating integer NOT incorrectly and applying Math.min to arrays.