AVATAR: A Parallel Corpus for Java-Python Program Translation

1 Introduction

• Manual cross-language rewriting is time-consuming, expensive, and requires expertise in both source and target languages.Source: §1 · p. 1
• Conventional transcompilers use ASTs and handwritten rules, and their development and adaptation require advanced knowledge, limiting their availability across languages.Source: §1 · p. 1
• Prior NMT and fine-tuned code language models motivate a supervised Java–Python parallel corpus.The cited work reports success with unsupervised translation and fine-tuning on small parallel examples.Source: §1 · p. 1
• AVATAR contains solutions for 9,515 programming problems and unit tests for 250 examples to evaluate functional correctness.The problems were collected from competitive programming sites, online platforms, and open-source repositories.Source: §1 · p. 1
• AVATAR provides 3,391 parallel functions for model training or fine-tuning and function-translation evaluation on Lachaux et al.'s dataset.Source: §1 · p. 2

2 AVATAR Construction

• The collected solutions include examples from competitive programming sites, online platforms, and open-source repositories, including AtCoder, AIZU, Google Code Jam, LeetCode, and Project Euler.Codeforces and GeeksforGeeks are also named among the collection sites.Source: §2 · p. 2
• Preprocessing removes comments and docstrings, filters solutions longer than 464 tokens, and keeps at most five solutions per problem.The retained solutions are selected for maximal difference from other solutions of the same problem.Source: §2 · p. 2
• The 9,515 problem statements are split into 7,133 training, 476 validation, and 1,906 test examples in a 75:5:20 ratio.Training pairs Java and Python solutions; validation and test use multiple ground-truth solutions and select the best metric performance.Source: §2 · p. 2
• Unit tests from AtCoder are provided for 250 of the 1,906 evaluation examples to assess functional accuracy.Source: §2 · p. 2
• AVATAR contains 3,391 parallel Java–Python functions, extracted from programs containing only one function, for training and evaluation.Function evaluation uses the dataset released by Lachaux et al. (2020).Source: §2 · p. 2

3.3 Results

• Match-based performance is strong, but computational accuracy remains far from perfect; PLBART has the highest execution and computational accuracy for Java-to-Python program translation.The table reports test-set results for Java-Python program translation.Source: Table 2 · p. 3, §3.3 · p. 4

3.1 Evaluation Metrics

• BLEU measures candidate-reference overlap, while Syntax Match, Dataflow Match, CodeBLEU, Execution Accuracy, and Computational Accuracy assess structural, flow, combined, executable, and output-equivalence properties.Computational Accuracy checks whether candidate and reference translations produce the same outputs for the same inputs.Source: §3.1 · pp. 2–3

3.2 Models

• The comparison includes zero-shot translation systems, models trained from scratch, and pre-trained models spanning decoder-only, encoder-only, and Seq2Seq architectures.The scratch models include Seq2Seq+Attn. and Transformer; the pre-trained group includes CodeGPT, CodeBERT, GraphCodeBERT, PLBART, and CodeT5.Source: §3.2 · p. 3
• TransCoder-ST is also fine-tuned because it is identified as the best translation model in the literature.Source: §3.2 · p. 3

3.3 Results

• For function translation, all models except those trained from scratch, CodeBERT, and GraphCodeBERT perform well across match-based, execution, and computational accuracy; fine-tuned TransCoder-ST leads, with PLBART closest.The table reports test-set results for Java-Python function translation.Source: §3.3 · p. 4, Table 3 · p. 4

4 Related Works

• Earlier Java-C# corpora mined similar classes or methods from open-source projects, while later work used libraries and transcompilers to construct parallel data.Examples include Nguyen et al. (2013), Karaivanov et al. (2014), Aggarwal et al. (2015), and Chen et al. (2018).Source: §4 · p. 4
• Unlike those efforts, AVATAR collects Java-Python programming-problem solutions from competitive-programming sites, online platforms, and open-source repositories.Source: §4 · p. 4

5 Conclusion

• Future work aims to enlarge the parallel corpus and add support for more programming languages.Source: §5 · p. 4

Limitations

• The training set is relatively small, limiting training deep neural models from scratch.Source: Limitations · p. 5
• The corpus covers only two languages and mostly data structures and algorithms; its utility for software projects with API dependencies remains unknown.Source: Limitations · p. 5
• Large language models were not evaluated because of limited computational resources, so AVATAR's value for them is unknown.Source: Limitations · p. 5

Ethics Statement

• AVATAR users must adhere to the licenses covering its examples, which vary by source.LeetCode examples are under MIT, AtCoder and AIZU examples under Apache-2.0, and GeeksforGeeks and Codeforces examples are released under CC BY-NC-SA 4.0; Project Euler and Code Jam examples lack license information.Source: Ethics Statement · p. 5
• Computational limits led the authors to avoid fine-tuning large models; nine models’ training jobs took 1–2 days on RTX 2080 Ti GPUs.They estimate 100 hours on one RTX 2080 Ti GPU produces approximately 7.5 kg of carbon emissions.Source: Ethics Statement · p. 5
• The authors remove natural-language comments and docstrings to address personally identifiable or offensive content, but do not manually check every example for such content in strings.The authors say string content may still contain such material.Source: Ethics Statement · p. 5

A Hyperparameters Details

• Models are fine-tuned separately for Java-to-Python and Python-to-Java program and function translation.The directional setup is described across the continuation of the appendix passage.Source: A Hyperparameters Details · p. 7
• Fine-tuning runs for at most 20 epochs with Adam, batch size 32, and learning rates selected from [1e −4, 5e −5, 3e −5, 1e −5].The final models are selected based on validation BLEU score.Source: A Hyperparameters Details · p. 7
• Inference uses beam decoding with beam size 10 across all models.Source: A Hyperparameters Details · p. 7

B Analysis

• The program-translation evaluation uses 250 AVATAR examples with unit tests, while function translation uses test examples from Lachaux et al. (2020).Source: B Analysis · p. 7
• For function translation, 464 Java-to-Python and 482 Python-to-Java examples have test cases; Table 4 reports success, failure, and error counts, and Table 5 breaks down compilation and runtime errors.Source: B Analysis · p. 7
• PLBART often omits Java imports, causing symbol-resolution failures; type mismatch is a primary compilation-error cause across models, and models also fail on longer programs.Examples of missing imports include StringTokenizer and BufferedReader.Source: B Analysis · p. 7

• PLBART translates Java’s Math.pow() to Python’s pow() and turns a Java class containing a function into a Python function.Source: C Qualitative Examples · p. 7
• In the Python-to-Java example, PLBART fails to translate correctly and adds an unnecessary InputReader class using BufferedReader.Source: C Qualitative Examples · p. 7
• Function translations are described as generally good, although some do not pass all unit tests.Source: C Qualitative Examples · p. 7
• For Python-to-Java translation, PLBART generates a class named Main or GFG, which the authors attribute presumably to generic class names in programming solutions and GeeksforGeeks examples.Source: C Qualitative Examples · p. 7