ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback

1 Introduction

• Restricted APIs for leading LLMs create barriers to new research, motivating translation capabilities built on open-source models.The introduction also describes LLMs' ability to move among translation-related tasks through natural-language instructions.Source: §1 · pp. 1–2
• ParroT reformulates human-written translation and feedback data as instructions and adds an optional Hint field for extra translation requirements.Its three instruction types are translation, contrastive, and error-guided instructions.Source: §1 · p. 2
• Translation instructions significantly improve vanilla LLM translation, especially for English-to-other-language directions, and error-guided instructions can further improve performance.The authors connect the latter result to learning from low-quality translations annotated by humans.Source: §1 · p. 2
• LoRA can prevent overfitting and improve dominant-language performance, but may slow learning for other languages.Source: §1 · p. 2
• The authors demonstrate the potential of COMET to provide translation-quality information for constructing error-guided instructions where human annotations are unavailable.Source: §1 · p. 2

2 Instruction Pool

• The paper presents translation instruction as the basis for translation ability and the other two instruction types as ways to align LLMs with human-written translation and feedback.Source: §2 · p. 2

2.1 Translation Instruction

• Translation instruction converts bilingual sentence pairs into an instruction-following format to establish basic translation ability.Source: §2.1 · p. 2, §2 · p. 2
• The translation-instruction template uses a fixed preface, task instruction, source input, and target response, with human-written translations chosen for pair quality.Source: §2.1 · p. 2

2.2 Contrastive Instruction

• Contrastive instruction uses multiple system translations and human evaluation scores to represent their relative quality.Source: §2.2 · p. 2
• Its response places the higher-scoring translation first and links it to a lower-quality alternative, while the Hint identifies the preferred translation.The second translation serves as a negative sample.Source: §2.2 · p. 3

2.3 Error-Guided Instruction

• Contrastive instruction conveys which of two translations is better but does not identify the errors responsible for the quality difference.Source: §2.3 · p. 3
• Error-guided instruction marks errors in a translation and uses the Hint to specify error types, drawing on human evaluation annotations.Source: §2.3 · p. 3

3.1 Training Data

• Training data combines Alpaca general instructions, WMT validation sentence pairs, and MQM human evaluation data for selected language directions.The WMT data covers Chinese-English and German-English; MQM data is used for English-to-German and Chinese-to-English.Source: §3.1 · p. 4
• For directions without MQM annotations, the authors use COMET scores to assign rough error levels and construct contrastive and error-guided instructions.They report 24K contrastive and 29K error-guided instructions from this automatically assessed data.Source: §3.1 · p. 4

3.2 Model Training

• Experiments compare Alpaca, translation-only ParroT-T, ParroT trained with all three instruction types, and ParroT-LoRA, across LLaMA and BLOOM model sizes.The LLaMA-7b ParroT-LoRA variant has 4.2M tunable parameters.Source: §3.2 · p. 4

3.3 Evaluation

• Evaluation uses a Flores subset and WMT22 test sets, with BLEU and COMET as automatic translation metrics.BLOOM models are evaluated only on WMT22 because Flores was used in BLOOMZ development.Source: §3.3 · pp. 4–5

4.1 Ablation Study

• Prompt-input is marginally better than prompt-no-input, TP1 beats TP3 on Alpaca, and beam search generally outperforms sampling, especially on BLEU.The authors adopt prompt-input, TP1, and beam search as the default inference setting.Source: §4.1 · p. 5

4.2 Main Results

• Translation instructions significantly improve vanilla LLaMA-7b, while Alpaca tuning fixes incomplete generations and small translation additions further boost performance, especially for En⇒Zh.The paper reports that Alpaca’s translation gains are mainly on high-resource directions, attributed to its English-dominant data.Source: §4.2 · pp. 5–6
• Error-guided instructions significantly improve ParroT across translation directions and test sets when it is asked to generate translations with no error.The authors speculate that the model learns the relationship between errors and translations, helping it avoid errors when given a no-error hint.Source: §4.2 · pp. 6–7
• LoRA helps Alpaca overfit less, but ParroT-LoRA beats full ParroT on De⇒En and performs worse on the other directions.The authors suggest the small tunable parameter set may hinder learning across translation directions and that LoRA hyperparameters need adjustment.Source: §4.2 · p. 7
• Larger LLaMA and BLOOM models perform better after instruction tuning; BLOOMZ-7b1-mt outperforms LLaMA-7b on Zh⇒En and En⇒Zh.The authors attribute the latter comparison mainly to BLOOM’s better Chinese modeling during pretraining.Source: §4.2 · p. 7
• COMET-based error-guided data yields considerable improvements on De⇒En and En⇒Zh, directions lacking MQM annotations.The authors present automatic evaluation as a potential source of translation-quality information for constructing these instructions.Source: §4.2 · p. 7

4.3 Analysis

• Translation quality worsens from no-error to minor-error to major-error hints, while no-hint outputs are usually comparable to the minor-error level.The no-error hint makes ParroT more likely to generate high-quality translations, but does not guarantee fully correct translations, especially for named entities.Source: §4.3 · p. 8
• Contrastive instructions produce lexical differences, but unpreferred translations score higher on BLEU and the pattern differs for COMET.The authors suggest that subtle quality differences between competitive WMT systems may be difficult for the LLM to learn effectively.Source: §4.3 · p. 8

5 Related Work

• ParroT targets instant chatbot translation without few-shot examples, unlike prior work on prompting for machine translation.Prior studies explored prompt templates, few-shot example selection, domain adaptation, and rare-word translation.Source: §5 · p. 8
• Instruction tuning has been used to make LLMs zero-shot learners across NLP tasks, with projects including Alpaca, Vicuna, and WizardLM.The cited studies include Mishra et al. (2022) and Wei et al. (2022); the named projects include Alpaca and WizardLM.Source: §5 · p. 9
• Unlike general-NLP chatbot work, ParroT targets machine translation and converts human feedback data into instructions that improve translation performance.The passage distinguishes instructions built from parallel translation data from instructions derived from human feedback.Source: §5 · p. 9

6 Conclusion

• ParroT adapts open-source LLMs for chat translation using human-written translation and feedback data.The framework reformulates translation data into instruction-following format and adds a Hint field for extra translation requirements.Source: §6 · p. 9
• Three instruction types—translation, contrastive, and error-guided—are proposed to regulate the translation process.The Hint field incorporates extra requirements into the instruction format.Source: §6 · p. 9
• Experiments on Flores subsets and WMT22 suggest translation instructions improve vanilla LLM translation, while error-guided instructions can lead to further improvement.The authors identify learning from human-annotated low-quality translations as important and propose exploring other hints, such as entity alignments.Source: §6 · p. 9

Limitations

• The paper characterizes its study of chatbot instant translation as a preliminary exploration.It lists instruction variants, contrastive translations, and LoRA effectiveness as areas for further improvement.Source: Limitations · p. 9
• Current instructions translate incoming sentences, while translating earlier chat records remains a possible extension.The proposed extension addresses users who struggle to understand responses in foreign languages.Source: Limitations · p. 9
• The study observed no performance improvement from contrastive instructions, possibly because of incorrect instruction formatting.The authors suggest exploring alternative formats, such as automatic post-editing, as a possible way to use contrastive translations.Source: Limitations · p. 9
• LoRA did not show consistent performance improvements over full model training in the current analysis.The authors suggest that the number of tunable parameters may need adjustment according to dataset size.Source: Limitations · p. 9

A Flores Full Sets

• The Flores subset was adopted to compare directly with ChatGPT and commercial MT systems from a previous evaluation report.The comparison is described in Section 3.3.Source: A Flores Full Sets · p. 11
• On full Flores sets of 1,012 sentences, the performance trend across systems still holds, and ParroT + Infer w/ No Err performs best.The full-set evaluation is presented as a more convincing evaluation than the subset comparison.Source: A Flores Full Sets · p. 11
• The full-set evaluation uses 1,012 Flores sentences.The results are listed in Table 8.Source: A Flores Full Sets · p. 11

• ParroT finetuning includes the Alpaca multitask dataset to preserve the model’s ability on general tasks.Source: B General Tasks · p. 11
• The examples show correct responses for question answering and code generation, though their wording differs.Source: B General Tasks · p. 11, Table 9 · p. 12
• A quantitative evaluation of general-task performance is deferred to future work.Source: B General Tasks · p. 11