PaperlayerView exact revision on arXiv

Source-linked AI summary

BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models

Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, Yang Feng

arXiv:2306.10968v2cs.CLcs.AI

TL;DR

English-centric training and instructions leave LLMs weaker in non-English languages, while collecting language-specific data is costly. BayLing uses interactive translation to transfer generation and instruction-following capabilities, and its 13B model performs comparably to GPT-3.5-turbo.

  • Problem

    English-dominated training data and English instructions are associated with inferior non-English LLM performance, while collecting language-specific data and instructions is difficult.

    Source: §1 · p. 2

  • Method

    BayLing uses automatically constructed, multi-turn interactive-translation instructions to transfer generation and instruction-following capabilities from English to other languages.

    Source: §1 · p. 2

  • Results

    Across translation, general tasks, and standardized tests, 13B BayLing performs comparably to GPT-3.5-turbo; on BayLing-80 it reaches 89% of GPT-3.5-turbo performance.

    Source: §1 · p. 2

  • Takeaways & Limitations

    BayLing offers a viable approach for transferring English-corpus foundation-model proficiency to other languages while enhancing multiple capabilities through composite-task instruction tuning.

    Source: §5 · p. 14

    BayLing-13B still has room to improve against ChatGPT on instructions with abstract meanings, including creative requirements and human suggestions.

    Source: §4.2.2 · p. 11

Abstract

from arXiv · show

Large language models (LLMs) have demonstrated remarkable prowess in language understanding and generation. Advancing from foundation LLMs to instructionfollowing LLMs, instruction tuning plays a vital role in aligning LLMs to human preferences. However, the existing LLMs are usually focused on English, leading to inferior performance in non-English languages. In order to improve the performance for non-English languages, it is necessary to collect language-specific training data for foundation LLMs and construct language-specific instructions for instruction tuning, both of which are heavy loads. To minimize human workload, we propose to transfer the capabilities of language generation and instruction following from English to other languages through an interactive translation task. We have developed BayLing, an instruction-following LLM by utilizing LLaMA as the foundation LLM and automatically constructing interactive translation instructions for instructing tuning. Extensive assessments demonstrate that BayLing achieves comparable performance to GPT-3.5-turbo, despite utilizing a considerably smaller parameter size of only 13 billion. Experimental results on translation tasks show that BayLing achieves 95% of single-turn translation capability compared to GPT-4 with automatic evaluation and 96% of interactive translation capability compared to GPT-3.5-turbo with human evaluation. To estimate the performance on general tasks, we created a multi-turn instruction test set called BayLing-80. The experimental results on BayLing-80 indicate that BayLing achieves 89% of performance compared to GPT-3.5-turbo. BayLing also demonstrates outstanding performance on knowledge assessment of Chinese GaoKao and English SAT, second only to GPT-3.5-turbo among a multitude of instruction-following LLMs. Demo, homepage, code and models of BayLing are available.

1 Introduction

English-centric training leaves LLMs weaker in non-English languages, while collecting language-specific training data and instructions is costly. BayLing uses interactive translation to transfer capabilities from English and reports performance comparable to GPT-3.5-turbo.

1 Introduction

  • English-dominated corpora and English instructions contribute to inferior non-English performance, while collecting language-specific data and instructions is difficult.Both generation training data and instructions across NLP tasks are identified as costly needs.Source: §1 · p. 2
  • Interactive translation is proposed to transfer generation and instruction-following capabilities from English, using automatically constructed instructions without manual construction.The authors argue that cross-lingual semantic alignment requires less data than acquiring generation capabilities, and that multi-turn translation reflects human preferences.Source: §1 · p. 2
  • BayLing-13B performs comparably to GPT-3.5-turbo across evaluated tasks and ranks second only to it on Chinese GaoKao and English SAT.The evaluation covers translation, general tasks, and standardized tests.Source: §1 · p. 2
  • BayLing reaches 95% of GPT-4 single-turn translation capability, 96% of GPT-3.5-turbo interactive translation capability, and 89% of GPT-3.5-turbo BayLing-80 performance.The first figure is from automatic evaluation, the second from human evaluation, and the third from GPT-4 evaluation of BayLing-80.Source: §1 · p. 2
  • Interactive-translation instructions can improve non-English generation, instruction following, translation, and general tasks, while math, coding, and reasoning remain areas for improvement.The reported generation improvement does not negatively affect English generation capabilities.Source: §1 · p. 2

2 Related Work

Prior work develops foundation LLMs through pretraining and aligns them with human instructions and preferences through instruction tuning and RLHF. BayLing targets cross-lingual alignment and multi-turn interaction through interactive translation.

2 Related Work

  • Foundation LLMs gain language understanding and generation from large-corpus pretraining, while instruction tuning and RLHF support instruction-following and preference alignment.ChatGPT and GPT-4 are cited as instruction-following examples.Source: §2 · p. 3
  • Because RLHF carries substantial labor costs, researchers have also pursued alignment through high-quality instruction tuning.The cited work reports noteworthy advances through this approach.Source: §2 · p. 3
  • BayLing addresses gaps in cross-lingual alignment and multi-turn interaction by using interactive translation to improve language alignment, instruction following, and interaction together.The study focuses on transferring English-learned capabilities to other languages through instruction tuning.Source: §2 · p. 3

3 Construction of BayLing

BayLing builds on LLaMA and fine-tunes it with interactive-translation and general-task instructions. Its construction uses multi-turn translation to combine cross-lingual alignment with instruction following and contextual comprehension.

3 Construction of BayLing

  • BayLing is built on LLaMA and fine-tuned with interactive-translation instructions together with general-task instructions.The interactive-translation task is intended to transfer language-generation capabilities across languages while following human feedback through multi-turn interaction.Source: §3 · p. 3

3.1 Foundation Model

  • BayLing uses LLaMA-7B and LLaMA-13B as the foundation models for BayLing-7B and BayLing-13B, respectively.LLaMA is described as trained on approximately 1.4 trillion English-dominant tokens.Source: §3.1 · pp. 3–4

3.2 Alignment with Interactive Translation Task

  • Interactive translation requires understanding user intent and generating translations accordingly; its multi-turn, human-centered interaction is intended to strengthen alignment and instruction following.The authors also argue that capabilities learned through the task can transfer to other tasks.Source: §3.2 · p. 4
  • The instruction dataset contains 160K interactive-translation instances covering vocabulary, grammar, style, and creation, supplemented with Alpaca single-turn and ShareGPT multi-turn data.Table 1 is identified as presenting statistics for the instruction data.Source: §3.2 · p. 4, Table 1 · p. 4

3.3 Training Details

  • BayLing-7B and BayLing-13B are fine-tuned on the instruction data for three epochs, with loss computed only on output parts in multi-turn interactions.Training uses a 2e-5 learning rate, batch size 32, and maximum text length 1024; 8-bit quantization enables BayLing-13B inference on a 16G GPU.Source: §3.3 · p. 5

4 Experiments

The experiments assess BayLing on seen and unseen translation directions, instruction-language robustness, and lexically constrained translation. Across these settings, results report strong translation performance, including comparisons with GPT models and other open-source systems.

4.1.1 Experimental Setup

  • BayLing is evaluated on multilingual translation, interactive translation, general tasks, and standardized tests; WMT22 covers Chinese⇔English and German⇔English, while additional zero-shot pairs are unseen during instruction tuning and assessed with SacreBLEU and COMET.BayLing-7B and BayLing-13B are compared with translation-specific models and general instruction-following LLMs.Source: §4 · p. 5, §4.1.1 · p. 5

4.1.2 Single-turn Translation

  • BayLing-13B achieved 94% and 97% of GPT-4 COMET capability on Chinese→English and English→Chinese, and 97% and 95% on German→English and English→German.Overall, BayLing achieved state-of-the-art translation performance among open-source LLMs.Source: Figure 3 · p. 6, §4.1.2 · p. 6, Figure 4 · p. 7

4.1.3 Zero-Shot Translation

  • On unseen directions, BayLing averaged above Vicuna-13B, reaching 93% of GPT-3.5-turbo for X⇒English versus 82% for English⇒X; comparator zero-shot status is uncertain.The authors interpret the difference as multilingual understanding transferring more readily than multilingual generation.Source: Figure 5 · p. 7, §4.1.3 · p. 7

4.1.4 Robustness in Instruction Language

  • BayLing and GPT-3.5-turbo achieved similar translation quality with English or Chinese instructions, unlike ChatGLM-6B and Vicuna-13B, which were instruction-language sensitive.ChatGLM-6B performed better when the instruction matched the target language; Vicuna-13B preferred a match with the source language.Source: Figure 6 · p. 8, §4.1.4 · p. 8

4.1.5 Lexically Constrained Translation

  • BayLing produced constraint-compliant translations in 55% of cases while maintaining translation quality; GPT-3.5-turbo and Vicuna-13B sometimes replicated constraints, reducing English-to-Chinese quality.The task requires specified words or phrases in the translation.Source: §4.1.5 · p. 8, Figure 7 · p. 9

4.2 Interactive Translation

The evaluation covers human-rated interactive translation and GPT-4-rated general tasks, including English and Chinese single- and multi-turn instructions. BayLing-13B is competitive with ChatGPT and GPT-3.5-turbo overall, with weaker results on some abstract or reasoning-intensive tasks.

4.2.1 Experimental Setup

  • Human evaluation used 60 source sentences, five systems, four interaction turns, and five categories, with annotators scoring and ranking translation, instruction following, and multi-turn interaction.The 60 cases included 30 Chinese and 30 English sentences; each interaction category comprised 12 cases.Source: §4.2.1 · pp. 9–10

4.2.2 Results from Human Evaluation

  • Against ChatGPT, BayLing-13B receives the highest human rating in 18% of translation cases, 30% of instruction-following cases, and 20% of multi-turn interaction cases, ranking second only to ChatGPT.The evaluation reports the proportion of cases where each system achieved Rank #1.Source: Figure 8 · p. 10, §4.2.2 · p. 10
  • On specific interaction types, BayLing-13B is comparable to ChatGPT for vocabulary, grammar, and style instructions, while creative requirements and suggestions remain areas for improvement.Figure 9 presents average scores across vocabulary, grammar, style, suggestion, and creation categories.Source: Figure 9 · p. 10, §4.2.2 · pp. 10–11

4.3.2 Results from GPT-4 Evaluation

  • BayLing-13B outperforms ChatGLM-6B in 80% of English-instruction cases and 50% of Chinese-instruction cases, and shows more pronounced advantages over other systems on multi-turn instructions.The GPT-4 comparison also reports advantages over Alpaca-7B in almost all cases and over Vicuna-13B, especially for Chinese instructions.Source: Figure 10 · p. 11, §4.3.2 · p. 12

4.3.1 BayLing-80 Test Set

  • The BayLing-80 test set extends 80 English single-turn instructions into English and Chinese single- and multi-turn sets, with second turns linked to the first by hidden references or logical relations.The test set is intended to evaluate multilingual and interactive capabilities on general tasks.Source: §4.3.1 · p. 11

4.3.2 Results from GPT-4 Evaluation

  • GPT-4 evaluated paired system responses on BayLing-80, scoring each response from 1 to 10 and selecting the higher-scoring system as the winner.The systems' responses were submitted in pairs for comparison.Source: §4.3.2 · pp. 11–12
  • Figure 11 compares BayLing-13B and GPT-3.5-turbo across nine task categories; BayLing-13B also needs improvement on Chinese counterfactual tasks.The paper attributes that scope for improvement to LLaMA's English-dominant pretraining.Source: Figure 11 · p. 12, §4.3.2 · p. 12
  • 89%: BayLing-13B achieves 89% performance of GPT-3.5-turbo across 80 general-task cases.It is comparable on generic, knowledge, common-sense, and writing tasks, but has a notable gap on math, coding, and Fermi tasks.Source: Table 2 · p. 12, §4.3.2 · p. 12

4.4 Standardized Tests

The standardized-test evaluation examines whether interactive-translation instruction tuning helps BayLing transfer English-stored knowledge into Chinese. BayLing-13B improves over Vicuna-13B in Chinese GaoKao results, while its relative performance varies by subject and comparator.

4.4 Standardized Tests

  • The paper presents interactive-translation instruction tuning as a way to align languages and transfer knowledge stored in English form into Chinese.It frames this as an efficient solution for enhancing Chinese ability without a separate Chinese knowledge pretraining stage.Source: §4.4 · p. 13

4.4.1 Experimental Setup

  • The evaluation uses AGIEval Chinese GaoKao and English tests, including SAT, LSAT, Civil Service Examination, GRE, and GMAT, in a zero-shot setting.Chinese GaoKao covers nine subjects; scores are percentages of correct answers.Source: §4.4.1 · p. 13

4.4.2 Results from Automatic Evaluation

  • BayLing-13B scores an average of 32.13 points on Chinese GaoKao and 35.31 points on the English examinations.The evaluations use zero-shot questions, with percentage correct as the score for each subject.Source: §4.4.2 · p. 13
  • Compared with Vicuna-13B, BayLing-13B improves by 3 points on average in Chinese GaoKao and by 8 points in the Chinese subject, while remaining comparable on English tests.Both systems are based on LLaMA; the paper reports BayLing-13B surpasses Vicuna-13B in most Chinese subjects.Source: §4.4.2 · p. 14
  • BayLing-13B outperforms ChatGLM-6B in physics, chemistry, and biology, but scores slightly lower in history and geography, where subject knowledge may be language-specific.The authors note that transfer cannot provide knowledge that was not initially present in LLaMA.Source: §4.4.2 · p. 14
  • The authors conclude that language alignment enables transfer of foundation-model English capabilities and knowledge into Chinese, improving Chinese proficiency and knowledge.They describe BayLing as a feasible solution for cross-language capability transfer.Source: §4.4.2 · p. 14

5 Conclusion

The conclusion presents BayLing as an instruction-following LLM with enhanced language alignment, instruction following, and multi-turn interaction. Across evaluations, its performance closely approaches GPT-3.5-turbo, and the authors identify cross-language transfer as a central contribution.

5 Conclusion

  • Across translation, interactive translation, general tasks, and standardized tests, BayLing performs exceptionally relative to existing open-source LLMs and closely approaches GPT-3.5-turbo.The authors highlight the potential of BayLing and the proposed instruction-tuning method.Source: §5 · p. 14
  • BayLing offers a viable way to transfer foundation models' proficiency and knowledge acquired from English corpora to other languages.The conclusion also identifies BayLing as a potential translation baseline and a foundation for follow-up LLMs.Source: §5 · p. 14
  • The paper presents composite-task instruction tuning as a way to enhance multiple LLM capabilities simultaneously, including language alignment and multi-turn interaction.It identifies BayLing as a candidate base for follow-up LLM development.Source: §5 · p. 14

A Translation Benchmarks

The appendix describes WMT22 as the basis for evaluating translation quality across Chinese-English and German-English domains and multilingual translation tasks. It reports that the resulting translations are released as potential research baselines.

A Translation Benchmarks

  • Translation quality is evaluated on the full WMT22 test sets for Chinese-English and German-English multi-domain benchmarks, as well as multilingual benchmarks.The paper says the translation outputs are released as open-source resources for future translation research.Source: A Translation Benchmarks · p. 20
  • Table 5 reports results on WMT22 Chinese-English and German-English translation tasks.Source: Table 5 · p. 20
  • Table 6 reports results on WMT22 multilingual translation tasks.Source: Table 6 · p. 21

B BayLing-80 Test Set

BayLing-80 is a Chinese-and-English multi-turn instruction test set augmented from Vicuna-80. Its constructed second turns test whether models understand context and adapt responses to added requirements.

B BayLing-80 Test Set

  • BayLing-80 combines the original 80 single-turn English instructions from Vicuna-80 with translated Chinese instructions and expanded multi-turn instructions in both languages.Its four components are the original English set, translated single-turn Chinese instructions, expanded multi-turn English instructions, and multi-turn Chinese instructions.Source: B BayLing-80 Test Set · p. 21
  • Constructed second turns are semantically related to the first while omitting references, testing context understanding in multi-turn interactions.The example’s second instruction adds a supplement or constraint to test whether the model adjusts its earlier response.Source: B BayLing-80 Test Set · p. 21
  • The added requirements test whether a model can revise its previous response to accommodate the user’s constraints.Source: B BayLing-80 Test Set · p. 21

C Templates of GPT-4 Evaluation

The evaluation templates compare two systems’ responses to the same user questions, with separate formats for single-turn and multi-turn interactions. GPT-4 scores the responses after the questions and corresponding answers are inserted.

C Templates of GPT-4 Evaluation

  • GPT-4 receives the questions and both systems’ corresponding responses in a template, then scores the two systems.Source: C Templates of GPT-4 Evaluation · p. 21
  • The appendix also points readers to the WMT22 translation results hosted in the project repository.Source: C Templates of GPT-4 Evaluation · p. 21
  • For single-turn evaluation, the template pairs one user question with one response from each of two systems.Q is the user question; R1 and R2 are the respective systems’ responses.Source: Figure 13 · p. 22
  • For multi-turn evaluation, the template pairs two user questions with each system’s responses to both turns.Q1 and Q2 are the questions; each system has a response corresponding to each turn.Source: Figure 14 · p. 23

D Cases on General Tasks

The section presents BayLing-80 response examples in English and Chinese, alongside GPT-4 reviews and scores. The listed cases span both languages.

D Cases on General Tasks

Loading 2306.10968v2…