Source-linked AI summary
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data
Canwen Xu, Daya Guo, Nan Duan, Julian McAuley
TL;DR
Restricted APIs and limited public multi-turn chat corpora impede research on capable chat models. The paper uses ChatGPT self-chat data and parameter-efficient LLaMA tuning to create Baize, further improved with Self-Distillation with Feedback; it reports good multi-turn dialogue performance with guardrails and releases the model and data for research.
Problem
Restricted access to capable chat models and limited availability of high-quality public multi-turn chat corpora hinder research and model refinement.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The released Baize models and data provide an accessible research resource for open-source multi-turn chat modeling.
Source: §1 · p. 2, §7 · p. 8Changing Baize’s inference prompt can potentially remove its guardrails, and releasing the weights creates risks of malicious use.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Chat models, such as ChatGPT, have shown impressive capabilities and have been rapidly adopted across numerous domains. However, these models are only accessible through a restricted API, creating barriers for new research and progress in the field. We propose a pipeline that can automatically generate a high-quality multi-turn chat corpus by leveraging ChatGPT to engage in a conversation with itself. Subsequently, we employ parameter-efficient tuning to enhance LLaMA, an open-source large language model. The resulting model, named Baize, demonstrates good performance in multi-turn dialogues with guardrails that minimize potential risks. Furthermore, we propose a new technique called Self-Distill with Feedback, to further improve the performance of the Baize models with feedback from ChatGPT. The Baize models and data are released for research purposes only at https://github.com/project-baize/baize-chatbot. An online demo is also available at https://huggingface.co/spaces/project-baize/chat-with-baize.
1 Introduction
The paper addresses restricted access to capable chat models and scarce public multi-turn corpora by generating self-chat data with ChatGPT and tuning LLaMA into Baize. It combines parameter-efficient tuning with Self-Distillation with Feedback to produce an accessible open-source chat model.
1 Introduction
- Restricted APIs and limited public high-quality multi-turn corpora constrain research and the refinement and evaluation of chat models.Source: §1 · p. 1
- ChatGPT can simulate both user and assistant roles to automatically generate a high-quality multi-turn corpus for training and evaluating chat models.A seed dataset can guide domain-specific sampling for areas such as healthcare or finance.Source: §1 · pp. 1–2
- Baize combines ChatGPT self-chat corpus generation, parameter-efficient LLaMA tuning, and Self-Distillation with Feedback for accessible multi-turn dialogue modeling.The model is designed to run on a single GPU.Source: §1 · pp. 1–2
- The pipeline is presented as a novel and reproducible public resource intended to fill the gap in multi-turn chat training data.Source: §1 · pp. 1–2
- Self-Distillation with Feedback is proposed alongside parameter-efficient tuning to enhance LLaMA in low-resource settings, producing the open-source Baize model.Source: §1 · pp. 1–2
2 Related Work
Prior chat models span open-domain systems, large pretrained dialogue models, and RLHF-optimized assistants, while newer efforts replicate ChatGPT with open-source foundations. Baize differs by emphasizing an affordable, reproducible parameter-efficient pipeline for multi-turn chat.
Language Models for Chat
- Earlier chat systems range from DialoGPT and Meena to LaMDA, ChatGPT, and GPT-4, using sources and training strategies including Reddit, public dialogue data, and RLHF.Source: §2 · p. 2
- Stanford Alpaca uses GPT-3.5-generated instruction data to fine-tune LLaMA, while Vicuna fine-tunes LLaMA on ChatGPT dialogues crawled from sharegpt.com.Source: §2 · p. 2
Parameter-Efficient Tuning
- Parameter-efficient methods reduce the cost of adapting large models through techniques such as adapters, bias tuning, sparse weights, prefix tuning, and LoRA.LoRA inserts tunable low-rank matrices into attention layers and is described as outperforming conventional GPT-3 fine-tuning.Source: §2 · p. 2
- Baize focuses on an affordable and reproducible way to efficiently tune a general-purpose language model for multi-turn chat.Source: §2 · p. 2
3 Data Collection via Self-Chat
The self-chat pipeline uses ChatGPT to generate multi-turn dialogues from seed questions, including domain-specific seeds, creating scalable training data while avoiding third-party sharing concerns.
Self-Chat Generation
- Self-chat has ChatGPT generate both user and assistant turns from a seed until the conversation naturally stops.The seed can be a question or key phrase that sets the topic.Source: §3 · p. 3
- 111.5k dialogues were collected for Baize v1 from approximately 55k Quora and Stack Overflow questions at an API cost of about $100.The corpus statistics include dialogue counts, average turns, and response lengths.Source: §3 · p. 3, Table 2 · p. 3
Domain-Specific Data
- 47k additional medical dialogues were generated from MedQuAD seeds to train a healthcare-specialized Baize model.Source: §3 · p. 3
Data Quality
- Generating responses one turn at a time produced longer, more detailed outputs for Baize v1.5 but increased API cost by repeatedly attaching context.The resulting corpus statistics are reported in Table 2.Source: §3 · p. 3, Table 2 · p. 3
Comparison with Other Data Sources
- Unlike user-shared ChatGPT dialogues, self-chat avoids third-party copyright concerns when the seed dataset has a proper license.The comparison identifies privacy and legal risks in data crawled from sharegpt.com.Source: §3 · p. 4
4 Model Training
Baize is trained with parameter-efficient LoRA adaptation of LLaMA and further improved through self-distillation with ChatGPT-ranked responses. SDF provides an alternative to RLHF that the paper reports as faster and free of reward-model training.
Parameter-Efficient Supervised Fine-Tuning
- LoRA adapts LLaMA by updating low-rank parameters while freezing the original model parameters.The paper applies LoRA to all linear layers and computes loss only on AI responses for Baize v1.5.Source: §4 · p. 4
Qualitative Study
- Table 4 shows Baize v2 producing a more detailed Lehman Brothers bankruptcy analysis than Baize v1, with a response similar to ChatGPT’s.The example is described as not cherry-picked, although some response details are omitted for space.Source: Table 4 · p. 5
Self-Distillation with Feedback
- SDF generates four Baize responses per instruction, asks ChatGPT to rank them, and fine-tunes new LoRA modules on the best response.The procedure produces Baize v2 from Baize v1.5.Source: §4 · p. 5
- SDF is reported as 3× faster than RLHF, does not require reward-model training, and distills Baize’s own generations for lower-loss fine-grained optimization.The authors also suggest that the technique could use human feedback instead of ChatGPT preferences.Source: §4 · p. 5
5 Model Settings
Baize uses fixed int8 LLaMA checkpoints, LoRA-based parameter-efficient tuning, and inference safeguards to support efficient conversational modeling.
5 Model Settings
- LoRA parameters are trained while the int8 LLaMA checkpoints remain fixed, reducing GPU memory consumption and improving training speed.Source: §5 · p. 5
- In a non-cherry-picked joke example, Baize and ChatGPT explain the joke successfully, whereas Alpaca fails.Source: Table 5 · p. 6
- Baize uses Adam with batch size 64 and model-specific learning rates of 2e-4, 1e-4, and 5e-5 for 7B, 13B, and 30B models.Source: §5 · p. 6
- Nucleus sampling with temperature 1 and top-p 0.95 is used to preserve response diversity while maintaining coherence.Source: §5 · p. 6
- An inference rule makes the assistant decline unethical, controversial, or sensitive topics to limit involvement with sensitive subjects.Source: §5 · p. 6
6 Evaluation
Baize is evaluated through GPT-4 dialogue scoring, language-model benchmarks, qualitative examples, and carbon-footprint estimates. The reported results include competitive dialogue performance, guardrail behavior, domain-specific responses, and the effectiveness of Baize’s data pipeline with a larger base model.
GPT-4 Score
- Baize v2 7B outperforms Vicuna 7B, while Baize v2 13B performs on par with Vicuna 13B despite Vicuna being fully fine-tuned.Source: §6 · p. 6
- GPT-4 scoring compares Baize v2 before and after Self-Distill with Feedback against ChatGPT and other models using 80 prompts across nine categories.Source: §6 · p. 6
- The evaluation pipeline places ChatGPT’s answer first because GPT-4 shows a positional preference for the first answer.Source: §6 · pp. 6–7
LM Evaluation Harness
- Falcon-40B-instruct ranked #1 on the leaderboard as of June 23, 2023, demonstrating the effectiveness of Baize’s data pipeline with a larger base model and full fine-tuning.Source: §6 · p. 7, Table 7 · p. 7
Qualitative Study
- Qualitative examples report that Baize rejects unethical requests, demonstrates coding ability, and receives practitioner-confirmed appropriate responses for Baize-Healthcare.Source: §6 · p. 8, Table 9 · p. 8
Carbon Footprint
- Estimated emissions were 0.83, 1.48, and 3.33 kg CO2 eq. for Baize v1 7B, 13B, and 30B, respectively, with additional emissions for later versions and SDF.Source: §6 · p. 8
7 Conclusion and Future Work
The paper presents a self-chat data pipeline, LoRA-based training, and self-distillation with feedback for Baize. Future work targets greater diversity in simulated user queries and improved self-chat quality.
7 Conclusion and Future Work
Limitations
Baize has limitations inherited from its foundation model, evaluation procedure, licensing, and prompt-dependent safety controls. These constraints include outdated or narrow knowledge, biased automatic evaluation, research-only artifact use, and risks if guardrails are bypassed.
Foundation Model
- At least 82% of LLaMA’s pretraining data predates 2020, and LLaMA supports only 20 languages with limited non-English data.Baize may therefore provide outdated answers and has limited non-English coverage.Source: Limitations · p. 9
Evaluation
- GPT-4-based automatic evaluation has a strong preference for longer responses and positional bias, making human evaluation potentially more rigorous and reliable.Human evaluation is described as expensive and time-consuming, while automatic evaluation remains open.Source: Limitations · p. 9
License and Legality
- Baize weights and its fine-tuning corpus are released for research use only, and the applicable licenses may change.Use with LLaMA’s original weights is also subject to Meta’s LLaMA License Agreement.Source: Limitations · p. 9
Safety and Access Control
- Changing Baize’s prompt can potentially remove its guardrails, creating risks of malicious use despite the default prompt’s safety behavior.The authors prohibit use outside research purposes while noting that further tuning is likely to reduce risks inherited from LLaMA.Source: Limitations · p. 9
- The authors do not perform access reviews, although Meta’s access application process may help control LLaMA distribution and minimize risks.The released weights are intended to facilitate research on fairness, toxicity, and social impacts.Source: Limitations · p. 9
A Self-Chat Template
The self-chat template instructs ChatGPT to simulate both sides of a conversation about a seed topic. It enforces speaker labels, topic continuity, and a natural stopping condition.
A Self-Chat Template
- The template defines a conversation between a human and an AI assistant centered on the seed topic.The seed is inserted into the template as the conversation topic.Source: A Self-Chat Template · p. 11
- Human statements use [Human] and AI assistant statements use [AI].The transcript must be completed in exactly this format.Source: A Self-Chat Template · p. 11
- The human asks related questions about the topic or prior conversation, while the AI assistant tries not to ask questions.The human stops when no further question remains.Source: A Self-Chat Template · p. 11
- The template includes a short example transcript beginning with “Hello!” and a helpful assistant reply.The example demonstrates the required alternating speaker format.Source: A Self-Chat Template · p. 11
B Inference Prompt
The inference prompt frames Baize as an open-source assistant that produces detailed Markdown responses while declining unethical, controversial, or sensitive requests. A healthcare variant adds domain-specific restrictions.
B Inference Prompt
- Baize’s inference prompt identifies it as an open-source AI assistant developed by UCSD and Sun Yat-Sen University.It names Baize after a mythical creature in Chinese folklore and specifies alternating human and AI turns.Source: B Inference Prompt · p. 11
- The prompt requires detailed responses in Markdown format and instructs Baize to decline unethical, controversial, or sensitive topics.These instructions function as an explicit behavioral guardrail.Source: B Inference Prompt · p. 11
Baize-Healthcare
- The Baize-Healthcare prompt identifies a specialized healthcare assistant and retains the detailed-response and safety requirements.The healthcare variant is presented as a separate inference prompt.Source: B Inference Prompt · p. 11
- Baize-Healthcare cannot help with doctor appointments and will never ask for personal information.It also declines unethical, controversial, or sensitive requests.Source: B Inference Prompt · p. 11
C Feedback Prompt for SDF
The SDF feedback prompt asks ChatGPT to score four assistant responses to the same question and explain the evaluation. It targets helpfulness, relevance, accuracy, and detail while reducing order effects.
C Feedback Prompt for SDF
- The feedback prompt presents one question and four assistant responses for comparative evaluation.Each response is delimited as Assistant 1 through Assistant 4.Source: C Feedback Prompt for SDF · p. 11
- A feedback system asks evaluators to rate four AI assistants’ helpfulness, relevance, accuracy, and level of detail, assigning each an overall score from 1 to 100.Higher scores indicate better overall performance.Source: C Feedback Prompt for SDF · p. 11
- The prompt requires four scores on one line followed by a comprehensive explanation that avoids bias and presentation-order effects.Scores must be output in Assistant 1–4 order.Source: C Feedback Prompt for SDF · p. 11