Source-linked AI summary
Platypus: Quick, Cheap, and Powerful Refinement of LLMs
Ariel N. Lee, Cole J. Hunter, Nataniel Ruiz
TL;DR
Platypus addresses the cost of refining base LLMs with limited compute and data. It combines a curated Open-Platypus dataset with LoRA fine-tuning, module merging, and contamination checks, achieving strong leaderboard performance while enabling a 13B model to train on one A100 GPU in 5 hours. The models remain subject to static-knowledge, language-coverage, factuality, harmful-content, and bias limitations.
Problem
Fine-tuning domain-specific LLMs can be time-consuming and costly, motivating a recipe that preserves generalized instruction-tuning benefits while adding specific domain knowledge.
Method
The paper curates a small STEM- and logic-focused dataset from open sources, applies LoRA fine-tuning and merging, and filters similar questions and benchmark contamination.
Results
Platypus2-70B-instruct secured first place on the Hugging Face Open LLM Leaderboard with an average score of 73.13.
Takeaways & Limitations
A 13B model can be trained on a single A100 GPU using 25k questions in 5 hours, while model merging produced especially strong enhancements for 13B variants.
Takeaways & Limitations
Platypus has static knowledge, limited and inconsistent non-English proficiency, and can generate non-factual, harmful, offensive, or biased content.
Abstract
from arXiv · showhide
We present $\textbf{Platypus}$, a family of fine-tuned and merged Large Language Models (LLMs) that achieves the strongest performance and currently stands at first place in HuggingFace's Open LLM Leaderboard as of the release date of this work. In this work we describe (1) our curated dataset $\textbf{Open-Platypus}$, that is a subset of other open datasets and which $\textit{we release to the public}$ (2) our process of fine-tuning and merging LoRA modules in order to conserve the strong prior of pretrained LLMs, while bringing specific domain knowledge to the surface (3) our efforts in checking for test data leaks and contamination in the training data, which can inform future research. Specifically, the Platypus family achieves strong performance in quantitative LLM metrics across model sizes, topping the global Open LLM leaderboard while using just a fraction of the fine-tuning data and overall compute that are required for other state-of-the-art fine-tuned LLMs. In particular, a 13B Platypus model can be trained on $\textit{a single}$ A100 GPU using 25k questions in 5 hours. This is a testament of the quality of our Open-Platypus dataset, and opens opportunities for more improvements in the field. Project page: https://platypus-llm.github.io
1 Introduction
Platypus targets the cost of refining base LLMs by combining parameter-efficient fine-tuning, curated domain data, and LoRA-module merging. The resulting models are designed to improve selected capabilities while retaining broad instruction-tuning benefits.
- Related approaches: Instruction tuning, knowledge distillation, and specialized task training are established strategies for improving LLM performance while addressing computational constraints.The related work discusses instruction-formatted data and domain-focused models for coding, quantitative reasoning, and biomedical knowledge.
- Motivation: Fine-tuning domain-specific datasets and merging models aims to reduce training time while improving selected task categories.The approach seeks to retain generalized instruction-tuning improvements while adding specific domain knowledge.
- Open-Platypus: Open-Platypus contains 11 open-source datasets focused on STEM and logic, mostly human-designed questions, enabling a 13B model to train on one A100 GPU in 5 hours.Approximately 10% of questions were generated by an LLM, and the dataset contains 25k questions for the stated training example.
- Data curation: Similarity exclusion removes redundant training questions and supports filtering intended to reduce contamination and memorization.The paper describes both dataset de-duplication and benchmark-test leak checks as part of the data curation process.
- Model refinement: The authors describe selecting and merging specialized fine-tuned LoRA modules to combine broad-model and domain-specific capabilities.LoRA-module merging is presented as a central contribution alongside dataset construction and contamination analysis.
2 Methods
The methods curate a small, STEM- and logic-focused instruction dataset, remove redundant or potentially contaminating questions, and organize data for model training. Contamination filtering uses similarity heuristics and manual categories to avoid benchmark leakage.
- 2.1 Curating Open-Platypus: The curation strategy prioritizes high-quality, diverse prompts and a small training set to balance output precision, relevance, training time, and model performance.The decisions were guided by the Superficial Alignment Hypothesis, the unsaturated state of LLaMa-2 base models, and the importance of high-quality input data.
- 2.1 Curating Open-Platypus: Open-Platypus is curated from 11 open-source datasets, emphasizing STEM and logic while supplementing specialized sources with keyword-filtered broader datasets.Its backbone is a modified MATH dataset supplemented with expanded step-by-step solutions from PRM800K.
- 2.2 Removing similar & duplicate questions: Questions are de-duplicated by removing exact instruction duplicates and pairs exceeding 80% cosine similarity in SentenceTransformers embeddings.When duplicates were found, the more verbose answer was retained because it was expected to provide more detailed explanations or step-by-step solutions.
- 2.3 Contamination Check: Potential benchmark leaks are manually filtered after identifying Open-Platypus questions with similarity above 80% to benchmark questions.The filtering scheme groups cases into duplicate, gray-area, and similar-but-different categories, removing all three as a cautious policy.
- 2.3 Contamination Check: Duplicate contamination denotes test questions copied exactly or with minor wording changes, and is the only category counted as true contamination.The number of such leaked questions is reported in Table 1, with examples shown in Figure 1.
- 2.3 Contamination Check: Gray-area cases are non-exact duplicates involving general knowledge, including synonymous answers or differently phrased questions whose detailed answers appear in training outputs.The paper leaves final judgment of these cases to the open-source community and notes that they may require expert knowledge.
- 2.3 Contamination Check: Similar-but-different questions can have high cosine similarity yet produce different answers because subtle structural changes alter the question’s result.The paper illustrates this category using changes to the rotation axis and quadrant definition.
Train Test
The methodology combines LoRA-based parameter-efficient training with selected transformer modules, using different learning rates for the 13B and 70B models.
- Fine-tuning & merging: LoRA freezes pretrained weights and adds low-rank matrices to transformer layers, reducing trainable parameters and training cost.The approach is described as compatible with model merging.
- Fine-tuning & merging: The fine-tuning strategy uses gate_proj, down_proj, and up_proj modules rather than the initially tested attention modules.This choice followed prior analysis reporting superior performance except when trainable parameters were below 0.1% of total parameters.
- Fine-tuning & merging: 0.27% and 0.2% of parameters were trainable in the 13B and 70B fine-tunes, respectively.The same module strategy was adopted for both model sizes.
- Fine-tuning & merging: The 70B model used a 3e-4 initial learning rate instead of 4e-4 because its loss reached zero after 15 steps.The initial learning rate was the only stated difference between the 13B and 70B models.
Train Test
The paper describes contamination-aware dataset curation and LoRA scaling choices while acknowledging that checks cannot fully verify merged models trained on undisclosed data.
- Fine-tuning & merging: The weight matrix is scaled by lora_alpha divided by lora_rank, with higher alpha assigning more weight to LoRA activations.These quantities describe the scaling of the LoRA contribution.
- Fine-tuning & merging: A LoRA rank of 16 and a 1:1 alpha-to-rank ratio were chosen to avoid overpowering the base model.The rank choice followed common practice in reviewed training scripts.
- Contamination checks: The authors avoided merging models trained on contaminated datasets and specifically merged Dolphin-70B after checking that no test questions had leaked.They performed contamination checks to the best of their abilities, but some source datasets were not publicly released.
- Contamination checks: The authors cannot provide absolute assurances for merged models trained with closed-source data.The limitation follows from unavailable training datasets.
3 Results
Platypus models achieved strong leaderboard and benchmark results, while model-merging outcomes varied substantially by model combination and domain. The experiments therefore emphasize careful merge selection and domain-specific evaluation.
- Leaderboard results: 73.13 average score placed Platypus2-70B-instruct first on the Hugging Face Open LLM Leaderboard data from 8/10/23.Stable-Platypus2-13B was the premier 13B model with an average score of 63.96.
- Fine-tuning and merging: Post-fine-tuning, both 13B and 70B models improved over base LLaMa-2 models, particularly on ARC and TruthfulQA.
- Fine-tuning and merging: The Stable Beluga merge exceeded its constituent models by at least 0.5% across most benchmarks, including a 2.91% TruthfulQA increase.Stable-Platypus2-13B also gained +1.05% over its base model.
- Interpretation: Merging consistently improved TruthfulQA, suggesting broader knowledge rather than enhanced reasoning capabilities.
- Task-level results: +4.12% was the largest ARC-Challenge improvement, achieved by Camel-Platypus2-70B merged with the Platypus adapter.
- Task-level results: MMLU outcomes varied by domain: Camel-Platypus2-70B gained +26.32% in machine_learning, while Platypus2-70B-instruct gained +18.18% in abstract_algebra and Camel-Platypus2-13B declined -15.62%.
- Merge selection: Similar baseline scores did not guarantee a superior merged model, and Dolphin-Platypus2-70B was the only merge that failed to outperform both base and adapter models.
- Task-level results: Merge effects were domain-specific, including minimal changes in marketing and declines of -20.93%, -13.16%, and -18.42% in college_physics for three 70B models.The authors recommend evaluating target-domain performance before finalizing a merge.
4 Broader Impacts & Future Work
The paper frames compact models, curated niche datasets, and adapter merging as ways to reduce resource demands. It identifies further investigation of LoRA/PEFT applications and merge behavior as future work.
- Broader impacts: Modern LLM training and inference can require computational resources that restrict access for limited-budget users.
- Broader impacts: LIMA proposes many small, carefully curated datasets for niche domains, supporting streamlined fine-tuning and rapid cosine-similarity searches across adapter inputs.
- Future work: Future work should investigate LIMA-style strategies in LoRA and PEFT, merge behavior for models with similar baseline scores, and larger LoRA merges such as Lazarus.
5 Limitations
Platypus retains limitations inherited from LLaMa-2 while adding risks associated with static, specialized, and publicly sourced training data. The authors also caution that contamination checks and deployment safety testing do not provide absolute assurances.
- Platypus lacks continuous knowledge updates after pretraining and fine-tuning, so its information can become outdated or incomplete.
- The model may generate non-factual, unqualified, harmful, offensive, or biased content, particularly for ambiguous, misleading, or sensitive prompts.
- Platypus primarily targets English-language data, making performance in other languages inconsistent and not guaranteed.
- Platypus may perform poorly outside STEM and logic topics because its specialized training concentrates on those domains.
- Users must check benchmark contamination independently because PRM800K training included both MATH train and test data after removing overly similar questions.
- Merged variants lack absolute assurances because some source datasets were closed, and deployment requires application-specific safety testing and tuning.
- Keyword and embedding-similarity filtering may miss contamination, leaving some leaked questions possible despite confidence in the cleaned data.
Appendix
The appendix includes Alpaca instruction-format examples, model abbreviation definitions, benchmark table captions, and a hyperparameter table. The benchmark tables report changes relative to the base model across specified evaluations.
- Alpaca formatting examples show instruction-response prompts both with and without an accompanying input.
- Tables 4 and 5 report ARC-Challenge, Hellaswag, and TruthfulQA-MC changes relative to the model receiving the merged adapters.
- Tables 6 and 7 report MMLU percent change and change in percent relative to the base model.
- C-P2-13, S-P2-13, D-P2-70, and C-P2-70 abbreviate the Camel, Stable, Dolphin, and Camel Platypus model variants.
- Table 8 lists hyperparameters for the 13B and 70B models.