Source-linked AI summary
Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs
Oded Ovadia, Menachem Brief, Moshik Mishaeli, Oren Elisha
TL;DR
LLMs contain broad factual knowledge but may lack nuanced domain expertise, motivating the question of how best to inject knowledge from a text corpus into a pretrained model. The paper compares fine-tuning and retrieval-augmented generation across specialized and unseen knowledge, finding RAG to be the more reliable approach.
Problem
The paper asks how best to teach a pretrained model knowledge from a text corpus, addressing limitations in broad factual knowledge and nuanced domain expertise.
Method
The study compares fine-tuning and retrieval-augmented generation for knowledge injection, using causal autoregressive continuation for fine-tuning and evaluating knowledge across multiple tasks and topics.
Results
RAG consistently outperformed fine-tuning across MMLU and current-events tasks, although fine-tuning improved results over base models in most cases.
Takeaways & Limitations
RAG is a more reliable choice than fine-tuning for injecting specialized and completely unseen knowledge.
Takeaways & Limitations
The experiments used three models, so generalization to other LLMs requires further testing.
Abstract
from arXiv · showhide
Large language models (LLMs) encapsulate a vast amount of factual information within their pre-trained weights, as evidenced by their ability to answer diverse questions across different domains. However, this knowledge is inherently limited, relying heavily on the characteristics of the training data. Consequently, using external datasets to incorporate new information or refine the capabilities of LLMs on previously seen information poses a significant challenge. In this study, we compare two common approaches: unsupervised fine-tuning and retrieval-augmented generation (RAG). We evaluate both approaches on a variety of knowledge-intensive tasks across different topics. Our findings reveal that while unsupervised fine-tuning offers some improvement, RAG consistently outperforms it, both for existing knowledge encountered during training and entirely new knowledge. Moreover, we find that LLMs struggle to learn new factual information through unsupervised fine-tuning, and that exposing them to numerous variations of the same fact during training could alleviate this problem.
1. Introduction
LLMs contain broad factual knowledge but remain static and potentially nonspecific, motivating knowledge injection from external corpora. This section frames fine-tuning and RAG as competing ways to teach pre-trained models such knowledge.
- LLMs capture vast factual information through massive pre-training datasets, but their knowledge is static and may lack domain-specific nuance.
- Knowledge injection asks how best to teach a pre-trained model factual knowledge from a text-corpus knowledge base.
- Fine-tuning adapts model weights using task-specific data, whereas RAG retrieves relevant knowledge and incorporates it into generated text.
- The study compares fine-tuning and RAG to evaluate LLMs’ ability to memorize, understand, and retrieve factual data.
2. Background
The background defines LLM knowledge operationally through factual question answering and distinguishes previously seen from entirely new knowledge. It also highlights factual-error sources and cautions that the evaluation overlaps with reasoning ability.
- The framework treats knowledge as answering factual questions accurately and consistently, while distinguishing true from false statements.
- For a question set Q, a model’s knowledge score is its standard accuracy, with higher accuracy indicating greater knowledge relative to another model.
- Knowledge injection may refresh previously seen information by biasing a model toward a domain rather than teaching entirely new facts.
- Knowledge and Reasoning: The evaluation is imperfect because multiple-choice knowledge tasks can reward reasoning-based educated guesses and therefore should complement broader benchmarks.
- Causes for Factual Errors: Factual errors can reflect domain deficits, outdated information, imemorization, catastrophic forgetting, or reasoning failure.
3. Injecting Knowledge to Language Models
The paper formulates knowledge injection as transforming a pre-trained model using an auxiliary corpus, then compares unsupervised fine-tuning with retrieval-augmented generation. Fine-tuning continues causal next-token training, while RAG retrieves relevant documents as query context.
- 3.1. Problem formulation: Knowledge injection seeks a transformation F that improves a pre-trained model’s performance on factual questions using a relevant auxiliary knowledge base.
- 3.1. Problem formulation: The study compares fine-tuning and RAG as alternative choices for the knowledge-injection transformation.
- 3.2. Fine-Tuning: Instruction tuning and reinforcement-learning methods improve response quality or behavior but do not necessarily expand the model’s breadth of knowledge.
- 3.2. Fine-Tuning: Unsupervised fine-tuning continues causal autoregressive training from a saved checkpoint, typically using a lower learning rate to reduce catastrophic forgetting.
- 3.3. Retrieval-Augmented Generation: RAG retrieves documents resembling the input query from an auxiliary knowledge base and appends them to the query as additional context.
- 3.3. Retrieval-Augmented Generation: In implementation, RAG embeds each knowledge-base document, retrieves the top-K dot-product neighbors for a query, concatenates them with the query, and returns the model output.
4. Knowledge Base Creation
The study assembled knowledge-intensive benchmarks from MMLU and newly created current-events questions, using Wikipedia-derived text and GPT-4-generated question and paraphrase augmentation.
- Task selection: The evaluation selected MMLU tasks emphasizing factual knowledge with minimal reliance on reasoning, alongside a current-events task targeting post-cutoff information.The current-events questions covered U.S. events from August–November 2023 and were designed to mostly ensure the models had not seen the facts during training.
- Knowledge source: The auxiliary knowledge base was built by retrieving relevant Wikipedia articles through the official API and cleaning them into small chunks.Cleaning removed HTML and URLs and supported both evaluation of understanding across domains and fine-tuning.
- Question construction: GPT-4 generated four specific multiple-choice questions per remaining Wikipedia chunk, selected the two best questions, and the authors manually verified them, yielding 910 questions.Each question was intended to have one correct answer, minimal ambiguity, and no need to identify its source context.
- Augmentation: GPT-4 also produced varied paraphrases that retained the original information while rewording the inputs.Different random seeds were used for each paraphrasing iteration.
- Augmentation: Two paraphrases per chunk were reserved for validation, while the current-events fine-tuning data used ten paraphrases per chunk.The validation sets were created from 240 randomly selected chunks for each task.
5. Experiments and Results
The experiments compared base models, retrieval augmentation, fine-tuning, and their combination across MMLU and current-events tasks. RAG consistently outperformed fine-tuning, while combining fine-tuning with RAG was unstable or sometimes inferior to RAG alone.
- Evaluation: The evaluation used standardized LM-Evaluation-Harness procedures and scored each multiple-choice option by its log probability.The highest-scoring option was treated as the model’s answer for accuracy calculation.
- Experimental framework: The evaluation compared base models, RAG, fine-tuning, and fine-tuning plus RAG across three 7B models, using 0-shot and 5-shot settings.The models were Llama2-7B, Mistral-7B, and Orca2-7B; RAG used bge-large-en embeddings with FAISS.
- Combined methods: Using the fine-tuned model as the RAG generator sometimes improved results further, but the effect was inconsistent, demonstrating fine-tuning instability.On the current-events task, combining RAG with fine-tuning performed worse than RAG alone.
- Prompting: 5-shot prompting produced a small improvement in most cases, with the same trend across the different approaches.This pattern was observed across the evaluated approaches in the MMLU experiments.
- MMLU and current events: RAG consistently performed significantly better than the base models and outperformed fine-tuning across the MMLU and current-events evaluations.Fine-tuning improved over the base model in most cases but was not competitive with RAG.
- Interpretation: Potential explanations for RAG’s advantage include its question-relevant context and fine-tuning’s possible effects on other model capabilities through catastrophic forgetting.The passage presents these as possible contributing factors rather than established causal findings.
6. The Importance of Repetition
For entirely new information, standard fine-tuning can degrade performance, whereas paraphrase augmentation improves accuracy as the number of paraphrases increases. The authors hypothesize that repeated, varied presentations help models learn knowledge beyond memorizing individual sentences.
- Standard fine-tuning degraded Llama2’s performance on the current events task containing information absent from pretraining.
- Accuracy increased monotonically with the number of paraphrases for all tested models.The authors associate this trend with information repetition through paraphrase augmentation.
- Training loss dropped significantly after each epoch, consistent with memorization and overfitting during training.
- The authors hypothesize that teaching new knowledge requires repeating it in numerous ways.
- Varied presentations may expose relationships in the data that sentence memorization alone does not entail.The authors connect this hypothesis to prior work on pretraining and the Reversal Curse, while noting that further research is needed.
7. Conclusion and Future Work
The paper compares fine-tuning and retrieval-augmented generation for adapting LLMs to specialized and completely unseen knowledge. It concludes that RAG is more reliable, while identifying several directions for future research.
- The study compares fine-tuning and retrieval-augmented generation for injecting specialized and completely unseen knowledge into LLMs.
- RAG was found to be a more reliable choice for knowledge injection, although fine-tuning remains useful for many use cases.
- The authors focused on unsupervised training rather than instruction-tuning or reinforcement-learning-based fine-tuning.They propose studying combinations of techniques and diverse auxiliary knowledge bases.
- Further research is needed on knowledge representation in LLMs, especially from a theoretical perspective.
- The empirical knowledge measure should be complemented by other definitions and perspectives on knowledge.
8. Limitations
The study’s conclusions are bounded by hyperparameter sensitivity, limited model coverage, and reliance on Wikipedia as the sole source of knowledge bases.
- Hyperparameter choices significantly affect results, so relevant hyperparameters should be optimized for each specific case.
- The experiments covered three models, and generalization to other LLMs still requires thorough testing.
- All knowledge-base sources came from Wikipedia, so other datasets may produce different results and require careful evaluation.
A. RAG Ablation Study
The RAG ablation study found no consistently optimal number of retrieved context chunks across models, prompting a practical warning about the instability of K.
- The study varied K from 0 to 5 but found no optimal value per model, shot setting, or task.Anatomy consistently worked well with K = 2, but no broader predictive pattern emerged.
- The gap between the best- and worst-performing K values can be large, making K an unstable RAG hyperparameter.The authors state that K cannot be ignored in practice.
B. Paraphrase Examples
The paraphrase examples show GPT-4 generating multiple fact-preserving rewrites from Wikipedia-based source text. The examples cover International Emmy submissions and the 2023 Indianapolis mayoral election.
- GPT-4 was prompted to generate multiple paraphrases while preserving meaning, factual information, and roughly the original length.
- The required output format was a JSON string containing one key, “paraphrases,” with a list of generated rewrites.
- The International Emmy example states that the 51st competition accepted submissions from December 7, 2022, through February 16, 2023.
- The Indianapolis example describes the November 7, 2023 mayoral election, its May 2 preliminary elections, and Joe Hogsett’s successful third-term campaign against Jefferson Shreve.
- The examples illustrate paraphrases that change wording while retaining the same election dates, candidates, offices, and outcomes.
C. Current Events Existing Knowledge Examples
The examples distinguish questions answerable through basic reasoning or prior knowledge from questions requiring specific unfamiliar facts. They also show that automatically generated questions can depend on pre-existing knowledge.
- Unseen information: The United Auto Workers strike question is easy to answer without event-specific knowledge because the stagnant-wages option is more plausible than the distractors.
- Prior knowledge: The Hawaii wildfires example shows how geographic and wildfire knowledge can lower the likelihood of two options and increase the chance of selecting the correct remaining answer.
- Evaluation caveat: Automatically generated questions may rely strongly on pre-existing knowledge, complicating their use as tests of newly acquired information.
- Existing knowledge: The September 2023 New York floods question may be answerable from pre-training because Hurricane Ida was the only listed event occurring in 2021.
- Unseen information: The Matthew Belk question is difficult to guess because identifying his “200-year event” description requires the specific flood information.