PaperlayerView exact revision on arXiv

Source-linked AI summary

Orca 2: Teaching Small Language Models How to Reason

Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, Hamid Palangi, Guoqing Zheng, Corby Rosset, Hamed Khanpour, Ahmed Awadallah

arXiv:2311.11045v2cs.AI

TL;DR

Small language models often imitate larger models without matching their reasoning or comprehension skills. Orca 2 teaches multiple reasoning strategies and strategy selection, outperforming similarly sized models and matching or exceeding models 5–10 times larger on reasoning tasks.

  • Problem

    Imitation-based training can leave smaller language models short of larger models’ reasoning and comprehension skills.

    Source: §1 · p. 3

  • Method

    Orca 2 teaches smaller models multiple reasoning techniques and how to select the most effective strategy for each task.

    Source: §1 · p. 3

  • Results

    Orca 2 significantly surpasses similarly sized models and matches or exceeds models 5–10 times larger, especially on reasoning tasks.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Tailored synthetic-data training can improve smaller language models’ reasoning, producing performance comparable to or exceeding much larger models on zero-shot reasoning tasks.

    Source: §8 · p. 22

    Without suitable safeguards, these models could be maliciously used to generate disinformation or harmful content.

    Source: §7 · p. 21

Abstract

from arXiv · show

Orca 1 learns from rich signals, such as explanation traces, allowing it to outperform conventional instruction-tuned models on benchmarks like BigBench Hard and AGIEval. In Orca 2, we continue exploring how improved training signals can enhance smaller LMs' reasoning abilities. Research on training small LMs has often relied on imitation learning to replicate the output of more capable models. We contend that excessive emphasis on imitation may restrict the potential of smaller models. We seek to teach small LMs to employ different solution strategies for different tasks, potentially different from the one used by the larger model. For example, while larger models might provide a direct answer to a complex task, smaller models may not have the same capacity. In Orca 2, we teach the model various reasoning techniques (step-by-step, recall then generate, recall-reason-generate, direct answer, etc.). More crucially, we aim to help the model learn to determine the most effective solution strategy for each task. We evaluate Orca 2 using a comprehensive set of 15 diverse benchmarks (corresponding to approximately 100 tasks and over 36,000 unique prompts). Orca 2 significantly surpasses models of similar size and attains performance levels similar or better to those of models 5-10x larger, as assessed on complex tasks that test advanced reasoning abilities in zero-shot settings. make Orca 2 weights publicly available at aka.ms/orca-lm to support research on the development, evaluation, and alignment of smaller LMs

1 Introduction

Orca 2 addresses the limitations of imitation learning by teaching smaller language models multiple reasoning strategies and how to select among them for each task. It evaluates this approach across 15 benchmarks and reports performance surpassing similarly sized models, sometimes matching or exceeding models 5–10 times larger.

Motivation

  • Imitation learning can reproduce larger models’ style while falling short on reasoning and comprehension, limiting smaller models’ ability to use capacity-appropriate solution strategies.The paper identifies this limitation as a motivation for moving beyond direct output replication.Source: §1 · p. 3

Contribution

  • Orca 2 teaches smaller models multiple reasoning techniques and how to choose the most effective strategy for each task.Strategies include step-by-step, recall-then-generate, recall-reason-generate, extract-generate, and direct-answer methods.Source: §1 · p. 3

Method

  • Prompt Erasure trains Orca 2 only on tasks and resulting behaviors, making it a cautious reasoner that selects task-suitable behaviors rather than naively imitating teacher prompts.More capable models are prompted to elicit tailored strategies, while the student does not see those original prompts.Source: §1 · p. 3

Evaluation

  • The evaluation covers 15 benchmarks, approximately 100 tasks, and over 36,000 unique prompts spanning understanding, reasoning, mathematics, comprehension, summarization, groundedness, truthfulness, and toxicity.The authors use this broad comparison to address limitations of evaluations based on few tasks or model-based pairwise judging.Source: §1 · p. 3

Results

  • Orca 2 significantly surpasses similarly sized models and matches or exceeds models 5–10 times larger, especially on reasoning tasks.Reported results use LLaMA-2 7B and 13B base models, which remain constrained by their underlying pretrained models.Source: §1 · p. 3

2 Preliminaries

Instruction tuning improves instruction following, generation quality, zero-shot performance, and reasoning, but may teach style without proportional gains in knowledge- or reasoning-intensive correctness. Explanation Tuning addresses this weakness by training students on richer reasoning signals elicited from capable models through system instructions.

2.1 Instruction Tuning

  • Instruction tuning improves instruction following on seen and unseen tasks, generation quality, and zero-shot reasoning abilities [47].Source: §2.1 · p. 4
  • However, behavior cloning can produce stylistically correct but incorrect outputs and may not proportionally improve small models on knowledge- or reasoning-intensive tasks [42] [5].Instruction tuning also does not add knowledge beyond pre-training, so enhanced small models are best used as reasoning engines over contextual knowledge or in narrow domains.Source: §2.1 · p. 4, §2.2 · p. 4

2.2 Explanation Tuning

  • Explanation Tuning elicits detailed reasoning from capable models using hand-crafted system instructions, then trains students to predict answers from those instructions and user prompts.The resulting triplets contain a system instruction, user prompt, and teacher answer, providing richer “Slow Thinking” demonstrations.Source: §2.2 · p. 4
  • Combining M prompt clusters with N system instructions yields M × N diverse teacher answers, supporting improvements over traditional instruction tuning in complex zero-shot reasoning [42].Source: §2.2 · p. 4

3 Teaching Orca 2 to be a Cautious Reasoner

Orca 2 is trained as a cautious reasoner that selects task-appropriate solution strategies rather than naively imitating larger models. Its Prompt Erasing procedure removes teacher-specific instructions so students learn the underlying reasoning strategy and abilities.

  • Cautious Reasoning means choosing between direct answering and Slow Thinking strategies such as step-by-step, guess-and-check, or explain-then-answer.The selected strategy should depend on the task’s difficulty and requirements.Source: §3 · p. 5
  • Smaller models should select strategies suited to each task because their optimal strategy may differ from a powerful model’s, making simple imitation sub-optimal.Training on multiple strategies, including step-by-step reasoning, increases flexibility beyond training only on explained answers.Source: §3 · p. 5
  • Training first uses Orca’s performance to assign strategies to diverse tasks, then generates teacher responses with task-specific system instructions and erases those instructions during student training.Prompt Erasing replaces the student’s detailed system instruction with a generic one, while teacher response generation may use multiple calls or highly detailed instructions.Source: §3 · p. 5
  • GPT-4’s answers vary substantially with system instructions: only a carefully crafted, multi-step instruction produces the correct story ordering in Figure 3.Default, chain-of-thought, and explain-your-answer instructions respectively produce wrong, incomplete, and incorrect answers despite a correct explanation in the third case.Source: §3 · p. 5, Figure 3 · p. 6
  • Prompt Erasing encourages students to learn the reasoning strategy and abilities underlying the teacher’s responses rather than merely reproducing the teacher’s instructional framing.This technique motivates the presentation of Orca 2 as a cautious reasoner.Source: §3 · p. 7

4 Technical Details

Orca 2 uses an 817K-instance dataset assembled from diverse sources and trains through progressive learning on increasingly specialized teacher-generated data. Its training pipeline combines LLaMA-based tokenization, sequence packing, and loss computation focused on teacher-generated responses.

4 Technical Details

  • Orca 2 introduces an approximately 817K-instance dataset and combines it with FLAN annotations and the Orca 1 dataset through progressive learning.The training subsets are obtained by combining these three sources.Source: §4 · p. 7

4.1 Dataset Construction

  • The dataset construction uses approximately 602K FLAN queries, 55K few-shot samples, 160K math problems, and 2,000 synthetic doctor-patient conversations.FLAN queries come from 1,448 high-quality tasks; few-shot samples use 3–5 in-context examples; synthetic conversations target specialized summarization skills.Source: §4.1 · pp. 7–8
  • FLAN tasks are organized into 23 categories and 126 sub-categories, while a generic cautious system instruction replaces the original system instructions.The cautious instruction emphasizes careful instruction-following, helpfulness, harmlessness, ethical guidelines, and positive behavior.Source: §4.1 · p. 7

4.2 Training

  • Progressive learning fine-tunes LLaMA-2-7B or LLaMA-2-13B sequentially on FLAN-v2, 5 million Orca 1 ChatGPT examples, then 1 million Orca 1 GPT-4 examples plus 817K Orca 2 examples.The stages use 1, 3, and 4 epochs, respectively.Source: §4.2 · p. 8
  • Training uses LLaMA BPE tokenization, packs examples into sequences capped at max_len = 4096 tokens, and computes loss only on teacher-generated tokens.The tokenizer vocabulary contains 32,003 tokens after adding padding and ChatML special tokens.Source: §4.2 · p. 8

5 Experimental Setup

Orca 2 is evaluated against instruction-tuned baselines across diverse capability benchmarks, using controlled zero-shot inference and standardized answer parsing. The setup spans reasoning, knowledge, completion, conversation, grounding, summarization, safety, truthfulness, and alignment, with explicit in-domain and evaluation limitations.

5.1 Baselines

  • Orca 2 is compared with instruction-tuned LLaMA 2, WizardLM, Orca 1, ChatGPT, and GPT-4 models, selected for stronger instruction following, reasoning, and zero-shot performance [47] [42].The LLaMA 2 and WizardLM baselines include 13B and 70B variants; Orca 1 is 13B, while ChatGPT and GPT-4 are also evaluated [57] [42].Source: §5.1 · pp. 8–9

5.2 Benchmarks

  • The benchmark suite covers advanced and basic capabilities, including reasoning, knowledge, language understanding, text completion, grounding, truthfulness, safety, and alignment in zero-shot evaluation.Reasoning tasks include AGIEval, DROP, CRASS, RACE, BBH, and GSM8K; knowledge and understanding tasks include MMLU and ARC.Source: §5.2 · p. 9, §5.2.1 · p. 9, §5.2.2 · p. 9

5.2.3 Text Completion / 5.2.4 Multi Turn Open Ended Conversations

  • Text-completion and dialogue evaluations use HellaSwag, LAMBADA, and MT-bench to test commonsense inference, long-range context, and multi-turn chat proficiency [48].MT-bench uses GPT-4 as the judge.Source: §5.2.3 · p. 10, §5.2.4 · p. 10

5.2.5 Grounding and Abstractive Summarization

  • Grounding and summarization are assessed with ACI-BENCH, MS-MARCO, and QMSum, covering clinical-note generation, web-question answering, and query-based meeting summarization [68].These tasks require generating notes from doctor-patient dialogues, answering questions from web documents, or selecting and summarizing relevant meeting spans.Source: §5.2.5 · p. 10

5.3 Evaluation Settings

  • All models are evaluated zero-shot without exemplars or chain-of-thought prompting, using greedy decoding and simple prompts with empty system messages except for answer-format guidance.Orca 2 is reported with both empty and cautious system messages, while larger models may benefit more from few-shot settings; detailed few-shot analysis is left for future work.Source: §5.3 · pp. 10–11
  • Answers are parsed uniformly as multiple-choice, exact-match/span-extraction, or no-extraction tasks, and evaluation additionally reports format-OK coverage.Regular-expression patterns extract answer options or spans, responses are matched against gold answers, and the authors note that better parsing could improve all models’ performance.Source: §5.3 · p. 11
  • Some evaluated datasets are treated as in-domain despite zero-shot testing, while the remainder are considered out-of-domain, and possible base-model data leakage cannot be completely ruled out.In-domain datasets include DROP, ARC, RACE, HellaSwag, LAMBADA, MS-MARCO, and GSM8K; the study reports instruction-tuned LLaMA-2 versions for reference.Source: §5.3 · p. 11

6 Evaluation Results

Across zero-shot evaluations, Orca 2 outperforms similarly sized instruction-tuned models and often matches or exceeds models 5–10 times larger, while showing strengths and limitations across reasoning, knowledge, completion, conversation, grounding, safety, and task specialization. Results also depend on evaluation prompts, endpoints, and whether tasks require grounded or multi-turn behavior.

6.1 Reasoning

  • Orca-2-13B improves zero-shot reasoning by 47.54% over LLaMA-2-Chat-13B and 28.15% over WizardLM-13B, exceeds LLaMA-2-Chat-70B, and is comparable to WizardLM-70B and ChatGPT.Orca-2-7B is better or comparable to LLaMA-2-Chat-70B on all reasoning tasks, while cautious system messages provide small gains for both Orca 2 sizes.Source: §6.1 · pp. 11–12, Figure 4 · p. 12

6.2 Knowledge and Language Understanding

  • Orca-2-13B surpasses same-size baselines on every knowledge and language-understanding benchmark, improving 25.38% over LLaMA2-Chat-13B and 44.22% over WizardLM-13B.It also outperforms both 70B baselines overall; on MMLU it scores 57.73% versus 58.54% for LLaMA2-Chat-70B and 55.00% for WizardLM-70B, while Orca-2-7B surpasses both 70B baselines on ARC.Source: §6.2 · p. 13, Table 2 · p. 13

6.3 Text Completion

  • Both Orca-2-7B and Orca-2-13B outperform the 13B and 70B baselines on HellaSwag, with Orca-2-13B improving 33.13% over LLaMA-2-Chat-13B and 61.94% over WizardLM-13B.The authors conjecture that chat models are poorly suited to HellaSwag because sampled chat responses often answer incorrectly or refuse for safety reasons.Source: §6.3 · pp. 13–14, Figure 5 · p. 14

6.4 Multi-Turn Open Ended Conversations

  • Orca-2-13B performs comparably to other 13B models in MT-Bench, but its average second-turn score is lower than its first-turn score.The authors attribute this decline to the absence of conversations in training data; packing multiple zero-shot examples can improve conversational ability, while GPT-4 endpoint differences affect assessments.Source: §6.4 · p. 15, Table 3 · p. 15

6.5 Grounding

  • Orca-2-13B has the lowest hallucination rate among Orca 2 variants and other 13B and 70B models, reducing hallucinations by 76.92% versus LLaMA-2-13B and 61.71% versus WizardLM-13B.Cautious system messages consistently increase hallucination rates because reasoning may extrapolate beyond the provided context, producing factually accurate but unsupported content.Source: §6.5 · p. 15, Figure 6 · p. 16

6.6 Safety

  • Orca-2-13B performs better than similarly sized models and comparably to much larger models on TruthfulQA, while both Orca-2-13B and Orca-2-7B outperform similarly sized models on HHH.In ToxiGen, Orca-2-7B classifies toxic statements better than neutral ones, whereas Orca-2-13B does not show this imbalance; Orca 2 family models exceed 96% instruction-following rates.Source: §6.6 · pp. 16–18

6.7 Effect of Task-Specific Data with Story Reordering

  • Mixing 5,000 prompt-erased, task-specific story-reordering samples with the training data demonstrates the potential to specialize Orca 2 for specific tasks using synthetic data.Evaluation uses a distinct ROCStories test set with FLAN-overlapping instances removed and compares Orca 2 against GPT-4 prompting variants, ChatGPT, Orca 1, LLaMA, and WizardLM.Source: §6.7 · p. 20, Figure 12 · p. 20

7 Limitations

Orca 2 retains limitations from the LLaMA 2 family, large language models, and its training process. These include risks involving bias, transparency, harms, hallucination, misuse, data distribution, system instructions, synthetic data, limited capacity, and research-only evaluation.

7 Limitations

  • Users should not rely on Orca 2 for critical decisions because it can fabricate content, while its susceptibility to hallucination in ungrounded generation remains unclear and under active study.The authors call for more rigorous measurement, understanding, and mitigation of hallucination risks, particularly for small models.Source: §7 · p. 21
  • Orca 2 may generate biased or unfair outputs, remain difficult to interpret as a black box, cause content harms, and be misused for disinformation or harmful content without safeguards.The authors recommend transparency review, content moderation, and stronger safeguards, regulations, and standards.Source: §7 · p. 21
  • Orca 2’s accuracy may be limited in underrepresented areas such as math and coding, and its performance varies with system instructions, including potentially non-deterministic responses.Its tuning-data distribution strongly correlates with performance, while model-size stochasticity contributes to response variance.Source: §7 · p. 21
  • Although Orca 2 performs strongly in zero-shot settings, it does not show the same few-shot gains as other, especially larger, models.Its training data mostly simulated zero-shot settings, which may limit transfer to few-shot use.Source: §7 · p. 21
  • Synthetic training data may transmit both the benefits and shortcomings of the generation models and methods, despite safety measures and guardrails incorporated during training.The authors state that detailed studies are needed to quantify these risks more precisely.Source: §7 · p. 21
  • Post-training improves task-solving and reasoning but does not add new knowledge, leaving Orca 2 mainly constrained by pretraining and better suited to contextual reasoning or narrow-domain fine-tuning.The model is intended solely for research, and downstream use requires additional analysis of potential harms and bias.Source: §7 · p. 21

8 Conclusions

Orca 2 shows that tailored synthetic-data training can improve smaller language models’ reasoning, enabling performance comparable to or exceeding much larger models on zero-shot reasoning tasks. The study also identifies synthetic post-training as a promising path for improving reasoning, control, and safety despite persistent base-model limitations.

Conclusions

  • Orca 2 models achieve performance comparable to, and often exceeding, much larger models on zero-shot reasoning tasks by using varied techniques and selecting effective task-specific strategies.Their approach teaches smaller models to reason rather than simply imitate larger-model outputs.Source: §8 · p. 22
  • Tailored, high-quality synthetic data created by more powerful models can support post-training with complex prompts and potentially multiple model calls.The authors present this as a broader route toward improving smaller language models’ capabilities.Source: §8 · p. 22
  • Despite limitations inherited from their base models, Orca 2 models offer promising opportunities to improve reasoning, control, and safety through synthetic-data post-training.They have not undergone RLHF for safety, but safety-filtered synthetic data may improve overall safety.Source: §8 · p. 22
  • Developing more capable smaller language models could enable applications requiring different deployment trade-offs between efficiency and capability, even as frontier models remain superior.The authors characterize this work as a step toward realizing the potential of small language models.Source: §8 · p. 22

A AGIEval Subtask Metrics

On zero-shot AGIEval tasks, Orca-2-13B outperforms several larger or comparable baselines, including LLaMA-2-70B, while smaller 7B variants remain competitive. Orca 2 also improves over Orca1-13B on average and in most tasks.

A AGIEval Subtask Metrics

  • +3.23 points: Orca-2-13B leads LLaMA-2-70B on average despite having around 5X fewer parameters.Orca-2-13B trails on LSAT-LR, LOGIQA, and SAT-English without passage, but leads notably on AQUA-RAT (+7.87 points) and SAT-MATH (+17.71).Source: Table 6 · p. 28, A AGIEval Subtask Metrics · p. 28
  • 49.93%: Orca-2-13B averages higher than its cautious-system-message variant (48.18%), while the 7B variants score 45.10% and 43.97%.The 7B iterations remain relatively competitive but are surpassed by their 13B counterparts.Source: A AGIEval Subtask Metrics · p. 28
  • +11.68 points: Orca-2-13B surpasses WizardLM-13B on average, while also leading on LSAT-RC (+15.99 points) and Gaokao EN (+12.74 points).Against LLaMA-2-Chat-13B, Orca-2-13B leads by +11.08 points on average, including LSAT-RC (+22.31 points), LSAT-LR (+10.20 points), and Gaokao EN (+14.70 points).Source: A AGIEval Subtask Metrics · p. 28
  • +4.24 points: Orca-2-13B improves over Orca1-13B on average and surpasses Orca1 in most tasks.The cautious-system-message variant leads Orca1 by +2.49 points on average, though it trails in LOGICQA (English) and SAT English.Source: A AGIEval Subtask Metrics · pp. 28–29
  • Orca-2-13B exceeds Orca1-13B by +0.59 points on LSAT-LR, whereas its cautious variant trails by -2.15 points; both variants fall short on GAOKAO-EN.The cautious variant also lags on LOGICQA (English) by -1.84 and SAT English by -0.49.Source: A AGIEval Subtask Metrics · pp. 28–29

B BigBench-Hard Subtask Metrics · C Evaluation of Grounding in Abstractive Summarization · C.1 Hallucination Rate Evaluation

The section reports zero-shot evaluation across BigBench-Hard reasoning tasks and abstractive summarization grounding, measuring accuracy and hallucination rates. Orca-2-13B is identified as the most effective model among evaluated Orca 2 variants and comparable-size baselines for hallucination reduction.

B BigBench-Hard Subtask Metrics

C Evaluation of Grounding in Abstractive Summarization

  • The hallucination evaluation covers zero-shot abstractive summarization on ACI-BENCH, QMSum [68], and MS MARCO.The evaluation measures summary quality and hallucination rates using methods proposed in and.Source: C Evaluation of Grounding in… · p. 31
  • The study frames abstractive summarization as a grounding test because summaries should respond generatively while remaining grounded in provided context.Grounding is presented as a central aspect of fabrication and hallucination in modern language models.Source: C Evaluation of Grounding in… · p. 31

C.1 Hallucination Rate Evaluation

  • Orca-2-13B achieves the best hallucination-rate performance among Orca 2 variants and comparable-size baseline models, with lower rates indicating better performance.Hallucination rates are judged by GPT-4 using an empty system message.Source: Table 11 · p. 31, §C.1 · p. 31
  • Table 11 compares hallucination rates for Orca 2’s 13B and 7B versions against baseline models.The table’s upper segment compares Orca 2 variants, while its lower segment reports baseline models.Source: Table 11 · p. 31

C.2 Evaluation of Generated Summaries

Orca 2 summaries are evaluated with BLEU, ROUGE-L, and perplexity, comparing direct and cautious system messages against other LLMs. Results vary by dataset and metric, with direct prompting generally outperforming cautious prompting.

  • BLEU, ROUGE-L, and perplexity are used to evaluate generated summaries, with lower perplexity indicating better performance.Table 12 compares Orca 2 under direct and cautious system messages with other evaluated LLMs.Source: §C.2 · p. 31, Table 12 · p. 32
  • Orca 2 outperforms both LLaMA-2-Chat variants on ACI-BENCH, matches WizardLM-70B, and exceeds LLaMA-2-Chat-70B and WizardLM-70B on QMSum.For MS-MARCO, Orca 2 achieves high n-gram-metric performance, while models without system messages attain perplexity comparable to larger models; this dataset is in-distribution.Source: §C.2 · p. 31, Table 12 · p. 32
  • Orca-2-13B performs best among Orca 2 models on ACI-BENCH and QMSUM n-gram metrics, while no Orca 2 perplexity winner emerges.GPT-4 leads ACI-BENCH n-gram metrics, Orca-1-13B leads MS-MARCO and QMSum among baselines, and ChatGPT or LLaMA-2-Chat-13B has the lowest baseline perplexity depending on dataset.Source: §C.2 · p. 32, Table 12 · p. 32
  • Direct system messages generally outperform cautious messages across Orca 2 versions and datasets, suggesting closer alignment with human-generated summaries.Cautious prompting can produce valid but differently worded conclusions through additional reasoning, and tends to rephrase or extrapolate the source.Source: §C.2 · pp. 31–32

D Evaluation of Safety · D.1 ToxiGen MCQ · E Prompts used in Evaluation

The appendices provide further safety-evaluation results, including target-identity-group breakdowns for ToxiGen MCQ, alongside the prompts used across evaluations. Tables document neutral and toxic statement classification, while evaluation prompts primarily clarify answer formatting and preserve task inputs where applicable.

D Evaluation of Safety

  • The safety appendix expands on the experiments reported in section 6.6.It presents additional details and results for those experiments.Source: D Evaluation of Safety · p. 33

D.1 ToxiGen MCQ

  • ToxiGen MCQ results are broken down by target identity group under the discriminative evaluation regime.These results decompose the aggregated findings reported in section 6.6.Source: §D.1 · p. 33
  • Table 13 covers neutral statement classification in the ToxiGen MCQ evaluation.Source: Table 13 · p. 33
  • Table 14 covers toxic statement classification in the ToxiGen MCQ evaluation.Source: Table 14 · p. 33

E Prompts used in Evaluation

  • The evaluation appendix provides a list of prompts used to assess the models.Source: E Prompts used in Evaluation · p. 34
  • Evaluation prompts are simple hints about answer format for parsing, while prompt-formatted questions use their inputs as provided.Examples from all datasets are shown in Appendix F.Source: Table 15 · p. 34

F Illustrative Example from Evaluation Benchmarks and Corresponding Model Output

The section illustrates Orca-2-13B outputs across diverse evaluation benchmarks, including reasoning, question answering, summarization, and toxicity-related tasks. Several examples show cautious outputs without detected hallucinations, while others contain specific factual errors identified by GPT-4 judges.

Reasoning and benchmark examples

  • Orca-2-13B with a cautious system message is illustrated on AGIEval SAT math, DROP, CRASS, RACE, BBH, GSM8k, MMLU, ARC, Hellaswag, LAMBADA, and MT-Bench examples.The examples span mathematical reasoning, reading comprehension, commonsense, language understanding, completion, and open-ended analysis tasks.Source: Figure 14 · p. 35, Figure 15 · p. 35, Figure 16 · p. 36

QMSum

  • In the QMSum example, Orca-2-13B has no detected hallucination, whereas the cautious variant correctly extracts the facts but adds two incorrect facts: the project manager “goes first” and participants draw animals like “liver”.Source: Figure 30 · p. 49

MS-MARCO

  • The MS-MARCO example shows a cautious output that compares battery capacities, although GPT-4 identifies the comparison as hallucinated because capacity is specified only for Procell batteries, not Coppertop.The output describes scanning passages, identifying their main ideas, and comparing definitions before answering.Source: Figure 31 · p. 50, Figure 33 · p. 52

ToxiGen

  • The ToxiGen examples cover both multiple-choice toxicity classification and content generation for Latino, Women, Asian, and LGBTQ categories, distinguishing toxic from neutral annotations at a score threshold of 2.5.The content-generation examples contrast hateful prompts with neutral prompts.Source: Figure 34 · p. 53

Related papers

Loading 2311.11045v2…