PaperlayerView exact revision on arXiv

Source-linked AI summary

LongForm: Effective Instruction Tuning with Reverse Instructions

Abdullatif Köksal, Timo Schick, Anna Korhonen, Hinrich Schütze

arXiv:2304.08460v3cs.CLcs.AIcs.LG

TL;DR

Instruction tuning data can be costly, academically narrow, or low quality when generated entirely by LLMs. LongForm-C pairs human-written corpus passages with LLM-generated instructions, and its fine-tuned models outperform prior instruction-tuned systems and a much larger untuned model on long-text generation.

  • Problem

    Instruction-tuning data can require costly human annotation, focus on academically limited tasks, or be low quality when instructions and outputs are generated entirely by LLMs.

    Source: §1 · pp. 1–2

  • Method

    LongForm-C uses LLMs to generate instructions for human-written corpus passages, then fine-tunes language models on the resulting instruction–output pairs.

    Source: §1 · p. 2

  • Results

    LongForm models outperform prior instruction-following models by over 63% in-domain and 35% out-of-domain relative METEOR, while LongForm-OPT-2.7B beats 10× larger OPT-30B on long-text generation.

    Source: §1 · p. 2

  • Takeaways & Limitations

    LongForm-C combines reverse-instructed corpus examples with structured-corpus and NLP-task examples, and the authors report gains in both long-text generation and language understanding.

    Source: §1 · p. 2, §6 · p. 9

    The datasets and models mainly focus on long-text generation and may have limitations on structured prediction with some LLMs; LongForm models may also hallucinate like LLMs.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Instruction tuning enables language models to more effectively generalize and better follow user intent. However, obtaining instruction data is costly and challenging. Prior work employs methods such as expensive human annotation, crowd-sourced datasets with alignment issues, and generating noisy examples via LLMs. We introduce the LongForm-C dataset, which is created by reverse instructions. We generate instructions via LLMs for human-written corpus examples using reverse instructions. First we select a diverse set of human-written documents from corpora such as C4 and Wikipedia; then we generate instructions for these documents via LLMs. This approach provides a cheaper and cleaner instruction-tuning dataset with natural output and one suitable for long text generation. Our models outperform 10x larger language models without instruction tuning on tasks such as story/recipe generation and long-form question answering. Moreover, LongForm models outperform prior instruction-tuned models such as FLAN-T5 and Alpaca by a large margin, and improve language understanding capabilities further. We publicly release our data and models: https://github.com/akoksal/LongForm.

1 Introduction

The paper proposes reverse instructions to turn human-written corpus passages into instruction–output pairs, addressing the cost and quality limits of existing instruction-data approaches. LongForm-C-trained models improve long-text generation over prior instruction-tuned models and larger untuned models, with reported gains extending to multilingual generation and language understanding.

1 Introduction

  • The approach targets costly human-annotated data, academically narrow instruction coverage, and low-quality examples generated entirely by LLMs.The introduction contrasts expensive annotation with limited real-world generation coverage and noisy fully synthetic datasets.Source: §1 · pp. 1–2
  • Reverse instructions pair human-written passages from C4 and English Wikipedia with LLM-generated instructions intended to elicit those passages or similar text.The authors describe this as a cost-effective, faster alternative to human annotation that yields higher-quality output than fully synthetic generation.Source: §1 · p. 2
  • LongForm models exceed prior instruction-following models by more than 63% in-domain and 35% out-of-domain relative METEOR, while LongForm-OPT-2.7B beats 10× larger OPT-30B on long-text generation.The paper also reports improved MMLU performance over prior datasets.Source: §1 · p. 2
  • LongForm models follow multilingual instructions and generate news articles in German, Spanish, French, and Russian better than prior models.The data and models are publicly released.Source: §1 · p. 2

2 Related Work

Prior instruction-data approaches include human annotation, reformulated NLP tasks, and LLM-generated examples; the paper positions LongForm-C as combining corpus texts and LLMs for general-purpose text generation.

Instruction-following models

  • Long outputs are scarce in reformulated academic tasks: only 7 of 1,762 BigBench tasks and 12 of 1,613 NIv2 tasks have English outputs averaging over 50 words.Reformulating existing tasks extends instruction coverage but remains limited to academic tasks.Source: §2 · p. 2
  • LLM-generated instruction data broadens task coverage, but Self-Instruct’s reported validity is 54% (Wang et al., 2023b).Human-annotated datasets such as Dolly remain costly to curate.Source: §2 · p. 2

Reverse Instructions

  • LongForm-C is presented as the first work to combine corpora and LLMs to automatically create a general-purpose text-generation dataset.Source: §2 · p. 3

Long Text Generation

  • Long-text generation requires modeling long dependencies and planning, while its evaluation remains challenging (Celikyilmaz et al., 2020).Prior task-specific work draws examples from resources such as story, recipe, essay, and long-form QA datasets.Source: §2 · p. 3

Reverse Instructions

3 The LongForm-C Dataset

LongForm-C combines reverse-instructed corpus passages with structured-corpus examples and reformulated NLP tasks, pairing diverse instructions with human-written outputs.

3 The LongForm-C Dataset

  • LongForm-C contains 15,000 reverse-instruction examples and 12,739 examples from structured corpora and NLP tasks, with human-written outputs paired to varied instructions.The dataset includes human-written, template-based, and LLM-generated instructions.Source: §3 · p. 3

3.1 Reverse Instructions (RI)

  • Reverse instructions select 10,000 C4 and 5,000 Wikipedia examples, generate instructions with zero-shot GPT-3 prompts in three styles, and add output-length cues to 30% of examples.C4 documents are selected through Reddit-upvote filtering and clustering; Wikipedia examples use the first paragraph or first two paragraphs. Average example length is 291±272 words overall.Source: §3.1 · p. 4

3.2 Structured Corpora (SC)

  • Structured-corpus examples add Stack Exchange question–answer pairs and WikiHow how-to examples, with templates controlling requested steps and target-text formats.Stack Exchange sampling spans 88 subdomains; WikiHow targets combine introductions, summaries, and step descriptions in varied combinations.Source: §3.2 · pp. 4–5

3.3 NLP Tasks (NLP)

  • The NLP component selects long-output tasks including helpful answers, story QA, poems, table-to-text, stories, summaries, and fact generation, plus email writing and grammar correction.It includes 600 BigBench samples and 3,684 NIv2 samples, as well as Enron email and BEA-2019 grammar-correction tasks.Source: §3.3 · p. 5

3.4 Dataset Analysis

  • The dataset is split into 23,652 training, 2,042 validation, and 2,045 test examples; 54% come from reverse instructions, 25% from structured corpora, and 21% from NLP tasks.The analysis parses 8,872 of 15,000 reverse-instruction prompts into verb–object or auxiliary–verb pairs, whose frequent examples span diverse topics.Source: §3.4 · p. 5

4 Experimental Setup

The evaluation compares LongForm models with instruction-tuned and raw language-model baselines across in-domain and held-out long-text generation, multilingual generation, and language-understanding tasks.

4.1 LongForm Models

  • The LongForm models are T5-3B, OPT-6.7B, and LLaMA-7B variants fine-tuned on LongForm-C.Source: §4.1 · pp. 5–6

4.2 Baselines

  • Baselines include T5-11B-based T0++, Tk-Instruct, and Flan-T5, LLaMA-7B-based Alpaca, and untuned OPT-30B.The compared models have the same or more parameters than the LongForm models.Source: §4.2 · p. 6

4.3 Evaluation

  • Evaluation includes LongForm-C test data, held-out recipe generation, story generation, and ELI5 question answering, plus MMLU and multilingual news generation in four languages.METEOR is the main generation metric because the authors report higher human correlation; they also report BLEU, ROUGE, and self-BLEU.Source: §4.3 · p. 6
  • Table 2 reports METEOR by LongForm-C test subset and overall average; all LongForm models outperform prior LMs, with LongForm-LLaMA-7B achieving the best results.LongForm-T5-XL is comparable on BEA-GEC, NIv2, and BigBench.Source: Table 2 · p. 7

5 Results

LongForm models outperform prior instruction-tuned baselines across LongForm-C and out-of-domain generation tasks. Reverse Instructions improve NLG in subset and cross-model comparisons, while NLU outcomes vary by dataset combination and model.

5.1 LongForm-C Results

  • LongForm-T5-XL, with 3B parameters, outperformed T5-based instruction-tuned models with up to 11B parameters across all LongForm-C subtasks.LongForm-LLaMA-7B also outperformed LongForm-OPT-6.7B despite similar parameter counts; OPT-30B failed to follow instructions as effectively as the ten-times-smaller LongForm-T5-XL and LongForm-OPT-2.7B.Source: §5.1 · p. 7

5.2 Out-of-domain Generalization

  • 35% higher relative METEOR than Alpaca-LLaMA-7B was achieved by LongForm-LLaMA-7B, while LongForm models consistently outperformed baselines across three out-of-domain tasks.These tasks were recipe generation, long-form question answering, and short story generation; Tk-Instruct and FLAN-T5 had seen recipe-generation training data.Source: §5.2 · pp. 7–8, Table 3 · p. 7

5.3 Language Understanding and Generation

  • LongForm-C improved NLG over FLAN by +10.6 for LLaMA-7B; combining it with FLAN raised MMLU by +2.3 but lowered NLG by -3.2 versus LongForm-C alone.With LLaMA 2-7B, LongForm-C improved NLG by +6.0 and exceeded FLAN’s MMLU score by +1.2; with Mistral 7B, it outperformed FLAN and the combination on NLG (+3.1, +2.6) and MMLU (+1.1, +0.5).Source: §5.3 · p. 8, Table 4 · p. 8

5.4 Ablation of LongForm-C Subsets

  • Adding SC to NLP tasks changed NLG by +5.0 and NLU by -0.4; adding Reverse Instructions then improved NLG by +6.3 and NLU by +0.1, yielding the best results.The ablation compared NLP Tasks, Structured Corpora (SC), and Reverse Instructions (RI) subsets.Source: §5.4 · pp. 8–9, Table 5 · p. 8

5.5 Data Evaluation

  • 97 of 100 generated instructions were judged relevant; detailed corpus examples sometimes led to overlong responses to simple questions, though this occurred infrequently.The reported validity for Self-Instruct was 54%.Source: §5.5 · p. 9

6 Conclusion

The paper introduces reverse instructions and LongForm-C, a dataset combining reverse-instruction examples, structured corpora, and NLP tasks. LongForm models outperform prior instruction-tuned baselines on a wide range of long-text generation tasks.

6 Conclusion

  • LongForm-C combines reverse-instruction examples, structured corpora, and NLP tasks as an instruction-following dataset for long-text generation.The paper reports that generated reverse instructions are highly relevant to corpus examples and cover diverse tasks.Source: §6 · p. 9
  • LongForm models outperform prior instruction-tuned baselines, including FLAN-T5 and Alpaca, on a wide variety of long-text generation tasks.The conclusion presents LongForm-C as the basis for instruction-tuning these models.Source: §6 · p. 9

Ethics Statement

LongForm models generate high-quality long texts by following instructions, which the paper identifies as a possible source of misuse risk. The authors share the models and datasets to support research, judging the potential research benefits to outweigh the risks.

Ethics Statement

  • LongForm models can generate long texts such as news articles, essays, and tutorials by following instructions.Source: Ethics Statement · p. 9
  • The authors recognize that these generation capabilities may raise concerns about malicious use.Source: Ethics Statement · p. 9
  • The authors publicly share the models and datasets to facilitate research, and state that the possible research benefits outweigh the possible risks.Source: Ethics Statement · p. 9

Limitations

The paper focuses mainly on long-text generation and notes possible structured-prediction limitations for some LLMs, alongside potential hallucination problems. Training details include an end-of-instruction token for autoregressive OPT and LLaMA variants.

Limitations

  • The datasets and models mainly target long-text generation and may have structured-prediction limitations with some LLMs; hallucination problems may also occur.The paper reports that LongForm-C complements existing datasets and, with more recent models, can outperform previous datasets on structured-prediction tasks.Source: Limitations · p. 9

A Training Details

  • For autoregressive OPT and LLaMA variants, training places an [EOI] token between the instruction and output.Finetuning used two to four A6000 48GB GPUs, batch sizes of 32 to 64, DeepSpeed, and bfloat16 training.Source: A Training Details · p. 15
  • Learning rates were 5e-5 for LongForm-T5-3B and 5e-6 for LongForm-OPT-6.7B and LongForm-LLaMA-7B.Source: A Training Details · p. 15

B Evaluation with Different Metrics

The evaluation adds BLEU, ROUGE, and self-BLEU alongside METEOR, and reports similar result patterns across the evaluated metrics.

B Evaluation with Different Metrics

C Multilingual News Generation

LongForm-OPT models outperform untuned OPT-30B on METEOR, while multilingual and qualitative evaluations examine generation quality, language matching, and the instruction templates used across tasks.

C Multilingual News Generation

D Smaller LongForm Models

F.1 LLM Query

  • LLM queries use text-davinci-003 with default temperature and top_p of 1, using instruction, informal-chatbot, and search-query prompt styles.Source: §F.1 · p. 17

F.2 WikiHow Templates

  • The LongForm-C dataset uses 18 WikiHow templates phrased as verb-form questions, often specifying a number of steps.Source: §F.2 · p. 17

F.3 Enron Templates

  • Enron email-writing instructions ask for an email using a supplied subject.Source: §F.3 · p. 17

E Qualitative Examples

  • Qualitative examples compare the best-performing LongForm model with Alpaca; LongForm outputs are described as well-structured, reliable, and creative, while Alpaca examples show task-specific issues.The cited examples include factual inaccuracies, shorter story responses, and omitted recipe ingredients for Alpaca.Source: E Qualitative Examples · p. 17, Figure 5 · p. 18

F.4 BEA-GEC Templates

  • For grammar correction, each input is prepended with a randomly selected instruction.Source: §F.4 · p. 18

G Diversity Analysis: LongForm-C, Dolly, FLAN

LongForm-C's instruction types differ from the task concentrations described for Dolly and FLAN, and the paper also reports a manual analysis of its reverse-instructions subset.

G Diversity Analysis: LongForm-C, Dolly, FLAN

H Manual Analysis of the LongForm-C Dataset

I Qualitative Examples

The section presents qualitative LongForm outputs and reports comparisons of instruction diversity and model performance. It also notes corpus failure cases and reviewer concerns about model recency and the scope of applicability.

I Qualitative Examples

  • The dataset includes rare corpus examples with irrelevant metadata or excessive detail that may make them unsuitable for instruction tuning.The authors identify these as representative failure cases and suggest additional filtering as future work.Source: Table 11 · p. 20
  • Figure 7 presents multilingual generation examples and reports that LongForm can follow instructions in languages other than English.The caption says the displayed outputs were obtained after changing random seeds for up to two iterations.Source: Figure 7 · p. 21
  • Examples include a multi-city trip plan, a trolley-problem answer, an opinion statement, and a dialogue about natural language processing.The section describes the examples as illustrating trip planning, long-form writing, scriptwriting, and open-book QA.Source: I Qualitative Examples · pp. 19, 22
  • LongForm-C consistently outperforms FLAN on NLG and NLU with LLaMA 2-7B and Mistral 7B.The authors also report that LongForm-C alone achieves better NLU performance than FLAN.Source: Table 1 · p. 24, I Qualitative Examples · p. 24
  • Reviewers questioned the earlier evaluation's model recency and whether its applicability was mainly to long-form generation.These concerns are quoted in the paper's review-response material.Source: I Qualitative Examples · p. 24
  • LongForm-C spans description, explanation, analysis, and creative instructions, while Dolly and FLAN show different task concentrations.Dolly is more focused on listing, naming, extraction, and identifying; FLAN is concentrated around classification, question answering, and sentiment analysis.Source: I Qualitative Examples · p. 24
Loading 2304.08460v3…