PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploring the Benefits of Training Expert Language Models over Instruction Tuning

Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, Minjoon Seo

arXiv:2302.03202v2cs.CL

TL;DR

MT LMs generalize to unseen tasks, but prior work treated the number of training tasks as a key route to stronger generalization. This paper trains and retrieves separate task experts, reporting that a single-task expert can outperform a multitask model across unseen tasks.

  • Problem

    Prior work treated scaling the number of training tasks as a key component in strengthening MT LMs, motivating a test of whether single-task experts can generalize to unseen tasks.

    Source: §1 · pp. 1–2

  • Method

    The paper trains experts independently for individual tasks and uses expert retrieval at inference, with adapters for prompt experts and full-model fine-tuning for dataset experts.

    Source: §3.1 · p. 4, §8 · p. 9

  • Results

    A single-task expert can outperform an MT LM trained on 300+ tasks on mean accuracy across 24 unseen tasks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Distributed experts are reported to avoid negative task transfer, support learning new tasks without retraining on previous tasks, and enable compositional instructions through merging.

    Source: §1 · pp. 2–3

    The experiments omit MT LMs above 11B parameters; retrieval assumes batch access to 32 unlabeled target-task examples, and composition assumes the optimal experts can be retrieved from concatenated instructions.

    Source: §7 · p. 9

Abstract

from arXiv · show

Recently, Language Models (LMs) instruction-tuned on multiple tasks, also known as multitask-prompted fine-tuning (MT), have shown the capability to generalize to unseen tasks. Previous work has shown that scaling the number of training tasks is the key component in making stronger MT LMs. In this work, we report an unexpected finding that an expert LM fine-tuned on just a single task can outperform an MT LM trained with 300+ different tasks on 11 different unseen datasets and on 13 datasets of the BIG-bench benchmark by a mean accuracy of 3.20% and 1.29%, respectively. This finding casts doubt on the previously held belief that simply scaling the number of tasks makes stronger MT LMs. Leveraging this finding, we further show that this distributed approach of training a separate expert LM per training task instead of a single MT LM for zero-shot inference possesses many benefits including (1) avoiding negative task transfer that often occurs during instruction tuning, (2) being able to continually learn new tasks without having to re-train on previous tasks to avoid catastrophic forgetting, and (3) showing compositional capabilities when merging individual experts together. The code is available at https://github.com/joeljang/ELM.

1. Introduction

The paper challenges the view that scaling the number of training tasks is key to unseen-task generalization, finding that single-task experts can outperform multitask models. It then examines how separately trained and retrieved experts address negative transfer, continual learning, and task composition.

Part 1 (Section 5)

  • 3.20% and 1.29% mean-accuracy gains over 300+-task T0-3B on 11 unseen datasets and 13 BIG-Bench datasets, respectively, came from the top single-task expert.The figure compares mean accuracy for experts trained on individual tasks with T0-3B.Source: Figure 1 · p. 1, §1 · p. 2

Part 2 (Section 6)

  • Selecting relevant experts improves mean accuracy by +10.4% on validation datasets of 36 training tasks compared with T0-3B.The paper links this benefit to avoiding negative task transfer through independent task learning.Source: §1 · p. 2
  • Independent training avoids degradation on seen tasks when adding eight experts for new generative tasks, without retraining on previous tasks.This addresses catastrophic forgetting during continual learning.Source: §1 · p. 3
  • Merging two individual-task experts outperforms mT0-3B by +2.71 mean ROUGE-L on five novel summarization and translation tasks.The paper contrasts this with MT LMs’ poor performance on concatenated compositional instructions.Source: §1 · p. 3

2. Related Work

Related work establishes instruction-tuned multitask models as a route to zero-shot generalization, while this paper proposes task features and distributed experts as complementary perspectives. It also builds on work in task-specific retrieval and model merging.

2.1. Multitask Prompted Fine-tuning of Language Models

  • Instruction tuning and scaling tasks, prompts, and model size have been used to improve zero-shot generalization; this paper argues task features may matter more than task count alone.The authors note related few-shot findings by Chan et al. (2022).Source: §2.1 · p. 3

2.2. Retrieving task-specific embeddings

  • Prior retrieval methods retrieve soft prompts for target-task adaptation; this work instead trains experts on vanilla pretrained LMs, replacing the instruction-tuning stage.Ye et al. (2022a) is identified as the motivating zero-shot retrieval approach.Source: §2.2 · p. 3

2.3. Distributed Training of Language Models

  • Prior work explores merging models trained across domains or tasks; this paper studies whether merging distributed task experts amplifies their generalization capability.The paper also explores dataset-level and prompt-level experts.Source: §2.3 · pp. 3–4, Figure 3 · p. 4

3. Expert Language Models

The framework trains task-specific experts, retrieves a relevant expert for inference, and explores merging experts. It represents tasks at prompt or dataset granularity and uses adapters or full-model fine-tuning accordingly.

3. Expert Language Models

  • The framework combines independent expert training, inference-time retrieval, and experiments on merging experts to examine distributed training.The explored expert units include prompts and datasets.Source: §3 · p. 4, §3.1 · p. 4

3.1. Training Experts

  • Prompt Experts (PE) are trained for individual prompts using adapters, while Dataset Experts (DE) are trained per dataset by fine-tuning the entire LM.For adapter experts, only the adapter feed-forward parameters are trainable; the remaining LM parameters are frozen.Source: §3.1 · p. 4

3.2. Retrieval-of-Experts (RoE)

  • Retrieval-of-Experts uses an Expert Library of training-instance embeddings linked to expert IDs, then selects the most frequently retrieved expert for target-task inference.Target instances are embedded and matched to library keys using maximum inner product search.Source: §3.2 · pp. 4–5

3.3. Merging of Experts

  • For merging, each fully fine-tuned model is represented as a task vector relative to the pretrained model, and weighted task vectors are added to pretrained parameters.Weights default to 1/N for uniform merging; in some cases, weights are selected using a held-out validation dataset.Source: §3.3 · p. 5

4. Experimental Setup

The experiments train prompt- and dataset-level experts using T0 datasets and evaluate them alongside T0 baselines on unseen tasks. The setup also includes additional generative tasks and specifies different metrics for classification and generation.

Training Setup

  • 36 training datasets and 296 prompts yield 36 Dataset Experts and 296 Prompt Experts; the original 38-dataset T0 setup was adjusted because four datasets could not be loaded.The study uses all available prompts from the Promptsource Library for each dataset.Source: §4 · p. 5
  • Classification and generative fine-tuning sample 50,000 and 10,000 instances per task, respectively, and train for five epochs at a constant 1e-4 learning rate.The Expert Library uses 100 randomly sampled training instances per expert.Source: §4 · p. 5

Evaluation Setup

  • Evaluation compares T0-3B, T0-11B, and T5-3B with experts on 11 unseen datasets and 13 BIG-Bench datasets, plus eight new generative tasks.Classification uses rank classification; generation uses ROUGE-L by default. Retrieval uses Q = 32, following a prior setting without separate S and Q ablations.Source: §4 · p. 5

5. Expert LMs Can Generalize to Unseen Tasks

The section finds that single-task experts can outperform a multitask model on unseen benchmarks, while expert selection and merging show task-dependent benefits. Its results also point to limits in the current proof-of-concept comparisons and retriever.

Experimental Scope

  • The experiments are proof-of-concept rather than exhaustive head-to-head comparisons, with broader baseline comparisons and ablations left for future work.Source: §5 · p. 5

Main Results

  • A single-task COS PE beats T0-3B by +3.20% mean accuracy on 11 unseen datasets and +1.29% on 13 BIG-Bench datasets.The 11-dataset comparison is shown in Table 1, and the BIG-Bench comparison in Table 2.Source: Table 1 · p. 6, Table 2 · p. 6, §5 · p. 6
  • Oracle ROE exceeds T0-3B, GPT-3(175B), and T0-11B by +11.94%, +4.37%, and +2.61% mean accuracy; practical ROE gains +2.05% over T0-3B.Practical ROE underperforms T0-3B by -5.37 mean score on the 8 unseen generative tasks, leaving room to improve retrieval.Source: Table 1 · p. 6, §5 · p. 6, Table 3 · p. 7
  • SAM PE beats T0-3B by +6.83 mean score on 8 unseen generative tasks, but underperforms it by -9.15% mean accuracy on unseen classification tasks.The paper uses this contrast to argue that high performance on generative tasks does not guarantee high performance on classification tasks.Source: Table 3 · p. 7, §5 · p. 7

Merging of Experts

  • Merging COS and SOC adapter experts yields mixed transfer, whereas merging full-parameter experts gives the best or second-best performance and a +0.20% mean-accuracy gain over COS DE.The authors note that using entire LM weights is computationally demanding and call for more parameter-efficient merging methods.Source: §5 · pp. 6–7, Table 4 · p. 7

Analysis of Experts

  • MCQA training tasks generally show the strongest generalization, while COSMOS-QA, SOCIAL-I-QA, and DREAM consistently yield high performance across PE and DE.The authors hypothesize that MCQA transfers well because the 11 evaluation datasets are classification tasks requiring question answering via instructions.Source: §5 · p. 7

6. Benefits of Expert LMs over MT LMs

The section presents expert LMs as an alternative to multitask LMs, reporting benefits for seen-task performance, continual learning, and compositional instructions through merged experts.

Seen Task Performance

  • +10.40% mean accuracy over T0-3B and +7.70% over T0-11B on validation datasets of 36 training tasks are reported for T5(3B) + PE W/ ROE.The authors attribute the result to retrieval being likely to select the best-performing expert for seen instructions; retrieval selected an expert from the same training dataset on 280 of 296 seen tasks.Source: §6 · p. 8

Continual Learning of New Tasks

  • With 8 new generative tasks and no access to original datasets, distributed training caused no seen-task degradation, -0.15% unseen-task degradation, and +1.08 mean performance on target tasks versus CT0-3B.It outperformed CT0-3B on 7 of 8 target tasks; the authors leave scaling to more new tasks for future work.Source: §6 · p. 8

Compositional Instructions

  • For compositional evaluation, the authors trained one English summarization expert and five translation experts, then merged the summarization expert with each translation expert.The five translation pairs were English-to-Spanish, French, Japanese, Chinese, and Korean; Table 8 provides example outputs.Source: §6 · pp. 8–9, Table 8 · p. 9
  • +2.71 mean ROUGE-L over MT0-3B was reported on five compositional tasks, with merged experts outperforming on 4 of 5 tasks.The paper attributes the result to low-resource Korean and Japanese being protected from negative transfer during distributed training. Table 7 compares summarization and translation composition using ROUGE-L.Source: §6 · p. 9, Table 7 · p. 9

7. Limitations and Discussions

The study does not evaluate MT LMs larger than 11B parameters and identifies assumptions in its inference settings for unseen and compositional tasks.

Model scale

  • The experiments do not cover MT LMs larger than 11B parameters, which may be less susceptible to negative task transfer because of greater capacity.The authors identify this as an untested model-size range.Source: §7 · p. 9

Inference assumptions

  • Unseen-task retrieval assumes batch inference with access to 32 unlabeled samples from the target task.This bounds the reported inference setting.Source: §7 · p. 9
  • Compositional experiments assume the two optimal experts can be retrieved from the concatenated instructions, an assumption that may fail for complex instructions requiring decomposition.The authors leave methods for retrieving optimal experts for future work.Source: §7 · p. 9

8. Conclusion

The conclusion reports that single-task experts can generalize strongly to unseen tasks, and summarizes benefits of distributed expert training while urging further exploration.

Conclusion

  • Single-task expert LMs show strong unseen-task generalization and surpass MT LMs trained on 300+ tasks by a non-trivial margin.The paper presents this as an unexpected finding.Source: §8 · p. 9
  • The authors report that distributed training and retrieving experts is more robust against negative task transfer, more adept at learning new tasks, and can perform compositional instructions.These are the three benefits highlighted in the conclusion.Source: §8 · p. 9
  • The conclusion urges further exploration of distributed and collaborative expert training, naming efficiency, privacy, and personalization as possible benefits not explored in the paper.Source: §8 · p. 9

A. Details of Training and Evaluation Datasets

The appendix describes 36 training datasets across eight task categories, alongside evaluation sets for unseen, BIG-bench, and generative tasks.

Training datasets

  • The experts were trained on 36 datasets across eight categories, including sentiment and topic classification, summarization, multiple-choice and extractive question answering, and closed-book question answering.The listed categories also include paraphrase identification and structure-to-text.Source: A. Details of Training and Evaluation… · p. 17

Evaluation datasets

B. Varying the Embedding Model and Text Format for Retrieval of Experts

The section tests off-the-shelf sentence embedding models and input text formats for retrieving experts. The best reported choices are ALL-MINILM-L6-V2 and a format combining the prompted instance with answer choice.

Embedding models

  • The comparison explores 13 sentence embedding models off the shelf, without supervision to train the embedding models.The models include Sentence Transformer, SimCSE, INSTRUCTOR, GTR, SentenceT5, and DiffCSE variants.Source: B. Varying the Embedding Model and… · pp. 17–18

Text formats

  • Table 10 compares text formats on 11 unseen datasets using Prompt Experts and fixes the embedding model to ALL-MINILM-L6-V2.The table evaluates 300 samples from each dataset for efficiency.Source: Table 10 · p. 18
  • The text-format comparison varies whether prompted instances, label lists, and answer choices are included or concatenated.Label lists use the actual label options, whereas answer choices format them with delimiters.Source: B. Varying the Embedding Model and… · p. 18

Results

  • ALL-MINILM-L6-V2 outperforms the other tested embedding options, while combining the prompted instance and answer choice gives the best text-format results.The authors suggest prompted instances' structure may explain the embedding-model result, and report that actual label options harm retrieval compared with answer choice formatting.Source: B. Varying the Embedding Model and… · p. 18

C. Details of Performing Compositional Instructions

The compositional-instruction examples use machine-translated XL-Sum inputs paired with target-language labels. The section also varies decoding strategies and presents examples across several target languages.

Experimental setup

  • Each language's compositional-instruction task has 400 instances, split into 300 validation and 100 test examples, with λ_i set to 1.0 for each task vector.The inputs come from XL-Sum via Google Translate, and the selected weights sum to 2.0.Source: C. Details of Performing Compositional… · p. 18

Decoding

Compositional examples

D. Full List of PE and DE ranked on the 11 unseen datasets

The section provides the full Dataset Expert and Prompt Expert rankings by mean accuracy on 11 unseen tasks.

Ranking scope

Dataset Experts

Prompt Experts

Loading 2302.03202v2…