PaperlayerView exact revision on arXiv

Source-linked AI summary

A Comprehensive Overview of Large Language Models

Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, Ajmal Mian

arXiv:2307.06435v10cs.CL

TL;DR

Rapid advances across LLM architectures, training, context length, fine-tuning, multimodality, efficiency, and applications have made the field difficult to survey. This paper addresses that challenge with a self-contained, comprehensive review, identifying major findings and design aspects across the literature while also discussing reported capabilities and ongoing challenges.

  • Problem

    Rapidly expanding LLM research spans many topics, making the field’s broader developments difficult to perceive and motivating a concise, comprehensive overview.

  • Method

    The paper conducts a self-contained survey covering architectures, training strategies, fine-tuning, efficiency, evaluation, multimodal and augmented LLMs, agents, datasets, and benchmarks.

  • Results

    The review summarizes major findings and design aspects across LLM architectures, datasets, training pipelines, fine-tuning, and zero-shot and few-shot performance.

  • Takeaways & Limitations

    The article serves as a comprehensive reference for researchers and practitioners seeking informative summaries of existing LLM work.

  • Takeaways & Limitations

    LLMs face limited and potentially obsolete pretrained knowledge, while their black-box operation limits effectiveness and trustworthiness, especially in sensitive domains.

Abstract

from arXiv · show

Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works encompass diverse topics such as architectural innovations, better training strategies, context length improvements, fine-tuning, multi-modal LLMs, robotics, datasets, benchmarking, efficiency, and more. With the rapid development of techniques and regular breakthroughs in LLM research, it has become considerably challenging to perceive the bigger picture of the advances in this direction. Considering the rapidly emerging plethora of literature on LLMs, it is imperative that the research community is able to benefit from a concise yet comprehensive overview of the recent developments in this field. This article provides an overview of the existing literature on a broad range of LLM-related concepts. Our self-contained comprehensive overview of LLMs discusses relevant background concepts along with covering the advanced topics at the frontier of research in LLMs. This review article is intended to not only provide a systematic survey but also a quick comprehensive reference for the researchers and practitioners to draw insights from extensive informative summaries of the existing works to advance the LLM research.

1. Introduction

LLMs emerged from advances in transformers, computation, and large-scale data, enabling coherent communication and broad task generalization. This survey organizes the rapidly expanding field and summarizes its architectures, training, applications, efficiency methods, and challenges.

  • Transformers, increased computational capabilities, and large-scale training data drove recent breakthroughs in language models.
  • LLMs process and generate coherent text while generalizing across multiple tasks.
  • Their broad capabilities require slow training and inference, extensive hardware, and higher running costs.
  • The survey provides a concise, comprehensive overview with fine-grained summaries of pretrained-model architectures and training details.
  • Coverage spans pre-training, fine-tuning, efficient methods, inference, evaluation, applications, challenges, multimodal LLMs, augmented LLMs, agents, datasets, and benchmarking.
  • The paper uses a loose convention in which pretrained LLMs generally have 10B parameters or more.

2. Background

The background section introduces the components, architectures, objectives, adaptation stages, and infrastructure underlying LLMs. It covers tokenization, attention, activations, normalization, distributed training, data preparation, and prompting.

  • Tokenization parses text into non-decomposing units such as characters, subwords, symbols, or words.
  • Attention computes token importance using query, key, and value mappings, weighting relevant values more heavily.
  • Activation functions discussed include ReLU and GLU variants such as GeGLU and SwiGLU.
  • Pre-layer normalization is applied before multi-head attention and is described as improving training stability.
  • Data preprocessing includes quality filtering and deduplication, while distributed training uses data, tensor, and pipeline parallelism.
  • Transformer architectures differ in how attention and transformer blocks connect, including encoder-decoder and causal-decoder designs.
  • LLM training objectives include full, prefix, masked, and unified language modeling.
  • Adaptation progresses from task-specific fine-tuning to instruction-tuning and alignment-tuning, with prompting available across stages.

3. Large Language Models

This section reviews LLMs by describing their architectures, training objectives, pipelines, datasets, and fine-tuning details.

  • The review covers LLM architectures, training objectives, pipelines, datasets, and fine-tuning details.

3.1. Pre-Trained LLMs

This survey summarizes prominent pre-trained LLMs, covering their architectures, training objectives, adaptation strategies, and reported capabilities across general-purpose, multilingual, multimodal, and coding applications.

  • The survey reviews well-known pre-trained LLMs and identifies findings that shaped research and development in natural language processing.It also summarizes architecture and training details and discusses design aspects intended to help practitioners use these models.
  • General Purpose: T5 uses unified text-to-text training, span masking, and adapter layers for downstream fine-tuning.Its encoder-decoder architecture places layer normalization outside the residual path, while masking consecutive spans with one mask produces shorter training sequences.
  • General Purpose: Reported comparisons show Gopher’s 280B model beats listed competitors on 81% of evaluated tasks, while OPT-175B is comparable to GPT-3-175B and MT-NLG beats GPT-3 on several evaluations.These results illustrate the survey’s emphasis on model-scale and training-design findings across multiple evaluations.
  • General Purpose: 70B compute-optimal training with four times more data outperforms the 280B Gopher model and other models on various downstream tasks after fine-tuning.The associated scaling relationship recommends doubling model size for every doubling of training tokens.
  • Coding: Codex generates 100 program versions through repetitive sampling and produces unit-test-passing solutions for 77.5% of the problems.Its fine-tuned variant is also reported to outperform PaLM, LLaMA, and LAMDA on HumanEval and MBPP benchmarks.

3.2. Fine-Tuned LLMs

Fine-tuned LLMs improve task performance through parameter-efficient adaptation, instruction and prompt tuning, continued pre-training, and training choices involving data, objectives, and model scaling. The surveyed findings also identify trade-offs involving convergence, safety, memorization, and computational cost.

  • Parameter-efficient tuning: Prompt fine-tuning updates few parameters and can match full-model fine-tuning, but converges more slowly.Prompt tokens can also provide additional context and aggregate information with the input text.
  • Efficiency: Mixture-of-experts and routed-expert designs reduce computation or deployment cost while preserving useful model capabilities.Randomly Routed Experts can maintain performance similar to the original model while enabling domain-specific deployment sub-models.
  • Scaling and architecture: Model and data scaling, architecture, and objectives jointly affect performance, including proportionate token scaling, sparse computation, and mixed denoising objectives.Reported findings include improved training stability, faster generation with MLA, and greater diversity from mixed denoisers.
  • Safety and trade-offs: Fine-tuning and alignment choices produce task-specific trade-offs: unfiltered data may improve downstream performance after fine-tuning but increases toxicity.RLHF further improves safety and resistance to jailbreak attacks, while adding dialogue data can reduce performance.
  • Instruction and prompt tuning: Instruction tuning improves zero-shot generalization, usability, and downstream performance across unseen tasks.Its benefits include compute-efficient improvement and stronger reasoning when combined with chain-of-thought data.

3.3. Increasing Context Window

LLMs struggle to generalize beyond training context lengths because attention is expensive and memory-intensive. The surveyed approaches address this through positional encodings and efficient attention mechanisms, each with stated limitations or design trade-offs.

  • Motivation: Limited-length training causes poor generalization to unseen inference lengths, motivating methods for context-window expansion.ALiBi supports zero-shot length extrapolation but has lower expressive power and inferior benchmark performance in the cited discussion.
  • Efficient attention: Efficient attention variants reduce the computation cost of training larger-context LLMs.LongT5 uses transient global attention over local and global tokens and pre-trains on sequences of length 4098.

3.4. Augmented LLMs

Augmented LLMs extend model capabilities by adding external context or iterative tool use. Retrieval augmentation supplies relevant information for generation, while tool augmentation enables planning, tool selection, execution, memory, and feedback loops.

  • Context augmentation: In-context learning lets LLMs adapt to unseen tasks from examples concatenated with the input, without fine-tuning.This capability supports context augmentation and few-shot prompting.
  • Retrieval augmented LLMs: Retrieval augmentation provides relevant, up-to-date external information so LLMs can answer with references and use more knowledge.The cited findings report smaller augmented models performing comparably to much larger models, including 11B versus 540B and 7.5B versus 280B comparisons.
  • Retrieval augmented LLMs: Retrieval pipelines retrieve relevant chunks or examples, encode them with the input, and may train the retriever, the LLM, or both.A study on RETRO reports that fine-tuning with RAG does not provide the gains obtained by pre-training with RAG.
  • Tool augmented LLMs: Tool-augmented LLMs iteratively decompose tasks, select tools, and act until completion rather than relying only on retrieved context.Their modules can include retrieval, tool execution, memory read-write, and feedback.
  • Tool augmented LLMs: Tool use can be represented as generated tool tokens that pause text generation and restart after tool execution outputs.ToolkenGPT concatenates tool embeddings with token embeddings and generates tool calls during inference.

3.5. LLMs-Powered Agents

LLMs serve as cognitive controllers for autonomous agents, supporting planning, reasoning, memory, and adaptation across diverse agent settings. Research combines fine-tuning, prompting, and auxiliary modules to improve these capabilities.

  • LLMs act as cognitive controllers that generate plans, reason about tasks, use memory, and adapt based on environmental feedback.
  • LLM-powered agents span web, coding, tool, embodied, and conversational applications, often requiring minimal to no fine-tuning.
  • Multi-Agents Systems: Multi-agent systems assign LLMs distinct user-defined roles so they can collaborate as domain-specific experts on complex tasks.
  • Agent architectures can combine short-term memory of recent responses with long-term memory containing summarized failed attempts for reflection.

3.6. Efficient LLMs

Efficient LLM research targets the high computational, memory, and deployment costs of large models. Parameter-efficient tuning, quantization, and pruning reduce adaptation or deployment burdens while introducing distinct performance and implementation trade-offs.

  • Parameter-Efficient Fine-Tuning: Parameter-efficient fine-tuning avoids updating all model parameters, reducing the cost of adapting models with tens or hundreds of billions of parameters.Compared with full fine-tuning, PEFT performs better in low-resource settings, comparably in medium-resource settings, and worse in high-resource settings.
  • Parameter-Efficient Fine-Tuning: LoRA learns low-rank decomposed matrices while freezing original weights, then fuses the learned weights for inference without added latency.
  • Parameter-Efficient Fine-Tuning: Prompt tuning adapts a pretrained LLM by training only 0.001%-3% additional parameters concatenated with model embeddings.P-Tuning encodes prompts through a learnable mapping, with only the prompt encoder trained.
  • Quantization: Post-training quantization requires minimal or no training while aiming to preserve model performance, but large-model outliers complicate quantization.LLM-8-bit uses full-precision multiplication for outlier-associated weights and 8-bit multiplication for the remaining features.
  • Pruning: Pruning compresses model size to reduce deployment costs, with structured and unstructured methods designed to retain reasonable performance across tasks.Wanda prunes weights using importance calculated from weights multiplied by the input norm and does not require fine-tuning.

3.7. Multimodal LLMs

Multimodal LLMs extend language models to image, video, and audio inputs, enabling richer contextual understanding and responses. Prompting is widely used because it supports specialized multimodal tasks without changing model parameters and reduces reliance on large multimodal datasets.

  • Multimodal LLMs process image, video, and audio information alongside language to produce responses with deeper contextual understanding and varied expressions.
  • Prompting: Prompting constructs multimodal LLMs by supplying context, examples, or instructions without changing model parameters.This approach can significantly reduce the need for large-scale multimodal data.
  • Prompting: Multimodal Chain-of-Thought methods prompt models to generate both reasoning processes and answers from multimodal inputs.Multimodal-CoT separates rationale generation from answer inference, while CoT-PT combines prompt tuning with visual bias.
  • Prompting: Multimodal descriptions and tools can divide complex tasks into smaller subtasks during prompting.

3.8. Summary and Discussion

The survey links LLM performance and training behavior to architectural choices and large-scale optimization strategies, while reporting important limits of generalized models. In particular, supervised models remain stronger on various NLP tasks, and instruction fine-tuning improves zero-shot generalization.

  • Architectural Design: Architectural modules such as layer normalization and positional encoding substantially affect LLM performance, training stability, time, and memory.
  • Architectural Design: BLOOM reports ALiBi outperforming learned and rotary positional encodings, whereas GLM-130B reports rotary positional encoding outperforming ALiBi.The survey concludes that the literature has no settled conclusion on positional encodings.
  • Training Strategies: Mixed precision reduces memory usage and improves training efficiency, but FP16 can cause loss spikes while BF16 offers greater range and stability with higher memory and hardware requirements.
  • Training Strategies: Training instability can be mitigated by restarting from earlier checkpoints, skipping 200-500 earlier data batches, reshuffling batches, or shrinking embedding-layer gradients.
  • Training Strategies: Lower learning rates with warmup and decay support stability, while 3D parallelism combines data, pipeline, and tensor parallelism for large-scale training.Typical learning rates are reported within 1e−4 to 8e−4.
  • Summary and Discussion: Supervised models still outperform generalized models by a large margin on various NLP tasks.
  • Zero-Shot vs Few-Shot: Instruction fine-tuning significantly improves zero-shot performance and generalization to unseen tasks, while pretraining alone is insufficient for strong zero-shot performance.

4. Model Configurations

The review summarizes pre-trained and instruction-tuned LLMs, including their publication, licensing, creators, training, and parallelism information. It separately provides architecture details for pre-trained models because instruction-tuned models retain their baselines’ architectures.

  • The review reports statistics for pre-trained and instruction-tuned models, including publication venue, license type, creators, training steps, and parallelism.
  • Architectural details are provided for pre-trained models because instruction-tuned models fine-tune pre-trained baselines and retain their architectures.
  • Pre-trained LLM summaries include model-specific information such as training data or tokens, data cleaning, and calculated training cost.
  • Instruction-tuned LLM summaries use the same abbreviation conventions as pre-trained models, while “S-” entries in Data/Tokens denote training samples.

5. Datasets and Evaluation

The review treats datasets and evaluation as central to LLM development because large-scale training data are expensive to generate and model performance depends on data quality, size, and diversity. It surveys training datasets, optimization settings, evaluation categories, and benchmarks spanning knowledge, reasoning, language understanding, generation, and dialogue.

  • Generating training and evaluation datasets is expensive because LLMs require large-scale data, making datasets and benchmarking important topics.
  • LLM performance depends substantially on training-data quality, size, and diversity, motivating pre-training and fine-tuning datasets summarized in the review.
  • The review organizes evaluation into natural language understanding and natural language generation, and compares datasets and benchmarks used across pre-trained and fine-tuned LLMs.
  • MMLU evaluates pretrained knowledge and problem solving across 57 subjects in zero-shot and few-shot settings.
  • SuperGLUE tests language understanding through question answering, inference, and coreference tasks while requiring progress in transfer, multitask, and self-supervised learning.
  • BIG-bench evaluates LLM abilities across reasoning, creativity, ethics, and domain-specific understanding.

6. Applications

The review surveys LLM applications across fields including medicine, scientific research, law, and finance. It also emphasizes that application quality remains dependent on the data used for training and that domain-specific training can improve industry-focused performance.

  • LLMs are being applied across AI research and industry in fields including medicine, scientific research, law, and finance.
  • In medicine, LLMs support clinical decision-making by analyzing patient data and medical literature to suggest diagnoses, tests, and treatments.
  • The review compares top-performing LLMs across natural language understanding and generation tasks using zero-shot or few-shot evaluation settings.
  • LLMs can assist scientific researchers by processing large datasets, formulating hypotheses, drafting documents, and improving scientific writing.
  • Domain-specific training improves industry-focused language processing, as illustrated by BloombergGPT’s performance on financial tasks and FinGPT’s open-source accessibility.

7. Challenges and Future Directions

The review identifies technical, operational, and societal challenges that accompany LLM deployment, including computational cost, scalability, privacy, hallucination, safety, and interpretability. It also describes future directions involving multimodality, transfer learning, continuous learning, ethical frameworks, and auditing.

  • LLMs face challenges in computational cost, adversarial robustness, interpretability, scalability, privacy, and real-time processing as tasks and environments become more complex.
  • Increasing computational resources improves performance, but improvement gradually decreases when model and dataset sizes remain fixed.
  • Future directions include multimodality, transfer learning, continuous learning, ethical policy frameworks, and auditing for responsible deployment.
  • Input-conflicting, context-conflicting, and fact-conflicting hallucinations describe outputs that diverge from user input, contradict earlier generations, or conflict with established world knowledge.
  • Prompt variations can substantially change model outputs, while pretrained knowledge may become obsolete and updating it through retraining is costly.
  • LLMs raise safety, controllability, security, privacy, and interpretability concerns because they may produce harmful or misleading content, leak information, and provide limited insight into their operation.

8. Conclusion

The article comprehensively reviews LLM developments and summarizes significant findings from existing literature. It analyzes design aspects including architectures, datasets, and training pipelines.

  • The article summarizes significant findings from existing LLM literature.
  • It analyzes LLM design aspects, including architectures, datasets, and training pipelines.
  • The review identifies architectural components and training strategies used by different LLMs.
Loading 2307.06435v10…