PaperlayerView exact revision on arXiv

Source-linked AI summary

Challenges and Applications of Large Language Models

Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, Robert McHardy

arXiv:2307.10169v1cs.CLcs.AIcs.LG

TL;DR

LLM research has expanded faster than researchers can systematically track its unresolved challenges and successful applications. This paper provides an opinionated review organized around open problems and application areas, concluding that limitations in data, training, behavior, evaluation, and deployment constrain practical use.

  • Problem

    The rapid growth of LLM research makes it difficult to identify remaining challenges and fruitful application areas systematically.

  • Method

    The paper reviews unresolved LLM challenges and current applications, examining how challenges constrain applications across multiple domains.

  • Results

    The review identifies challenges including opaque datasets, fine-tuning overhead, inference latency, brittle behavior and evaluation, and context-window limits for code generation.

  • Takeaways & Limitations

    The paper aims to help researchers understand the field more quickly, transfer ideas across application domains, and target future research at identified limitations.

  • Takeaways & Limitations

    The review focuses on text-trained LLMs for technical researchers and excludes political, philosophical, and moral perspectives.

Abstract

from arXiv · show

Large Language Models (LLMs) went from non-existent to ubiquitous in the machine learning discourse within a few years. Due to the fast pace of the field, it is difficult to identify the remaining challenges and already fruitful application areas. In this paper, we aim to establish a systematic set of open problems and application successes so that ML researchers can comprehend the field's current state more quickly and become productive.

1 Introduction

The paper organizes a rapidly expanding LLM literature around unresolved challenges and current applications. It groups challenges into Design, Behavior, and Science, and surveys ten application fields for technical researchers.

  • The paper asks which LLM problems remain unresolved and where LLMs are currently being applied.
  • Challenges are grouped into three categories: Design, Behavior, and Science.
  • Applications surveyed include chatbots, computational biology, programming, creative work, knowledge work, law, medicine, reasoning, robotics, and social sciences.
  • The review targets technical researchers familiar with LLMs, focuses on text-trained models, and excludes political, philosophical, and moral perspectives.

2 Challenges

The paper surveys challenges spanning data, multitask adaptation, tokenization, and multilingual use. These include poorly understood datasets, contamination and memorization risks, transfer trade-offs, and language-dependent inefficiencies.

  • Unfathomable Datasets: Modern datasets are too large for individuals to thoroughly read or quality-assess, leaving practitioners with limited understanding of training data.
  • Unfathomable Datasets: Near-duplicates can degrade performance, while deduplication reduced emitted memorization of C4 sequences by 10x in one study.
  • Unfathomable Datasets: Training-test overlap can inflate benchmark metrics because models may memorize and regurgitate evaluation data, while finding every overlap is difficult.
  • Data Mixtures: Pre-training domain mixtures remain underexplored; poor mixtures can reduce downstream transferability and increase reliance on spurious correlations.
  • Fine-Tuning Task Mixtures: Multitask fine-tuning can improve generalization with little additional compute but may suffer negative transfer and catastrophic forgetting.
  • Tokenizer-Reliance: Tokenization can make API costs and results unequal across languages, especially disadvantaging low-resource and non-space-separated languages.
  • Tokenizer-Reliance: BPE and WordPiece iteratively learn merge rules, while byte-level BPE supports vocabulary sharing across languages.
  • Tokenizer-Reliance: Charformer learns soft subword tokenization from bytes and performs comparably to subword models with less computational overhead than other byte or subword models.

2.3 High Pre-Training Costs

LLM training is constrained by extreme compute costs and uncertain scaling behavior. The paper reviews compute-optimal training, prediction limits, parallelism, and pre-training objectives that trade data efficiency against task suitability.

  • High Pre-Training Costs: Training a single LLM can require hundreds of thousands of compute hours, millions of dollars, and substantial energy.
  • Scaling: Model size, dataset size, and compute exhibit power-law scaling with diminishing returns when one size is fixed.
  • Compute-Optimal Training Recipes: Compute-optimal training asks how model and corpus sizes should be chosen for a fixed budget, with scaling laws offering empirical guidance.
  • Compute-Optimal Training Recipes: OpenAI predicted full-size GPT-4 performance from smaller models using at most 10,000x less compute.
  • Compute-Optimal Training Recipes: Scaling-law coefficients remain debated, and pre-training loss does not always correlate well with downstream performance.
  • Masking Strategies: Attending bidirectionally is data-efficient for understanding but unsuitable for generation, whereas next-token prediction supports generation with lower data efficiency.
  • Pre-Training Objectives: Masked language modeling predicts masked tokens from bidirectional context, while span corruption replaces contiguous spans and can shorten training sequences.
  • Pre-Training Objectives: Mixture of Denoisers improves performance when continuing pre-training of LLMs for relatively few steps.

2.4 Fine-Tuning Overhead

Fine-tuning adapts pretrained LLMs effectively to downstream tasks, but full-model adaptation creates substantial memory, storage, and computation overhead. PEFT reduces updated parameters and storage requirements, yet still requires full-network gradient computation.

  • Fine-tuning adapts pretrained LLMs to specific domains or tasks using comparatively small datasets.
  • Full Fine-Tuning: Full-model fine-tuning requires storing and loading a separate model copy for each task.
  • Parameter-Efficient Fine-Tuning: PEFT updates only a small subset of parameters while keeping the base LLM fixed.
  • Parameter-Efficient Fine-Tuning: Adapters match full fine-tuning competitively on 26 text-classification tasks while updating only 3% of parameters.
  • Parameter-Efficient Fine-Tuning: (IA)3 outperforms full-model fine-tuning across datasets while updating only 0.01% of model parameters.
  • Remaining Overhead: PEFT reduces memory complexity but still requires full gradient computation, limiting applications such as personalization on smaller devices.

2.5 High Inference Latency

LLM inference remains costly because autoregressive decoding is poorly parallelizable and models have large memory footprints. The reviewed approaches reduce active computation, model size, or memory bandwidth, but some introduce practical or routing limitations.

  • LLM inference latency remains high because generation is weakly parallelizable and requires large model and decoding-state memory footprints.
  • Memory Reduction: Quantization reduces memory footprint or increases throughput by lowering the precision of weights and activations.
  • Memory Reduction: 3 to 4 bits per weight enable 175B-parameter models to run on a single GPU.
  • Sparsity: SparseGPT prunes up to 60% of parameters while maintaining roughly the same model performance.
  • Conditional Computation: Mixture-of-Experts models reduce inference computation by activating only a subset of experts for each input.
  • Limitations: Expert collapse and underconstrained routing remain issues, and learned expert assignments do not always outperform random assignments.

2.6 Limited Context Length

Limited context lengths constrain applications requiring long inputs, while quadratic attention costs and uncertain positional extrapolation hinder scaling. Efficient attention, positional schemes, and Transformer alternatives extend context, but long-context performance remains uneven.

  • Long-Context Performance: Many open-source models show severe performance degradation on long contexts despite being architecturally capable of processing them.
  • Long-Context Performance: Relevant information placed in the middle of a context can be harder for decoder-only LLMs to access than information at the beginning or end.
  • Limited context lengths hinder applications such as novel writing, textbook writing, and summarization.
  • Length Generalization: Transformer attention costs grow quadratically with sequence length, motivating models that generalize from short training sequences to much longer inference sequences.
  • Positional Embeddings: Interpolating RoPE and briefly fine-tuning RoPE-based models extends pretrained LLM context windows to up to 32,768 tokens.
  • Positional Embeddings: ALiBi trained on sequences up to 1,024 tokens matches sinusoidal-encoding perplexity on test sequences up to 2,048 tokens.

2.7 Prompt Brittleness

Prompting methods steer LLM behavior without necessarily changing model parameters, but outputs are highly sensitive to wording and demonstration order. The field offers many techniques while lacking a systematic theory of prompt design.

  • Prompt syntax and semantics can significantly change an LLM’s output, including through wording, example selection, and example ordering.
  • Open Challenge: Prompt engineering still relies heavily on experimentation, and robustness to prompt style and format remains unsolved.
  • In-Context Learning: Few-shot learning vastly outperforms zero-shot learning even when demonstration labels are randomly flipped.
  • In-Context Learning: In-context learning performs task recognition from demonstrations and can acquire unseen input-label mappings through task learning.
  • In-Context Learning: LLMs are highly sensitive to permutations in the order of few-shot demonstrations.
  • Prompting Methods: Chain-of-thought prompting constructs few-shot prompts with intermediate reasoning steps before the final answer.
  • Prompting Methods: Tree of Thoughts maintains multiple reasoning paths and supports systematic exploration with search, lookahead, and backtracking.

2.8 Hallucinations

Hallucinations are fluent outputs that contradict provided context or remain unsupported by it, making factual accuracy increasingly important as LLMs answer everyday questions. The section reviews retrieval augmentation and decoding strategies, which reduce but do not eliminate hallucinations.

  • Hallucination: Hallucinations are fluent, inaccurate outputs that either contradict source content or are under-determined by it.Intrinsic hallucinations contradict the source; extrinsic hallucinations cannot be verified from the provided source.
  • Retrieval Augmentation: Retrieval augmentation separates knowledge storage from processing by retrieving relevant passages and feeding them to the language model with the prompt.This approach grounds model inputs on external knowledge through retriever modules and large text corpora.
  • Retrieval Augmentation: Over 60% average and up to 85% out-of-distribution reductions in hallucinated responses were achieved by Shuster et al.’s best retrieval-augmented models.The results concern GPT-3 and retrieval-augmented architectures.
  • Retrieval Augmentation: Standard retrieval augmentation can still produce factually inaccurate answers even when retrieved documents are relevant.A browsing example retrieved two related papers, but the final response misstated the second paper’s title and summary.
  • Retrieval Augmentation: Retrieval can fail when no passage directly answers a question, motivating frameworks that combine retrieval augmentation with multi-turn prompting.The combined framework targets more complex questions programmatically.
  • Decoding Strategies: Decoding research links hallucinations to sampling randomness and response diversity, while uncertainty-aware methods penalize high predictive uncertainty.Dziri et al. report a positive correlation between response-generation diversity and hallucinations.

2.9 Misaligned Behavior

Misaligned behavior concerns whether LLMs conform to human values, objectives, and expectations without causing unintended harms. The section surveys detection and alignment methods, including human-feedback training, instruction tuning, RLHF alternatives, interpretability, bias mitigation, and prompt-injection risks.

  • Overview: Alignment work primarily detects misaligned behavior or trains models to align with human values, objectives, and expectations.Detection includes evaluation, auditing, interpretability, and red teaming; alignment includes human-feedback pre-training, instruction fine-tuning, and RLHF.
  • Alignment Methods: Instruction-tuned LLMs often achieve state-of-the-art downstream performance and improve over their non-instruction-tuned counterparts.Instruction fine-tuning uses natural-language instructions paired with desired responses based on human judgment.
  • Alignment Methods: RLHF uses human rankings to train a reward model and optimize a pretrained language model toward human preferences.It is commonly applied after supervised fine-tuning for complex downstream tasks.
  • Alignment Risks: RLHF fine-tuning can increase agreement with users’ political views and the expression of particular political or religious views, alongside a greater stated desire not to be shut down.These effects are reported as unwanted side effects of RLHF.
  • Alignment Methods: DPO implicitly optimizes the same objective as RLHF while replacing reward-model fitting and reinforcement learning with classification on preference data.This addresses an ongoing debate about whether the reinforcement-learning component of RLHF is necessary.
  • Red Teaming: Red-teaming methods often assume harmful behavior is precisely specified, so Casper et al. propose exploring behavior, measuring undesired behavior, and exploiting model flaws in three stages.Their approach relaxes the assumption that the adversary knows the harmful behavior precisely.
  • Emergent Capabilities: Emergent-capability research matters for identifying when unsafe abilities might arise, although some alleged emergence disappears under different metrics or better statistics.The latter finding challenges whether such abilities reflect fundamental scaling behavior.
  • Biases: Pre-training data can encode political discourse, hate speech, discrimination, and other biases, while quantitative evaluations and debiasing toolkits address these issues.Reported bias categories include race, gender, ethnicity, and age.

2.10 Outdated Knowledge

Outdated or inaccurate pre-training knowledge is costly to replace through retraining or fine-tuning. Retrieval augmentation offers updateable external memory, whereas model editing remains limited by unintended associations, weak generalization, and architecture-dependent performance.

  • Problem: Updating inaccurate or outdated pre-training knowledge is expensive through retraining and non-trivial through fine-tuning.The issue includes facts becoming outdated as the world changes, such as changes in political leadership.
  • Limitations: Model edits can create unintended associations, limiting their specificity when only one faulty or outdated fact should change.Related information should reflect the update while unrelated information remains unchanged.
  • Solutions: Model editing aims to efficiently change model behavior by modifying parameters or adding an external post-edit model.Parameter methods include locate-then-edit and meta-learning; parameter-preserving methods add post-edit components or weights.
  • Limitations: Model-editing methods show limited non-trivial generalization and varying performance across architectures; ROME and MEMIT work well empirically only on decoder-only LLMs.These limitations constrain performance and applicability across model architectures.
  • Solutions: Retrieval-augmented language modeling can update knowledge at inference time by hot-swapping non-parametric indices.An updated memory enabled answering questions about world leaders who changed between memory-collection dates.

2.11 Brittle Evaluations

LLM evaluations are brittle because small changes to prompts, problems, or protocols can reverse results, making holistic capability assessment difficult. Broader benchmark suites and human-oriented tests improve coverage, but evaluation choices still materially affect reported performance.

  • Challenge: Small changes to a benchmark problem or prompt can produce opposite results because LLMs have an uneven capabilities surface.This makes it difficult to infer related capabilities from success on one problem.
  • Evaluation Approaches: Holistic suites such as HELM standardize evaluation across scenarios and tasks while seeking broad coverage of capabilities and risks.LLMs are also increasingly tested on human examinations and competition mathematics.
  • Evaluation Approaches: GPT-4 achieves human-level performance on several tests in the AGIEval benchmark.AGIEval rigorously tests LLM abilities on exams such as the SAT, LSAT, and mathematics competitions.
  • Evaluation Limitations: MMLU benchmark results vary significantly with the evaluation method used for multiple-choice answers.Methods include checking the generated first token or comparing log-probabilities of answer choices.
  • Evaluation Limitations: Larger and instruction-fine-tuned models are more sensitive to small prompt variations in some evaluations.Prompt changes include appending requests such as “Please answer yes or no”.

2.12 Evaluations Based on Static, Human-Written Ground Truth

Static, human-written ground truth and benchmarks are increasingly inadequate for evaluating capable LLMs, especially in domains lacking reliable expert references. The review highlights regularly updated tasks, model-generated evaluation data, and model-based judging as alternatives, while noting important robustness limitations.

  • Human-written ground truth is scarce in expert domains such as programming and mathematics, making human-level comparisons harder as models surpass benchmark performance.
  • Benchmark datasets become outdated, saturated, or overfit as model capabilities change, reducing their usefulness for measuring further improvement.
  • BIG-Bench addresses benchmark aging by regularly admitting new tasks, including programmatically evaluated tasks.
  • Model-generated evaluation tasks: LLMs can generate benchmark questions and evaluation prompts, with reward models filtering generated datasets for quality.
  • Model-generated scores: Model-based judges can grade answers or compare their relative quality, but results vary substantially with the judge model and remain poor substitutes for human judgment.

2.13 Indistinguishability between Generated and Human-Written Text

The review examines how to distinguish LLM-generated from human-written text through post-hoc detectors and watermarking. Paraphrasing attacks undermine both approaches, although retrieval from stored generations can remain fairly robust because paraphrasing preserves semantics.

  • Detecting whether text was generated by an LLM matters for limiting misinformation, plagiarism, impersonation, identity theft, scams, and fraud.
  • The field uses post-hoc detectors for arbitrary text and watermarking schemes that modify generation to make detection easier.
  • Post-hoc Detectors: DetectGPT identifies generated passages by exploiting their tendency to occupy negative-curvature regions of a model’s log-probability function.
  • Watermarking: Watermarking hashes preceding tokens, partitions the vocabulary into green and red lists, and excludes red-list tokens during generation.
  • Watermarking: Watermarks remain detectable after rewriting or document mixing given sufficient tokens, making the required text length a crucial metric.
  • Paraphrasing Attacks: Paraphrasing removes revealing LLM signatures while approximately preserving meaning, enabling attacks against machine-generated text detectors.
  • Paraphrasing Attacks: 70.3% to 4.6%: an 11B paraphrase model sharply reduced DetectGPT’s detection accuracy, while database retrieval remained fairly robust because semantics were preserved.
  • Paraphrasing Attacks: Detection remains unreliable in practice and theory, since paraphrasing can defeat watermarks and repeated queries can expose schemes for spoofing detectors.

2.14 Tasks Not Solvable By Scale

The review surveys tasks and research practices that challenge simple scaling narratives, including inverse or uneven scaling, memorization effects, high-dimensional design choices, and uncontrolled experiments. It also notes that many proposed limits remain speculative and that larger models do not uniformly close compositionality gaps.

  • Inverse Scaling is task performance worsening as model scale and training-loss performance increase, with proposed causes including memorization, undesirable imitation, distractor tasks, and spurious correlations.
  • Four of eleven tasks remained inverse-scaling at higher compute, while six exhibited U-shaped scaling with later performance recovery.
  • Compositional tasks: Larger GPT-3-family models solve subproblems faster than composed problems, leaving the compositionality gap unimproved.
  • Memorization vs. Generalization: Counterfactual tasks expose memorization-like effects: performance declines as altered input-output conditions become less common.
  • Tasks Not Solvable By Scale: The existence of tasks permanently unsolvable by scaling remains speculative, so the review examines possible patterns rather than asserting specific examples.
  • Lack of Controlled Ablations: Many LLM papers lack controlled ablations because training enough models is prohibitively costly, weakening scientific conclusions about which design choices matter.
  • Uncontrolled Experiments: One-factor-at-a-time ablations are expensive, while zero-shot hyperparameter transfer offers a cheaper potential remedy.
  • Curse of Dimensionality: LLM design spaces become high-dimensional across data, tokenization, objectives, architecture, and hyperparameters, making factorial experiments combinatorially expensive.

2.16 Lack of Reproducibility

LLM research faces repeatability problems in both distributed training and closed-source API inference. Nondeterministic training, changing black-box services, and stochastic outputs can make empirical results difficult to reproduce exactly.

  • Repeatability concerns both training runs and generations from closed-source API-served models, defined here as repeating experimental outcomes exactly.
  • Training Repeatability: Distributed training can be nondeterministic because of scheduling, communication, parallelism, floating-point precision, and other randomness sources.
  • Training Repeatability: Web-indexed pre-training datasets can change between curator collection and user download, compromising dataset reproducibility.
  • Inference Repeatability: Black-box API providers can change models without public notice, restrict access to older versions, and produce stochastic outputs even at zero temperature.
  • Inference Repeatability: 97.6% to 2.4%: GPT-4’s prime-number identification accuracy changed dramatically between March and June 2023, while GPT-3.5 improved over the same period.
  • Irreproducible API Inference: Using exclusively open-source LLMs is presented as an easy fix for API-related reproducibility problems.

3 Applications

LLMs are being applied across domains including dialogue, code, genomics, creativity, summarization, knowledge work, law, medicine, and reasoning, with results varying substantially by task and prompting strategy. These applications also expose practical constraints involving context windows, latency, reliability, quantitative reasoning, and access to internal data.

  • Cross-domain architectures: LLMs support diverse application architectures, including retrieval-augmented dialogue, synthetic conversation generation, and external-code or tool use through API definitions.API definitions can reduce required context, while LLM-generated functions and self-debugging extend the framework.
  • Scientific applications: Domain-specific models can substantially improve specialized tasks, with xTrimoPGLM outperforming existing approaches on 13 of 15 protein tasks and HyenaDNA modeling sequences up to 1 million tokens.HyenaDNA also outperforms Transformer-based models with multiple orders of magnitude more parameters while retaining in-context learning capabilities.
  • Deployment constraints: Practical deployment remains constrained by context-window limits, inference latency, reliability, missing internal data, and weak quantitative-task performance.These constraints affect code generation, chatbot responsiveness, email management, and potential knowledge-work applications such as financial services or accounting.
  • Knowledge and creative work: Application results range from near-human creative and summarization performance to weaker quantitative and academic-review capabilities.LLMs performed comparably to 100 human participants on idea generation, while GPT-4 detected 7 of 13 paper errors and had limited ability to distinguish better abstracts.

4 Related Work

The paper situates itself among broad and domain-specific surveys of LLMs, including work on augmented models, AutoML, detection, and applications such as chatbots, medicine, law, and programming.

  • Concurrent surveys cover large language models and associated topics, augmented language models, AutoML, and LLM-generated text detection.
  • Previous domain-specific surveys address chatbots, computational biology, programming, medicine, law, knowledge work, and reasoning.

5 Conclusion

The paper identifies unresolved LLM challenges, surveys current applications, and discusses how those challenges constrain application areas. It aims to support future research and transfer ideas across domains.

  • The paper identifies several unsolved challenges, overviews current applications, and discusses how challenges constrain applications.
  • The authors hope that highlighting limitations will foster future research addressing them.
  • Reviewing approaches across applied areas is intended to facilitate transfer of ideas between domains and target further research.
Loading 2307.10169v1…