PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models

Guangji Bai, Zheng Chai, Chen Ling, Shiyu Wang, Jiaying Lu, Nan Zhang, Tingwei Shi, Ziyang Yu, Mengdan Zhu, Yifei Zhang, Xinyuan Song, Carl Yang, Yue Cheng, Liang Zhao

arXiv:2401.00625v4cs.LG

TL;DR

LLMs’ growing resource demands make efficient training and deployment challenging, while existing approaches and evaluations lack a cohesive, specialized framework. This survey organizes efficiency techniques across resource types and the LLM lifecycle, and reviews evaluation benchmarks and metrics. It concludes that holistic system design and practical evaluation matter to resource efficiency.

  • Problem

    LLMs demand substantial computational, memory, energy, and financial resources, while existing efficiency evaluations rely heavily on general NLP benchmarks that may not capture efficiency-specific concerns.

    Source: §1 · p. 2, §9.2 · p. 40

  • Method

    The survey reviews resource-efficient LLM techniques across the lifecycle and organizes them by resource type, alongside benchmarks and evaluation metrics.

    Source: §1 · pp. 4–5

  • Results

    The survey concludes that holistic system design must consider both hardware and software, and that practical applications and evaluations connect techniques to the resource taxonomy.

    Source: §11 · p. 44

  • Takeaways & Limitations

    The survey’s resource taxonomy and evaluation framework provide a structured basis for comparing efficiency methods and identifying open research directions.

    Source: §1 · pp. 4–5

    Many LLMs use proprietary training data or restrict parameter access, limiting some efficiency-improvement techniques and direct optimization.

    Source: §1 · p. 3

Abstract

from arXiv · show

The burgeoning field of Large Language Models (LLMs), exemplified by sophisticated models like OpenAI's ChatGPT, represents a significant advancement in artificial intelligence. These models, however, bring forth substantial challenges in the high consumption of computational, memory, energy, and financial resources, especially in environments with limited resource capabilities. This survey aims to systematically address these challenges by reviewing a broad spectrum of techniques designed to enhance the resource efficiency of LLMs. We categorize methods based on their optimization focus: computational, memory, energy, financial, and network resources and their applicability across various stages of an LLM's lifecycle, including architecture design, pretraining, finetuning, and system design. Additionally, the survey introduces a nuanced categorization of resource efficiency techniques by their specific resource types, which uncovers the intricate relationships and mappings between various resources and corresponding optimization techniques. A standardized set of evaluation metrics and datasets is also presented to facilitate consistent and fair comparisons across different models and techniques. By offering a comprehensive overview of the current sota and identifying open research avenues, this survey serves as a foundational reference for researchers and practitioners, aiding them in developing more sustainable and efficient LLMs in a rapidly evolving landscape.

1 Introduction

LLMs’ growing scale raises computational, energy, memory, and financial demands, while the field lacks a cohesive framework for comparing resource-efficiency methods. The survey reviews existing work, organizes techniques and resources, and outlines evaluation resources and open questions.

1 Introduction

  • LLM scale drives high resource demands, while autoregressive generation, quadratic attention, memory constraints, and multi-resource evaluation complicate efficiency.The introduction also identifies diminishing returns, generalization concerns, proprietary data, and closed-model access as challenges.Source: §1 · pp. 2–4
  • The survey reviews resource-efficient LLM techniques across the lifecycle and organizes them by resource type, alongside standardized evaluation metrics and datasets.It also discusses bottlenecks, unresolved challenges, and future research directions.Source: §1 · pp. 4–5

1.1 Related work

  • Prior surveys cover foundational LLM topics, compression and acceleration, or efficient deep neural networks, but the introduction identifies gaps in comprehensive, current LLM-specific coverage.The text notes that deep-neural-network techniques do not directly transfer to LLMs because of their scale and transformer architecture.Source: §1.1 · pp. 5–6

1.2 Outline

  • The survey outline spans preliminaries and taxonomy, architecture, pretraining, fine-tuning, inference, system design, resource-based technique evaluation, and benchmarks and metrics.It also includes a closing section on open challenges and future directions.Source: §1.2 · p. 6
  • The resource taxonomy covers computation, memory, energy, money, and network communication across LLM use.The outline identifies system design and benchmarking as additional survey topics.Source: §1.2 · p. 6

2 Preliminary and taxonomy

The section introduces transformer and LLM preliminaries, then frames resource efficiency through five resource domains and a lifecycle-oriented taxonomy of techniques. The taxonomy spans architecture design, pretraining, fine-tuning, inference, and system design.

2 Preliminary and taxonomy

  • The section first establishes transformer and LLM preliminaries before introducing its taxonomy of efficiency and acceleration techniques.The taxonomy is presented as the framework for the survey’s subsequent discussion.Source: §2 · p. 7

2.1.1 Transformer model

  • Transformers use self-attention to weight input words, with query, key, and value matrices and key-vector dimension defining the operation.Multi-head attention computes multiple attention heads in parallel and concatenates their outputs.Source: §2.1.1 · pp. 7–8

2.1.2 Large Language Models (LLMs)

  • LLMs follow pretrained-language-model designs and may be encoder-only, encoder-decoder, or decoder-only.The text associates encoder-only models with understanding tasks, encoder-decoder models with sequence-to-sequence tasks, and decoder-only models with autoregressive generation.Source: §2.1.2 · pp. 8–9

2.2.1 Taxonomy of key resources involved with using LLMs.

  • Resource efficiency is organized across computation, memory, energy, money, and network communication.The section describes computation in terms of processing power, memory as RAM and storage, and energy as electricity consumed during the model lifecycle.Source: §2.2.1 · pp. 9–10

2.2.2 Taxonomy of techniques for resource-efficient LLMs

  • The survey organizes efficiency techniques into five lifecycle categories: architecture design, pretraining, fine-tuning, inference, and system design.The taxonomy’s branches include efficient architecture types, memory and data efficiency, fine-tuning approaches, inference strategies, and deployment and infrastructure.Source: §2.2.2 · p. 10, Figure 1 · p. 11

3 LLM architecture design

The section surveys architecture-level approaches to efficiency, including approximate and hardware-optimized attention and non-transformer alternatives. These methods target computational or memory demands through changes to attention, model structure, or execution.

3 LLM architecture design

  • The section presents both efficient transformer designs and alternative architectures as approaches to reducing resource demands for practical LLM deployment.Its discussion includes Reformer, Linear Transformer, AFT, KDEformer, hardware-optimized attention, and non-transformer models.Source: §3 · pp. 11–12

3.1 Efficient transformer architecture

  • Efficient transformer work focuses on throughput, with attention identified as a major influence on processing speed.Source: §3.1 · p. 12

3.1.1 Approximate attention.

  • Approximate attention methods target quadratic attention costs with alternative time or memory complexities, including linear-time AFT-simple and memory-efficient attention.The section also describes Reformer, Linear Transformer, EfficientAttention, and KDEFormer; Table 1 summarizes selected complexity changes over classical Transformer.Source: §3.1.1 · pp. 12–13, Table 1 · p. 14

3.1.2 Hardware optimized attention.

  • Hardware-optimized attention work improves implementation through kernel fusion, matrix multiplication optimization, IO-aware tiling, work partitioning, and paging.The section names FlashAttention, FlashAttention-2, vLLM’s PagedAttention, and MobileLLM among the approaches.Source: §3.1.2 · p. 14

3.2 Non-transformer architecture

  • Alternative architectures have emerged alongside dominant Transformer models to address challenges or provide different advantages.Source: §3.2 · p. 14

3.2.1 Modular network

  • Mixture-of-Experts models use sparse routing to activate selected experts, allowing parameter growth while reducing computation and memory requirements.GLaM is described as having 1.2T parameters, seven times GPT-3’s size, while requiring half the FLOPs for inference.Source: §3.2.1 · pp. 14–15

3.2.2 Other architecture

  • Non-transformer architectures include RWKV, Hyena, Mamba, and other designs that replace or avoid standard attention mechanisms.The section describes Mamba as scaling linearly in sequence length and YOCO as reducing GPU-memory scaling through one-time key-value caching.Source: §3.2.2 · pp. 15–16

4 LLM pre-training

The survey reviews resource-efficient LLM pre-training, covering distributed computation, mixed precision, and ways to use training data more efficiently. It groups data-efficiency work around importance sampling, augmentation, and training objectives.

4 LLM pre-training

  • The pre-training overview combines advanced hardware and data/model parallelism with selective sampling, pruning, and quantization to reduce computational, data, and memory demands.Source: §4 · p. 16

4.1.1 Distributed training

  • Distributed training splits work across nodes; data parallelism partitions data but duplicates model states, while model parallelism spreads layers or tensors across accelerators.ZeRO, PaLM, and Fairscale partition model states to reduce data-parallel memory redundancy.Source: §4.1.1 · p. 16

4.1.2 Mixed precision training

  • Mixed precision uses 16-bit and 32-bit floating-point types to accelerate training, with BF16 allocating more exponent bits than FP16 to mitigate performance degradation.The survey reports that 16-bit numbers reduce memory consumption compared with 32-bit or 64-bit numbers.Source: §4.1.2 · p. 17

4.2 Data efficiency

  • Data efficiency concerns how effectively training data is used, affecting the number of steps and overall training cost.Source: §4.2 · p. 17

4.2.1 Importance sampling

  • Importance sampling prioritizes informative training instances by estimating per-sample importance; gradient norms are one estimation approach.More recent methods aim to accelerate the importance-sampling process.Source: §4.2.1 · p. 18

4.2.2 Data augmentation

  • Data augmentation creates modified copies of existing data; examples include augmenting interaction graphs and profiles for recommendation and generating diverse data for few-shot entity recognition.The passage also reports downstream benefits from augmentation in vision-language pre-training.Source: §4.2.2 · p. 18

4.2.3 Training objective

  • Pre-training objectives depend on architecture, input/target construction, and masking strategy; skipping some masked-token processing can improve training efficiency.Source: §4.2.3 · p. 19

5 LLM fine-tuning

The survey presents fine-tuning strategies that trade parameter updates and resource use against task performance. It covers parameter-efficient methods and memory-oriented optimizations for full-parameter fine-tuning.

5.1 Parameter-efficient fine-tuning

  • Parameter-efficient fine-tuning updates only part of an LLM’s parameters; masking-based methods freeze the rest, while adapter-based methods train added lightweight layers.LoRA adds low-rank matrices alongside frozen pre-trained weights and introduces no inference latency [125].Source: §5.1 · pp. 19–21

5.2 Full-Parameter fine-tuning

  • Full-parameter fine-tuning generally costs more than PEFT but can perform better, though that advantage may not hold on simple, language-limited datasets.One report puts its time cost at about 3–5 times that of LoRA [127].Source: §5.2 · p. 21
  • HiFT enables fine-tuning 7B models on 24GB devices without additional memory-saving techniques by updating parameter blocks, while LOMO and MeZO also reduce memory demands.LOMO fine-tunes a 65B model with less than 192GB GPU memory; MeZO trains a 30B model with 55GB GPU memory.Source: §5.2 · p. 21
  • Full-parameter fine-tuning can distort pre-trained features and underperform on data distributions not seen during fine-tuning.Source: §5.2 · p. 22

6 LLM inference

The survey frames inference efficiency as reducing computation and memory use while maintaining output quality. It reviews compression and dynamic methods for resource-constrained deployment.

6 LLM inference

  • Inference applies trained LLMs to text generation, question answering, and other language tasks.Source: §6 · p. 22
  • The section focuses on reducing computational load and memory usage while maintaining high-quality outputs.Source: §6 · p. 22
  • Approaches include pruning, quantization, and dynamic inference that adjusts computation to input data.Source: §6 · p. 22

6.1 Model compression

The survey reviews model compression approaches intended to reduce storage, latency, and inference costs. It covers pruning, quantization, distillation, and low-rank approximation, with trade-offs shaped by deployment needs.

6.1.1 Pruning

  • Pruning removes model structures or individual weights; structured pruning is more hardware-friendly, while unstructured sparsity may require specialized deployment support.Source: §6.1.1 · p. 22
  • ZipLM iteratively shrinks weight matrices to meet achievable target speedups on specified hardware, while contextual pruning uses input-dependent parameters to approximate dense-model outputs.Deja Vu dynamically predicts contextual sparsity and uses asynchronous, hardware-optimized execution.Source: §6.1.1 · pp. 23–24

6.1.2 Quantization

  • Quantization represents weights or activations with fewer bits to reduce memory and computation, accepting some precision loss that careful methods can limit.The survey also describes fixed-point quantization for weights and activations.Source: §6.1.2 · pp. 24–25

6.1.3 Knowledge distillation

  • White-box distillation uses teacher parameters, whereas black-box distillation relies on predictions; black-box methods generate instruction-following queries and answers for smaller models.A cited white-box example compresses a 1.5-billion-parameter model to 70 million parameters for click-through-rate prediction.Source: §6.1.3 · p. 26

6.1.4 Low-rank approximation

  • SVD reduces low-rank matrix storage from O(mn) to O((m + n) × r); the survey also describes low-rank methods for quantization error and model storage.Source: §6.1.4 · pp. 26–27

6.2 Dynamic acceleration

Dynamic acceleration reduces inference work without removing model parameters, through early exit, input pruning, or token parallelism. The surveyed methods use varied criteria and mechanisms, with reported speedups in specific settings.

6.2 Dynamic acceleration

  • Dynamic acceleration methods comprise early exit, token pruning, and token parallelism, which respectively shorten model depth, input length, or sequential token generation.Unlike static methods that reduce parameters, these methods can depend on the input sequence.Source: §6.2 · p. 28

6.2.1 Early exit

  • Early-exit methods use criteria such as prediction entropy, softmax scores, stable intermediate predictions, confidence with patience, or layer-wise input similarity.MuE adapts early exit to multimodal LLMs using layer-wise input similarity because encoder-decoder dependencies complicate exit decisions.Source: §6.2.1 · pp. 28–29

6.2.2 Input pruning

  • Input pruning scores tokens for removal or retention, or inserts prediction modules before layers; some added modules can introduce inference-latency overhead.Examples include importance-based top-k retention, learned thresholds, prediction modules, and multimodal token merging.Source: §6.2.2 · p. 29
  • LazyLLM progressively prunes tokens during prefilling and decoding, reporting 2.34× TTFT speedup on Llama 2 multidocument QA without sacrificing accuracy.GRIFFIN reports 1.25x–1.29x speed-ups on Llama 2 and Gemma, while LLMLingua-2 reports 1.6x–2.9x latency reductions at 2x–5x compression.Source: §6.2.2 · p. 30

6.2.3 Token parallelism

  • Speculative decoding uses a smaller approximation model to propose prefixes that a larger target model expands in parallel while preserving the output distribution.SpS generates multiple tokens per transformer call and reports 2 to 2.5 times acceleration without altering the model.Source: §6.2.3 · p. 30
  • 3.16x lower single-batch decoding latency was reported for staged speculative decoding on a 762M-parameter GPT-2-L without compromising output quality.The method structures speculative batches as trees and adds a stage of speculative decoding.Source: §6.2.3 · p. 31

7 System design

The section surveys system-level approaches for efficient LLM inference and deployment, including hardware offloading, collaborative inference, support infrastructure, and edge-device methods.

7.1 Deployment optimization

  • Hardware offloading moves temporarily unused LLM data from accelerators to larger, slower storage, enabling operation with restricted GPU memory but adding communication overhead.FlexGen searches offloading strategies and compresses weights and KV cache to 4 bits; ZeRO-Inference streams layers from DRAM or NVMe into GPU memory as needed.Source: §7.1 · p. 31
  • Collaborative inference pools participants’ computing resources or data; PETALS distributes model layers across servers and uses 8-bit compression.Other deployment approaches include priority-based KV offloading, low-bit inference co-design, and smartphone I/O optimization.Source: §7.1 · p. 32

7.2 Support infrastructure

  • Support infrastructure includes libraries and frameworks for parallelism, heterogeneous memory management, distributed training, and inference-serving simulation.Examples include DeepSpeed, Megatron-LM, Colossal-AI, Mesh-TensorFlow, GPT-NeoX, MaxText, Alpa, and LLMServingSim.Source: §7.2 · pp. 32–33
  • Edge-device approaches address limited resources through memory and computation reductions, federated learning, model partitioning, and mobile inference optimization.Examples include MobileLLM’s depth-over-width design, EdgeShard’s heterogeneous edge-cloud partitioning, and Any-Precision LLM’s multiple bit-widths within a single-model-like memory footprint.Source: §7.2 · pp. 33–34

7.3 Other systems

  • Tabi routes heterogeneous queries to multiple DNNs rather than invoking the same costly LLM for every query.The section also notes min-hash for more efficient and scalable near-duplicate sequence search.Source: §7.3 · p. 34

8 Technique categorization by resources

The survey maps efficiency techniques to computation, memory, energy, financial cost, and network communication, distinguishing direct from indirect impacts. It describes resource-specific techniques and relationships across these categories.

8.1 Computation efficiency

  • Approximated and hardware-aware attention and pruning directly target computation, while data parallelism and parameter-efficient fine-tuning contribute indirectly.These approaches reduce operation complexity, tailor computation to hardware, remove redundant computation, distribute workloads, or reduce parameter updates.Source: §8.1 · pp. 34, 36

8 Technique categorization by resources

  • The resource mapping distinguishes techniques’ direct and indirect impacts across five resource categories: computation, memory, energy, financial cost, and network communication.Table 2 uses bold checkmarks for direct impacts and regular checkmarks for indirect impacts.Source: §8 · p. 34, Table 2 · p. 35

8.2 Memory efficiency

  • Pruning and quantization directly improve memory efficiency by reducing model size, while distillation and distributed training also contribute to memory management.Distributed data and model parallelism spread memory use across devices.Source: §8.2 · p. 36

8.3 Energy efficiency

  • Structured pruning and quantization reduce energy use by reducing operations and data size; contextual pruning and approximated attention also reduce unnecessary computation.The section describes contextual pruning and approximated attention as energy-saving techniques.Source: §8.3 · p. 36

8.4 Financial cost efficiency

  • Data-efficiency methods and dynamic inference can indirectly reduce monetary costs, while SpotServe reports 54% lower costs and improved latency versus existing systems.SpotServe adapts parallelization to instance availability and workload fluctuations on preemptible GPU instances.Source: §8.4 · pp. 36–37

8.5 Network communication efficiency

  • Mixed-precision training and weight quantization directly reduce communicated data, while collaborative inference can indirectly optimize network data transfer and processing.These techniques address communication efficiency in distributed training and cloud-based deployment.Source: §8.5 · p. 37

9.1 Evaluation metrics

The survey presents metrics for evaluating resource efficiency across computation, memory, energy, financial cost, network communication, and other trade-offs. It emphasizes measures that can inform model selection and comparisons.

9.1.1 Computation

  • Computation metrics include FLOPs, training time, inference latency, throughput, and speed-up ratio.Latency can be end-to-end or next-token generation latency; throughput can be measured in tokens or queries per second.Source: §9.1.1 · pp. 37–38

9.1.2 Memory

  • Memory metrics include parameter count and model size, with model size often measured through peak memory usage during training or inference.The section notes that larger models may not fit standard hardware memory.Source: §9.1.2 · p. 38

9.1.3 Energy

  • Energy consumption can be reported in watt-hours or joules, while carbon emissions can be estimated from energy use and local grid carbon intensity.The survey encourages reporting energy and carbon across the LLM lifecycle, including inference, whose cumulative costs may matter for intensively used models.Source: §9.1.3 · pp. 38–39

9.1.4 Financial cost

  • The proposed dollars-per-parameter metric divides total training or running cost by parameter count to support fairer cost-efficiency comparisons.The survey also suggests accounting for hardware, software, electricity, cloud compute, and personnel costs.Source: §9.1.4 · p. 39

9.1.5 Network communication

  • Communication volume measures the data transmitted across a network during LLM execution or training, typically using traffic monitoring or system logs.The section notes that high communication costs can slow distributed training and increase infrastructure expenses.Source: §9.1.5 · p. 39

9.1.6 Other metrics

  • Other metrics include compression ratio, teacher-student loyalty or fidelity, and robustness measures such as after-attack accuracy and query count.The survey also describes Pareto optimality as a framework for balancing resource efficiency against task performance.Source: §9.1.6 · pp. 39–40

9.2 Benchmarks

The section reviews benchmarks that combine model quality with efficiency measures, while noting that existing evaluations do not yet provide comprehensive, specialized coverage of LLM resource efficiency. The examples span multiple tasks and resource dimensions.

Dynaboard and EfficientQA

  • Dynaboard combines quality with memory, throughput, fairness, and robustness metrics; EfficientQA compares QA systems under model-size and accuracy constraints.EfficientQA tracks the most accurate model under 6 GB, the most accurate under 500 MB, and the smallest model reaching 25% accuracy.Source: §9.2 · p. 40

SustaiNLP Shared Task

  • SustaiNLP evaluates inference energy consumption alongside SuperGLUE performance across eight NLU tasks.It uses experiment-impact-tracker to measure energy consumption during inference.Source: §9.2 · p. 41

ELUE

  • ELUE evaluates model performance, average FLOPs, and parameter counts across six datasets, and accepts Python model-definition files for online evaluation.Its upload requirement is less costly for users than submitting a containerized model and environment.Source: §9.2 · p. 41

VLUE

  • VLUE evaluates vision-language models on task performance and inference time, exposing their efficiency-performance trade-off.Its tasks include image-text retrieval, visual question answering, visual reasoning, and visual grounding.Source: §9.2 · p. 41

Long-Range Arena

  • Long-Range Arena tests efficient Transformers on 1,000–16,000-token contexts across modalities and reasoning tasks, under memory or execution-time constraints.Tasks include text, natural language, synthetic images, and mathematical expressions.Source: §9.2 · p. 41

Efficiency-aware MS MARCO

  • Efficiency-aware MS MARCO adds per-query latency and cost budgets to accuracy, aggregating the measures into a Dynascore.The leaderboard applies this approach to the MS MARCO information retrieval benchmark.Source: §9.2 · p. 41

10 Open challenges and future directions

The survey identifies open challenges spanning trade-offs among resource types, combining optimization methods, evaluating efficiency consistently, and maintaining trustworthy, deployable LLMs. It also highlights automation, edge deployment, and theoretical understanding as research directions.

10.1 Managing resource type disagreements

  • Efficiency methods can improve one resource dimension while worsening another, motivating holistic strategies that balance computation, parameter count, and memory.For example, reducing computation may increase memory demands when sparse data structures are involved.Source: §10.1 · p. 42

10.2 Combining techniques for resource efficiency

  • Research on cohesively combining methods for different aspects of LLM resource efficiency remains scarce.The section points to integrating energy analysis into software development and resource allocation with task assignment as relevant ideas.Source: §10.2 · p. 42

10.3 Standardized and unified evaluation

  • Existing efficiency benchmarks lack comprehensive, consistent comparisons because studies vary in speed-up, parameter counts, accuracy, and hardware.The proposed need is for standardized benchmarks covering computational resources, memory, energy, and processing power.Source: §10.3 · pp. 42–43

10.4 Explainability and robustness

  • Efficient LLM methods can improve reasoning performance without clarifying why they work, motivating explainability and robustness research [289] [290].The section connects transparency with reliability and predictability in real-world applications.Source: §10.4 · p. 43

10.5 AutoML for resource-efficient LLMs

  • Distillation, pruning, weight sharing, and low-rank factorization often rely on expert heuristics, motivating Meta-Learning and NAS to automate optimization [180].The section notes that designing distillation losses and pruning saliency scores involves human judgment [138] [173].Source: §10.5 · p. 43

10.6 Edge computing with LLMs

  • Edge deployment must contend with device limits in battery life, compute, and memory, as well as privacy and network latency.The section calls for resource-efficient methods supporting on-device training and operation.Source: §10.6 · p. 43

10.7 Theoretical insights into scaling laws

  • Theoretical understanding of scaling laws and generalization remains underexplored and is presented as important for resource-efficiency methods beyond compression.The section calls for deeper insight into how performance scales with model size and complexity.Source: §10.7 · p. 44

11 Conclusion

The survey synthesizes resource-efficient LLM techniques across model-development stages and system design, and relates practical applications and evaluations to a resource taxonomy. It also reviews benchmarks and metrics and identifies open challenges and future research opportunities.

Conclusion

  • The survey provides a comprehensive overview of state-of-the-art resource-efficient LLM techniques and methodologies.It establishes the challenges and needs that motivate the survey’s examination of architecture design, pre-training, fine-tuning, inference, and system design.Source: §11 · p. 44
  • The survey examines resource-efficiency approaches in both transformer and non-transformer architectures and across pre-training, fine-tuning, and inference.These stages each present distinct opportunities and challenges for resource optimization.Source: §11 · p. 44
  • The survey treats holistic system design as involving both hardware and software, and links practical applications and evaluations to its resource taxonomy.Source: §11 · p. 44
  • The survey reviews benchmarks and evaluation metrics and identifies open challenges and future directions for resource-efficient LLMs.It frames these directions as opportunities for further research and innovation toward efficient models accessible across applications.Source: §11 · p. 44
Loading 2401.00625v4…