Source-linked AI summary
Large Language Models Meet NLP: A Survey
Libo Qin, Qiguang Chen, Xiachong Feng, Yang Wu, Yongheng Zhang, Yinghui Li, Min Li, Wanxiang Che, Philip S. Yu
TL;DR
Although LLMs have shown impressive NLP capabilities, their potential has not been systematically investigated. This survey analyzes LLMs across NLP tasks using a unified taxonomy, compares application paradigms, and identifies frontiers and challenges for future research.
Problem
A systematic investigation of how LLMs are applied to NLP tasks, whether traditional tasks are solved, and what future directions remain is limited.
Method
The paper provides a comprehensive task-oriented survey organized around parameter-frozen and parameter-tuning paradigms, while synthesizing frontiers and challenges.
Results
The survey finds that zero-shot, few-shot, full-parameter, and parameter-efficient approaches differ in efficiency, in-domain performance, resource demands, and generalization.
Takeaways & Limitations
The taxonomy, frontier analysis, and curated resources provide a unified perspective and practical support for studying and building LLMs for NLP.
Takeaways & Limitations
Current LLM-based NLP research faces challenges including low-resource language performance, cross-lingual alignment, tool selection and planning, and hallucinations that harm reliability.
Abstract
from arXiv · showhide
While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to address this gap by exploring the following questions: (1) How are LLMs currently applied to NLP tasks in the literature? (2) Have traditional NLP tasks already been solved with LLMs? (3) What is the future of the LLMs for NLP? To answer these questions, we take the first step to provide a comprehensive overview of LLMs in NLP. Specifically, we first introduce a unified taxonomy including (1) parameter-frozen paradigm and (2) parameter-tuning paradigm to offer a unified perspective for understanding the current progress of LLMs in NLP. Furthermore, we summarize the new frontiers and the corresponding challenges, aiming to inspire further groundbreaking advancements. We hope this work offers valuable insights into the potential and limitations of LLMs, while also serving as a practical guide for building effective LLMs in NLP.
1 Introduction
This survey examines how LLMs are applied across NLP tasks and organizes existing work through parameter-frozen and parameter-tuning paradigms. It also identifies emerging research frontiers and challenges while providing curated resources for further study.
- LLMs support diverse NLP tasks through a unified generative paradigm, including mathematical reasoning, summarization, translation, information extraction, and sentiment analysis.
- The survey analyzes LLMs from the perspective of independent NLP tasks to investigate their current applications, progress, and future directions.
- Parameter-frozen methods include zero-shot and few-shot learning, while parameter-tuning methods include full-parameter and parameter-efficient tuning.
- The proposed taxonomy distinguishes parameter-frozen methods, which use prompting without parameter tuning, from parameter-tuning methods.
- The survey discusses new research frontiers and associated challenges to stimulate future breakthroughs in LLM-based NLP.
- It provides a curated collection of open-source implementations, relevant corpora, and research papers through a publicly available resource website.
2 Background
LLM-based NLP adaptation is organized into parameter-frozen and parameter-tuning paradigms, which differ in whether model parameters are updated. These paradigms trade off resource consumption, in-domain performance, deployment efficiency, and out-of-domain generalization.
- Paradigm taxonomy: Parameter-frozen adaptation applies prompting without parameter tuning, whereas parameter-tuning adaptation adjusts LLM parameters for NLP tasks.The parameter-frozen category includes zero-shot and few-shot learning; parameter-tuning includes full-parameter and parameter-efficient tuning.
- Parameter-frozen paradigm: Zero-shot learning uses an instruction prompt without demonstrations, while few-shot learning adds demonstrations to support in-context task solving.Zero-shot prompting maps an input to an output; few-shot prompting additionally conditions on demonstrations E.
- Parameter-tuning paradigm: Full-parameter tuning fine-tunes all model parameters on training data, while parameter-efficient tuning updates selected or added parameters.Examples of parameter-efficient methods include Bottleneck Adapter, LoRA, Prefix-tuning, and QLoRA.
- Comparison of paradigms: Zero-shot learning offers the highest application efficiency and strongest out-of-domain generalization, while full-parameter tuning provides the best in-domain performance at the highest cost.The comparison describes zero-shot learning as resource-light and full-parameter tuning as resource-intensive, with opposite strengths across domain settings.
- Comparison of paradigms: Few-shot learning generally improves in-domain performance over zero-shot learning but requires more computation and generalizes less effectively to novel domains.Its advantage in in-domain performance is accompanied by lower overall efficiency and reduced out-of-domain generalization.
- Comparison of paradigms: Parameter-efficient tuning balances resource use and performance, sometimes matching or exceeding full-parameter tuning while offering higher efficiency and improved beyond-domain generalization.This balance is especially described for settings with limited resources.
3 Natural Language Understanding
The survey organizes NLP understanding tasks under parameter-frozen and parameter-tuning paradigms, covering sentiment, information extraction, dialogue, and table understanding. Across these tasks, prompting, demonstrations, structured representations, and tuning are used to improve LLM performance.
- Parameter-Frozen Paradigm: Zero-shot prompting supports sentiment analysis, information extraction, dialogue understanding, and table understanding without additional parameter tuning.Studies use instructions, task decomposition, chain-of-thought, table segmentation, parsing statements, and external knowledge to support these tasks.
- Parameter-Tuning Paradigm: Full-parameter tuning customizes LLMs for sentiment, information extraction, dialogue, and table-understanding tasks.Reported strategies include unified instruction formats, mixed datasets, structured textual representations, and task-specific fine-tuning.
- Parameter-Frozen Paradigm: Few-shot demonstrations improve task relevance and structured-output adherence, especially when LLMs must perform sequence labeling or dialogue-state tracking.Retrieval modules select pertinent examples, while agent and in-context-learning approaches extend dialogue understanding.
- Parameter-Tuning Paradigm: Parameter-efficient tuning targets selected prompt or model components to improve adaptation and deployment efficiency.Examples include prompt tuning for lifelong information extraction and LoRA for emotional systems.
4 Natural Language Generation
The survey examines LLMs for summarization, code generation, machine translation, and mathematical reasoning through parameter-frozen and parameter-tuning approaches. It concludes that LLMs provide a unified generative paradigm, yet remain behind smaller supervised models while benefiting substantially from continued fine-tuning.
- Task coverage: LLMs support summarization, code generation, machine translation, and mathematical reasoning through a unified generative paradigm.These tasks are reviewed across zero-shot, few-shot, full-parameter, and parameter-efficient tuning settings.
- Parameter-Frozen Paradigm: Zero-shot and few-shot methods demonstrate strong capabilities in summarization, code generation, translation, and mathematical reasoning.Examples include factually accurate summaries, zero-shot programming, multilingual instruction tuning, and zero-shot chain-of-thought prompting.
- Parameter-Tuning Paradigm: Parameter-efficient tuning adapts LLMs for summarization and code tasks while reducing parameter and resource requirements.Prefix tuning, prompt tuning, adapters, and LoRA are discussed as approaches for domain adaptation, controllability, and computational efficiency.
- Machine Translation: Machine-translation research uses instruction tuning, fine-tuning, preference optimization, and reinforcement learning to improve multilingual and context-aware translation.The surveyed work addresses polysemy, long-context translation, low-resource languages, and inference efficiency.
- Takeaways: LLMs still lag behind smaller supervised learning models on NLP tasks, while continued fine-tuning brings substantial improvements.The survey presents this as a central takeaway across the reviewed generation tasks.
5 Future Work and New Frontier
The paper identifies future frontiers for LLMs in NLP, including multilingual, multimodal, tool-using, reasoning, hallucination, and safety research. It pairs these frontiers with challenges involving cross-lingual alignment, complex reasoning, tool planning, evaluation, creativity, and safety benchmarks.
- 5.1 Multilingual LLMs for NLP: Multilingual LLM research must address poor low-resource-language performance and improve alignment between English and other languages.The paper highlights multilingual pretraining, supervised fine-tuning, reinforcement learning, few-shot approaches, and zero-shot instructions as existing strategies.
- 5.2 Multi-modal LLMs for NLP: Multimodal LLMs need stronger complex reasoning and more effective interaction mechanisms beyond simple recognition, projection, or prompting.Current work often focuses on simple multimodal reasoning, while complex reasoning and inference-time multimodal interaction remain open problems.
- 5.3 Tool-usage in LLMs for NLP: Practical NLP deployment requires appropriate tool selection and coordinated multi-tool planning rather than static or single-tool usage.The paper illustrates tool planning with flight, train, and bus booking tools whose coordination should reduce travel time and cost.
- 5.4 X-of-thought in LLMs for NLP: X-of-thought research seeks better complex logical reasoning, but universal step decomposition and integration of diverse prompting knowledge remain challenging.The approach refines processing and reasoning through decomposed steps for difficult NLP tasks.
- 5.5 Hallucination in LLMs for NLP: Hallucinations reduce reliability in practical NLP settings, motivating unified evaluation benchmarks and research on using hallucinations to stimulate creativity.The paper defines hallucinations as outputs deviating from world knowledge, user requests, or self-generated context.
- 5.6 Safety of LLMs for NLP: LLM safety research must address copyright, hate toxicity, social bias, psychological safety, sparse benchmarks, and multilingual risks.The paper specifically identifies safety benchmark construction and cross-language and cultural risk mitigation as major challenges.
- 5.7 Long Chain-of-Thought: Long Chain-of-Thought extends standard chain-of-thought prompting with deeper reasoning, multiple solution paths, and reflection on intermediate outcomes.Hierarchical levels or segmented sub-chains support complex NLP challenges and compositional reasoning beyond conventional chain-of-thought.
6 Conclusion
The paper presents a systemic overview of LLMs in NLP through a unified taxonomy of parameter-frozen and parameter-tuning paradigms. It also identifies research frontiers and challenges and maintains a public resource website to support future work.
- 6 Conclusion: The paper offers a systemic overview of LLMs in NLP organized by parameter-frozen and parameter-tuning paradigms.This taxonomy is intended to provide insights and resources for building effective LLMs in NLP.
- 6 Conclusion: The paper highlights new research frontiers and challenges to facilitate future research.These areas are presented as directions for continued investigation and advancement.
- 6 Conclusion: A publicly available resource website tracks developments in the LLM-for-NLP literature.The resource is intended to support researchers working on LLM-based NLP.