A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks

1 INTRODUCTION

• LLMs have achieved strong performance across many NLP tasks, drawing research and industry interest (Chang et al., 2023).Source: §1 · p. 1
• Prompt engineering uses natural-language instructions to elicit LLM knowledge and does not require extensive task-specific parameter retraining or fine-tuning.It enables researchers to interact with models through natural-language exchanges to pursue NLP task goals.Source: §1 · pp. 1–2
• The survey reviews 44 papers on 39 prompting methods across 29 NLP tasks, grouping methods by task and discussing datasets, LLMs, and potential SoTA methods.It distinguishes its task-level categorization from broader application groupings in Sahoo et al. (2024), the privacy-focused scope of Edemacu & Wu (2024), and the narrower method coverage of Chen et al. (2023).Source: §1 · p. 2

2 PROMPT ENGINEERING TECHNIQUES

• The section distinguishes zero-shot prompting, which relies on pretrained knowledge, from few-shot prompting, which supplies examples but requires curation; basic prompting directly submits a query without added engineering.Basic prompting is also called Standard or Vanilla prompting. The survey notes that techniques may have zero-shot or few-shot variants, or may inherently support only one; curated few-shot examples can improve results but may induce unexplained bias.Source: §2 · p. 2, §2.1 · p. 2

2.2 CHAIN-OF-THOUGHT (COT)

• Reported gains include CoT’s maximum differences from Basic prompting of around 39% for Mathematical Problem Solving and 26% for Commonsense Reasoning, and Complex CoT’s average 5.3% accuracy gain and up to 18% improvement.Plan-and-Solve beats CoT by at least 5% on almost all zero-shot Mathematical Problem Solving datasets and by at least 5% on Commonsense Reasoning, with around 2% better accuracy on Multi-Hop Reasoning. MathPrompter raises performance from 78.7% to 92.5%; Contrastive CoT gains 10% over CoT on Mathematical Problem Solving, while Contrastive Self-Consistency gains over 15% over Self-Consistency; both exceed 10% gains over conventional counterparts on Multi-Hop Reasoning. Federated methods beat CoT by at least 10% and up to 20% on Mathematical Problem Solving.Source: §2.2 · p. 3, §2.6 · p. 4, §2.11 · p. 5

2.3 SELF-CONSISTENCY

• Self-Consistency samples diverse CoT reasoning paths and selects the most consistent answer; Auto-CoT generates reasoning examples from clustered queries, while PoT delegates computation to Python.Self-Consistency gains 11% on Mathematical Problem Solving, 3% on Commonsense Reasoning, and 6% on Multi-Hop Reasoning over CoT. Ensemble Refinement produces and refines multiple generations, outperforming CoT and Self-Consistency across Context-Free Question-Answering datasets. Auto-CoT matches or outperforms few-shot CoT; PoT gains around 12% over CoT across four task types. Least-to-Most decomposes and sequentially solves subproblems, and outperforms CoT and Basic prompting on four task types. Chain-of-Symbol uses symbolic spatial representations and gains up to 60.8% accuracy on Spatial Question-Answering.Source: §2.3 · p. 3, §2.4 · p. 3, §2.5 · p. 3

2.10 STRUCTURED CHAIN-OF-THOUGHT (SCOT)

• Across code, reasoning, and commonsense tasks, structured CoT and prompting variants improve over CoT or baselines, with reported gains reaching 13.79% for SCoT and 20% for Maieutic.Analogical Reasoning gains an average accuracy 4% over CoT across Mathematical Problem Solving, Code Generation, Logical Reasoning, and Commonsense Reasoning; Synthetic prompting achieves up to 15.6% absolute gains on datasets spanning Mathematical Problem Solving, Commonsense Reasoning, and Logical Reasoning. Tree-of-Thoughts achieves 65% better success rate than CoT on Mathematical Problem Solving, around 40% better success rate on Logical Reasoning datasets, and coherency scores of 7.56 versus CoT’s 6.93 on average on Free Response; Logical Thoughts surpasses CoT by maxima of 3.7% on Mathematical Problem Solving, 16.2% on Commonsense Reasoning, 2.5% on Logical Reasoning, 15.8% on Causal Reasoning, and 10% accuracy on Social Reasoning, while Maieutic prompting achieves up to 20% better Commonsense Reasoning accuracy than Basic, CoT, Self-Consistency, and GKP.Source: §2.10 · p. 4, §2.15 · p. 5, §2.16 · p. 6

2.21 REASON + ACT (REACT)

• ReAct interleaves reasoning and actions, Active-Prompt selects uncertain examples for annotation, and ThoT summarizes context sections before answering queries.ReAct combined with CoT or Self-Consistency improves on CoT, and exceeds reinforcement-learning methods by more than 10% absolute success rate on Language-Based Task Completion. Active-Prompt outperforms Self-Consistency, CoT, Auto-CoT, and Random CoT across the reported task datasets. ThoT achieves around 0.56 exact match on Context-Free Question-Answering and an average score of 3.8 on Dialogue System. Implicit RAG asks the LLM to retrieve context chunks and reports SoTA or near-SoTA on biomedical contextual QA datasets. S2A regenerates context to remove irrelevant information; Instructed prompting tells the model to ignore irrelevant information and scores 88.2 normalized micro accuracy on Truthfulness, surpassing its reported counterparts.Source: §2.21 · p. 7, §2.22 · p. 7, §2.23 · p. 7

2.27 CHAIN-OF-VERIFICATION (COVE)

• Verification and code-execution approaches address hallucinations or computation by checking answers, adapting knowledge, or executing generated programs.CoVe generates verification questions and revises its initial response, outperforming CoT and Basic prompting by around at least 10% across three tasks. CoK prepares rationales, adapts knowledge when answers lack majority consensus, and consolidates an answer. CoC selectively simulates interpreter output and surpasses CoT and other baselines across varied tasks; PAL interleaves natural-language and program statements for execution. Binder maps queries to programs and executes them, while Dater decomposes tables and queries, beating fine-tuned methods by at least 2% on Table-Based Truthfulness and at least 1% on Table-Based Question-Answering, and outperforming Binder on both.Source: §2.27 · p. 8, §2.28 · p. 8, §2.29 · p. 9

2.33 CHAIN-OF-TABLE

• Chain-of-Table plans table operations; DecomP decomposes and delegates subproblems; THOR uses staged sentiment reasoning, while metacognitive prompting consistently excels CoT and PS across numerous NLP tasks.Chain-of-Table averages around 3% better performance on Table-Based Question-Answering and 1.5% on Table-Based Truthfulness than prior SoTA; DecomP averages 25% higher exact match than CoT and Least-to-Most on Commonsense Reasoning.Source: §2.33 · p. 10, §2.34 · p. 10, §2.35 · p. 10

2.38 BASIC WITH TERM DEFINITIONS

• Vatsal et al. (2024) found that adding medical-term definitions did not help and could confuse LLMs.The proposed explanations are their narrow scope relative to the LLM's knowledge and the way medical definitions can change with context.Source: §2.38 · p. 11

2.39 BASIC + ANNOTATION GUIDELINE-BASED PROMPTING + ERROR ANALYSIS-BASED PROMPTING

• Hu et al. (2024)'s method averaged 0.57 exact match F1 across multiple Named Entity Recognition datasets.Source: §2.39 · p. 11
• Hu et al. (2024) combined task and output-format instructions with annotation definitions, linguistic rules, and training-data error-analysis instructions.The authors also tested versions using different combinations of these components.Source: §2.39 · p. 11

3 PROMPT ENGINEERING ON DIFFERENT NLP TASKS

• Each dataset is assigned to the single NLP task with which it most strongly associates, avoiding entangled cross-task analyses.The taxonomy diagram reflects this organization.Source: §3 · p. 11, Figure 1 · p. 13
• Because prompting performance varies with the LLM, the survey lists the models used alongside prompting strategies for each dataset.Source: §3 · p. 11
• The survey designates a method as a dataset’s SoTA when a listed LLM-and-method combination achieved the best reported performance, even if that exact LLM was not tested with every method.Source: §3 · p. 11
• Different versions of the same dataset across studies make absolute comparisons between prompting techniques difficult, so SoTA selection relies on the authors’ best judgment.Source: §3 · p. 13

3.1 MATHEMATICAL PROBLEM SOLVING

• Mathematical problem solving measures a model’s ability to perform mathematical computation in a non-tabular setting.Source: §3.1 · p. 13
• The surveyed datasets span arithmetic, word problems, and other mathematical tasks, including GSM8K, MATH, SVAMP, AQuA, Game of 24, and MathQA.The passage also names ASDiv, MAWPS, MultiArith, AddSub, SingleEq, Multi-Step Arithmetic, GSM-HARD, and SingleOp.Source: §3.1 · p. 13
• Table 1 lists the surveyed datasets, prompting methods tested on them, and the best-performing strategy for each dataset.Source: §3.1 · p. 13

3.2 LOGICAL REASONING

• Logical reasoning measures whether a model can follow natural-language commands with inputs and solve a problem.Source: §3.2 · p. 14
• The surveyed logical-reasoning datasets include tasks such as word sorting, temporal sequences, formal fallacies, object counting, and logical deduction.The survey also includes Boolean expressions, tracking shuffled objects, Web of Lies, Dyck Languages, Geometric Shapes, Repeat Copy Logic, and Mini Crosswords.Source: §3.2 · p. 14
• Table 2 catalogs datasets, prompting techniques tested on them, and the best-performing prompting method for each dataset.Source: §3.2 · p. 14, Table 2 · p. 15

3.3 COMMONSENSE REASONING

• Commonsense reasoning measures judgments based on practical knowledge, rather than solving a problem to arrive at an answer.Source: §3.3 · p. 16
• The surveyed datasets include commonsense tasks such as CSQA, Sports Understanding, Coin Flip, Com2Sense, CSQA 2.0, and Creak.Other named datasets include Reasoning about Colored Objects, Date Understanding, Last Letter Concatenation, Odd One Out, Disambiguation QA, Hyperbaton, and List Reversal.Source: §3.3 · p. 16
• Table 3 presents the surveyed datasets, prompting strategies tested on them, and the best-performing method for each dataset.Source: §3.3 · p. 16, Table 3 · p. 16

• Multi-hop reasoning assesses whether a model can connect evidence from different parts of a context to answer a query.Source: §3.4 · p. 17
• The surveyed datasets include StrategyQA, HotpotQA, Bamboogle, CommaQA-E, MuSiQue, 2WikiMultihopQA, and Ho et al. (2020).Source: §3.4 · p. 17
• Table 4 lists the prompting methods tried on these datasets and the best-performing strategy for each.Source: §3.4 · p. 17

3.5 CAUSAL REASONING

• Causal reasoning assesses a model’s ability to handle cause and effect across Cause And Effect and Causal Judgement datasets.Source: §3.5 · p. 18

3.6 SOCIAL REASONING

• Social reasoning tests models’ ability to reason about human social interactions, with SocialQA as the surveyed dataset.Source: §3.6 · p. 18

3.7 CONTEXTUAL QUESTION-ANSWERING

• Contextual question-answering tests answering from a supplied context and surveys datasets including ProcessBank, BioMRC, MASH-QA, CliCR, MultiSpanQA, FinQA, TAT-QA, Patient Case Reports, Drop, and BoolQ.Source: §3.7 · pp. 18–19

3.8 CONTEXT-FREE QUESTION-ANSWERING

• Context-free question-answering relies on a model’s embedded or an open-source knowledge base, rather than only supplied context; surveyed datasets include PopQA, MedQA, PubMedQA, MMLU, and AI2 Reasoning Challenge.Table 8 lists datasets, experimented prompting strategies, and the best-performing strategy. For PubMedQA and MMLU, the supplied fragments list Basic, CoT, Self-Consistency, and ER.Source: §3.8 · pp. 19–21

3.9 SPATIAL QUESTION-ANSWERING

• Spatial question-answering measures spatial reasoning, with surveyed datasets including Brick World, NLVR-Based Manipulation, Natural Language Navigation, Spartun, and Navigate.Table 9 catalogs the prompting methods tried on these datasets and the best-performing strategy.Source: §3.9 · p. 21, Table 9 · p. 22

3.10 CONVERSATIONAL CONTEXTUAL QUESTION-ANSWERING

• Conversational contextual question-answering evaluates responses to interconnected queries about a text extract, where an answer may depend on previous answers; the surveyed dataset is ConvFinQA.Table 10 lists the prompting methods tried and the best-performing strategy.Source: §3.10 · pp. 21–22, Table 10 · p. 22

3.11 DIALOGUE SYSTEM

• The dialogue-system task covers language generation in user-to-machine conversation or answering from an existing conversation, and the survey includes MTCR.The paper keeps dialogue systems separate from conversational contextual question-answering despite possible overlap between tasks.Source: §3.11 · p. 22, Table 11 · p. 23

3.12 CODE GENERATION

• Code generation covers cases where the model receives or produces programming-language code, with Codeforce Scraping, HumanEval, MBPP, and MBCPP among the surveyed datasets.Table 12 catalogs tested prompting methods and the best-performing strategy.Source: §3.12 · p. 22, Table 12 · p. 23

3.13 FREE RESPONSE

• Free response assesses a model’s ability to generate unconstrained textual responses, with Creative Writing and Longform Generation of Biographies among the surveyed datasets.Table 13 lists tested prompting strategies and the best technique.Source: §3.13 · p. 23, Table 13 · p. 23

3.14 TRUTHFULNESS

• Truthfulness assesses factual communication and misinformation avoidance rather than understanding a supplied context, using SycophancyEval, GSM-IC, and Fever.Table 14 lists tested prompting techniques and the best-performing technique.Source: §3.14 · p. 23, Table 14 · p. 24

3.15 TABLE-BASED TRUTHFULNESS

• Table-based truthfulness extends truthfulness to a tabular setting, and TabFact is the surveyed dataset.Table 15 lists tested prompting strategies and the best-performing strategy.Source: §3.15 · p. 24, Table 15 · p. 24

3.16 TABLE-BASED QUESTION-ANSWERING

• Table-based question-answering covers tabular QA outside more specific categories such as table-based truthfulness and mathematical problem solving, with FeTaQA and WikiTQ as surveyed datasets.Table 16 lists tested prompting methods and the best-performing strategy.Source: §3.16 · p. 24, Table 16 · p. 25

3.17 TABLE-BASED MATHEMATICAL PROBLEM SOLVING

• Table-based mathematical problem solving extends mathematical problem solving to computations in a tabular setting, with TabMWP and Penguins in a Table among the surveyed datasets.Table 17 lists tested prompting methods and the best-performing strategy.Source: §3.17 · p. 24, Table 17 · p. 25

3.18 RECOMMENDER SYSTEM

• The recommender-system task measures whether a model can select the most relevant items from a list, and the survey covers Movie Recommendation.Table 18 lists tested prompting strategies and the best-performing technique.Source: §3.18 · p. 25, Table 18 · p. 25

3.19 EMOTION/SENTIMENT UNDERSTANDING

• Emotion/sentiment understanding covers Ruin Names, SemEval14 Laptop and Restaurant, and Forex, with prompting strategies and best-performing strategies tabulated.Source: §3.19 · p. 25, Table 19 · p. 26

3.20 MACHINE TRANSLATION

• Machine translation covers Salient Translation Error Detection, FLORES, WMT21, Multi-Domain, and PDC, with prompting methods and best-performing strategies tabulated.Source: §3.20 · p. 26, Table 20 · p. 26

3.21 NAMED ENTITY RECOGNITION

• Named entity recognition covers MTSamples, VAERS, Research Papers, and BC5CDR-chem; table fragments separately list GLM-130B and Basic + Variations.Source: §3.21 · pp. 26–27, Table 21 · p. 27

3.22 WORD SENSE DISAMBIGUATION

• Word sense disambiguation is represented by WiC, with prompting methods and a best-performing method included in the task table.Source: §3.22 · p. 27, Table 22 · p. 28

3.23 SUMMARIZATION

• Summarization is represented by CCTC, with prompting methods and a best-performing strategy included in the task table.Source: §3.23 · p. 28, Table 23 · p. 28

3.24 PARAPHRASING

• Paraphrasing rewrites input with different words while preserving its semantics, unlike summarization, which aims to shorten output; the survey lists QQP 2.Source: §3.24 · p. 28, Table 24 · p. 28

3.25 STANCE DETECTION

• Stance detection assesses whether a text's author favors or opposes a target, using SemEval-2016, VAST, and P-Stance datasets.Source: §3.25 · p. 28, Table 25 · p. 29

3.26 NATURAL LANGUAGE INFERENCE

• Natural language inference classifies a hypothesis as entailment, contradiction, or neutral given a premise, with QNLI and MedNLI datasets.Source: §3.26 · p. 29, Table 26 · p. 29

3.27 RELATION EXTRACTION

• Relation extraction identifies semantic relationships between predefined entity classes, and the survey covers the DDI dataset.Source: §3.27 · p. 29, Table 27 · p. 29

3.28 LANGUAGE-BASED TASK COMPLETION

• Language-based task completion evaluates following language-based navigational commands to choose actions, using ALFWorld, WebShop, SayCan, and Scan.Source: §3.28 · p. 29, Table 28 · p. 30

3.29 MULTILABEL TEXT CLASSIFICATION

• Multilabel text classification assigns inputs to predefined labels; its survey coverage excludes datasets suitably categorized under previously discussed tasks.Source: §3.29 · p. 30, Table 29 · p. 30

4 CONCLUSION

• The survey analyzes 44 papers covering 39 prompting strategies across 29 NLP tasks, with a taxonomy, task-and-dataset categorization, and potential state-of-the-art methods listed for each dataset.Source: §4 · p. 30