PaperlayerView exact revision on arXiv

Source-linked AI summary

Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models

Jiaao Chen, Xiaoman Pan, Dian Yu, Kaiqiang Song, Xiaoyang Wang, Dong Yu, Jianshu Chen

arXiv:2308.00304v3cs.CL

TL;DR

LLMs can struggle to compose existing skills into solutions for harder unseen problems. SKiC demonstrates foundational skills and grounded compositions together in the prompt; the paper reports near-perfect systematic generalization across tasks and zero-shot weak-to-strong generalization with SKiC-style fine-tuning.

  • Problem

    Although LLMs show strong zero- and few-shot performance across many NLP tasks, they still struggle to use existing skills to solve complex unseen problems.

    Source: §1 · p. 1

  • Method

    SKiC places foundational skills and exemplars that explicitly ground reasoning steps in those skills together in one prompt context.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    SKiC achieves near-perfect systematic generalization across many tasks, can use internal pretraining skills, and supports zero-shot weak-to-strong generalization after fine-tuning.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The results indicate that SKiC can help models compose skills beyond those provided in context and transfer to new tasks.

    Source: §1 · p. 2

    Reliance on internal knowledge may limit SKiC where external tools offer advantages, such as real-time retrieval or calculations beyond the model’s internal knowledge base.

    Source: §7 · p. 9

Abstract

from arXiv · show

We investigate how to elicit compositional generalization capabilities in large language models (LLMs). Compositional generalization empowers LLMs to solve complex problems by combining foundational skills, a critical reasoning ability akin to human intelligence. However, even the most advanced LLMs currently struggle with this form of reasoning. We examine this problem within the framework of in-context learning and find that demonstrating both foundational skills and compositional examples grounded in these skills within the same prompt context is crucial. We refer to this prompt structure as skills-in-context (SKiC). With as few as two exemplars, this in-context learning structure enables LLMs to tackle more challenging problems requiring innovative skill combinations, achieving near-perfect systematic generalization across a broad range of tasks. Intriguingly, SKiC also unlocks the latent potential of LLMs, allowing them to more actively utilize pre-existing internal skills acquired during earlier pretraining stages to solve complex reasoning problems. The SKiC structure is robust across different skill constructions and exemplar choices and demonstrates strong transferability to new tasks. Finally, inspired by our in-context learning study, we show that fine-tuning LLMs with SKiC-style data can elicit zero-shot weak-to-strong generalization, enabling the models to solve much harder problems directly with standard prompting.

1 Introduction

Although LLMs perform well on many tasks, they struggle to compose existing skills for harder unseen problems. SKiC places foundational skills and grounded composition examples alongside the test problem, and the paper reports near-perfect generalization across broad tasks.

1 Introduction

  • LLMs still struggle to use existing skills to solve complex unseen problems, despite strong zero- and few-shot performance across many NLP tasks.Source: §1 · p. 1
  • Earlier prompting approaches often fail on problems significantly harder than those demonstrated, while decomposition methods are limited to sequential subproblems.Source: §1 · p. 1
  • SKiC combines foundational skills, exemplars showing how to compose them, and a test problem in one prompt.The exemplars explicitly ground reasoning steps in the foundational skills.Source: §1 · p. 2, Figure 1 · p. 2
  • SKiC achieves near-perfect systematic generalization across many tasks by teaching LLMs to ground reasoning steps in mastered skills.The prompting structure also supports generalization beyond skills shown in the context and transfer to new tasks.Source: §1 · p. 2
  • SKiC-style fine-tuning elicits zero-shot weak-to-strong generalization, letting models solve harder problems with standard prompting.Source: §1 · p. 2

2 SKiC: Elicit Compositionality with In-Context Skills and Grounding

SKiC demonstrates foundational skills and examples of composing them in a shared context, explicitly grounding reasoning steps in those skills. The paper presents its one-stage format as applicable beyond linear decomposition and shows models invoking skills absent from the prompt.

Demonstration of Composition

  • SKiC demonstrates foundational skills and their explicit composition in one context, grounding each reasoning step in the corresponding skill.The skill list need not cover every skill required for a solution; the exemplars demonstrate grounded composition.Source: §2 · p. 3

Comparison to existing approaches

  • Unlike sequential decomposition, SKiC solves complex problems in one stage and can handle tasks with general computation graphs.Decomposition can itself be treated as a basic skill, and the one-stage format can replace chain-of-thought prompting or combine with ensemble techniques.Source: §2 · p. 3

Construction of the skills

  • Foundational skills are non-exhaustive and can be manually or automatically constructed; SKiC typically uses 2–10 composition exemplars.The paper reports that automatic construction by prompting LLMs can still achieve good performance.Source: §2 · p. 3

Grounding the composition

  • In the MATH example, generated reasoning invokes Angle Bisector Theorem and Heron’s Formula although neither appears in the prompt or exemplars.The model generates these skill names automatically, illustrating use of pre-trained internal skills.Source: §2 · p. 4, Figure 2 · p. 4

3 Analysis of Compositional Abilities

The analysis tests both composition over skills supplied in the prompt and complex reasoning that can draw on pretrained skills. SKiC improves generalization across these settings, while error analysis identifies missing skills and difficult compositions as important remaining sources of error.

3 Analysis of Compositional Abilities

  • Evaluation separates composition when all needed skills are in-context from complex reasoning that must also draw on pretrained skills, including GSM8K and MATH.The systematic tasks include last-letter concatenation, arithmetic, CommaQA-E, and dynamic programming.Source: §3 · p. 4

3.1 Near-Perfect Systematic Generalization

  • SKiC achieves nearly perfect generalization on last-letter concatenation, addition, and dynamic programming with text-davinci-003, ChatGPT, or GPT-4, unlike baselines that degrade on harder inputs.Figure 3 distinguishes in-distribution tests matching the context's complexity from increasingly harder out-of-distribution tests.Source: §3.1 · p. 5, Figure 3 · p. 5

3.2 Enhanced Complex Reasoning

  • SKiC significantly boosts GSM8K accuracy even with incomplete prompt skills, and generated reasoning uses provided skills absent from examples as well as pretrained skills.The analysis describes these behaviors as using prompt skills and pretrained knowledge compositionally.Source: §3.2 · p. 5, Figure 5 · p. 5
  • On MATH, 24% of skills used in GPT-4 reasoning outputs come from internal knowledge, alongside in-context skills.The reported internal skill activation rate measures the share of skills in generated reasoning that originate in pretrained knowledge rather than the prompt.Source: §3.2 · pp. 5–6, Table 1 · p. 6

3.3 Synergy between Skills and Composition

  • LLM-discovered skills are effective on RTE and last-letter tasks, while ChatGPT-generated skills improve ChatGPT's MATH accuracy and internal-skill activation over GPT-4-generated skills.The MATH comparison tests the same prediction model with skills generated by ChatGPT or GPT-4.Source: §3.3 · p. 6, Table 2 · p. 6, Table 3 · p. 6
  • GSM8K-designed prompts transfer better than CoT to unseen MATH and FOLIO tasks, and removing either skills or explicit grounding lowers dynamic-programming performance.The ablation compares SKiC with variants retaining only one of those two components.Source: §3.3 · p. 7, Table 4 · p. 7, Table 5 · p. 7
  • SKiC is robust to exemplar choices and ordering, with selection robustness similar to CoT while achieving better overall performance.The studies evaluate choices on last-letter tasks and ordering on GSM8K.Source: §3.3 · p. 7, Table 6 · p. 7, Table 7 · p. 7

3.4 Error Analysis

  • Unseen basic skills are the most common errors; mastery failures worsen with skill complexity, composition errors occur in complex GSM8K reasoning, and copying errors rise with longer contexts.Math reasoning is also difficult because it requires varied, problem-specific skill compositions.Source: §3.4 · p. 7, Figure 6 · p. 8

4 Beyond In-Context Learning

The paper applies SKiC-style data in instruction tuning, training Llama2 on GSM8K and evaluating it on harder MATH problems. This fine-tuning improves MATH accuracy over CoT-annotated training data.

4 Beyond In-Context Learning

  • Fine-tuning Llama2 on GSM8K data with SKiC-grounded reasoning improves accuracy on harder MATH problems compared with GPT-4-generated CoT reasoning data.The authors suggest SKiC could potentially replace CoT in instruction tuning for stronger reasoning and weak-to-strong generalization.Source: §4 · p. 8

5 Related Work

Prior work explores compositional generalization through neuro-symbolic methods and prompting, while decomposition-based prompting faces limits on harder or nested problems. SKiC is presented as a one-stage, single-agent alternative.

5 Related Work

  • Neuro-symbolic methods combine symbolic and distributed representations, while prompting approaches aim to unlock LLM compositional generalization.The passage situates these approaches within a longer history of compositional-generalization research.Source: §5 · p. 8
  • Least-to-Most (Zhou et al., 2022a) and Decomp (Khot et al., 2022) decompose hard problems into easy-to-hard subproblems, but performance degrades as problems grow harder.Their applications are limited to problems decomposable into subproblems, and highly nested problems make prompt and exemplar construction challenging.Source: §5 · p. 8
  • Unlike multi-stage or multi-agent prompting, SKiC is described as a one-stage, single-agent strategy that can replace standard or CoT prompting.The paper also distinguishes its emphasis on explicit grounding of reasoning steps in basic skills.Source: §5 · pp. 8–9

6 Conclusion

SKiC teaches LLMs to ground reasoning steps in foundational skills by presenting those skills and grounded compositional examples together. The paper reports broad near-perfect systematic generalization and stronger complex reasoning, including use of skills learned during pretraining.

6 Conclusion

  • SKiC places foundational skills and compositional examples grounded in those skills within the same prompt context.The paper identifies explicit grounding of each reasoning step on foundational skills as crucial.Source: §6 · p. 9
  • SKiC achieves near-perfect systematic generalization across many tasks and enhances complex reasoning capabilities.Source: §6 · p. 9
  • With SKiC, LLMs can use skills and knowledge acquired during earlier pretraining to solve unseen complex problems.The conclusion also says SKiC can be used in fine-tuning to improve easy-to-hard generalization.Source: §6 · p. 9

7 Limitations

The paper focuses on easy-to-hard compositional generalization, defining unseen tasks as harder variants than the demonstrations. It reports near-perfect generalization with SKiC, while noting challenges in scaling to domains with many skills and limits where external tools could help.

7 Limitations

  • Here, unseen tasks are harder variants than the demonstrated problem size; some tasks may have appeared in pretraining as simpler examples.The authors note that harder test cases, such as last letters of 12 words, differ from simpler examples that might have been crawled into pretraining.Source: §7 · p. 9
  • SKiC boosts easy-to-hard generalization to near-perfect levels, unlike CoT and Least-to-Most prompting.Source: §7 · p. 9
  • Scaling SKiC beyond problems with relatively clear, limited skill sets remains challenging, and manual or semi-automatic distillation may not suit intricate real-world scenarios.The authors identify higher-quality skills, broader skill coverage, and more composition examples as directions for improvement.Source: §7 · p. 9
  • SKiC’s focus on internal skills may limit its applicability when external tools offer advantages, such as real-time retrieval or complex calculations.Source: §7 · p. 9

A Comparison to Existing In-Context Learning Strategies

  • Unlike decomposition-based prompting, SKiC handles complex problems in one stage, which could alleviate error propagation and apply to tasks difficult to decompose linearly.The authors also say SKiC can replace one-stage CoT prompts and combine with ensemble techniques.Source: A Comparison to Existing In-Context… · p. 13

B Details about the Construction of Skills

The paper describes manual and LLM-assisted ways to construct SKiC skills and composition examples. It positions internal-skill composition as complementary to tool use, while noting the scope and practical limits of both approaches.

B Details about the Construction of Skills

  • Manual skill distillation summarizes skills from fewer than 10 problems, illustrates them with examples, then adds compositional examples grounded in those skills.The approach is described as generally applicable to narrow domains requiring a limited set of skills.Source: B Details about the Construction of… · p. 13
  • LLM-assisted distillation prompts a model to generate skills, followed by human review and composition examples grounded in contextual or inherent skills.For MATH, the authors report that about 24% of skills used in reasoning steps came from the LLM’s internal pretrained knowledge.Source: B Details about the Construction of… · p. 13, Figure 9 · p. 15

C Comparison to Tool-Using Works

  • SKiC focuses on composing internal skills, while tool use is complementary and could be integrated as a basic skill.The authors suggest a hybrid approach combining internal skills and external tools for more complex problems.Source: C Comparison to Tool-Using Works · pp. 14–15
  • The authors argue SKiC may suit ambiguous problems and broader task types, whereas external tools can add inference latency and may be hard to enumerate for complex tasks.They also note that external tools may be advantageous in settings such as real-time data retrieval or complex calculations.Source: C Comparison to Tool-Using Works · pp. 14–15

D Experimental Setup

The experiments assess systematic generalization from easier demonstrations to harder problems, as well as generalization beyond contextual skills. They cover symbolic, arithmetic, question-answering, dynamic-programming, and mathematical reasoning tasks across multiple models and prompting baselines.

D Experimental Setup

  • The evaluation separates systematic generalization using provided skills from complex reasoning that also requires internal skills beyond the prompt.Systematic tasks include symbolic manipulation, arithmetic, question answering, and dynamic programming; the broader-reasoning setting includes GSM8K and MATH.Source: D Experimental Setup · pp. 15–16

D.1.1 Symbolic Manipulation: Last Letters

  • Last-letter concatenation tests whether models can extend demonstrations with one or two words to harder inputs of up to 12, 50, or 100 words.SKiC prompts include skills for forming a word list and extracting last letters, followed by two composition examples.Source: §D.1.1 · pp. 16–17, Table 12 · p. 17

D.1.2 Arithmetic Operation

  • Addition and multiplication evaluations test generalization beyond demonstrations with two- or three-digit numbers, using longer digit lengths for held-out problems.The addition tests span two through seven digits, while multiplication tests span two through five digits.Source: §D.1.2 · pp. 17–18, Table 13 · p. 17, Table 14 · p. 18

D.1.3 Long-Context Question Answering: CommaQA-E

  • CommaQA-E tests multi-hop question answering over synthetic entities, including unseen compositional questions, against standard, CoT, and Decomp prompting.The SKiC construction treats both breaking questions into simpler ones and answering them as basic skills.Source: Table 10 · p. 16, §D.1.3 · p. 18, Table 15 · p. 18

D.1.4 Dynamic Programming

  • Dynamic programming tests finding the highest-sum non-adjacent subsequence, with demonstrations at sequence lengths 4 and 5 and harder evaluations at lengths 6–8.The SKiC prompt presents list-length, maximum-number, and single-digit addition skills, followed by two composition examples.Source: Table 11 · p. 16, §D.1.4 · pp. 18–19, Table 16 · p. 19

D.2 Enhanced Complex Reasoning: Generalization Beyond In-Context Skills

  • The enhanced-reasoning setting asks whether models can generalize beyond provided skills by using internal capabilities acquired during pretraining.The authors motivate this setting by noting that complex tasks require varied compositions across many skills, impractical to enumerate in context.Source: §D.2 · p. 19

D.2.1 GSM8K

  • GSM8K evaluation uses a limited skill set with eight compositional examples and compares SKiC with several prompting strategies across text-davinci-003, ChatGPT, and GPT-4.Source: §D.2.1 · p. 19

D.2.2 MATH

  • For MATH, the authors use LLM-generated skills because the benchmark’s broad range of topics makes manual enumeration infeasible, then craft grounded composition examples.The seven exemplars comprise one per category; reasoning is grounded partly in contextual skills and partly in internal skills such as Pascal’s Triangle.Source: §D.2.2 · p. 19

E Detailed Results for Systematic Generalization (Last Leter, Addition, Multiplication, Commaqa-E and DP)

SKiC generally improves performance on harder-than-exemplar problems, with near-perfect generalization reported for several tasks. The results also examine model and skill-source effects, and robustness to exemplar choices.

E Detailed Results for Systematic Generalization (Last Leter, Addition, Multiplication, Commaqa-E and DP)

  • SKiC improves 7-digit summation by over 68.9% versus baselines with text-davinci-003, and achieves nearly perfect generalization on last-letter concatenation, addition, and dynamic programming.On 12-word last-letter concatenation tests, 4-shot standard prompting achieves only 10% accuracy with textdavinci-003.Source: E Detailed Results for Systematic… · p. 20
  • Compared with fine-tuned scratchpad baselines, SKiC performs better out of distribution but worse in distribution.The authors note that the baseline was fine-tuned directly on input sequences of lengths 4 and 5, whereas SKiC was not fine-tuned.Source: E Detailed Results for Systematic… · p. 20, F The Performance of SKiC on LLAMA2 · p. 20
  • For CommaQA-E, SKiC presents question decomposition and simple question answering as skills with composition examples in one context, unlike Decomp’s multi-stage prompting.The authors report that early-stage errors in Decomp can propagate to wrong predictions, while SKiC answers the questions in one context.Source: E Detailed Results for Systematic… · p. 20

F The Performance of SKiC on LLAMA2

  • LLAMA2-70B generally outperforms LLAMA-65B and follows exemplars more strongly, though open-source LLAMA models still trail text-davinci-003, ChatGPT, and GPT-4.The evaluation covers last-letter concatenation, addition, multiplication, CommaQA-E, and dynamic programming.Source: F The Performance of SKiC on LLAMA2 · pp. 20–21

G Different Sources of In-context Skills

  • ChatGPT’s MATH accuracy and internal skill activation rate are both higher when its SKiC skills come from ChatGPT rather than GPT-4, despite GPT-4 skills generally having higher quality.Internal skill activation rate is the average proportion of skills used per question that come from pretrained knowledge rather than the prompt context.Source: G Different Sources of In-context Skills · p. 21, Table 20 · p. 21

H Robustness of Exemplars in SKiC

  • SKiC’s performance improvements persist when few-shot exemplars are changed, and shuffling exemplar order is reported to matter little.The authors tested randomly selected exemplars and shuffled four-example GSM8K prompts.Source: H Robustness of Exemplars in SKiC · p. 21

I Generalization to New Tasks

The authors test whether SKiC prompts designed for GSM8K transfer to unseen tasks, including math reasoning and logical inference.

I Generalization to New Tasks

  • ChatGPT is evaluated on unseen MATH and FOLIO tasks using skills and prompts designed for GSM8K.MATH is described as competition-level math reasoning, while FOLIO evaluates logical inference.Source: I Generalization to New Tasks · p. 21
  • The reported transfer targets span competition-level math reasoning and logical inference rather than GSM8K alone.The experiments apply the GSM8K-designed prompts directly to MATH and FOLIO.Source: I Generalization to New Tasks · p. 21
  • The associated table reports accuracy on MATH and FOLIO using prompts designed for GSM8K with ChatGPT models.The supplied table caption identifies the evaluated tasks and model family but does not provide accuracy values.Source: I Generalization to New Tasks · p. 21, Table 23 · p. 22

J Ablation of Different SKiC Components

The paper tests SKiC components, semantic parsing, automatically discovered skills, and SKiC-style instruction tuning. It reports that SKiC-labeled training data improves zero-shot MATH performance over CoT-labeled data.

J Ablation of Different SKiC Components

  • Removing either the basic skill list or grounding reasoning steps in those skills reduces Dynamic Programming performance.The ablations separately retain grounding while removing skills, or retain the skills while removing grounding.Source: J Ablation of Different SKiC Components · p. 22

K Applying SKiC to Semantic Parsing

L LLMs can automatically discover skills

  • ChatGPT can discover skills from two RTE examples, and prompts built from discovered skills produce results similar to manually constructed prompts on last-letter tasks.The discovered RTE skills include Context Understanding and Inference Evaluation; the last-letter skill set includes identifying words, determining and concatenating last letters, and forming a new sequence.Source: L LLMs can automatically discover skills · pp. 22–23

M SKiC Helps Instruction Tuning

  • Fine-tuning LLAMA2 on GSM8K reasoning steps labeled in SKiC format improves MATH performance over CoT-labeled data in zero-shot standard prompting.MATH evaluation problems are described as significantly harder than the GSM8K training problems; the authors frame this as better weak-to-strong generalization.Source: M SKiC Helps Instruction Tuning · p. 23

O The Most Frequently Used Skills by GPT-4 for Solving MATH Benchmark

The section describes a GPT-4 MATH skill-frequency table that separates skills supplied in the SKiC context from skills drawn from pretrained internal knowledge.

O The Most Frequently Used Skills by GPT-4 for Solving MATH Benchmark

Loading 2308.00304v3…