Source-linked AI summary
Progressive-Hint Prompting Improves Reasoning in Large Language Models
Chuanyang Zheng, Zhengying Liu, Enze Xie, Zhenguo Li, Yu Li
TL;DR
LLM reasoning methods do not fully exploit generated answers to guide later responses. PHP addresses this gap by iteratively using prior answers as hints, improving benchmark performance and combining with CoT and self-consistency. Its prompts are handcrafted, and the paper identifies automated hint construction as future work.
Problem
Existing reasoning prompts and self-consistency methods do not fully exploit LLM-generated answers to guide subsequent responses.
Method
PHP sequentially combines a question with prior answers as hints, repeating interactions until answers stabilize, and can operate alongside CoT and self-consistency.
Results
PHP improves performance across reasoning benchmarks, achieving state-of-the-art results with GPT-4 on SVAMP, GSM8K, AQuA, and MATH.
Takeaways & Limitations
PHP provides an automatic sequential-interaction approach that can further improve LLM reasoning when combined with established prompting and sampling methods.
Takeaways & Limitations
The progressive-hint prompts are handcrafted, and the paper leaves automatic hint construction for future work.
Abstract
from arXiv · showhide
The performance of Large Language Models (LLMs) in reasoning tasks depends heavily on prompt design, with Chain-of-Thought (CoT) and self-consistency being critical methods that enhance this ability. However, these methods do not fully exploit the answers generated by the LLM to guide subsequent responses. This paper proposes a new prompting method, named Progressive-Hint Prompting (PHP), that enables automatic multiple interactions between users and LLMs by using previously generated answers as hints to progressively guide toward the correct answers. PHP is orthogonal to CoT and self-consistency, making it easy to combine with state-of-the-art techniques to further improve performance. We conducted extensive and comprehensive experiments on seven benchmarks. The results show that PHP significantly improves accuracy while remaining highly efficient. For instance, with text-davinci-003, we observed a 4.2% improvement on GSM8K with greedy decoding compared to Complex CoT, and a 46.17% reduction in sample paths with self-consistency. With GPT-4 and PHP, we achieve state-of-the-art performances on SVAMP (89.1% -> 91.9%), GSM8K (92% -> 95.5%), AQuA (76.4% -> 79.9%) and MATH (50.3% -> 53.9%).
1 Introduction
Progressive-Hint Prompting (PHP) addresses iterative reasoning refinement by reusing generated answers as hints, complementing Chain-of-Thought and self-consistency. It improves reasoning performance, including state-of-the-art results with GPT-4 and Complex CoT.
- 1 Introduction: PHP complements CoT and self-consistency, which respectively improve reasoning through designed prompts and majority voting over sampled answers.The paper presents these approaches as compatible rather than conflicting.
- 1 Introduction: PHP reuses generated answers as hints in sequential interactions, stopping when two consecutive answers match.It combines the question with a base answer to obtain subsequent answers and uses answer stability as the stopping signal.
- 1 Introduction: 91.9% on SVAMP, 95.5% on GSM8K, 79.9% on AQuA, and 53.9% on MATH were achieved with GPT-4, Complex CoT, and PHP.The paper reports these as state-of-the-art performances after adding PHP.
- 1 Introduction: The method was evaluated through extensive experiments, baseline comparisons, and ablation studies across four language models.The listed models include text-davinci-002, text-davinci-003, GPT-3.5-Turbo, and GPT-4.
2 Related Work
Related work studies how prompting and answer aggregation elicit or recover reasoning paths in language models. PHP builds on these strands by iteratively reusing model outputs rather than relying only on prompt design or sampled-path aggregation.
- 2 Related Work: Multi-step reasoning is especially relevant because larger models and few-shot prompting can outperform smaller models and full-dataset fine-tuning, respectively.The passage contrasts these gains with more limited improvements on tasks such as sentiment classification.
- 2 Related Work: Chain-of-Thought, Least-to-Most, Complex CoT, Auto-CoT, and prompt phrasing methods improve multi-step reasoning through structured or increasingly complex prompts.These approaches vary in how they decompose problems, select examples, or reduce manual prompt construction.
- 2 Related Work: Self-Consistency samples multiple reasoning paths and aggregates their answers, extending reasoning-path methods beyond a single generation.The related work also describes task-specific graph, retrieval, annotation, and extraction approaches.
3 Progressive-Hint Prompting
PHP creates subsequent reasoning interactions by combining a question with prior answers as hints, using prompt variants built from Standard, CoT, or Complex CoT. The interaction ends when consecutive responses stabilize.
- 3 Progressive-Hint Prompting: Table 1 illustrates PHP-CoT by contrasting Base CoT with the handcrafted hint added to the designed prompt.Its example solves a tree-counting problem, with the base answer concluding that six trees were planted.
- 3 Progressive-Hint Prompting: PHP combines a base answer with the question to generate subsequent answers, repeating the process until two consecutive responses are identical.This operationalizes sequential re-evaluation of the question using previous answers as hints.
- 3 Progressive-Hint Prompting: The pipeline has two stages: base-answer generation with a base prompt, followed by subsequent-answer generation with a corresponding PHP prompt.The method defines PHP-CoT and PHP-Complex CoT from CoT and Complex CoT base prompts.
- 3 Progressive-Hint Prompting: PHP prompt design considers both hints matching the correct answer and hints differing from it, requiring the model to retain correct hints or escape incorrect ones.The design principle explicitly addresses both potential hint conditions.
- 3 Progressive-Hint Prompting: PHP is presented as a pipeline intended to improve response quality and communication in question-answer scenarios.
4 Experiments
Experiments across seven reasoning datasets evaluate PHP with multiple models, prompts, and decoding settings. PHP generally improves performance, especially with stronger models and prompts, while also improving self-consistency efficiency.
- 4.1 Main Results: PHP improves reasoning performance across models and prompting methods, with larger and more consistent gains for stronger models and prompts.With text-davinci-003, PHP-Complex CoT improved performance by 4.6%, compared with 3.6% for text-davinci-002; Complex CoT also produced larger gains than weaker prompts.
- 4.2 Impact of the Hint Quality: PHP-Complex CoT improved GSM8K from 16.0% with Standard base answers to 50.2% with CoT and 60.3% with Complex CoT.The results show that the quality of the base answer used as a hint substantially affects final performance.
- 4.2 Impact of the Hint Quality: 96.8% accuracy on MultiArith was achieved by CoT with PHP-Complex CoT, while Complex CoT with PHP-CoT reached 95.6%.These results indicate that PHP can perform better when its subsequent prompt is not derived from the same base prompt.
- 4.3 Ablation Study: Adding P1 and P2 improved Complex CoT on five of six datasets, including SVAMP from 78.0% to 80.0% and GSM8K from 68.3% to 71.6%.The ablation identifies both answer-hint statements as useful components, particularly with stronger prompts and models.
- 4.3 Ablation Study: Non-merge PHP-Complex CoT with P1 and P2 achieved the best performance among Complex CoT-based methods, whereas CoT-Merge was strongest among CoT-based methods.The comparison supports non-merge designs when prompts are more powerful.
- 4.1 Main Results: PHP improved GPT-3.5-Turbo by 2.3% on GSM8K and 3.2% on AQuA, while GPT-4 achieved new state-of-the-art results on SVAMP, GSM8K, AQuA, and MATH.GPT-4 also required fewer interactions than GPT-3.5-Turbo.
5 Conclusion
PHP improves mathematical reasoning, achieves state-of-the-art results on several benchmarks, combines with CoT and self-consistency, and motivates further work on hint and prompt design.
- 5 Conclusion: PHP achieves substantial improvements on math reasoning tasks and state-of-the-art results on several reasoning benchmarks.The method also benefits more powerful models and prompts, according to the paper’s conclusion.
- 5 Conclusion: PHP can be combined with Chain-of-Thought and self-consistency to further improve performance.
- 5 Conclusion: Future research should improve handcrafted hints, prompt sentences, and additional hints that help LLMs reconsider questions.
6 Implementation Details
The supplementary materials contain the paper’s code.
- 6 Implementation Details: The paper provides its code in the supplementary materials.
7 Limitation and Further Work
The paper identifies handcrafted prompts, answer-only hints, and a minimum of two interactions as limitations of PHP, motivating automated hints and cost-reduction strategies.
- The paper discusses these limitations and possible directions for improving progressive-hint prompting.
- Handcrafted progressive-hint prompts motivate future work on automatically building and updating hints during testing.The authors propose Auto Progressive Hint to improve efficiency.
- PHP defines hints as previous answers, while future hints could also include model confidence, reasoning paths, or interaction numbers.
- At least two interactions are required to determine final answers, which can increase greedy-decoding costs relative to self-consistency.The authors suggest using a powerful model for the base answer and cheaper models afterward to reduce overall cost.
8 Broader Impacts
The paper describes PHP as applicable to educational assistance, complex mathematical problem solving, and other reasoning-related applications.
- PHP may assist students, particularly those from low-income areas, in learning more effectively and obtaining accurate answers.
- The method may also aid mathematicians solving complex mathematical problems and support other reasoning-related applications.
9 Model Response with Different Hints
The example shows that PHP's answer hints can either mislead or improve an LLM response depending on how close the hint is to the correct answer.
- Hints far from the correct answer can mislead the model, whereas more effective hints can produce more precise outcomes.
10 Interaction Cases
The interaction cases illustrate PHP repeatedly feeding prior answers back as hints, with outcomes depending on hint quality and the model's subsequent reasoning.
- PHP interaction examples repeatedly prepend earlier answers as hints before asking the model to solve the question again.
- A track-relay example shows successive hints of -2, 145, -2, and 2 eventually yielding the correct 2-second margin.
- A shopkeeper example uses hints near candidate answers, including (B) and (C), to guide repeated reasoning about restoring a discounted price.
- The interaction cases also include feed-allocation and probability questions, demonstrating PHP across varied quantitative problems.
11 Standard Prompt
The standard-prompt examples pair direct answers with PHP prompts that feed earlier answers back as hints. Across arithmetic and multiple-choice questions, the PHP examples retain the corresponding answers while illustrating iterative hint use.
- 11.1 For AddSub, MultiArith, SingleEQ, SVAMP and GSM8K: The base standard prompts answer arithmetic questions directly, including tree planting, car counts, chocolate totals, and lollipop differences.
- 11.1 For AddSub, MultiArith, SingleEQ, SVAMP and GSM8K: The remaining base examples cover toy totals, computer installations, golf-ball losses, and money remaining after purchases.
- Progressive-Hint Prompting Standard (PHP-Standard): PHP-Standard appends one or more answer hints to the tree, car, chocolate, and lollipop questions before producing the same final answers.
- Progressive-Hint Prompting Standard (PHP-Standard): PHP-Standard similarly uses prior-answer hints for toy, computer, golf-ball, and money questions, while returning their displayed answers.
- 11.2 For AQuA: The AQuA base prompts provide selected answers for mean shifting, a ratio system, travel distance, and keystroke counting.
- Progressive-Hint Prompting Standard (PHP-Standard): AQuA PHP-Standard examples insert answer-choice hints before solving the mean, ratio, travel, and keystroke questions.
12 Chain-Of-Thought Prompt
The Chain-of-Thought examples show direct step-by-step derivations, while PHP-CoT prepends previous-answer hints to the same reasoning process. The examples span elementary arithmetic and AQuA-style quantitative questions.
- Base Chain-Of-Thought: Base Chain-of-Thought derives answers by explicitly showing arithmetic for tree planting, car counts, chocolate totals, and lollipop subtraction.
- Base Chain-Of-Thought: Further Base Chain-of-Thought examples calculate toy totals, computer growth, golf-ball losses, and remaining money through intermediate operations.
- Progressive-Hint Prompting Chain-Of-Thought (PHP-CoT): PHP-CoT adds answer hints to the arithmetic questions while preserving explicit derivations for tree planting, car counts, and lollipop subtraction.
- Progressive-Hint Prompting Chain-Of-Thought (PHP-CoT): The chocolate PHP-CoT example separates the hint-bearing prompt from the subsequent derivation, which computes 32 + 42 - 35 = 39.
- Progressive-Hint Prompting Chain-Of-Thought (PHP-CoT): PHP-CoT also combines hints with step-by-step solutions for toy totals, computer installations, golf-ball losses, and money remaining.
- Base Chain-Of-Thought: Base Chain-of-Thought solves the AQuA mean, ratio, travel, and keystroke questions by presenting intermediate calculations before the selected answer.
- Progressive-Hint Prompting Chain-Of-Thought (PHP-CoT): PHP-CoT adds answer hints to AQuA reasoning, with the supplied examples showing explicit derivations for the mean and ratio problems.
13 Complex Chain-Of-Thought Prompt
Complex Chain-of-Thought examples provide extended multi-step solutions, while PHP-Complex CoT adds progressive answer hints to the same style of reasoning. The examples cover planning, counting, pricing, profits, and comparisons.
- Base Complex Chain-Of-Thought: Base Complex Chain-of-Thought applies extended arithmetic to study planning, basketball scoring, marble and frisbee counts, fruit baskets, and fruit pricing.
- Base Complex Chain-Of-Thought: The base examples explicitly report 140 total items for Bella’s collection and 115 dollars in profit for Sam’s highlighter sales.
- Base Complex Chain-Of-Thought: The base follower comparison calculates 170 followers for Susy and 180 for Sarah after three weeks.
- Progressive-Hint Prompting Complex Chain-Of-Thought (PHP-Complex CoT): PHP-Complex CoT adds answer hints to the basketball problem and then derives a total score of 201 points.
- Progressive-Hint Prompting Complex Chain-Of-Thought (PHP-Complex CoT): PHP-Complex CoT similarly supplies hints for Bella’s item count and the fruit-basket calculation while continuing the multi-step solution format.
- Progressive-Hint Prompting Complex Chain-Of-Thought (PHP-Complex CoT): The hinted apple-pricing example uses the relation 1W=4A and concludes that one apple costs $1.
- Progressive-Hint Prompting Complex Chain-Of-Thought (PHP-Complex CoT): The hinted follower comparison carries out the same three-week calculations after presenting 180 and 160 as answer hints.