PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives

Wenqi Zhang, Yongliang Shen, Linjuan Wu, Qiuying Peng, Jun Wang, Yueting Zhuang, Weiming Lu

arXiv:2401.02009v3cs.CLcs.AI

TL;DR

Without external feedback, LLM self-reflection is unreliable, and self-evaluated feedback can be overconfident or inconsistent. Self-Contrast generates diverse perspectives, compares their responses, and uses discrepancies to build a checklist; experiments report improvements across reasoning and translation tasks.

  • Problem

    Without external feedback, LLMs have difficulty correcting prior responses, and self-evaluated feedback can be overconfident or inconsistent.

    Source: §1 · pp. 1–2

  • Method

    Self-Contrast adaptively generates request-tailored solving perspectives, contrasts their responses, and turns discrepancies into a checklist for reflection.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Self-Contrast improves mathematical reasoning by an average +7.2% (vs. -0.51% for self-reflection) and creative translation by +0.95 (vs. -1.6).

    Source: §4.3 · p. 7

  • Takeaways & Limitations

    Contrasting discrepancies can surface potential errors or overlooked details and provide more informative instructions for re-examining responses.

    Source: §1 · p. 2

    For weaker instruction-following models such as Llama2-7B, Self-Contrast may be less effective than ensemble strategies.

    Source: Limitations · p. 9

Abstract

from arXiv · show

The reflection capacity of Large Language Model (LLM) has garnered extensive attention. A post-hoc prompting strategy, e.g., reflexion and self-refine, refines LLM's response based on self-evaluated or external feedback. However, recent research indicates without external feedback, LLM's intrinsic reflection is unstable. Our investigation unveils that the key bottleneck is the quality of the self-evaluated feedback. We find LLMs often exhibit overconfidence or high randomness when self-evaluate, offering stubborn or inconsistent feedback, which causes poor reflection. To remedy this, we advocate Self-Contrast: It adaptively explores diverse solving perspectives tailored to the request, contrasts the differences, and summarizes these discrepancies into a checklist which could be used to re-examine and eliminate discrepancies. Our method endows LLM with diverse perspectives to alleviate stubborn biases. Moreover, their discrepancies indicate potential errors or inherent uncertainties that LLM often overlooks. Reflecting upon these can catalyze more accurate and stable reflection. Experiments conducted on a series of reasoning and translation tasks with different LLMs serve to underscore the effectiveness and generality of our strategy.

1 Introduction

Without external feedback, LLM self-reflection is unreliable, in part because self-evaluation often produces overconfident or inconsistent feedback. Self-Contrast instead compares responses from diverse perspectives and turns discrepancies into a checklist for revision.

1 Introduction

  • Only 15.1% of incorrect responses are corrected through reflection; self-evaluations are overconfident in 46.7% of cases or inconsistent in 45.7%.The authors identify inaccurate self-evaluation as a bottleneck for reflection.Source: §1 · p. 2
  • Self-Contrast creates diverse solving perspectives, contrasts their responses, and uses the discrepancies to generate re-examination instructions.The authors argue that discrepancies can point to potential errors, overlooked details, or pitfalls.Source: §1 · p. 2
  • Across mathematical reasoning and challenging translation, Self-Contrast shows significant improvements and stability compared with vanilla reflection.The introduction presents these experiments as evidence of the strategy’s effectiveness across tasks.Source: §1 · p. 2

2 Evaluation of Intrinsic Reflection

Across the evaluated benchmarks, intrinsic reflection yielded little performance change. Feedback analysis found that erroneous responses often remained uncorrected amid overconfident or inconsistent self-evaluation, motivating a contrastive alternative.

2.1 Performance Pre- and Post-Reflection

  • GPT-3.5’s GSM8K accuracy changed by -0.8% and SVAMP accuracy by +0.7% after reflection, both statistically insignificant.Across models and benchmarks, changes were negligible; reasoning often declined slightly, while translation changed little.Source: §2.1 · pp. 2–3

2.2 Feedback Analysis

  • Invalid reflections outnumbered valid corrections 269 to 48, while invalid-case feedback was often overconfident (53.5%) or highly inconsistent (45.3%).Toxic-reflection evaluations were highly inconsistent in 65.4% of cases.Source: §2.2 · p. 3

2.3 From Self-Evaluate to Self-Contrast

  • On GSM8K, contrasting two responses and generating a checklist modestly improved standard reflection and shifted the t-test p-value from 0.6613 to 0.0933.The section frames this comparison as validation of contrastive evaluation before expanding it into full Self-Contrast.Source: §2.3 · p. 4

3 Self-Contrast

Self-Contrast develops multiple request-specific perspectives, compares their responses, and uses the resulting discrepancies to guide revision toward consistency. Its checklist directs re-examination toward potential errors, overlooked details, and biases.

3 Self-Contrast

  • Self-Contrast has three stages: create diverse perspectives, contrast inter-perspective discrepancies, and eliminate discrepancies using a checklist.The checklist guides examination of random errors or intrinsic biases behind inconsistent results.Source: §3 · p. 4

3.1 Create Diverse Perspectives

  • For each request, the LLM adaptively creates multiple prompts with tailored perspectives and samples responses; prompt count and design are left to the model.Perspectives can vary by role, personality, or thought style, broadening problem-solving approaches and mitigating single-prompt biases.Source: §3.1 · pp. 4–5

3.2 Contrast Inter-Perspective Discrepancies

  • Similar responses are filtered out, then selected responses are compared pairwise to identify differences, their causes, and potential errors.K-Medoids clustering selects representative responses from semantic-similarity clusters.Source: §3.2 · pp. 5–6

3.3 Eliminate Discrepancies

  • The LLM turns pairwise differences into a detailed checklist for re-examining intent, contradictions, or calculations, then revises candidates toward consistent answers.The checklist is more specific than a generic instruction to check a previous response.Source: §3.3 · p. 6

4 Experiments

Experiments test Self-Contrast on mathematical reasoning and creative translation across several LLMs, against prompting, reflection, ensemble, and multi-agent baselines. The authors report stronger and more stable performance than vanilla reflection, generality across most tested models, and lower call overhead than multi-agent debate.

4.1 Settings

  • The evaluation covers GSM8K, SVAMP, and CommonMT, using answer precision for mathematical reasoning and BLEURT for translation.Tests use GPT-3.5-Turbo-0613, GPT-4-0613, and Llama2-Chat at 7B, 13B, and 70B scales, with temperature 0.2.Source: §4.1 · p. 6

4.2 Baselines

  • Baselines include standard Chain-of-Thought, self-reflection, multi-agent debate, ExpertPrompt, Hint-Prompt, Math-Prompt, and three Self-Consistency variants.The variants sample multiple responses and then vote, select an answer, or reflect on the candidates.Source: §4.2 · p. 6

4.3 Main Results

  • Self-Contrast improves over vanilla reflection by an average of +7.2% on mathematical reasoning and +0.95 on creative translation, while self-reflection changes by -0.51% and -1.6, respectively.The reported gains are also pronounced and consistent against multi-agent and ensemble baselines.Source: Table 4 · p. 7, Table 5 · p. 7, §4.3 · p. 7
  • Self-Contrast achieves the best results on most tested models except Llama-2-7B, and applies to both reasoning and generative tasks, including translation.For GPT-3.5, improvements are 7.8% on GSM8K and 9.2% on SVAMP; for Llama-2-70B, they are 11.6% and 9.3%, respectively.Source: §4.3 · p. 7
  • Compared with multi-agent debate, Self-Contrast reports more significant improvements with more than 10% lower call overhead.The method dynamically designs perspectives for requests without manually preconfiguring agent roles and quantities.Source: §4.3 · p. 7

5 The Effect of the Different Components

The component analysis examines self-curated prompts and checklist generation. Both contribute to the reported performance, with the checklist’s effect larger for mathematical reasoning than for translation.

The Effect of the Different Components

  • The full strategy combines self-curated diverse perspectives with checklist generation to support re-examination and discrepancy resolution.The analysis treats these as distinct components of Self-Contrast.Source: §5 · p. 7

Self-curated Prompt Vs. Sampling Multiple Responses

  • With self-curated prompts, accuracy reaches 84.4%, compared with a maximum of 81.8% from sampling five responses.Accuracy rises as more responses are sampled, but top-ranked responses can be too similar to make contrast effective.Source: §5 · pp. 7–8

Reflection Without Checklist

  • Removing checklist generation reduces mathematical reasoning performance by 3.5% and translation performance by 0.1%.Without a checklist, the model is instructed to reflect directly on perspective differences.Source: §5 · p. 8

6 Analysis

The analysis links Self-Contrast’s reflection gains to fewer invalid and toxic reflections and to the contrast between dissimilar solutions. It also reports that adaptive, diverse perspectives produce varied results for subsequent comparison.

6.1 Reducing Invalid and Toxic Reflections

  • On GSM8K with GPT-3.5, Self-Contrast reduces toxic reflections by 78.9% and invalid reflections by 30.8%.The authors report that exploration, comparison, and summarization reduce uncertainty and enhance error-correction capability.Source: §6.1 · p. 8

6.3 Diverse Solving Perspectives Maximize Contrast Effect

  • For most requests, the model generates four prompts, and mathematical reasoning prompts adaptively produce unique perspectives and varied results.The authors argue that these dissimilar results maximize the efficacy of contrast; Figure 3 shows prompt-number distribution and prompt-name keywords.Source: Figure 3 · p. 8, §6.3 · p. 9

6.2 Contrasting Incorrect Solutions is also Instructive

  • In the experiments, contrasting a correct and an incorrect solution or two incorrect solutions with different errors yields significant enhancements of 13.5% and 5.4%, respectively.Comparing two solutions with similar errors produces no perceptible change; the authors attribute the improvement to contrasting dissimilar solutions.Source: §6.2 · p. 8

7 Discussions

The discussion frames Self-Contrast as difference-finding followed by checklist-based re-examination. It reports an improvement from direct contrastive evaluation and illustrates the process with a translation example.

7 Discussions

  • On GSM8K, changing from direct evaluation to contrastive evaluation raises the reported result from 75.8 to 77.5, with the accompanying value changing from 0.66 to 0.09.The passage describes the contrastive approach as more effective for reflection.Source: §7 · p. 9
  • After comparing solutions, the model summarizes their differences into a checklist that directs reflection toward logical pitfalls and related issues.The authors say this supports clearer, more purposeful reflection.Source: §7 · p. 9
  • The translation example contrasts “This plan was shot to death” with “This plan was axed,” prompting scrutiny of metaphor and English usage conventions.The former is described as a rigid translation that misses the metaphor in the military term.Source: §7 · p. 9

8 Conclusion

The paper finds that LLMs struggle to correct prior responses without external feedback, in part because self-evaluation can be inaccurate or inconsistent. It introduces Self-Contrast and reports performance across varied scenarios and models.

8 Conclusion

  • Without external feedback, LLMs struggle to correct prior errors, and inaccurate or overconfident and inconsistent self-evaluations impede reflection.The paper identifies self-evaluated feedback quality as a bottleneck.Source: §8 · p. 9
  • Self-Contrast compares multiple perspectives and uses their differences to produce a checklist for re-examining responses.The checklist is intended to help resolve discrepancies.Source: §8 · p. 9
  • Experiments show that Self-Contrast performs well across a variety of scenarios and with different LLMs.Source: §8 · p. 9

Limitations

Self-Contrast may be less effective with some smaller-scale LLMs whose instruction-following is weaker. The authors identify external comparison tools as a possible direction for future work.

Limitations

  • Weaker instruction-following in some smaller-scale LLMs can hinder precise comparisons and reflection.Source: Limitations · p. 9
  • With Llama2-7B, Self-Contrast performs marginally lower than self-consistency.Source: Limitations · p. 9
  • The authors leave external tools for comparing perspectives, such as difflib or rule-based strategies, as future work.Such comparisons could operate at different granularities, including character level.Source: Limitations · p. 9

A.1 Detail For Manual Feedback Evaluation

The appendix details manual feedback evaluation, tests whether initial-response quality affects reflection, compares contrasting with direct evaluation, and examines response-selection strategies.

A.1 Detail For Manual Feedback Evaluation

  • Manual evaluation classifies reflection outcomes as Invalid, Valid, Toxic, or Other, then assesses feedback quality within each category.The categories correspond to wrong-to-wrong, wrong-to-right, right-to-wrong, and right-to-right outcomes.Source: §A.1 · p. 15
  • More than seven of ten feedbacks identifying errors counts as accurate error identification; more than seven denying errors counts as overconfidence.More than three distinct opinions among ten feedbacks counts as inconsistent feedback; remaining cases are classified as stubbornly erroneous.Source: §A.1 · p. 15
  • Two senior PhD students conduct the human evaluation, with one categorizing cases and the other verifying those categories.Source: §A.1 · p. 15

A.2 LLM is More Likely to Trust Previous Response

  • -8.4% for GSM8K when using Llama2-70b→gpt3.5→gpt3.5 instead of gpt3.5 for all three phases.The reflection results are severely impacted by the initial response's quality, and weaker substituted LLMs correspond to poorer post-reflection performance.Source: Figure A1 · p. 15, §A.2 · p. 15

A.3 Self-Evaluate Vs. Self-Contrast

  • In manually checked comparison of contrasting two solutions with evaluating an incorrect solution, contrasting was more accurate: 171 correct Vs. 140 incorrect.Contrast succeeded where direct evaluation failed in 39 cases; evaluation was correct alone in only 8 samples.Source: §A.3 · p. 16

A.4 Ablation Study For Selection Strategy

  • Four response-selection strategies are compared, and several perform worse than Self-Contrast.The strategies are random selection, clustering plus random selection, clustering plus LLM selection, and clustering plus a negative perspective.Source: §A.4 · p. 16, Table A1 · p. 16

B.1 Benchmarks

This section describes mathematical reasoning and creative translation benchmarks, along with task-adaptive prompts for generating, contrasting, and revising diverse candidate responses. It also discusses revising inconsistent solutions through Self-Contrast, in the context of concerns about self-correction without external feedback and the method’s combination of breadth and depth reasoning.

B.1 Benchmarks

  • GSM8K and SVAMP evaluate mathematical reasoning at different difficulty levels, while CommonMT’s hard Chinese-to-English examples test culturally nuanced creative translation.GSM8K involves more complex operations; SVAMP uses combinations of arithmetic operations. CommonMT examples often include idioms and metaphors.Source: §B.1 · pp. 16–17

B.2 Other Details

  • For each request, the method generates 2–9 prompts, selects one result from each of three categories, and turns their differences into a checklist in one pass; translation also uses a deliberately careless negative perspective.The negative perspective generates common mistakes as a negative demonstration for later selection and reflection.Source: §B.2 · p. 17

C Baseline Prompts

  • CoT and self-consistency use a simple standard prompt and are run 10 times with averaged results, while the self-reflection baseline uses 10 evaluation-and-revision prompts.The reflection baseline follows an initial response–evaluation–revision pattern and keeps the initial-response prompt consistent with the standard prompt.Source: C Baseline Prompts · p. 17

D.1 Prompt for Self-Curated Process

  • The self-curated process dynamically generates 2–9 request-specific prompts that guide distinct translation or mathematical reasoning perspectives and emphases.Translation prompts can vary style, tone, emotion, or background; math prompts can vary reasoning approach and focus on entities, numbers, units, or percentages.Source: §D.1 · p. 18

D.2 Prompt for Contrasting Process

  • The contrasting process compares candidate translations across semantics, syntax, and word choice, or candidate math solutions across steps, then summarizes discrepancies and their causes as checklist instructions.The checklist guides re-examination of the input and candidates to eliminate discrepancies.Source: §D.2 · p. 19

D.3 Prompt For Reflection Stage

  • At reflection, the model revises inconsistent solution steps according to the checklist and repeats revision until all revised solutions have the same answer and inconsistencies are removed.The prompt requests a new solving process for each solution.Source: §D.3 · p. 19
  • The reflection-stage prompt asks the model to revise inconsistent solutions and output the revised solutions in JSON format.Source: §D.3 · p. 20

E.1 Self-correction Ability of LLM

  • Prior studies report that LLMs struggle to amend prior responses through inherent reflection when external feedback is unavailable.Source: §E.1 · p. 20

E.2 Prompting for Better Problem-Solving

  • Self-Contrast combines breadth reasoning, by creating multiple perspectives, with depth reasoning, by reflecting on their differences to improve problem-solving reliability.The passage contrasts this combination with approaches that primarily sample diverse reasoning or iteratively refine reasoning.Source: §E.2 · p. 20

E.3 Agent-based Methods

Agent-based methods use multiple roles or agents to build more comprehensive responses, while Self-Contrast lets the LLM determine roles and their number from the request. Mathematical-reasoning work spans prompting strategies and fine-tuning approaches.

E.3 Agent-based Methods

  • Role personas and multi-role dialogues have been used to elicit higher-quality responses and more comprehensive answers (Li et al., 2023; Deshpande et al., 2023; Xu et al., 2023a; Du et al., 2023; Xiong et al., 2023; Wang et al., 2023e; Fu et al., 2023; Liang et al., 2023; Schick et al., 2022; Dong et al., 2023; Park et al., 2023; Liu et al., 2023).Other agent-based work decomposes complex tasks such as code generation into subtasks assigned to agents with different identities (Chen et al., 2023a).Source: §E.3 · pp. 20–21
  • Unlike agent-based approaches requiring manually designed roles and interaction patterns, Self-Contrast lets the LLM determine roles and their number from the user request.This offers greater flexibility.Source: §E.3 · p. 21

E.4 Learning Mathematical Reasoning

  • Chain-of-Thought and other prompting strategies have augmented mathematical reasoning, alongside approaches that separate computation from reasoning, generate diverse reasoning paths, optimize demonstrations, or use search trees and self-evaluation (Zhu et al., 2023; Yuan et al., 2023b; Frieder et al., 2023; Zhou et al., 2022; Gao et al., 2022; Chen et al., 2022; Imani et al., 2023; Zhao et al., 2023; Zhang et al., 2022d; Fu et al., 2022; Shao et al., 2023; Pitis et al., 2023; Yao et al., 2023; Xie et al., 2023).PAL and Program-of-Thoughts use code as an intermediate process; other methods generate text, code, or equations, or extend Chain-of-Thought into a search tree.Source: §E.4 · p. 21
  • Fine-tuning open-source models with insights from closed-source LLMs and process-supervised training are also approaches to improving mathematical reasoning (Yuan et al., 2023a; Luo et al., 2023; Yue et al., 2023; Wang et al., 2023b; Yu et al., 2023; Gou et al., 2023; Lightman et al., 2023; Wang et al., 2023c).Fine-tuning centers on developing quality question-response datasets.Source: §E.4 · p. 21
Loading 2401.02009v3…