On the Intersection of Self-Correction and Trust in Language Models

1 INTRODUCTION

• LLMs’ complexity, limited transparency, and susceptibility to misinformation and bias raise concerns about their trustworthiness.The paper also notes that biased or incorrect training information may be reflected in model responses.Source: §1 · p. 1
• The study investigates whether LLM self-correction can address trustworthiness concerns, focusing on toxicity and truthfulness.LLMs are prompted to self-correct and their performance is analyzed.Source: §1 · pp. 1–2
• Self-correction can improve toxicity and truthfulness, but the extent of improvement varies with the trustworthiness aspect and task.The study also reports instances of self-doubt during correction.Source: §1 · p. 2
• During self-correction, models sometimes change initially correct responses, a behavior the authors describe as “self-doubt.”The authors identify this as a challenge alongside potential performance improvements.Source: §1 · p. 2

Trustworthy ML

• Research on trustworthiness seeks to understand, measure, and mitigate concerns about complex models, whose decisions can be difficult to understand.The related-work discussion connects these concerns to LLMs and other complex generative models.Source: §2 · p. 2

In-Context Learning

• In-context learning and chain-of-thought prompting have been associated with improved LLM performance across challenging tasks.The section describes both human-annotated chain-of-thought examples and instructions to think step by step.Source: §2 · pp. 2–3
• Prior self-correction studies examine reducing prediction errors, but the authors identify limited research on using these abilities to improve LLM trustworthiness.The cited studies describe models searching for correct answers with additional context or feedback.Source: §2 · p. 3

3 SELF-CORRECTION

• The framework aims to let LLMs reassess responses and improve them without external human supervision.It comprises a Start Prompt followed by a Self-Correction Prompt.Source: §3 · p. 3
• The Start Prompt introduces the task, indicates an expected method such as step-by-step reasoning or few-shot learning, and supplies a test sample.This resembles prompting the model once without a self-correction stage.Source: §3 · p. 3
• The Self-Correction Prompt re-presents the start prompt and the model’s response so it can reassess and improve its answer.The study uses minimal human involvement, letting the LLM provide feedback and evaluate its response, and repeats this stage N times.Source: §3 · p. 3

4 EXPERIMENTAL EVALUATION

• Truthfulness is evaluated on TruthfulQA, while toxicity is examined on samples labeled challenging in Real-Toxicity.The paper selects these as two aspects of trustworthy machine learning.Source: §4 · p. 3
• The toxicity start prompt asks for non-toxic completions, while the truthfulness prompt asks for step-by-step answers without factual errors.The prompts specify how the final response should be presented.Source: §4 · p. 4
• Experiments use gpt-3.5-turbo-0613, with task-specific start and self-correction prompts and temperatures of 0.1 and 0.8.Truthfulness is measured by accuracy, toxicity by the Perspective API, and experiments use seven self-correction steps because additional steps would exceed the model’s maximum context length.Source: §4 · pp. 3–4

5.1 TOXICITY

• After one self-correction step, toxicity declined from 0.31 at self-correction-0.1 and 0.27 at self-correction-0.8 to 0.18 and 0.17, respectively—a 40–45% decline.Source: §5.1 · p. 5
• Most toxicity reduction occurred in early correction steps, through more decreases in toxicity than increases.The analysis attributes most score changes to detoxification; average increases were barely changed across iterations.Source: §5.1 · p. 5
• About 70% of samples became less toxic after the first correction, versus about 10% that became more toxic; increases grew with later steps but remained less common.The authors interpret the increases as signs of self-doubt in toxicity judgments.Source: §5.1 · p. 6
• A highly toxic prompt remained toxic despite a non-toxic continuation, and the model struggled to reduce its toxicity during self-correction.In another sample, toxicity fell from 0.7 to 0.1 after one correction step.Source: §5.1 · pp. 7–8

5.2 TRUTHFULNESS

• Cumulative TruthfulQA accuracy rose with correction steps, with T=0.1 outperforming T=0.8; per-step accuracy sharply declined, and random cumulative accuracy surpassed T=0.8 after five steps.The random baseline performed worst for accuracy at each individual step.Source: Figure 5 · p. 6, §5.2 · p. 8
• At the first correction step, more than 0.5 of samples changed answers under self-correction, compared with 0.37 under the random baseline.The authors suggest self-correction prompting can prompt answer flips.Source: §5.2 · p. 8
• TruthfulQA correct-to-incorrect shifts were about 0.45, versus about 0.15 for incorrect-to-correct shifts; the difference narrowed over later steps.The difference remained close to the same for the random baseline.Source: §5.2 · p. 8
• One TruthfulQA answer changed from correct to incorrect and then back to correct, with the model's reasoning suggesting confusion.The authors present this sequence as an example of self-doubt during self-correction.Source: §5.2 · pp. 8–9

• Self-correction can improve both truthfulness and toxicity, but the extent of improvement varies by aspect and task.Source: §6 · p. 9
• The study uncovers instances of self-doubt during self-correction, raising challenges that need to be addressed.Source: §6 · p. 9
• The experiments investigate self-correction in relation to two trustworthiness aspects: truthfulness and toxicity.Source: §6 · p. 9