Teaching Large Language Models to Self-Debug

• Single-attempt code generation is difficult, while reranking methods typically require tens of sampled programs and repair approaches require separately trained repair models.Although prior language-model feedback methods critique outputs in other domains, large language models were reported unable to correct code without external feedback such as tests or human instructions.Source: §1 · pp. 1–2
• SELF-DEBUGGING uses few-shot prompting to execute predicted code, explain it in natural language, inspect execution results, and iteratively refine the program without additional training.When unit tests are unavailable, feedback can rely purely on code explanation, resembling rubber duck debugging.Source: §1 · p. 2, Figure 1 · p. 2
• 2−3% gains on Spider, 9% higher accuracy on the hardest SQL queries, and up to 12% accuracy improvements on TransCoder and MBPP establish state-of-the-art performance.SELF-DEBUGGING also matches or outperforms baselines sampling more than 10× predictions.Source: §1 · p. 2
• The method is evaluated across GPT-family models and StarCoder on text-to-SQL generation, code translation, and text-to-Python generation.On Spider, code explanation improves performance without unit tests; on TransCoder and MBPP, unit tests combined with explanation provide additional gains.Source: §1 · p. 2

Few-shot prompting

• Few-shot prompting uses several input-output demonstrations to guide an LLM toward generating code in the demonstrated format (Brown et al., 2020).For text-to-SQL, the prompt prepends the target question with question–SQL pairs; prompts may also include high-level task instructions (Ouyang et al., 2022; Sanh et al., 2022; Suzgun et al., 2022).Source: §2 · pp. 2–3

Execution-based code selection

• When multiple predictions are available, the method selects the code with the most frequent execution result among candidates without execution errors, then applies SELF-DEBUGGING.This follows execution-based majority voting rather than reranking schemes (Li et al., 2022; Shi et al., 2022; Zhang et al., 2022; Ni et al., 2023; Zeng et al., 2022).Source: §2 · p. 3
• When unit tests are provided, programs that fail any required input-output test are filtered out before execution-based majority voting (Chen et al., 2021a; Austin et al., 2021; Li et al., 2022; Hendrycks et al., 2021).A program passes when it produces the specified output for every test input.Source: §2 · p. 3

Framework

• SELF-DEBUGGING uses three iterative steps—Generation, Explanation, and Feedback—to debug candidate programs without finetuning the pretrained language model.The process terminates when feedback indicates correctness or when the maximum number of debugging turns is reached.Source: §3 · p. 3

Feedback types

• Feedback can provide only correctness judgments, while unit-test feedback adds runtime errors and failed-test execution results for richer debugging information.Simple feedback omits the Explanation step, whereas unit-test feedback is applicable when problem descriptions include unit tests.Source: §3 · p. 3

Code Explanation feedback

• Code Explanation feedback lets the model identify bugs by describing generated code and comparing it with the problem, especially when unit tests are unavailable.The approach is designed as rubber duck debugging rather than explicit error-message prediction.Source: §3 · p. 4

Execution trace feedback

• When unit tests are available, Execution Trace feedback asks the model to explain intermediate execution steps line by line without accessing intermediate execution states.Both execution traces and line-by-line explanations are generated by the model rather than obtained through code execution.Source: §3 · p. 4

4 APPLICATIONS

• The evaluation spans text-to-SQL, code translation, and text-to-Python generation under differing unit-test availability.Text-to-SQL uses explanation-based correctness checking, code translation uses execution feedback, and text-to-Python requires inference beyond visible tests.Source: §4 · p. 4

4.1 TEXT-TO-SQL GENERATION

• Text-to-SQL debugging summarizes the question, executes the query, explains its returned table, and compares that explanation with the question without unit tests.The process predicts correctness from the inferred return type, SQL explanation, and question description, terminating when the query is judged correct or debugging turns are exhausted.Source: §4.1 · p. 4

4.2 CODE TRANSLATION

• Code translation evaluates C++-to-Python generation on 560 problems with 10 unit tests each, iteratively repairing predictions that fail the tests.SELF-DEBUGGING uses unit-test feedback and code explanations until all tests pass or the maximum debugging turns are reached.Source: §4.2 · p. 5

4.3 TEXT-TO-PYTHON GENERATION

• Text-to-Python evaluation uses 500 MBPP problems with one visible unit test while hiding two tests for full evaluation.Unlike code translation, the model must infer correctness even when generated code passes the visible test, optionally using execution results in feedback.Source: §4.3 · pp. 5–6

5.1 MAIN RESULTS

• 75.6 with UT + Expl. exceeds 72.2 for Codex and 68.9 for LEVER, showing SELF-DEBUGGING consistently improves prior ranking approaches.On Spider, the feedback variants progress from 73.6 with Simple to 75.2 with UT and 75.6 with UT + Expl.Source: §5.1 · pp. 7, 9, Table 1 · p. 9
• Over 12% improvement on TransCoder and 8% on MBPP show GPT-4’s self-debugging gains are on par with Codex, while StarCoder gains 6% on MBPP.GPT-4 generates stronger initial MBPP code than Codex and GPT-3.5, but performs worse than Codex on Spider, where few-shot prompting is less effective.Source: §5.1 · p. 9

5.2.1 SELF-DEBUGGING IMPROVES THE SAMPLE EFFICIENCY

• SELF-DEBUGGING improves sample efficiency: Spider debugging from 8 samples outperforms the baseline using 32 samples, while one turn usually suffices.Debugging greedy predictions matches the baseline accuracy obtained with 16 samples, and the improvement after one turn is within 0.1%.Source: Figure 6 · p. 10, §5.2.1 · p. 10

5.2.2 IMPORTANCE OF CODE EXECUTION

• Code execution is crucial: Codex improves by up to 5% without execution, execution traces consistently outperform simple feedback, and GPT-4 improves MBPP by 3.6%.Without unit tests, GPT-3.5 and GPT-4 tend to be overconfident in their initial predictions, though self-generated feedback can still improve performance.Source: §5.1 · p. 8, §5.2.2 · p. 10, Table 3 · p. 10

5.2.3 ERROR TYPES FIXED BY SELF-DEBUGGING

• SELF-DEBUGGING increases Spider accuracy by 9% on extra-hard problems by exposing small SQL discrepancies, such as missing WHERE conditions or SELECT columns.Code explanation helps models identify mismatches between the question and predicted SQL, as illustrated by a corrected extra-hard example.Source: §5.2.3 · pp. 11–12, Figure 7 · p. 11
• On TransCoder and MBPP, 60%-70% of successful fixes address output mismatches in near-correct programs, including language implementation differences and argument-order or type errors.Over 30% of successful TransCoder fixes address cross-language implementation differences, while over 10% of MBPP programs are fixed by changing argument order and matching types.Source: §5.2.3 · p. 12, Table 4 · p. 12

Language models for code

• SELF-DEBUGGING differs from prior code-generation systems by targeting LLMs’ limited understanding of code execution and using execution to improve sample efficiency rather than only select candidates.Recent models achieve strong coding performance across tasks, but generated programs can violate provided unit tests; prior work mainly reranks or selects among candidates using execution outputs. (Devlin et al., 2017; Chen et al., 2019; Yu et al., 2018; Roziere et al., 2020; Chen et al., 2021a; Chowdhery et al., 2022; Nijkamp et al., 2023; Zheng et al., 2023; Xu et al., 2022; Athiwaratkun et al., 2023; Orlanski et al., 2023; Austin et al., 2021; Li et al., 2022; Shi et al., 2022; Ni et al., 2023; Zhang et al., 2022; Chen et al., 2019; 2021b; Roziere et al., 2022; Chen et al., 2023b)Source: §6 · p. 12

Prompting techniques

• SELF-DEBUGGING resembles chain-of-thought prompting through line-by-line code explanation, while decomposing debugging into separate explanation and feedback-generation stages.Unlike general methods that elicit intermediate reasoning or delegate subproblems, SELF-DEBUGGING uses explanation specifically to facilitate code analysis for debugging. (Wei et al., 2022; Nye et al., 2021; Kojima et al., 2022; Khot et al., 2022; Zhou et al., 2022; Gao et al., 2022; Chen et al., 2022)Source: §6 · pp. 12–13

Code repair

• Unlike neural repair systems trained as separate models, SELF-DEBUGGING teaches pretrained code LLMs to repair predictions through few-shot prompting with execution results and self-generated explanations.Prior repair methods use code alone or add execution traces, compiler errors, or execution errors as repair information. (Gupta et al., 2020; Wang et al., 2018; Le et al., 2022)Source: §6 · p. 13

Training with feedback

• Unlike feedback-training approaches that refine outputs through additional training or human feedback, SELF-DEBUGGING generates feedback messages at test time without extra training.Related work includes RLHF, Constitutional AI, and systems trained to optimize code, support multi-turn generation, or repair programs from human feedback. (Ziegler et al., 2019; Korbak et al., 2023; Ganguli et al., 2023; Bai et al., 2022; Ouyang et al., 2022; Welleck et al., 2023; Liu et al., 2023; Madaan et al., 2023a; Yao et al., 2019; Nijkamp et al., 2023; Iyer et al., 2017; Elgohary et al., 2020; Chen et al., 2023a; Le et al., 2022)Source: §6 · p. 13

Prompting with feedback

• SELF-DEBUGGING extends prompting-with-feedback methods to code by combining self-generated and execution-derived feedback, using rubber-duck-style explanation without requiring explicit error diagnoses or fixes.Unlike reflection, iterative revision, state-grounded control, or multi-agent dialogue methods, it teaches the model to notice prediction errors through code explanation. (Bai et al., 2022; Ganguli et al., 2023; Shinn et al., 2023; Madaan et al., 2023b; Kim et al., 2023; Nair et al., 2023; Humphreys et al., 2022)Source: §6 · p. 13

Contributions

• SELF-DEBUGGING achieves state-of-the-art performance across several code-generation domains while improving sample efficiency.It enables models to iteratively debug their own predictions rather than generate correct code from scratch.Source: §7 · p. 14

Results

• 2−3%: code explanation consistently improves text-to-SQL baseline performance, with a 9% gain on the hardest problems without unit tests.For code translation and text-to-Python tasks with unit tests, SELF-DEBUGGING increases baseline accuracy by up to 12%.Source: §7 · p. 14

Mechanism

• SELF-DEBUGGING teaches models to understand code, identify errors, follow error messages, and fix bugs without human instructions.The work highlights iterative self-debugging as a promising alternative to generating correct code in one attempt.Source: §7 · p. 14

Future Work

• Future work should improve code explanation and feedback, including high-level semantics, implementation details, and more informative predicted error messages.Preliminary results found that semantic-error feedback added no benefit beyond line-by-line explanation.Source: §7 · p. 14

Setup

• GPT-3.5 generates stronger initial Python code, whereas Codex supplies the initial Spider code because GPT-3.5 performs substantially worse on that benchmark.For Spider, the GPT-3.5 SELF-DEBUGGING evaluation uses Codex-generated initial code and zero-shot SELF-DEBUGGING prompts.Source: A SELF-DEBUGGING WITH INITIAL SAMPLES… · p. 19, Table 5 · p. 19

Results

• Codex matches or exceeds GPT-3.5 for SELF-DEBUGGING on TransCoder, MBPP, and Spider, while code explanation further improves GPT-3.5 on Spider.Codex is on par with GPT-3.5 on TransCoder, close on MBPP, and outperforms it for both Simple and UT; on Spider, Codex also performs better for text-to-SQL.Source: A SELF-DEBUGGING WITH INITIAL SAMPLES… · p. 19, Table 5 · p. 19

Analysis

• The cross-model results support the effectiveness of few-shot SELF-DEBUGGING prompts and suggest Codex better learns from few-shot exemplars with long contexts.GPT-3.5 uses zero-shot SELF-DEBUGGING on Spider because its few-shot performance is similar, while Codex benefits from the few-shot setup.Source: A SELF-DEBUGGING WITH INITIAL SAMPLES… · p. 19

B TRANSCODER COMPARISON TO BASELINE MODELS

• Table 6 compares SELF-DEBUGGING against baseline models without debugging for C++-to-Python translation on TransCoder.The supplied passages specify the comparison scope but do not provide numerical results.Source: B TRANSCODER COMPARISON TO BASELINE… · p. 19, Table 6 · p. 19

C.1 CODE TRANSLATION

• SELF-DEBUGGING with greedy decoding outperforms the baseline accuracy with 5 samples and approaches the baseline accuracy with 10 samples.Source: §C.1 · p. 20
• Combining unit test execution and code explanation improves debugging performance.Examples are presented in Figures 9 and 10 in Appendix D.Source: §C.1 · p. 20
• Around 12% accuracy improvement over greedy decoding comes from a single full-feedback debugging turn on TransCoder.The first debugging turn provides the major improvement.Source: Figure 8 · p. 20, §C.1 · p. 20
• Code explanation alone, without SELF-DEBUGGING, consistently improves performance by 2 −3% across different numbers of samples.This ablation is identified as Codex+Expl. in Figure 8b.Source: Figure 8 · p. 20, §C.1 · p. 20

C.2 TEXT-TO-PYTHON GENERATION

• Over 8%: SELF-DEBUGGING increases greedy-decoding accuracy on MBPP, with unit-test execution and code explanation consistently improving performance.The initial Python generation uses the same 3-shot prompt as (Ni et al., 2023).Source: §C.2 · p. 20
• Code explanation contributes less on MBPP because unit-test feedback makes errors easier to recognize and context limits reduce the prompt from 6 to 3 exemplars.When initial code is far from correct, the model often generates a new program rather than relying on the initial code snippets.Source: §C.2 · p. 20

D EXAMPLES OF MODEL PREDICTIONS WITH SELF-DEBUGGING

• Figures 9–11 provide examples of model predictions produced with SELF-DEBUGGING.The examples cover unit-test feedback, code explanation, and substantial changes to an initial MBPP prediction.Source: D EXAMPLES OF MODEL PREDICTIONS WITH… · p. 20
• Unit-test feedback fixes a code-translation error that simple feedback fails to correct.This example is presented in Figure 9.Source: Figure 9 · p. 21
• Code explanation yields a correct translation where the baseline prompt fails, specifically by checking for the end of a string.This example is presented in Figure 10.Source: Figure 10 · p. 22
• On MBPP, SELF-DEBUGGING can produce a prediction very different from the initial code.This example is presented in Figure 11.Source: Figure 11 · p. 22

E.1 BASELINE PROMPT (5-SHOT)

• The cited examples pair database schemas and sample records with “Translate the following question into SQL” prompts and SQL answers, including a join with grouping and an INTERSECT query.Source: §E.1 · pp. 23–26

E.2 SIMPLE FEEDBACK PROMPT (9-SHOT)

• The simple-feedback prompt extends demonstrations to 9 shots by marking SQL predictions as correct or wrong and showing corrected queries after errors.Examples include replacing an incorrect COUNT(*) output, using INTERSECT instead of contradictory predicates, and revising a station query after negative feedback.Source: §E.2 · pp. 28, 30, 31
• The simple feedback prompt provides demonstrations where SQL predictions are marked correct or wrong and revised, including DISTINCT counting and filtering users with tweets.For example, COUNT(status) is corrected to COUNT(DISTINCT status), and the average follower query is restricted to users appearing in tweets.Source: §E.2 · pp. 32–33

E.3 EXPLANATION FEEDBACK PROMPT (9-SHOT)

• The explanation feedback prompt asks the model to describe a query’s output columns and operations, then compare that explanation with the question to detect mismatches.In one example, the model notices that returning COUNT(*) adds an unwanted column and revises the query to select only the creation year.Source: §E.3 · p. 34
• Explanation-based feedback also identifies logically incompatible predicates and repairs them with set intersection, changing mutually exclusive status filters into an INTERSECT query.The same process detects a mismatched destination value and revises the flight query from "HONO" to "Honolulu".Source: §E.3 · pp. 36–37
• Feedback compares the SQL explanation and query with the question, identifies a mismatch over average longitude for stations never exceeding 10 bikes, and requests correction until correct.Source: §E.3 · p. 38

E.4 PROMPT FOR QUESTION EXPLANATION

• The question-explanation prompt decomposes requested fields and operations to infer the expected return type, including the number of output columns.Demonstrations identify three columns for an English-song query, one column for procedures meeting combined conditions, three columns for department information, and two columns for department name and employee count.Source: §E.4 · pp. 42–43
• Question explanations decompose natural-language requests into component outputs and verify the resulting column count, including joins, aggregation, filtering, and ordering.Examples derive outputs ranging from two to four columns and explain how clauses such as grouping, ordering, and conditions affect the requested result.Source: §E.4 · pp. 44, 45, 46, 49

E.5 PROMPT FOR SQL EXPLANATION

• SQL explanations infer the returned table’s columns and meanings from execution results, then relate them to operations such as GROUP BY, ORDER BY, LIMIT, JOIN, filtering, and aggregation.The examples explain outputs for grouped averages, intersections, extrema, joins with subqueries, and ordered minimum or maximum results.Source: §E.5 · pp. 50–52
• The SQL explanation prompt teaches the model to describe returned columns, filters, joins, grouping, ordering, limits, and empty results from executions.Examples explain both populated and empty tables, including mutually exclusive conditions that produce no results.Source: §E.5 · pp. 52, 54

E.6 CHAT SIMPLE FEEDBACK PROMPT (0-SHOT)

• The simple feedback prompt asks the model whether generated SQL matches the question and, if not, to generate corrected SQL.It provides the database schema and question before requesting the initial query and follow-up verification.Source: §E.6 · p. 54

E.7 CHAT EXPLANATION FEEDBACK PROMPT (0-SHOT)

• The explanation feedback prompt compares the SQL execution’s return type with the question’s inferred return type before requesting correction when they do not match.The prompt separately summarizes execution output and infers the question’s expected return type.Source: §E.7 · p. 54

F.1 BASELINE PROMPT (3-SHOT)

• The 3-shot baseline prompts direct the model to translate C++ programs into Python, using the setup and temperature from Chowdhery et al. (2022).The examples include incrementing a number, computing factorials, and summing pairwise products.Source: §F.1 · p. 55

F.2 BASELINE + EXPL. PROMPT (3-SHOT)

• The baseline-plus-explanation prompt first requires line-by-line explanations of C++ examples before requesting their Python translations.The demonstrations explain bitwise incrementation, recursive factorial computation, and nested-loop pairwise summation.Source: §F.2 · pp. 56–57

F.3 SIMPLE FEEDBACK PROMPT (2-SHOT)

• The simple-feedback prompt presents incorrect Python translations and asks the model to correct them using an explicit statement that they differ from the C++ behavior.Examples target factorial base cases and loop bounds in pairwise-product summation, followed by a placeholder template.Source: §F.3 · pp. 57–58

F.4 UNIT TEST FEEDBACK PROMPT (2-SHOT)

• The unit-test feedback prompt supplements incorrect translations with failed assertions and actual outputs, then asks the model to correct the Python code.The examples expose a factorial RecursionError at n = 0 and a pairwise-sum mismatch where the actual result is 16 instead of 25.Source: §F.4 · p. 59

F.5 UNIT TEST + EXPL. FEEDBACK PROMPT (2-SHOT)

• The unit-test-plus-explanation prompt asks the model to explain the original C++ and Python translations line by line before using failed tests to correct them.The demonstrations identify incorrect factorial and pairwise-summation behavior through explanations and test failures.Source: §F.5 · pp. 60–61

F.6 UNIT TEST + TRACE FEEDBACK PROMPT (2-SHOT)

• The unit-test-plus-trace prompt requires tracing a failing translation’s execution to diagnose the error before correcting the code.For factorial at n = 0, the trace shows the incorrect n = 1 base case causes recursive calls toward -1 indefinitely, motivating a base case at n = 0.Source: §F.6 · p. 62
• F.6 demonstrates unit-test and trace feedback in two shots, identifying a Python loop that starts at 0 instead of the C++ loop’s starting value of 1 and correcting it to range(1, n + 1).The example reports a failed test with actual result 16 versus expected 25, traces the execution, diagnoses the off-by-one loop start, and proposes the corrected range.Source: §F.6 · p. 63

F.7 CHAT SIMPLE FEEDBACK PROMPT (0-SHOT)

• F.7 uses a zero-shot chat prompt that repeatedly tells the model the Python translation differs from the C++ code and asks it to correct the translation.The initial request asks for code only in a Markdown code block, followed by repeated correction requests.Source: §F.7 · pp. 63–64

F.8 CHAT UNIT TEST FEEDBACK PROMPT (0-SHOT)

• F.8 extends the simple chat feedback prompt by supplying one failed unit test and its execution result before repeatedly asking the model to correct the Python translation.Its feedback message explicitly tests whether Python outputs match the C++ program’s outputs.Source: §F.8 · p. 64

F.9 CHAT UNIT TEST + EXPL. FEEDBACK PROMPT (0-SHOT)

• F.9 asks the model to explain both the original C++ code and its Python translation line by line, then uses failed unit-test feedback to request correction.After correction, the prompt again requests a line-by-line explanation of the Python translation.Source: §F.9 · p. 64

F.10 CHAT UNIT TEST + TRACE FEEDBACK PROMPT (0-SHOT)

• F.10 combines failed unit-test feedback with execution tracing before asking the model to correct the Python translation, repeating this cycle across attempts.The prompt supplies a failed test and execution result, requests a trace of the failed function call, then requests correction and repeats the trace-and-correction sequence.Source: §F.10 · p. 65

G.1 BASELINE PROMPT (3-SHOT)

• The baseline prompt is adapted from Ni et al. (2023) for generating initial Python code.Source: §G.1 · p. 65
• The prompt asks the model to write a Python function that completes the task and passes assertion tests.Source: §G.1 · p. 65
• Each demonstration pairs a natural-language task description and assertion with a Python function implementation inside task delimiters.Source: §G.1 · pp. 65–66
• One demonstration implements similar_elements by returning the set intersection of two tuples.Source: §G.1 · p. 65
• A second demonstration implements is_not_prime using trial divisibility checks up to the square root of the input.Source: §G.1 · p. 65
• A third demonstration implements heap_queue_largest with heapq.nlargest to select the largest integers from a list.Source: §G.1 · p. 66

G.2 SIMPLE FEEDBACK PROMPT (6-SHOT)

• The simple-feedback demonstrations pair several tasks with assertions or problem descriptions, while one final template leaves assertions, problem description, and original code as placeholders.This establishes the prompt format independently of a specific task instance.Source: §G.2 · pp. 66–68
• The simple-feedback prompt teaches iterative repair by presenting a task, candidate function, and explicit “wrong” or “correct” feedback before a revised solution.Examples include domino tiling, one-bit difference, regular-expression matching, rotations, squaring, and selecting the smallest items.Source: §G.2 · pp. 66–67
• In the simple-feedback examples, incorrect candidates are replaced with implementations that receive explicit confirmation as correct, including dynamic programming and regular-expression solutions.The prompt also demonstrates correct candidates for bit testing, squaring, rotations, and selecting the n smallest items.Source: §G.2 · pp. 66–67

G.3 UNIT TEST FEEDBACK PROMPT (6-SHOT)

• Both prompt variants conclude with a placeholder template containing insertion points for assertions, the problem description, and the original code.This preserves a reusable six-shot structure beyond the illustrated examples.Source: §G.2 · p. 68, §G.3 · p. 71
• The unit-test-feedback prompt evaluates candidates against concrete assertions, explicitly stating whether each assertion passes before labeling the code wrong or correct.For example, it identifies failed outputs for count_ways and find_Rotations, then reports passing assertions for bit testing and list squaring.Source: §G.3 · pp. 68–69
• The unit-test demonstrations cover the same broad task types as the simple prompt, including domino tiling, bit comparison, regular expressions, squaring, rotations, and selecting the smallest items.The examples provide concrete assertion values and feedback tied to observed outputs.Source: §G.3 · pp. 68–71
• Unlike simple feedback, the unit-test prompt exposes execution failures such as `NameError: name ’re’ is not defined` and directs the model to add the missing import.The repaired regular-expression function is then judged against the stated assertion.Source: §G.3 · pp. 69–70

G.4 UNIT TEST + EXPL. FEEDBACK PROMPT (3-SHOT)

• The 3-shot prompt asks the model to write Python code that completes a task and passes supplied assertion tests.The prompt explicitly includes the programming task and assertions before the generated function.Source: §G.4 · p. 71
• The prompt requires line-by-line natural-language explanations of generated code, including function arguments and regular-expression behavior.For find_char_long, the explanation describes the function input and the regex boundaries, word characters, and minimum length.Source: §G.4 · pp. 71–72
• Execution feedback identifies a missing re import, after which the revised find_char_long implementation passes its assertion.The initial prediction raises NameError: name ’re’ is not defined; adding import re yields the asserted word list.Source: §G.4 · p. 72
• The example fixes a wrong count_ways implementation for filling a 3 x n board with 2 x 1 dominoes, after feedback notes that the code is wrong.The repaired implementation initializes A and B with boundary values and updates them using A[i] = A[i - 2] + 2 * B[i - 1] and B[i] = A[i - 1] + B[i - 2].Source: §G.4 · pp. 72–73
• The prompt also demonstrates correcting find_Rotations when the initial implementation returns 0 instead of the asserted 1 for "aaaa".The revised approach tests rotations i from 1 through n and returns the first rotation reproducing the original string.Source: §G.4 · pp. 74–76

G.5 CHAT SIMPLE FEEDBACK PROMPT (0-SHOT)

• The 0-shot chat prompt begins with a Python task containing one unit test, then asks the model to fix the code whenever that test fails.Failure feedback can be repeated through successive chat turns using “The code above is wrong. Please fix it.”Source: §G.5 · p. 76
• After a prediction passes its unit test, the 0-shot prompt permits one additional correctness check and retains the new code only if it also passes.If the extra revision fails, the original passing prediction is used for evaluation, and failure-feedback iterations stop.Source: §G.5 · p. 76

G.6 CHAT UNIT TEST FEEDBACK PROMPT (0-SHOT)

• G.6 distinguishes failure feedback, which supplies the failed test and execution result, from success feedback, which asks whether the code is correct.Both branches can request another completion, but only a passing revision replaces the original prediction.Source: §G.6 · pp. 76–77
• G.6 uses failed unit tests and execution results to prompt repeated Python fixes, while successful tests trigger a correctness check and possible repair.The original prediction is evaluated if the revised code does not also pass the unit test.Source: §G.6 · p. 77

G.7 CHAT UNIT TEST + EXPL. FEEDBACK PROMPT (0-SHOT)

• G.7 mirrors this failure-versus-success branching while inserting code explanations into both paths before or after the repair request.The explanation is explicitly requested line by line.Source: §G.7 · p. 77
• G.7 adds line-by-line Python explanations before and after unit-test-driven repair on failures, and also before correctness checking on successes.As in G.6, a revised prediction is used only if it passes the unit test.Source: §G.7 · p. 77

G.8 CHAT UNIT TEST + TRACE FEEDBACK PROMPT (0-SHOT)

• G.8 combines failed-unit-test feedback with repeated execution tracing of the failed function call before requesting Python fixes.The trace-and-fix sequence is repeated for subsequent failures.Source: §G.8 · p. 78