Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail

Esmail Gumaan

Source-linked AI summary

Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail

Esmail Gumaan

arXiv:2608.23651v1Aug 24, 2026cs.SEcs.AI

TL;DR

Agent harnesses append failed tool calls and their error messages, assuming this feedback will correct the next action. The paper measures that assumption with controlled probes and counterfactuals, finding that repetition increases across tested small instruction-tuned models because the failed call’s surface form dominates the semantic error signal. It then evaluates structural harness changes that retain the diagnosis while withholding or blocking the failed string.

Problem
Agent frameworks append failed calls and runtime feedback to transcripts, but whether this execution feedback actually shifts models away from the failed action has not been controlledly measured.
Method
The paper scores fixed teacher-forced probe items across simulated tool calling and MBPP repair, comparing failure, success, neutral, and structural harness variants.
Results
Failure feedback increased repetition for every tested model in both environments; the failed call’s surface form accounted for 83% of the effect, while semantic effects were small and inconsistent.
Takeaways & Limitations
Harnesses should preserve failure diagnoses while withholding failed action strings or making them unreachable, rather than relying on explicit prohibitions or clean restarts.
Takeaways & Limitations
The study characterises the small-model regime only up to 1.7B parameters and cannot yet test whether the probe predicts repetition across models.

Abstract

from arXiv · show

Agent harnesses record a failed tool call and its error message in the transcript and ask the model to continue, on the assumption that the error is corrective information. We measure whether it is. Defining the corrective gain of a failure record as the change in log-probability of re-emitting the action that just failed, we find the gain is negative for every instruction-tuned model we tested (6 checkpoints, 135M-1.7B, 4 families) in two environments: simulated tool calling and MBPP program repair. Normalised by action length the effect is about -1.03 nats per action token, a factor of 2.8 in the odds of each token, and holds on 90%-100% of individual items, not only on average. Over a fixed candidate set the probability of repeating the failed call rises from 0.06 to 0.54, and greedy decoding reproduces it token for token on 19% of items after the failure versus 0% before. Counterfactuals pairing the same call with a failure message, a success message, or a neutral acknowledgement separate two effects: the failed call's surface form accounts for 83% of the damage, while the semantic contribution of marking it failed is small and inconsistent in sign across environments. The problem is in the harness, not the model's grasp of error messages, and that predicts which remedies work. Replacing the verbatim call with a runtime-generated description of the failure removes 76% of the inversion at no token cost, and making previously-failed strings unreachable at the decoder acts on the same term. Two plausible remedies do not: an explicit "do not repeat" instruction leaves the measured quantity where it was, and deleting the failed attempt to retry from a clean context, the standard prescription for context contamination, is the worst harness we measured for repetition, because it restores the context that produced the failure. The study runs end to end on a CPU; all artefacts are released.

1 Introduction

The standard harness appends failed calls and runtime feedback, but the failed call’s exact surface form can make repetition more likely. Across small instruction-tuned models, the paper measures this inversion, identifies its source, and tests structural remedies.

Measurement: −17.38 nats: recording a failed call and its error message increased the call’s re-emission probability across 6 models, 4 families, and two environments.Normalised by action length, the effect was about −1.03 nats per action token, or a 2.8-fold change in each token’s odds.
Measurement: The effect held on at least 90% of individual items, rather than arising only from a few extreme cases.This consistency was observed across the tested models and environments.
Measurement: 0.06 to 0.54: over four fixed candidate actions, the probability of repeating the failed call rose after failure feedback.Greedy decoding reproduced the failed call token for token on 19% of items after failure, versus 0% before.
Cause: 83% of the effect came from the failed call’s surface form, while the semantic contribution of marking it failed was small and inconsistent across environments.The decomposition separates the copied token sequence from the failure assertion; the latter was slightly positive on tool calls and slightly negative on program repair.
Remedies: Deleting the failed attempt and retrying from a clean context was the worst harness, multiplying exact repetition by 2.6 without changing task success.The clean restart restores the context that produced the failure, so deterministic behaviour can reproduce the same action.
Remedies: Replacing the verbatim failed call with a runtime-generated failure description removed 76% of the inversion, while decoder bans acted on the same term.Both approaches retain the diagnosis while withholding or blocking the failed string.

2 Related work

Prior work documents agent repetition, weak self-correction, context contamination, and negative-instruction effects, but does not provide the paper’s controlled token-level measurement and decomposition of failure feedback.

Agents and feedback: ReAct established the dominant pattern of interleaving written actions with environment observations in an append-only transcript.Reflexion, Self-Refine, self-debugging, and tool-feedback systems build on this template.
Self-correction: Prior self-correction studies ask whether models can judge their own outputs, whereas this paper holds the environment’s ground-truth verdict fixed and measures the next-token response.The distinction is between judging an action and responding appropriately to an external execution result.
Context management: Context-contamination work finds retries retaining failed attempts perform worse and recommends clearing context before retrying.This paper evaluates clean restart as a baseline rather than treating it as the final remedy.
Agent repetition: Other work reports stuck agents, difficulty forming alternative plans after failures, and improved embodied-agent success when observations are degraded.The paper adds a token-level mechanism and a fix that preserves observations rather than corrupting them.
Copying and negation: Repetition, copying, and negative-instruction research predicts that strings appearing in context become more likely, even when prompts prohibit them.The paper measures this effect inside agent trajectories and tests the corresponding prohibition.
Small models and constraints: Small-model agent research emphasizes harness design, while constrained decoding is usually used for formatting and can impede free-form reasoning.This paper instead uses decoding constraints to make previously failed action strings unreachable.
Positioning: The missing contribution is controlled measurement of whether execution feedback shifts action probabilities correctly, decomposition into surface and semantic terms, and an intervention derived from that decomposition.This distinguishes the paper from prior aggregate policy evaluations and intuition-based fixes.

3 Problem formulation

The paper formalizes an agent loop in which actions and observations extend transcript context, defines corrective gain, and decomposes feedback inversion into copying and semantic effects.

3.1 The agent loop and its feedback record: At each step, the harness builds context C_t, the policy samples action a_t, and the environment returns observation o_t with a success flag.The transcript grows from prior system, goal, action, and observation turns.
3.1 The agent loop and its feedback record: Equation (1) appends the action and observation to the next context: C_t+1 = C_t ⊕ (a_t, o_t).The paper focuses on the literal verbatim inclusion of both the failed action and runtime output.
3.2 Corrective gain: The harness intends failure feedback to make the failed action less likely, so positive corrective gain is the desired outcome.Negative gain means the record instead increases the probability of repeating the action.
3.2 Corrective gain: Corrective gain compares the failed action’s summed token log-probability before and after its failure record is appended.Because the same string is scored in both contexts, the difference represents the change in its log-odds and is invariant to tokenisation length.
3.3 Decomposing the gain: Appending a failed action and observation combines two influences: access to the action’s token sequence and an assertion that the action failed.These influences can pull in opposite directions, while the net corrective gain alone cannot separate them.
3.3 Decomposing the gain: Counterfactual neutral and successful observations isolate semantic and polarity effects while holding the same failed action fixed.A valence-free observation acknowledges the call without saying whether it succeeded or failed.
3.4 What a fix would have to look like: If the semantic term dominates, clearer messages or better models are relevant; if copying dominates, structural context or decoding changes are required.The paper evaluates structural remedies against a natural-language prohibition.
3.4 What a fix would have to look like: Table 1 positions the paper as measuring the net failure-record effect and choosing interventions according to which component dominates.Its distinguishing comparison is between prior aggregate measurement and decomposition-guided intervention.

4 Measuring the gain

The paper uses deterministic teacher-forced probe items and controlled transcript variants to measure repetition, then evaluates harnesses that preserve, abstract, instruct against, or decode-block failed calls.

Probe construction: Teacher-forced probe items score fixed failing actions without sampling, eliminating decoding noise and making measurements bit-reproducible.Each item includes a successful prefix, failing action, runtime observation, and reference action.
Probe construction: The failing action is generated by a fixed perturbation of the reference action, so every model is tested on byte-identical items.This avoids confounding model size with model-specific mistake difficulty.
Controlled conditions: All conditions share the same prompt, goal, and successful prefix, differing only in how the failed attempt enters the transcript.The conditions include pre, fail, succ, and neut observations.
Controlled conditions: Counterfactual success observations are generated from the failing call’s own arguments to avoid incoherence cues unrelated to error reading.This keeps the polarity comparison tied to the same call.
Measurements: The paper reports log-probability differences alongside normalised repeat probability and exact greedy repeat rate.The derived measures make repetition easier to interpret.
Measurements: Normalised repeat probability is computed over a fixed candidate set containing the failed action, correct action, and two alternative wrong actions.This creates a proper probability comparable across models.
Measurements: Exact greedy repeat records whether greedy decoding reproduces the failed action token for token, using quantities already computed during scoring.It therefore requires no additional decoding pass.
Harnesses: The evaluated harnesses include standard verbatim feedback, an explicit prohibition, runtime-generated abstraction, and decoder-level suppression of failed strings.The intervention comparison is designed around the surface-form decomposition.

5 Experimental setup

The study evaluates failed-action feedback in two execution-based environments, using controlled counterfactual observations and fixed probe items across six CPU-run instruction-tuned checkpoints. Teacher-forced scoring and task-clustered bootstrap statistics quantify model behavior while preserving reproducibility.

Environments: Two environments execute actions for real: ToolShed simulates typed office tools, while CodeRepair runs mutated MBPP functions against their assertions.ToolShed returns runtime-generated errors; CodeRepair uses interpreter output and real tracebacks.
Environments: ToolShed uses six task templates, twelve typed tools, and eight perturbation operators producing twelve distinct error families.Experiments draw balanced subsamples from these probe pools.
Experimental rationale: The environments support failure, success, and neutral observations for the same action, enabling the paper’s counterfactual decomposition without relying on public policy-scoring benchmarks.All three observations come from the same renderer to avoid experimenter expectations leaking into the data.
Models: The evaluation covers 6 instruction-tuned checkpoints spanning 135M–1.7B parameters across 4 model families, all run in float32 on CPU.Within-family pairs allow size trends to be separated from family effects.
Evaluation and reproducibility: Teacher-forced probe scoring is deterministic, and uncertainty uses task-level cluster bootstrap intervals with 10,000 resamples and Holm correction for headline tests.A single base seed with named sub-streams makes reruns reproduce the same items and records.

6 Execution feedback inverts

Execution feedback increases the likelihood that models repeat failed actions, primarily because the failed call remains visible in the transcript rather than because the error message is misunderstood. This inversion appears across environments and suggests structural harness changes over message-level instructions.

The headline measurement: −17.38 nats: recording a failed call and its error message makes the model more likely to re-emit that exact call.The intervals exclude zero for all 6 tested models.
It is the string, not the message: 83% of the inversion comes from the failed call’s surface form being present in context, while the semantic failure signal is small and inconsistent across environments.The semantic term is slightly positive on ToolShed but slightly negative on CodeRepair.
Copying does not simply raise everything: The failed call specifically gains against the correct action: the gold-minus-failed log-probability gap shrinks by −13.12 nats after failure feedback.The correct action’s probability also rises by 4.30 nats on average, but the failed action gains more.
Scale: Within model families, larger checkpoints show milder gain, and the weakening is attributed to reduced copying rather than improved error reading.The pooled scaling fit is treated as descriptive, not predictive.
The second environment, and a unit that travels: Per-action-token corrective gain lands near one nat in both tool calling and program repair, despite programs being roughly five times longer than tool calls.The program-repair environment reports −73.55 raw nats across 3 models, so raw values are not directly comparable.
Harness variants: Removing the verbatim failed program can reverse the inversion, whereas message manipulations move the estimate little.Figure 3 identifies removal of the call’s surface form as the manipulation that moves the estimate by an order of magnitude.

7 What moves the effect, and what does not

The effect is driven primarily by the failed call’s surface form, while changing message content or recency has little influence. Structural interventions that remove or block the failed string help, whereas explicit prohibitions do not, and deleting the failed attempt worsens repetition.

Surface form versus message content: −0.28 [−0.68, +0.11] nats from shortening the error and −0.20 [−0.48, +0.08] nats from expanding it both have intervals containing zero.Message wording, including a corrective hint, does not measurably change repetition.
Surface form versus message content: +0.71 [+0.45, +0.97] nats results when the runtime echoes the offending call inside the error message, making the failed call appear twice.This contrasts with near-zero effects from rewriting message content.
Surface form versus message content: +3.33 [+2.95, +3.72] nats after two failures and +4.03 [+3.60, +4.48] nats after three show that repetition compounds across failures.Each additional failure deepens the repetition effect even though a corrective signal remains present.
Surface form versus message content: −0.88 [−1.41, −0.37] nats from moving the failed step earlier shows recency is not the explanation, although it accounts for about three percent of the total.The failed call need not be the most recent transcript item for the effect to persist.
Interventions: Replacing the verbatim call with a runtime-generated failure description drives the exact greedy repeat rate to zero for every model, while a natural-language prohibition does not help.The description preserves the diagnosis while removing the failed token sequence; the prohibition changes the measured quantity slightly in the wrong direction.

8 Does it matter when the agent drives?

In free-running rollouts, repetition is a substantial failure mode, and interventions targeting the failed call’s surface form reduce it. However, lower repetition does not necessarily improve task success, while clean restart makes repetition worst.

42% task success and 31% byte-for-byte repeats among failed actions show that repetition is a substantial rollout failure mode.
The do-not-repeat instruction leaves repetition essentially unchanged, despite increasing task success by +17 [+4, +33] points.Its success effect is measured but unexplained at this sample size, whereas its repetition result supports the narrower conclusion that it is not a repetition remedy.
From 31% to 80%, clean restart sharply increases exact repetition without improving task success.Deleting the failed step restores the context that produced the failure, encouraging deterministic re-emission of the same action.
Runtime-generated failure descriptions reduce exact repeats to 16%, while combining abstraction and banning reduces them further to 7%.The two interventions act on complementary mechanisms: context-level copying and residual re-derivation.
From 31% to 8%, decoder bans reduce exact repetition and eliminate loops, but task success remains unchanged.The ban costs 69 generated tokens per rollout versus 67 under the standard harness.
Across harnesses, loop rates rank abstract+ban best at 4% and clean restart worst at 67%, while task success does not follow this ordering.The mismatch reflects other model failures, including prose where a tool call was expected.

9 Cost

The recommended interventions are inexpensive to run on the study’s CPU setup. Their costs are shaped by token generation, context length, decoding checks, and hardware limits on model size and parallelism.

The two recommended interventions are close to free, making their low operational cost a central practical result.
The abstract harness adds no model call or latency while shortening each failure context.It requires only runtime knowledge of the error type already used to format the error message.
The decoder ban adds no generated tokens and only one ban-list pass per decoding step.The implementation’s added time is below the noise floor of wall-clock measurements.
The abstraction shortens context per step but can take more steps because it does not terminate early on a loop.Generated and prompt tokens accumulate over rollout steps, so context size should be compared per step.
Prefix reuse removes 87% of token positions in the probe, making distribution measurement two orders of magnitude cheaper than sampling.
Models above roughly 2B parameters do not fit alongside the workload in float32, and multi-threaded inference is unavailable on the study machine.

10 Where the interventions fail

The interventions reduce repetition unevenly: decoder bans prevent exact repeats but can produce paraphrases, while abstraction can discard needed evidence and clean restarts worsen repetition. The effect weakens with scale, and repetition is only one class of failure.

Where the interventions fail: Blocking exact strings can displace repetition into near-identical paraphrases with reordered arguments, altered quoting, or trivially changed values.The paper therefore reports canonical repeat rates alongside exact rates and does not recommend the ban alone.
Where the interventions fail: The abstract harness can remove evidence needed to repair failures when the diagnosis alone does not reveal what the model wrote incorrectly.Abstraction is lossless only for error families whose repair is fully determined by the diagnosis, such as unknown tools or missing required arguments.
Where the interventions fail: The inversion shrinks monotonically with model size within every tested family, but the study does not locate where the gain crosses zero.The model ladder stops at 1.7B parameters because of available CPU memory and runtime.
Where the interventions fail: 18% of actions are prose when a tool call was expected, limiting how much repetition-focused interventions can improve.A further 5% are truncated by the 32-token generation budget, an experimental artifact that depresses absolute success.
Where the interventions fail: The standard harness is relatively safe when the failed action was initially implausible, because copying starts from a low base and absolute repetition remains small.The inversion is most dangerous for already-plausible near-miss errors.

11 Limitations

The study’s scope is limited by its small-model ladder, constructed environments and synthetic perturbations, greedy decoding, and summed log-probability measurement. It also documents numerical safeguards after detecting a silent NaN failure mode.

Scale: The model ladder stops at 1.7B parameters because of CPU memory and runtime constraints, so the study characterizes small models without locating a capability threshold.The observed inversion decreases monotonically with model size within the tested ladder.
Environments: ToolShed uses author-designed perturbations, error messages, and task templates, while CodeRepair provides a check using MBPP problems, real mutations, and Python-interpreter observations.Neither environment is a long-horizon, realistic agent benchmark.
Perturbations: Synthetic failing actions enable byte-identical cross-model items but reduce realism because failures are constructed rather than sampled from each model.Sampling model-specific failures would confound model size with item difficulty.
Greedy decoding: Greedy decoding yields one trajectory per model-harness pair and omits variance from sampling, so the rollout results do not cover every deployed decoding regime.Intervals are obtained by resampling tasks rather than random seeds.
Log-probabilities of whole strings: Summed token log-probabilities grow with action length, so raw nat values are comparable across conditions and models only within an environment.Normalised repeat probability and greedy repeat rate avoid this cross-environment length caveat.
Numerical validation: A multithreaded PyTorch setting produced silent all-NaN logits, prompting finite-logit checks and verification that rejects non-finite or positive log-probabilities.One complete probe run was lost before the numerical safeguards were added.

12 Broader impact

The interventions reduce repeated failed actions and the associated wasted resources, but persistence after suppression can increase harm in side-effecting environments.

Broader impact: Reducing repetition lowers wasted tool calls, wasted tokens, and wall-clock cost in stuck loops, but continued experimentation after failures may cause more damage in side-effecting environments.The direct effects are mostly beneficial, while the persistence trade-off is a second-order concern.

13 Conclusion

The paper concludes that failed-call repetition is driven mainly by the call’s surface form in the harness, not by the model’s inability to use failure semantics. It supports preserving diagnostic information while removing or blocking the failed string, and reports the study’s paired evaluation and reproducible CPU-based setup.

Conclusion: Keeping the diagnosis while withholding the failed string, or making it unreachable during decoding, targets the dominant repetition term; an explicit do-not-repeat instruction does not.The conclusion treats these as harness changes rather than model retraining.
Conclusion: Clean-context retry is the worst option because it restores the context that produced the failure; useful post-failure context must differ without reproducing the failed action.This principle follows from comparisons among abstraction, decoder banning, instruction, standard, and clean-restart harnesses.
Reproducibility: The released workflow runs on CPU with pinned environments, resumable stages, deterministic named random streams, and greedy rollout settings.The reported setup uses Python 3.12 and PyTorch 2.4.0+cpu; version-drift checks found disagreements far below the study’s smallest effect.
Evaluation design: The study uses paired item-level changes in log π(a×), with identical scored actions and controlled transcript observations across probe conditions.Conditions include standard failure, counterfactual success, neutral acknowledgement, and abstraction without the verbatim call.
Evaluation design: The probe suite varies failure rendering, repetition count, timing, failed-action identity, abstraction, and explicit instructions while holding the decision point constant.The condition table defines the shared baseline and transcript variants used for comparison.

D Per-error-family results

The section examines whether repetition effects vary by error family and reports repetition readouts for CodeRepair using the same fixed candidate-set construction. It also illustrates how banning failed strings changes trajectories without necessarily producing reconsidered calls, while task-level variance motivates paired comparisons across tasks.

Per-error-family results: The error-family table tests whether corrective-gain differences are broadly uniform rather than driven by one error message’s wording.Rows represent the error provoked by the failing call, and the table is sorted by pooled mean.
Per-error-family results: CodeRepair repetition probabilities use prep for the failed call’s normalized candidate probability and greedy for exact reproduction under greedy decoding.Both quantities are computed over scored candidate actions.
Example trajectories: Under verbatim, drop, and abstract harnesses, one illustrated task repeats the same call six times; ban harnesses prevent that exact repetition but produce calls with error fragments or chained replacements.The example is explicitly described as illustrative rather than demonstrative, with quantitative claims based on aggregate paired comparisons.
Interpretation and scope: Task-level variance is large: abstraction can fail to change one item even when it reduces repetition across the task set, while the instruction can change one trajectory despite no aggregate effect.The paper therefore bases its claims on paired comparisons over all tasks rather than examples.