Wink: Recovering from Misbehaviors in Coding Agents

Rahul Nanda; Chandra Maddila; Smriti Jha; Euna Mehnaz Khan; Matteo Paltenghi; Satish Chandra

Source-linked AI summary

Wink: Recovering from Misbehaviors in Coding Agents

Rahul Nanda, Chandra Maddila, Smriti Jha, Euna Mehnaz Khan, Matteo Paltenghi, Satish Chandra

arXiv:2602.17037v2Feb 19, 2026cs.SEcs.AIcs.HCcs.PL

TL;DR

Autonomous coding agents often misbehave by drifting from instructions, reasoning unproductively, or failing in tool use, creating a need for manual correction. The paper introduces a production-grounded taxonomy and Wink, an asynchronous self-intervention system that provides targeted course correction. Wink achieved 90% recovery for single-intervention cases and reduced several operational costs in a live A/B test.

Problem
Coding agents can get stuck, pursue incorrect paths, modify unintended files, or require manual steering, making reliable and autonomous task completion difficult.
Method
The paper derives three misbehavior classes from production trajectories and developer feedback, then uses Wink to detect misbehaviors during execution and provide corrective guidance.
Results
90% of single-intervention conversations recovered, while production A/B testing showed statistically significant reductions in Tool Call Failures, Tokens per Session, and Engineer Interventions per Session.
Takeaways & Limitations
Automated self-intervention is a viable strategy for improving the efficacy and reliability of software engineering agents.
Takeaways & Limitations
The system and taxonomy were tested and deployed only at Meta, so the results and categories may not generalize to other platforms.

Abstract

from arXiv · show

Autonomous coding agents, powered by large language models (LLMs), are increasingly being adopted in the software industry to automate complex engineering tasks. However, these agents are prone to a wide range of misbehaviors, such as deviating from the user's instructions, getting stuck in repetitive loops, or failing to use tools correctly. These failures disrupt the development workflow and often require resource-intensive manual intervention. In this paper, we present a system for automatically recovering from agentic misbehaviors at scale. We first introduce a taxonomy of misbehaviors grounded in an analysis of production traffic, identifying three primary categories: Specification Drift, Reasoning Problems, and Tool Call Failures, which we find occur in about 30% of all agent trajectories. To address these issues, we developed a lightweight, asynchronous self-intervention system named Wink. Wink observes agent trajectories and provides targeted course-correction guidance to nudge the agent back to a productive path. We evaluated our system on over 10,000 real world agent trajectories and found that it successfully resolves 90% of the misbehaviors that require a single intervention. Furthermore, a live A/B test in our production environment demonstrated that our system leads to a statistically significant reduction in Tool Call Failures, Tokens per Session and Engineer Interventions per Session. We present our experience designing and deploying this system, offering insights into the challenges of building resilient agentic systems at scale.

1 Introduction

Coding agents can fail by drifting from instructions, reasoning unproductively, or misusing tools, often requiring manual steering. Wink uses runtime self-intervention to detect these misbehaviors and provide course correction, achieving strong recovery and production improvements.

Coding Agents: Coding agents iteratively reason and act through software-development tools to complete natural-language engineering objectives.Their trajectories consist of internal rationale, actions such as tool invocations, and environmental observations.
Agent Misbehaviors: Users may need to manually steer agents that become stuck, pursue incorrect paths, modify unintended files, or consume unnecessary steps.Successful assistance is perceived as correct, efficient, and autonomous.
Self-Intervention: Wink uses a concurrent runtime intervention mechanism to detect deviations from user instructions and nudge agents toward recovery.The intervention reflects on the trajectory and supplies corrected guidance while the main agent continues operating.
Self-Intervention: In the illustrated specification-drift case, the intervention redirects the agent to invoke the requested review_code tool with the correct parameters, after which the task succeeds.The agent had initially gathered diff details and read code instead of following the explicit tool instruction.
Results: 90% of single-intervention conversations recovered, while online A/B testing found statistically significant reductions in tool call failures, token usage per session, and engineer interventions per session.Multiple-intervention conversations achieved about 80% recovery.

2 Misbehaviors in Software Engineering Agents

The paper develops an enterprise-oriented taxonomy of coding-agent misbehaviors from production feedback and trajectory analysis. It focuses on specification drift, reasoning problems, and tool call failures as actionable classes for runtime intervention.

Taxonomy Design: Prior taxonomies classify software-engineering-agent failures across reasoning, execution, and planning, while this work adapts such categories to enterprise contexts.The adaptation accounts for proprietary languages, organization-specific frameworks, and heterogeneous legacy systems.
Taxonomy Design: The authors use production trajectories and developer feedback to quantify prevalent failure modes and identify classes suitable for runtime interventions and tooling.The goal is construct validity and operational utility in large-scale industrial codebases.
Misbehavior Classes: Manual review of disliked trajectories yields three main misbehavior classes: Specification Drift, Reasoning Problems, and Tool Call Failures.The review uses explicit user Like or Dislike feedback and inspection of hundreds of trajectories.
Specification Drift: Specification Drift captures divergence from the user’s task, including ignored constraints, omitted context, tangential solutions, and excessive scope.The category includes failures to incorporate user feedback or adhere to explicit intent.

Did Not Follow Instructions (DNF)

Unrequested Changes are a form of specification drift in which the agent modifies files or content outside the user’s intended scope.

Unrequested Changes: Unrequested Changes occur when an agent edits unrelated files or makes modifications that the user did not request.These changes may not align with the user’s intent and can require intervention to revert.

Unrequested Changes (UC)

Reasoning Problems include failures that prevent meaningful progress, such as infinite loops of repetitive actions or reasoning.

Reasoning Problems: Reasoning Problems affect an agent’s ability to make meaningful progress toward task completion.Infinite loops are characterized by repetitive actions or reasoning with little or no progress.
Infinite Loops: Infinite loops can involve repeated tool calls, repeated edits, or recurring attempts to fix self-introduced errors and merge conflicts.A key indicator is invoking the same or similar tool calls three or more times consecutively.

Infinite Loops

Tool Call Failures describe repeated unsuccessful tool interactions caused by agent errors or unresponsiveness, including malformed requests and failure to adapt after errors.

Infinite Loops: Tool Call Failures involve repeated failures to interact with tools because of agent errors or unresponsiveness.They include malformed, invalid, or incorrect tool-invocation parameters.
Infinite Loops: Agents may ignore tool-invocation failures and fail to adjust their strategy in response.

2.2 Methodology for Misbehavior Prevalence Calculation

The prevalence analysis uses calibrated LLM classifiers on production and historical trajectory datasets. It finds roughly 29% overall misbehavior prevalence, with specification drift and tool-call failures highest on the static set, while model changes affect categories unevenly.

2.2 Methodology for Misbehavior Prevalence Calculation: LLM-based binary classifiers were calibrated for each misbehavior category using trajectory history up to a given step.Different frontier models offered flexibility when high precision or high recall was required.
2.2 Methodology for Misbehavior Prevalence Calculation: 8k trajectories, representing 10% of daily production traffic, were classified after validating the classifiers.
2.2 Methodology for Misbehavior Prevalence Calculation: 42,920 real-user trajectories were randomly sampled from five consecutive weeks to establish historical prevalence trends before interventions.
2.2 Methodology for Misbehavior Prevalence Calculation: 29% overall misbehavior prevalence was observed on the static set, with Specification Drift and Tool Call Failures highest on average.The static dataset contains real user sessions sampled from five consecutive weeks.
2.2 Methodology for Misbehavior Prevalence Calculation: Trajectory-level classification allows one trajectory to contain multiple misbehaviors, although many categories are mutually exclusive.About 65% of tool-call-failure trajectories and 45% of instruction-following-problem trajectories were exclusive to those categories.
2.2 Methodology for Misbehavior Prevalence Calculation: Opus 4.5 improved Specification Drift prevalence but increased infinite-loop prevalence, while Tool Call Failures showed no statistically significant change.The reported p-value for Tool Call Failures was 0.878; other listed changes were statistically significant.

3 Self intervention

Self-intervention asynchronously inspects recorded agent trajectories, detects misbehavior, and injects targeted guidance into the agent’s subsequent input. The design aims to correct behavior without blocking the main loop, and defines recovery by whether the specific misbehavior recurs.

3 Self intervention: A trajectory records user messages, assistant reasoning, tool actions, and environment observations as the agent progresses through a task.
3 Self intervention: The observer asynchronously inspects the trajectory at fixed intervals and returns misbehavior feedback without blocking the main agent harness.This design targets latency protection while the agent continues operating.
3 Self intervention: The detection system consumes the trajectory and misbehavior taxonomy, then returns a binary misbehavior decision and identified classes.
3 Self intervention: Guidance consists of dynamically generated DOs and DONTs that usually prompt self-reflection and an alternative action or path.
3 Self intervention: The guidance is appended to the trajectory and passed as the next agent input to generate subsequent reasoning and action pairs.
3 Self intervention: Recovery occurs when the specific misbehavior does not recur after intervention, whereas recurrence defines non-recovery and later distinct misbehaviors trigger separate interventions.

4 Experiment setup

The experiments evaluate whether asynchronous self-intervention resolves misbehaviors, reduces their recurrence, and improves agent-session outcomes in production. They combine intervention-triggered trajectory analysis, shadow-mode comparison, live A/B testing, and qualitative coding.

Research questions: The study asks whether self-intervention resolves misbehaviors, reduces their magnitude, improves standard agent metrics, and clarifies recovery conditions.These questions are evaluated through multiple complementary experiments.
Recovery evaluation: 10,554 intervention-triggered trajectories were categorized by whether they received a single or multiple self-interventions.This distinction supports recovery analysis across failure types.
Recovery evaluation: 90.93% of single-intervention trajectories recovered, compared with 79.07% for trajectories requiring multiple interventions.Recovery was judged from post-intervention behavior, with conservative labels when evidence was insufficient.
Misbehavior magnitude: 15.14% misbehavior prevalence with course-correction was lower than 18.61% with detection alone, a statistically significant difference.The shadow-mode comparison used a two-proportion z-test with p-value = 0.00003274.
Production A/B test: 5.07% tool call failure rate in treatment was a 4.2% reduction from the control rate of 5.29%, with statistical significance.The live A/B test reported a 99% confidence interval and p-value = 0.0096.
Production A/B test: Token Usage per Session decreased by 5.3% and Engineer Interventions per Session decreased by 4.2%, while execution time improved directionally but was not statistically significant.The corresponding p-values were 0.003, 0.014, and 0.073.
Recovery analysis: Qualitative coding found frequent recovery from infinite loops and failures linked to ignored guidance, premature termination, tool mechanics, merge conflicts, and false negatives.Recovery interventions included updated plans, scope-avoidance guidance, and corrected tool arguments.

5 Threats to validity

The evaluation is limited by Meta-specific deployment, incomplete misbehavior coverage, and possible uncontrolled influences during the prevalence experiment.

5.1 Generalizability: The system was tested and deployed only at Meta, so its observed improvements and taxonomy may not generalize to other platforms.The paper specifically notes that results from VSCode trajectories and Meta-specific categories may not transfer elsewhere.
5.2 Internal Validity: The experiment’s time period may have included uncontrolled concurrent experiments that influenced observed prevalence metrics.The authors attempted to reduce bias with weekly breakdowns and A/B tests, but cannot guarantee complete bias elimination.
5.2 Internal Validity: The taxonomy covers the most prevalent misbehaviors rather than all possible forms, leaving untested categories outside its scope.The paper states that the intervention’s effect on excluded categories remains unknown.

6 Related work

Related work frames agent misbehavior as a trajectory-level diagnostic problem and studies runtime intervention, while this work combines production instrumentation with calibrated classification.

6 Related work: Contemporary coding agents extend beyond code synthesis to repository navigation, build and test orchestration, and automated program repair.This broader capability set establishes the context for studying multi-step agent behavior.
6 Related work: Prior taxonomies distinguish reasoning, execution, and planning errors, while other frameworks address open-source issues, multi-agent failures, and automated diagnosis.These lines of work motivate principled error classes for agent behavior.
6 Related work: Trajectory studies analyze interleaved reasoning, tool actions, and observations using pattern mining, coherence assessment, and LLM-based judging.These methods provide scalable ways to inspect agent behavior beyond final outcomes.
6 Related work: This work instruments production systems, combines quantitative action-pattern analysis with qualitative reasoning evaluation, and calibrates classifiers on human-annotated samples.The approach extends trajectory analysis toward high-fidelity traces from real production systems.
6 Related work: Runtime mitigation methods include process reward models that detect trajectory errors such as redundant exploration, tool-use loops, and failure to terminate.Examples include SWE-PRM and AgentPRM, which score or model intermediate progress during inference.

7 Conclusion

The paper presents a production-derived taxonomy and self-intervention system for recovering coding-agent misbehaviors. It finds automated intervention effective while identifying delayed or complex cases as targets for more sophisticated strategies.

7 Conclusion: The paper introduces a trajectory-derived taxonomy and a self-intervention system designed to automatically recover agentic misbehaviors.The contribution is based on large-scale analysis of real-world software-engineering trajectories.
7 Conclusion: Delayed interventions can be redundant when agents recover independently, while specification-drift corrections may increase user turns despite being justified.These observations motivate more timely and selective guidance.
7 Conclusion: Complex multi-turn misbehaviors motivate hierarchical intervention mechanisms that escalate from simple nudges to comprehensive plan revisions.The paper presents this escalation strategy as a future improvement direction.