Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix

1 Introduction

• Stale instructions can repeatedly produce code that diverges from the project and, when conventions affect correctness, can lead to broken behavior.The instruction file is read at the start of every session, so the conflicting account shapes each change.Source: §1 · p. 2
• 20 of Codex CLI’s 80 runs rewrote existing modules to match the stale file, and the original tests failed in 17 of those runs.Claude Code typically named the conflict and still used the file’s convention.Source: §1 · p. 2
• The authors attribute this behavior to the instruction file reaching the model as an instruction while existing code arrives as tool output, which models are trained to rank lower.They add a user-level sentence giving existing code precedence; across 40 projects and two agents, agents followed the code in nine out of ten choices and did not rewrite existing code.Source: §1 · p. 2

2 How instruction files go stale

• 73.8% of AI configuration files in 441 company repositories were committed once and never modified, while a checker flagged outdated code references in 23.0% of 356 public repositories.The same issue appeared in large projects: a Supabase instruction file described React 18 for 13 days after a move to React 19, and Codex CLI’s AGENTS.md referenced a deleted document for 24 days.Source: §2 · p. 2
• A stale instruction can persist because correcting files depends on someone noticing which lines have become wrong, a difficult task in a large project.Claude Code’s documentation recommends periodically reviewing instruction files and removing outdated lines.Source: §2 · p. 3
• Stale conventions can make new code diverge from existing code, require reviewer corrections, and recur across tasks while the file remains unchanged.A stale rule about monetary amounts led agents to use floating-point arithmetic despite existing code using integer cents.Source: §2 · p. 4

Instruction files describe the code in the form of commands

• A stale line keeps its imperative wording even after a code change makes the description it records false.The paper frames instruction files as descriptions of code written in the form of commands.Source: Instruction files describe the code in… · p. 4

Agent software delivers the file as an instruction

• Claude Code and Codex CLI deliver project instruction files in user messages, while existing code reaches the model through file-reading tool output.The tools place the user-level file and project file in the same message, with the user-level file first.Source: Agent software delivers the file as an… · pp. 4–5

Models give instructions priority over tool output

• Models are trained to prioritize system and user messages over tool output, an ordering intended to protect against prompt injection.The paper notes that agents can act on tool output when no instruction addresses the matter, but follow an existing instruction when one does.Source: Models give instructions priority over… · p. 5
• A user-level instruction that gives existing code precedence makes following the code an instruction for this specific documentation-code conflict.The paper predicts agents will follow stale files even after noticing the conflict, and that the added sentence will reverse the outcome.Source: Models give instructions priority over… · p. 5

Projects

• Each project consistently used one convention in its existing code while its instruction file asked for another, with both conflict directions represented across 40 projects.Projects had three to eight working modules, passing tests, and feature requests that did not mention conventions.Source: Projects · p. 5

The sentence

• The added sentence directs agents to follow existing code when documentation disagrees and to identify each inconsistency in the final reply.Every project was run both with and without this sentence in the user-level instruction file.Source: The sentence · pp. 5–6

Agents

• Claude Code with Claude Sonnet 5.5 and Codex CLI with GPT-6 Sol completed feature requests in non-interactive runs with edits approved automatically.Each project was run twice per agent and condition, for 320 runs overall.Source: Agents · p. 6

Measures

• The measures classified code choices, checked whether final replies named conflicts, counted rewrites, and recorded test results, tool calls, and runtime.A run counted as rewriting existing code if it removed at least 20 lines from pre-existing Python files; original tests and held-out feature tests were run after each run.Source: Measures · p. 6

5.1 Agents follow the stale file

• Without the sentence, 90.1% of choices matched the stale file and 4.2% matched existing code.The file prevailed for both agents in 14 of 16 choice types.Source: §5.1 · p. 7

5.2 Claude Code reports the conflict and still follows the file

• Claude Code named the conflict in 79 of 80 runs, yet 86.2% of its choices in those runs still matched the file.Figure 6 summarizes what each run did across agents and conditions.Source: §5.2 · p. 8, Figure 6 · p. 8

5.3 One sentence makes agents follow the code

• With the sentence, 91.4% of choices matched existing code, 87.2 percentage points more than without it, when just 4.2% did.Both agents reported conflicts in all 160 runs with the sentence.Source: §5.1 · p. 7, §5.3 · p. 8

5.4 Without the sentence, Codex CLI rewrote working code

• Without the sentence, Codex CLI rewrote existing code in 20 of 80 runs, and the original tests failed in 17 of those runs.Figure 7 shows one such rewrite; with the sentence, no Codex CLI run rewrote existing code.Source: §5.4 · p. 8, Figure 7 · p. 9

5.5 Tests pass in every run with the sentence

• Tests passed in 89.4% of runs without the sentence and in all 160 runs with it.Every failing run was a Codex CLI rewrite; runs with the sentence also used slightly fewer tool calls and less time.Source: §5.5 · p. 9
• With the file, its wording, and its context held constant, only the precedence statement changed; the outcome followed that statement, consistent with instruction priority.Codex CLI followed the file in 93.9% of choices without the sentence, despite not using Claude Code’s emphatic preamble.Source: §5.5 · p. 9

Users

• Users can add the precedence sentence once to a file they already maintain, where it then applies to every project.The sentence gives existing code precedence when it conflicts with documentation.Source: Users · p. 9

Makers of agent software

• Agent makers can place the precedence rule in the agent’s system prompt, making it reach users without action on their part.Figure 8 summarizes steps available to each group.Source: §6 · p. 9, Makers of agent software · p. 10

Makers of models

• Model makers can have models check code descriptions against the code, avoiding the problem without user action.Instruction files mix rules for the agent with descriptions of the code.Source: Makers of models · p. 10

7 Related work

• Recent studies examine instruction-file contents and maintenance [4] [8] [10], and effects on task success [17], running time [18], rule type [19], and rule-following reliability [12].Earlier work by the authors measured responses to skill documents [20] [21], test feedback [22], and tool error messages [13].Source: §7 · p. 10
• Existing tools detect outdated documentation references; Treude and Baltes [9] applied one to AI instruction files, but a checker helps only when run.The paper contrasts this with the sentence, which takes effect in every task.Source: §7 · p. 10
• Mohammadi et al. [5] showed agents follow written rules over contradicting code; this study finds they can recognize the conflict and traces the behavior to instruction hierarchy [6] [7].The authors report correcting it with one sentence.Source: §7 · pp. 10–11

• The study covers 40 Python projects in which the instruction file is the outdated source.Source: §8 · p. 11
• The projects span 16 kinds of convention.Source: §8 · p. 11
• Feature tasks were completed in a single unattended run by Claude Code with Claude Sonnet 5.5 and Codex CLI with GPT-6 Sol.Source: §8 · p. 11

• 4.2% to 91.4%: one user-level sentence raised the share of choices that follow existing code.Source: §9 · p. 11
• The authors attribute stale-file compliance to instruction files reaching the model as instructions and code reaching it as tool output.Source: §9 · p. 11
• The proposed sentence gives existing code precedence without requiring users to first find and correct the stale file.Source: §9 · p. 11

Funding.

• The research received no specific grant from funding agencies.Source: Funding · p. 11

Competing interests.

• The authors declare no competing financial interests or personal relationships that could have influenced the work.Source: Competing interests · p. 11

Ethics approval and consent to participate.

• The study used no human participants or personal data, so ethics approval and consent were not applicable.Source: Ethics approval and consent to… · p. 11

Data and code availability.

• The projects, tasks, run records, and analysis scripts are available in the stated data repository.Source: Data and code availability · p. 11

Agent commands

• Both agents ran in fresh project copies with the feature request on standard input and a 1,800-second time limit.Source: Agent commands · p. 11
• Claude Code ran with the Sonnet model, while Codex CLI ran with GPT-6 Sol and a dedicated user-level instruction-file configuration.Source: Agent commands · pp. 11–12
• Python and pytest commands were allowed, while package-install commands were disallowed.Source: Agent commands · p. 11

Failed attempts

• Runs stopped by usage limits or login problems were restarted from fresh copies; timeout, crash, or sandbox-rejection failures were retried once, and all 320 runs were valid.Source: Failed attempts · p. 12

Conflict judge

• Final replies were shuffled and judged under opaque identifiers using a criterion fixed before the first run.Source: Conflict judge · p. 12
• The judge returned YES or NO, with a sentence stating the conflict as evidence for YES.Source: Conflict judge · p. 12