Adversarial Review: Structured Disagreement for Grounded Agentic Code Review

1. Introduction

• Reliable automated review requires surfacing the right concerns, motivating lightweight cooperation without the overhead of large role-separated teams.Scaling peer-agent counts can yield diminishing or negative returns through coordination overhead. Production-grade coding agents instead often invoke subagents as tools rather than independent collaborators (Anthropic, 2024).Source: §1 · p. 1
• AR is built incrementally from zero-shot code generation, with each added design choice addressing a measurable failure of the preceding step.Construction is evaluated on LiveCodeBench (Jain et al., 2024), then tested for review quality on SWE-PRBench and repository-level repair on SWE-bench Verified (Jimenez et al., 2024). A Python orchestrator controls comparisons on the first two benchmarks; SWE-bench Verified uses portable, pure-text SKILL.md protocols followed by autonomous agents.Source: §1 · p. 1
• AR freezes the artifact while reviewer and critic exchange review text, separating review convergence from the main agent’s artifact edits.A first-pass review that converges without identifying flaws accepts the artifact immediately. Otherwise, the main agent edits the artifact, updates the change log, and repeats the process.Source: Figure 1 · p. 2
• AR’s minimal reviewer–critic loop improves agentic coding and program repair, while structured disagreement improves code generation and review quality across three benchmarks.The protocol preserves agent interaction while constraining communication rather than expanding a role-separated team.Source: §1 · pp. 1–2
• AR identifies false consensus as a core cooperative-agent failure: agents can optimize for agreement rather than correctness.Structured disagreement is presented as the remedy for this failure mode.Source: §1 · p. 2

Self-refinement and external verification for code

• AR extends test-time refinement by separating reviewer and critic roles and making disagreement explicit rather than an optional self-check.Self-Refine keeps generation, feedback, and revision within one model’s judgment loop (Madaan et al., 2023). Subsequent work finds that models do not reliably self-correct reasoning errors without external feedback (Huang et al., 2024).Source: §2 · p. 2

Multi-agent debate and structured disagreement

• AR preserves agent interaction while restricting communication to one channel with explicit disagreement types, seeking evidence-grounded pushback before edits rather than open-ended debate.Multi-agent debate exchanges arguments to improve reasoning or decisions (Du et al., 2024; Liang et al., 2024), but can depend on protocol choices and underperform cheaper baselines. MARS instead aggregates independent reviewers through a meta-reviewer to reduce debate overhead.Source: §2 · p. 2

Multi-agent systems for code generation and review

• AR asks whether a minimal reviewer–critic pair can provide reliable, evidence-grounded review within a main-agent-with-subagents workflow, rather than adding many specialized coding roles.Earlier systems specialize planning and debugging (MapCoder; Islam et al., 2024), implementation and testing (AgentCoder; Huang et al., 2023), simulation-driven generation (CodeSim; Islam et al., 2025), and refinement and repair (CodeCoR; Pan et al., 2025). CodeAgent adds a QA-Checker to keep review contributions aligned with the original question (Zhang et al., 2024).Source: §2 · pp. 2–3

Agentic coding systems and portable protocols

• AR is evaluated through a Python orchestrator for controlled comparisons and a pure-text SKILL form testing autonomous adherence without external enforcement.This design follows main-agent systems using tools and subagents (Anthropic, 2024), alongside focused subagent instances and portable Skills activated through SKILL.md files (Anthropic, 2026; 2025).Source: §2 · p. 3

Human-agent collaboration in coding workflows

• AR makes subagent disagreement explicit so developers receive review findings grounded in concrete code evidence rather than unchecked speculation.Related work studies progress communication, ambiguity, and user feedback (Li et al., 2022); code review requires findings developers can act on, accept, or redirect.Source: §2 · p. 3

Benchmarks for coding and review

• AR’s benchmark selection separates standalone programming, repository repair, and useful review comments, evaluating it as both a coding protocol and a cooperative-oversight protocol.LiveCodeBench measures code generation with a contamination-resistant, updating benchmark (Jain et al., 2024); SWE-PRBench assesses review quality against pull-request feedback. SWE-bench tests repository edits that resolve real GitHub issues and pass tests (Jimenez et al., 2024).Source: §2 · p. 3

3.1. Zero-shot →Self-Refine: adding verification

• 77% pass-rate for both Zero-shot and Self-Refine (Madaan et al., 2023) shows that adding self-verification did not improve LCB performance.Self-critique can miss the generator’s own errors, motivating a separate reviewer call.Source: §3.1 · p. 3

3.2. Self-Refine →Single-reviewer: external verification

• 77% pass-rate and 36/57 pass-on-hard leave Single-reviewer near Zero-shot despite adding an external review followed by one main-agent edit.A single reviewer may approve without checking or focus on minor issues, motivating multiple independent reviewers.Source: §3.2 · p. 3

3.3. Single-reviewer →Two-reviewers: independence reduces variance

• 75% pass-rate and 34/57 pass-on-hard keep Two-reviewers within noise of the earlier methods rather than improving performance through independent reviews.Two reviewers inspect the code independently, and the main agent edits once using their combined feedback; the result motivates diversity rather than additional independent samples.Source: §3.3 · p. 3

3.4. Two-reviewers →MARS: diversity but with a limit

• 85% pass-rate and 43/57 pass-on-hard distinguish MARS from the four-method cluster, but its five-agent token cost and lack of interaction motivate AR.MARS consolidates three independent structured reviews through a meta-reviewer, revising after rejection for at most K=2 rounds. It avoids MAD’s message-passing overhead, while scaled MAD can also lose pass-rate.Source: §3.4 · pp. 3–4

3.5. MARS →AR: interaction with two agents

• AR holds the artifact fixed while reviewer and critic reconcile reviews, then lets the main agent edit only after receiving a stable review.The inner loop ends at convergence or five rounds; immediate agreement without flaws triggers acceptance. Otherwise, the main agent revises the artifact, updates its change log, and restarts review.Source: §3.5 · p. 4
• 87% pass-rate and 43/57 pass-on-hard make AR the highest-scoring tested method, using two reviewing roles versus MARS’s four.AR’s reviewer and critic interact, whereas MARS’s reviewing agents do not.Source: Table 1 · p. 4, §3.5 · p. 4

3.6. The compact interactive structure

• AR’s three-role structure is the smallest tested configuration retaining interaction: a main agent, reviewer, and critic, with the latter two interacting.Removing the critic yields Single-reviewer; adding reviewers or rounds moves toward MARS-style or MAD-style structures. All subsequent agent and subagent calls use Claude Sonnet 4.5 Medium Reasoning.Source: §3 · p. 3, §3.6 · p. 4

4.1. Experiment Setup

• 100 real GitHub PR diffs comprise SWE-PRBench, where review quality is measured by F1 over comments matched against human reviews.The GPT-5.2 judge achieves Cohen’s kappa = 0.75 against human annotators. The comparison includes AR, MARS, Two-reviewers, and Single-reviewer, excluding Zero-shot and Self-Refine because they do not apply to review-only evaluation.Source: §4.1 · pp. 4–5

4.5. Result

• 0.533 F1 makes AR with text constraint the highest-scoring method in the four-method subset, with only prompts changed.The protocol retains one reviewer and one critic. The case studies motivate explicit pushback because agents seeking agreement can converge without finding the truth.Source: Table 2 · p. 5, §4.5 · p. 5

4.2. AR underperforms naively

• 0.457 F1 places naive AR last among the four methods, while the other three methods reach approximately 0.50 F1.AR’s LiveCodeBench win therefore does not transfer naively to review quality.Source: §4 · p. 4, §4.2 · p. 5

4.3. Two failure modes (case studies)

• 0.250 F1 in the over-decomposition case accompanies three of five comments judged fabricated after the reviewer–critic loop adds speculative findings without filtering.The critic confirms the reviewer’s hedged concerns and adds another speculative bug; formatting then converts each flag into a separate comment.Source: §4.3 · p. 5
• 0.286 F1 for AR versus 0.667 F1 for MARS illustrates a real bug lost when the critic yields to the reviewer’s unsupported rebuttal.The reviewer cites no code, yet the critic switches to AGREE and drops a manually verified concern. Pressure to agree turns disagreement into false agreement.Source: §4.3 · p. 5

4.4. AR with text constraint

• The critic now distinguishes AGREE, DISAGREE EVIDENCE with a code citation, and DISAGREE CONCERN with an epistemic objection.AGREE preserves every flag; DISAGREE EVIDENCE requires revision based on cited code. For DISAGREE CONCERN, the reviewer must cite diff code confirming the bug to retain it or refuting it to drop it.Source: §4.4 · p. 5, Table 2 · p. 5

5.1. Experimental Setup

• 500 real GitHub issues in Python repositories comprise SWE-bench Verified (Chowdhury et al., 2024), evaluated using pass@1 through the official Docker harness.Claude Code uses full tool access and a text-only SKILL.md protocol, letting the main agent choose reviewer/critic invocations and editing rather than following fixed Python orchestration. Estimated runtimes exceeding 300 hours per method limited evaluation to Zero-shot, MARS, and AR; MARS likewise used an autonomous SKILL-style protocol.Source: §5.1 · pp. 5–6

5.2. Result

• 75.2% pass-rate for AR exceeded Zero-shot’s 71.6% and MARS’s 72.6% on SWE-bench Verified.AR used about 4.5× the tokens of Zero-shot, trading token efficiency for performance.Source: Table 3 · p. 6, §5.2 · p. 6

5.3. Case studies

• About 20 lines comprised AR’s root-cause patch, the only patch passing hidden tests after the critic corrected the reviewer’s initial symptom-focused approach.Zero-shot’s 45-line and MARS’s 50-line caller patches both failed hidden tests. AR instead repaired the callee, preserving tick state rather than reapplying visibility settings after an over-broad reset.Source: §5.3 · p. 6
• 32 lines in AR’s RST-writer patch included unnecessary changes and failed hidden tests, while Zero-shot’s scoped 24-line patch passed.The critic did not reject the reviewer’s speculative concern about separate read() calls, prompting unnecessary removal of a stable class variable and addition of read(). The authors suggest making scope discipline explicit alongside evidence-grounded disagreement.Source: §5.3 · pp. 6–7

• AR lies on the cost-quality Pareto frontier across all three benchmarks: no tested method is both cheaper and better.Source: §6 · p. 7
• Figure 2 compares per-task median token usage against performance separately for each benchmark.Source: §6 · p. 7
• AR uses more tokens than Zero-shot on all three benchmarks, trading additional token cost for performance.Source: §6 · p. 7

Narrow cooperation

• AR’s central mechanism is narrow cooperation: a reviewer critiques frozen code, a critic audits that review, and the main agent edits only after review stabilizes.This separation prevents joint rewriting while agents still disagree about the problem. Independent reviewers do not reliably improve LCB, while five-agent MARS achieves a smaller gain than AR.Source: §7 · p. 7

Between tools and teams

• AR bridges passive subagents and large teams: reviewer and critic interact before the main agent acts, and SWE-bench Verified gains persist with a SKILL.md protocol.Improvement without a Python orchestrator suggests that simple, text-expressible roles and decision rules may support portability across agentic systems.Source: §7 · p. 7

False consensus

• False consensus occurs when reviewer and critic agree without sufficient evidence, making conversational convergence look like independent validation.AR performs best on LCB but worst among review-centered methods on SWE-PRBench. Cases show critics endorsing weak findings or abandoning real concerns after confident rebuttals, exposing an interaction-level failure rather than individual-agent weakness.Source: §7 · pp. 7–8

Evidence-grounded disagreement

• Textual constraints improve SWE-PRBench performance by making disagreement explicit and requiring evidence-backed reviewer responses, rather than adding agents.Critics must distinguish agreement, evidence-backed disagreement, and concern-based disagreement, making the path to convergence auditable. However, agents can forget or ignore these instructions, especially near or beyond context-window capacity.Source: §7 · p. 8

Cost and adaptive cooperation

• AR uses more tokens than Zero-shot on all three benchmarks; its tested Pareto-frontier position does not justify review loops for every task.Easy tasks may waste computation, while narrowly scoped tasks risk over-editing; complex repositories and root-cause localization better justify review. Proposed extensions include uncertainty or confidence gates, high-risk-code triggers, scope-checking critics, planning integration, and training from disagreement traces.Source: §7 · p. 8

Scope of the evidence

• AR’s cooperation claims are empirical, not universal: three complementary evaluations do not establish superiority across all coding tasks or agent scaffolds.LCB probes constructive progression, SWE-PRBench probes review quality, and SWE-bench Verified probes skills-form repository repair. Python orchestration tests the mechanism under strict workflow, whereas skills-form execution tests whether it survives delegation to an autonomous agent.Source: §7 · p. 8

Evaluation limitations

• Benchmark and judge limitations leave maintainability, patch minimality, and long-term code quality incompletely assessed, requiring broader evaluation before deployment.LCB measures competitive-programming correctness, SWE-PRBench measures similarity to human feedback, and SWE-bench Verified measures hidden-test repair success. SWE-PRBench’s LLM judge may penalize valid divergent comments or reward superficial matches; case studies only partly address these measurement gaps.Source: §7 · p. 8

• AR achieves the highest measured score among tested methods on all three benchmarks while retaining a compact, interactive main-agent-with-subagents structure.Source: §8 · p. 8
• Structured, explicit, evidence-grounded disagreement may improve both code generation and code review quality more effectively than naively increasing agent count.Source: §8 · p. 8
• The step-by-step construction on LCB progresses from Zero-shot through Self-Refine, Single-reviewer, Two-reviewers, and MARS to AR.Source: §8 · p. 8

A. Prompt templates

• The appendix includes every prompt used by every method, grouped by benchmark.Source: A. Prompt templates · p. 11
• The orchestrator substitutes Python-style format-string placeholders at run time.Placeholders include {specification}, {entry point}, {frozen ask}, {diff}, {problem statement}, {reviewer latest}, and {prior exchange}.Source: A. Prompt templates · p. 11
• The prompts will be released alongside the implementation upon acceptance.Source: A. Prompt templates · p. 11

A.1.1. ALL METHODS: MAIN-AGENT FIRST-VERSION WRITE (LCB)

• Every method initially generates complete Python code without prose or Markdown fences, preserving the exact required entry-point name for hidden tests.The code runs verbatim against hidden tests; a renamed entry point causes NameError regardless of semantic correctness. Necessary imports belong at the top, and undefined helper utilities must be implemented inline.Source: §A.1.1 · p. 11

A.1.2. ALL METHODS: MAIN-AGENT EDIT-FROM-REVIEW (LCB)

• Methods with reviews revise the previous code using the supplied consistent review, returning the complete updated Python code without explanation or Markdown fences.The edit prompt labels its review input as the consistent review from the reviewer–critic loop.Source: §A.1.2 · p. 11

A.1.3. ZERO-SHOT (LCB)

• Zero-shot uses only the initial code-generation prompt, with no reviewer or critic prompts.Source: §A.1.3 · p. 11

A.1.4. SELF-REFINE: SELF-CRITIQUE (LCB)

• Self-Refine critiques its own code for correctness without rewriting, listing specific bugs, missed edge cases, or logical errors.The critique must end with NEEDS_REVISION: <one-line summary> on its own line.Source: §A.1.4 · p. 12

A.1.5. SELF-REFINE: REVISE-FROM-CRITIQUE (LCB)

• Self-Refine then uses the specification, its previous code, and its self-critique to rewrite complete code addressing every flagged issue, without prose or Markdown fences.Source: §A.1.5 · p. 12

A.1.6. SINGLE-REVIEWER, TWO-REVIEWERS, AND AR: REVIEWER PROMPT (LCB)

• Single-reviewer, two-reviewers, and AR require correctness-only reviews that identify bugs, edge cases, and logic errors using line numbers or code snippets, excluding style feedback.Reviewers receive the verbatim original request, code artifact, and any previous inner-loop exchange. The required final line is NEEDS_CHANGES: <one-line summary of top issue>.Source: §A.1.6 · p. 12

A.1.7. AR: CRITIC PROMPT (LCB)

• The AR critic checks whether flagged issues are genuine bugs and identifies missed bugs, then ends with a prescribed verdict line.Disagreement must use DISAGREE: followed by a one-line summary; agreement requires naming no additional missed bugs.Source: §A.1.7 · p. 13

A.1.8. MARS: AUTHOR PROMPT (LCB)

• MARS reuses the initial writing prompt in round one and the edit-from-review prompt thereafter, inserting the meta-reviewer’s Suggestions into the consistent-review slot.These are the existing main-agent prompts from A.1.1 and A.1.2, respectively.Source: §A.1.8 · p. 13

A.1.9. MARS: REVIEWER PROMPT (LCB)

• Three reviewer subagents assess each candidate Python program independently and in parallel, without seeing one another’s outputs.Their Decision, Confidence, Justification, and Errors schema follows Wang et al. (2025) and is shared across benchmarks; only artifact framing and filled slots differ. Confidence is self-reported on 1-5, justification requires evidence, and rejection requires an explicit error list.Source: §A.1.9 · pp. 13–14
• MARS reviewers first interpret the original ask before reading the candidate, then check requirement satisfaction, missed edge cases, logic errors, and unrelated additions.This ordering requires reviewers to form an independent understanding of correct behavior before assessing the generated code.Source: §A.1.9 · p. 13

A.1.10. MARS: META-REVIEWER PROMPT (LCB)

• The MARS meta-reviewer consolidates the completed reviews into one decision and supplies concrete revision suggestions when rejecting the program.Its Decision, Justification, Suggestions, and Recommended-answer schema follows Wang et al. (2025). Suggestions remain blank on acceptance, while Recommended-answer summarizes the correct program.Source: §A.1.10 · p. 14
• The MARS meta-reviewer resolves disagreements using concrete program evidence rather than vote counts or confidence scores alone.Confidence scores are signals, not deciders, because a reviewer can be confidently wrong.Source: §A.1.10 · p. 14

A.2. SWE-PRBench – review-only inner loop

• F1 = 0.533 for AR with text constraint exceeds baseline AR’s F1 = 0.457 on SWE-PRBench.The artifact is a real PR diff, and the reviewer–critic interaction produces a review without a main-agent edit step.Source: §A.2 · p. 14

A.2.1. SINGLE-REVIEWER, TWO-REVIEWERS, AND AR: REVIEWER PROMPT (SWE-PRBENCH)

• Single-reviewer, Two-reviewers, and baseline AR share a prompt requiring diff-specific bugs, logic errors, and missing edge-case handling to be identified with code citations.Single-reviewer invokes it once, Two-reviewers makes two independent calls, and AR invokes it within a reviewer–critic inner loop. Reviewers should say when the diff looks correct.Source: §A.2 · pp. 14–15, §A.2.1 · p. 15

A.2.2. AR: CRITIC PROMPT (SWE-PRBENCH)

• AR’s baseline critic audits the review for both spurious flagged issues and real bugs the reviewer missed.This audit operates within AR’s reviewer–critic inner loop.Source: §A.2.2 · p. 15, §A.2 · p. 15

A.2.3. AR WITH TEXT CONSTRAINT: REVIEWER PROMPT (SWE-PRBENCH)

• The text-constrained reviewer preserves accepted findings and resolves objections through code evidence rather than capitulation.Only a critic-response block changes: contradicting code prompts revision or removal, while epistemic doubts require code confirming or refuting the flag. Bugs identified by the critic must be incorporated under any verdict.Source: §A.2.3 · p. 16

A.2.4. AR WITH TEXT CONSTRAINT: CRITIC PROMPT (SWE-PRBENCH)

• The text-constrained critic separates acceptance, DISAGREE_EVIDENCE, and DISAGREE_CONCERN, using epistemic doubt to request evidence rather than suppress findings.Only the verdict-format block changes from baseline; acceptance requires every flag to be real and nothing to add. DISAGREE_EVIDENCE requires grounding in visible code, whereas DISAGREE_CONCERN signals doubt without code refuting the flag.Source: §A.2.4 · p. 17

A.2.5. MARS: REVIEWER PROMPT (SWE-PRBENCH)

• MARS uses a separate prompt for three parallel, mutually independent reviewers, who assess the diff against an independently formed understanding of the intended fix.They check problem coverage, edge cases, regressions, and unrelated refactoring. Outputs contain Decision, Confidence (1-5), Justification, and Errors.Source: §A.2 · p. 15, §A.2.5 · pp. 17–18

A.2.6. MARS: META-REVIEWER PROMPT (SWE-PRBENCH)

• MARS submits its meta-reviewer’s evidence-weighted consolidation to the judge, rather than relying on reviewer votes or confidence scores.One round follows three independent reviews; the meta-reviewer reads their outputs and the diff. Justification and Errors-style content is submitted for judging, while Suggestions and Recommended-answer are produced but unscored.Source: §A.2 · p. 15, §A.2.6 · p. 18

A.3. SWE-bench Verified – repository-level coding tasks

• Each method uses an inlined, self-contained launch prompt; review methods combine a shared wrapper, method-specific body, and shared overrides.The overrides specify matched models and repeat final constraints. AR uses the baseline two-verdict protocol, without the later text-constraint upgrade.Source: §A.3 · p. 19

A.3.1. ZERO-SHOT, MARS, AND AR: TOP OF EVERY PROMPT (SWE-BENCH WRAPPER)

• The shared wrapper specifies a fresh checkout, issue-only scope, no test edits or commits, and an approximately 30-minute wall-clock budget.The runner grades git diff HEAD after exit, and agents must not load preinstalled skills.Source: §A.3.1 · pp. 19–20

A.3.2. ZERO-SHOT (SWE-BENCH VERIFIED)

• Zero-shot uses only the shared wrapper: the agent reads the issue, edits the repository, and exits without a review-protocol body or overrides.Source: §A.3.2 · p. 20

A.3.3. MARS: SKILL.MD (SWE-BENCH VERIFIED)

• MARS uses three independent reviewers and a meta-reviewer to guide author revisions for at most two rounds.Reviewers cannot see one another’s outputs; the meta-reviewer weighs reasoning and concrete evidence rather than counting votes.Source: §A.3.3 · pp. 20–22

A.3.4. AR: SKILL.MD (SWE-BENCH VERIFIED)

• AR freezes the artifact while a reviewer and critic converge on a review, leaving edits to the main agent between outer iterations.Reviewer and critic receive the same full context, including the diff, project documentation, and related code.Source: §A.3.4 · pp. 23–24
• AR terminates only after first-pass reviewer approval with zero flaws and first-pass critic agreement; otherwise, the main agent revises and restarts review.The inner loop permits at most seven rounds before escalation to the user.Source: §A.3.4 · pp. 23–24
• The reviewer assesses clarity, consistency, feasibility, architecture, best practices, correctness, security, and testability.Every reviewer and critic invocation receives the current artifact, project documentation, related code, and prior reviews from the current inner loop. Fresh subagents have no memory, so full context must be supplied each time.Source: §A.3.4 · p. 24
• AR requires separate plan and code reviews, with agreement among the main agent, reviewer, and critic at each phase.Artifacts remain frozen during reviewer–critic exchanges. Termination requires first-pass reviewer approval and immediate critic agreement; edits require a fresh review loop.Source: §A.3.4 · pp. 24–25

A.3.5. AR: SWE-BENCH INSTANTIATION (CONCRETE TWO-PHASE WORKFLOW)

• The SWE-bench plan lives in untracked APPROACH.md and records root cause, proposed fix, alternatives, risks, edge cases, and files to change.Root-cause analysis includes file:line references. Keeping APPROACH.md untracked excludes it from the runner’s captured diff; staging it is explicitly prohibited.Source: §A.3.5 · p. 26
• 5 inner rounds and 2 outer iterations per phase are the SWE-bench AR limits for both plan review and code-diff review.The agent first explores the repository without edits and writes an untracked APPROACH.md. It then implements the agreed plan as a minimal fix before reviewing git diff HEAD.Source: §A.3.5 · p. 26

A.3.6. ZERO-SHOT, MARS, AND AR: BOTTOM OF EVERY PROMPT (SWE-BENCH OVERRIDES)

• The matched-model experiment requires Sonnet 4.5 for the main agent and every subagent.Shared AR/MARS overrides prohibit unrelated changes, test-file edits, and commits, with approximately 30 minutes total wall-clock time. Zero-shot spawns no subagents and omits this override block.Source: §A.3.6 · p. 27