PaperlayerView exact revision on arXiv

Source-linked AI summary

A Harness Reducing over-Defensive Writing in Agents by Isolating Objections

Xiaochen Chang

arXiv:2610.36717v1cs.CL

TL;DR

Agents expanding reports may add qualifications that answer objections the task has not raised. The paper separates review, objection selection, and writing; in a 25-case evaluation, the harness reduced over-defensive propositions by 96.61% relative to direct writing.

  • Problem

    Report-writing agents can add qualifications that address anticipated objections rather than the source’s content, including after prompts prohibit defensive writing.

    Source: §1 · pp. 1–2

  • Method

    The harness separates review, admission of selected concerns, and writing, so the writer expands the source within an explicit content boundary.

    Source: §5.1 · p. 5

  • Results

    Over-defensive propositions fell by 96.61% relative to direct writing, with no unsupported factual additions identified and source-claim retention 12.77 percentage points higher than direct writing.

    Source: §4.4 · p. 5

  • Takeaways & Limitations

    The admission threshold lets the report adjust how extensively it addresses potential reader concerns.

    Source: §4.4 · p. 5

    The evaluation covers 25 expansion cases using an LLM assessor; a larger test set and human annotation can assess judgment consistency.

    Source: §5.2 · p. 5

Abstract

from arXiv · show

When expanding a report, writing with agents may introduce a stronger claim than the article makes and then disclaim it. This is called over-defensive writing. We assume that this is because the agent is writing for responding to an adversarial AI reviewer, instead of a human reader. The defensive context may also influnce the consequent when we simply ban defensive writing in the output through prompts. In order to reduce over-defensive writing, we produce a harness that separates the reviewer, filter, and writer. The writer agent receives only the problems choosen by the filter, which is an independent context. We evaluate this harness on 25 business-project drafts expanded into formal reports, the result shows that the defensive propositions decreased from 59 to 2, which is a 96.61\% reduction, no unsupported factual additions were identified while the source-claim retention rises from 80.35% to 93.12%.

1 Introduction

The paper frames over-defensive writing as adding or preserving exclusions that answer objections the task has not raised. It proposes separating objection selection from writing so only relevant issues enter the writer’s context.

1 Introduction

  • Over-defensive additions can introduce a stronger claim than the source and then disclaim it, diverting attention from the project explanation readers need.The example shifts from explaining a delivery-time change to denying that objectives have been achieved.Source: §1 · p. 1
  • Review-oriented objectives can prompt agents to add qualifications that careful readers do not need, turning anticipated criticism into a writing goal.The paper reports a writing run in which concerns raised by an unrequested reviewer subagent appeared in the report.Source: §1 · p. 1
  • Style-focused instructions to avoid defensive writing leave residual cases because qualifications can remain while changing form.Table 2 illustrates a qualification persisting after an output prohibition.Source: §1 · pp. 1–2, Table 2 · p. 2
  • Showing the writer every concern with a probability label can increase defense: a concern rated 10% still appeared in the output.Supplying only high-probability points and bounding responses to them reduced over-defense.Source: §1 · p. 2
  • The proposed workflow separates review, selection, and writing, with the writer expanding the source within an explicit content boundary.The study asks whether the writer can address specified clarifications without adding further defensive content.Source: §1 · p. 2

2 Study Design

The study evaluates agents expanding short project drafts into formal reports and defines criteria for identifying over-defensive writing. It also distinguishes defensive targets from the wording used to express them.

2.1 Materials and Annotation Criteria

  • The task expands 25 business-project drafts into approximately 500-word English reports, with GPT-6.1 Sol handling review, selection, and writing and DeepSeek-V4.1-Flash-1017 assessing outputs.Direct expansion serves as the baseline.Source: §2.1 · p. 2
  • The first three criteria cover denying unstated claims, adding unsupported doubts or limitations, and repeating qualifications that already establish scope.Repeated restrictions count when they add no factual, explanatory, or decision-relevant information.Source: §2.1 · p. 2
  • The remaining criteria identify weakening a supported claim and diverting the argument to preempt objections rather than advance or explain it.The criteria retain uncertainty already present in the source.Source: §2.1 · pp. 2–3

2.2 Defense Targets and Implicit Expression

  • Defense may be explicit or implicit: evaluation follows the proposition a passage answers, including when an affirmative statement realizes the same target.The example distinguishes a processing-time target from the claim that delivering technology establishes achievement of business objectives.Source: §2.2 · p. 3
  • Because an output prohibition can change wording while preserving the same defensive function, evaluation tracks targets across semantic rewrites.The paper uses semantic matching to locate potential issues in intervention outputs.Source: §2.2 · p. 3

3 Hypotheses and Tests

The tests examine whether content boundaries can separate writing from reviewing, whether approved review points trigger additional defense, and whether responses stay within the source’s evidence.

3 Hypotheses and Tests

  • The tests bound the writer’s response to review points to determine whether review can be separated from writing without introducing new defensive targets.A simple prohibition on additional defensive review leaves such additions in the output.Source: §3 · p. 3

3.1 Separating Writing from Reviewing

  • A content-boundary instruction almost eliminates defensive propositions, whereas instructions simply to avoid defensive review or writing do not remove them completely.The instruction specifies what the writer needs to address and changes the criterion for including a response.Source: Table 3 · p. 3, §3.1 · p. 3

3.2 Introducing Review Points into Writing

  • Atomic review points perform better than compound points, and an added instruction not to weaken unaddressed source claims reduces unrelated defensive additions to zero.Table 4 reports writing with review points.Source: Table 4 · p. 3, §3.2 · p. 3

3.3 Preventing Invented Evidence

  • Review points can prompt writers to present unconfirmed details or actions as established facts, so responses need an evidential boundary as well as a topic boundary.Direct writing and the source-bounded instruction did not overextend source claims across the 25 cases.Source: §3.3 · p. 3
  • Requiring responses not to exceed the material’s evidential strength removes the problem by distinguishing discussion of an issue from evidence that its premise or solution is true.A proposed action can be described as a proposal, but an unsupported arrangement cannot be presented as accomplished fact.Source: §3.3 · p. 4

4 A Harness That Separates Review and Writing

The harness separates review, filtering, and writing so the writer receives only admitted objectives in a fresh context. Across 25 business-project drafts, it reduced over-defensive propositions and retained more source claims than direct writing.

4 A Harness That Separates Review and Writing

  • The harness separates reviewer, filter, and writer agents, with the writer running in a fresh context restricted to approved content.The writer is instructed not to conduct another defensive review.Source: §4 · p. 4

4.1 Review: Generate Atomic Questions

  • The review agent turns the source files and task into atomic points, separating questions that need independent answers and grouping redundant paraphrases.Each point concerns one concrete object and one relationship, claim, condition, mechanism, or information request.Source: §4.1 · p. 4

4.2 Filter: Admit Useful Response Targets

  • The filter estimates likely reader misunderstandings or challenges and constructive follow-ups, then admits selected points as separate main-text or future-work objectives.The complete audit remains outside the writer’s input; only admitted objectives are passed forward.Source: §4.2 · p. 4

4.3 Writing: Develop Approved Content

  • Writing develops the source and approved main-text objectives first, then adds approved future-work discussion separately without changing the main text.Responses must remain within the evidence in the current material; review objectives authorize responses but do not establish new facts.Source: §4.3 · p. 4

4.4 Results

  • The harness reduced over-defensive propositions by 96.61% versus direct writing, had no assessor-identified unsupported factual additions versus 51 under the output-level prohibition, and retained 12.77 percentage points more source claims than direct writing.Source: §4.4 · p. 5

5 Conclusion and Discussion

The conclusion presents separating review, admission, and writing as a way to reduce over-defense while retaining more source claims than an output-level prohibition. The discussion frames admission as the step between generating feedback and using it, and notes that the evaluation remains limited to 25 expansion cases assessed by an LLM.

5.1 Conclusion

  • The conclusion attributes reduced over-defense and greater source-claim retention than an output-level prohibition to separating review, admission, and writing before generation.Review identifies questions requiring clarification, selection turns some into objectives, and the writer develops the source and approved objectives.Source: §5.1 · p. 5

5.2 Discussion and Further Work

  • The workflow makes admission explicit between feedback generation and writing, connecting review-point selection to feedback-based refinement and instructional-distraction research.The discussion says the three research questions test separation, uptake of approved questions, and how much pre-writing review can cover.Source: §5.2 · p. 5
  • The evaluation covers 25 expansion cases assessed by an LLM; a larger test set and human annotation could assess judgment consistency.Source: §5.2 · p. 5

A Core Prompts

The prompt set separates report writing from assessment, review-question generation, and review-point selection, with staged instructions to keep judgments and writing grounded in supplied material, approved objectives, and evidence. It also distinguishes the main report from an optional future-work addition.

A.1 User Task

  • The user task is to expand each project draft into an approximately 500-word rigorous English report for a supervisor, outputting only the report.Source: §A.1 · p. 7

A.2 Assessing over-Defense

  • The assessor flags five forms of excess: disclaimed stronger claims, unsupported caveats, needless repetition, weakened claims, and preemptive objections or concessions.Source: §A.2 · p. 7
  • Assessors judge against the source and task, count distinct semantic propositions across the whole report, and separate clear findings from materially debatable borderline cases.Necessary source boundaries, useful explanations, grounded discussion, and constructive proposals are not inherently excessive; no minimum finding count is required.Source: §A.2 · p. 7
  • Each finding requires exact report quotations and occurrences, source comparison, a concise judgment, and location; the output records criteria, coverage, counts, and a first-pass receipt.Empty findings are valid, and reports must not be revised.Source: §A.2 · pp. 7–8

A.3 Review Agent

  • The technical reviewer creates relevant, atomic, nonredundant questions from supplied material and task, noting source basis and whether answers are supplied, partial, or absent.It must not assume missing information proves a defect or write the report, use external information, or delegate.Source: §A.3 · p. 8

A.4 Selection Agent

  • The selection agent first estimates two independent subjective probabilities for each point: critical challenge or misunderstanding, and constructive follow-up during ordinary reading.The reader is a careful, field-familiar human without access to absent project-specific facts; estimates concern reactions to the current source, not a future rewrite.Source: §A.4 · p. 9
  • In round two, it freezes round-one estimates, chooses one rigor-based threshold from 0.33 to 0.67, and admits points separately to body and future-work streams using P1 and P2.Use T = 0.50 absent a stated strictness preference; apply the same threshold to both streams.Source: §A.4 · pp. 9–10
  • Admitted points become affirmative objectives, not factual evidence: body objectives clarify supported claims, while future objectives propose additional work without implying it is completed or disproves a claim.If a point enters both streams, it receives distinct body and future objectives.Source: §A.4 · p. 10
  • The Selection Agent section provides an “ATOMIC REVIEW POINTS {{REVIEW_POINTS}}” input placeholder.Source: §A.4 · p. 10

A.5 Writing Agent

  • The writer completes the main report first, then adds optional future-work paragraphs in a second round without altering the frozen main text.Source: §A.5 · p. 11
  • For the main text, the writer uses original material and approved body-response objectives, ignoring unapproved doubts and preserving unchallenged original claims.Source: §A.5 · p. 11
  • Responses must stay within the original material’s evidential strength; probabilities guide emphasis, while proposed actions remain proposals rather than established results.Source: §A.5 · p. 11
  • The prompts also require professional, technically accurate writing, prohibit external retrieval and further review, and restrict each round’s output to its assigned text.Source: §A.5 · pp. 11–12
  • In the future-work round, the writer addresses approved future objectives prospectively and must not imply that proposed actions are completed or outcomes guaranteed.Source: §A.5 · pp. 11–12
  • Future work must not add criticism or disclaimers about the main text, introduce unapproved claims, or duplicate body responses already present.Source: §A.5 · p. 12

Related papers

Loading 2610.36717v1…