PaperlayerView exact revision on arXiv

Source-linked AI summary

Measuring and Reducing Unusable Replies of LLM Agents

Yilin Jiang, Fei Tan, Xuanyu Yin, Guanchu Wang, Xia Hu, Yutao Yue

arXiv:2610.19659v1cs.CLcs.AIcs.HC

TL;DR

LLM-agent evaluations measure task completion and report faithfulness, but not whether requesters can understand and use the reply they receive. The paper measures reply usability from the requester’s position and finds that failures are common, while explicit requirements and secondary-model rewrites reduce them.

  • Problem

    Current evaluation scores task completion and report faithfulness but leaves open whether requesters can understand and use an agent’s reply.

    Source: §1 · p. 1

  • Method

    The paper defines reply usability using requester context, language, and whether the reply answers the request, and measures it with a reader model.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    54% of 480 replies failed across eight models from seven families; requirements reduced failure from 0.52 to 0.43, and rewrites to 0.36.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Reply usability can be measured alongside task success and improved with an appended requirements message or a secondary-model rewrite.

    Source: §8 · p. 9

    The collaborations are scripted, the reader represents field-informed requesters who have not seen the agent’s work, and sampling analysis covers one open-weight model.

    Source: Limitations · p. 9

Abstract

from arXiv · show

People delegate multi-step work to LLM agents and read only the reply that the agent hands back. Current evaluation scores task completion and the faithfulness of reports, and it leaves open whether the requester can understand and use the reply. We call this property reply usability and measure it with a reader model that takes the requester's position and tests whether the reply can be interpreted from what the requester has seen, is in the requester's language and answers the request. Its agreement with the majority of five annotators lies within the range of the individual annotators. Across eight models from seven families, 54% of 480 replies fail, and the best model still fails on 30%. Failed replies mostly point to material the requester has not seen or use labels that the agent coined during its work. Stating explicit requirements for the reply reduces the failure rate from 0.52 to 0.43 together with errors and omissions, and a secondary model that rewrites the reply reduces it to 0.36, which human annotators confirm in a blind comparison. Reply usability can thus be scored alongside task success and improved with one appended message or one call to a secondary model.

1 Introduction

Because requesters often see only an agent’s final reply, task completion and faithfulness do not establish that they can understand and use it. The paper defines reply usability from the requester’s position, measures it with a reader model, and finds that failures are common while explicit requirements and rewriting reduce them.

Motivation

  • Current evaluations score task completion or faithfulness, but a reply can satisfy both and still be unusable to the requester.Examples include invented run names, tool-only findings, and changes reported without answering the question.Source: §1 · pp. 1–2

Definition and measurement

  • A reply fails if the requester cannot interpret it from what they have seen, it is in the wrong language, or it does not answer the request.The reader model takes the requester’s position; its agreement with the majority of five annotators is κ = 0.60, within the range of individual annotators.Source: §1 · p. 2

Main findings

  • 54% of 480 replies failed across eight models from seven families, mostly because replies used material or labels from the agent’s private work.Explicit reply requirements reduced the failure rate from 0.52 to 0.43; secondary-model rewrites reduced it to 0.36, with annotators confirming the rewrite reduction.Source: §1 · p. 2

Contributions

  • The paper contributes a requester-centered usability measure, evidence that unusable replies occur across evaluated model families, and two remedies: requirements and rewriting.The authors also state that they will release the collaborations, reader, and judgments.Source: §1 · p. 2

2 Related Work

Related work studies common ground, decontextualized text, agent self-reports, audience instructions, and readability or faithfulness. This paper distinguishes its focus by asking whether a requester who has not seen the agent’s trajectory can understand and use the report.

Common ground and decontextualization

  • The paper adapts question-answering approaches to decontextualization by replacing an empty reference context with what the requester has seen.Related work examines common ground in conversation and whether text can be interpreted outside its original document.Source: §2 · p. 2

Reports of agents about their own work

  • Prior work on agent reports evaluates faithfulness and completeness against work trajectories, while this paper tests whether a requester without that trajectory can use the report.The paper describes requester-side usability as complementary to faithfulness.Source: §2 · pp. 2–3

Instructions about the audience

  • Prior studies report limited effects from audience or persona instructions; this paper compares a reader notice with concrete reply requirements under the same context and information.Source: §2 · p. 3

Readability and faithfulness

  • Because rewriting can introduce factual errors or omissions, the paper checks replies against work records and varies reply-writing context separately from requirements.Related work also studies simplification and changes to information presentation.Source: §2 · p. 3

3 Measuring Reply Usability

The section defines reply usability from the requester’s context and describes a reader model for detecting failures, alongside a checker for faithfulness. The reader’s judgments align with human ratings, and replies with more detected information gaps are more often judged failed.

3.1 Reply Usability

  • A reply is usable when the requester can understand it from requester context R, it is in the requester’s language, and it answers the request.An information gap is an expression that requires information unavailable in either the reply or R.Source: §3.1 · p. 3

3.2 Measuring Usability with a Reader Model

  • The reader generates questions tied to reply spans, searches the nearby text, the full reply, and R for quoted answers, then checks language and whether the request is answered.Source: §3.2 · p. 4
  • The reader agrees with the majority label at Cohen’s κ = 0.60 and 82% accuracy, within the range of individual annotators’ agreement with the other annotators.Five annotators labeled a 300-reply gold set; their agreement was Fleiss’ κ = 0.63.Source: §3.2 · p. 4
  • Majority failure rises from 26% with no reader-found gaps to 71% with one and 94% with two or more; 68% of detected gaps overlap annotator-marked spans.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3 Measuring Faithfulness with a Checker Model

  • A separate checker compares each reply with the agent’s work record and counts factual errors plus key results omitted from the reply.Source: §3.3 · p. 4

4 Experimental Setup

The experiments use scripted multi-session collaborations in research and business projects, giving models the same requests and projects. Eight models from seven families each work across all four projects, yielding 32 collaborations and 480 replies.

4.1 Collaborations

  • Each collaboration contains 15 requester messages over three sessions, with the agent’s first message after each request without tool calls treated as its reply.Long histories can be replaced by agent-written summaries, and new sessions load workspace documents as project memory.Source: §4.1 · pp. 4–5
  • The four scripted projects span research and business tasks, and every model receives the same projects and requests.Research tasks include literature-index searches and simulated studies; business tasks use a reference library and design simulator.Source: §4.1 · p. 5

4.2 Models

  • Eight models from seven families, including closed and open-weight models, each work on four projects, producing 32 collaborations and 480 replies.Source: §4.2 · p. 5

5 Replies Fail Across Model Families

Across model families, more than half of default replies fail, often because they rely on information or vocabulary from the agent’s private work. Failures are more frequent after extended tool use, and coined labels often originate in the agent’s own files or memory.

5.1 Prevalence

  • 258 of 480 replies fail (0.54); seven of eight models fail on at least 48% of replies, with rates from 0.30 for GPT-5.6 Sol to 0.82 for Claude Opus 5.The system prompt gives no particular reply-style instruction.Source: §5.1 · p. 5
  • 222 of 258 failed replies contain an information gap, 73 do not answer the request, and 17 use the wrong language; replies can meet multiple conditions.Source: §5.1 · p. 5
  • Replies following three or more tool-calling turns fail at 0.70, versus 0.48 with no tool call and 0.40 after one or two turns.In six of eight models, replies after three or more turns fail more often than replies without a tool call.Source: §5.1 · p. 5

5.2 What the Requester Is Missing

  • Pointers to unseen material and coined labels each occur in more than 40% of failed replies.At least a third of failed replies in six of seven families contain pointers to unseen material, and at least a quarter contain coined labels.Source: §5.2 · p. 5
  • 53.0% of coined labels first appear in agent-written files, tool-call arguments, or project memory loaded in a later session.The agent’s notes and memory preserve these working names, which can reach replies as unexplained labels or source identifiers.Source: §5.2 · p. 6

6 Reducing Failures

Requirements appended to a reply prompt reduce failures, while a secondary model can rewrite an existing reply with a further reduction. Requirements also improve sampling outcomes, and rewriting reduces errors and omissions.

6.1 Writing Conditions

  • The interventions were tested at 256 reply points; conditions included writing in the agent’s context and separate passes with the request and work record.The requirements message is appended at the point of reply; separate-pass variants differ in whether they receive extracted results or the work record and requirements.Source: §6.1 · pp. 6–7

6.2 Explicit Requirements

  • Explicit requirements reduce the failure rate from 0.52 to 0.43, while errors plus omissions fall from 1.41 to 1.07 per reply.The requirements specify an answer first, what was done, the result, next steps, evidence locations, and definitions of unseen names.Source: Table 4 · p. 7, §6.2 · p. 7, §6.1 · p. 7
  • The difference between the two conditions containing the requirements message is 0.004 [−0.069, 0.075], so a separate writing step adds no measurable reduction.Neither of the three conditions without the requirements message differs significantly from the default.Source: §6.2 · p. 7

6.3 Rewriting by a Secondary Model

  • Rewriting reduces errors plus omissions per default reply from 1.41 to 1.13; errors fall from 0.73 to 0.54, while the cited passage gives a 0.68 omission baseline.The rewriting model receives the request, work record, and default reply, then rewrites it once under the requirements.Source: §6.3 · p. 7
  • Rewriting reduces the failure rate from 0.52 for default replies to 0.36, a reduction of 0.168; human annotators also report fewer failures in a blind comparison.The reader’s reduction is lower in seven of eight models and six of seven families; annotator failure share falls from 0.52 to 0.32.Source: §6.3 · p. 7, Figure 4 · p. 8

6.4 Sampling

  • Under requirements, pass@k is 0.52 for one sample, 0.84 for four, and 0.97 for sixteen; four samples cover more points than sixteen default samples.Default pass@k is 0.36, 0.65, and 0.81 for one, four, and sixteen samples, respectively.Source: §6.4 · p. 8

7 Discussion

The discussion locates reply failures at the transition from working to reporting: agents retain a private vocabulary that requesters may not know. It argues that final-environment evaluation can miss this usability problem and that the reporting step merits its own instruction.

7 Discussion

  • Agents carry run names, configuration labels, and source identifiers from their work into replies, where requesters may lack the referents.Notes and memory can preserve this vocabulary across sessions.Source: §7 · p. 8
  • Evaluating only the final environment can credit completed work even when the requester cannot use its report.A reader model taking the requester’s position supplies a check on this gap.Source: §7 · p. 8
  • Because memory and compaction carry working names into later replies, the discussion argues for an instruction specifically at the point of return to the requester.A secondary model can enforce that instruction when the agent’s prompt is fixed.Source: §7 · p. 8

8 Conclusion

The paper defines reply usability from the requester’s position and finds failures common across evaluated model families, often because replies rely on the agent’s private work. Explicit requirements and secondary-model rewriting reduce these failures.

8 Conclusion

  • Across eight models from seven families, more than half of replies fail, mostly because they use terms from the agent’s private work.The paper presents reply usability as measurable alongside task success and faithfulness.Source: §8 · p. 8
  • A reader model measures whether a requester can understand and use the reply from their own position.Its agreement with human annotators is described as being at the level of individual annotators.Source: §8 · p. 8
  • Explicit requirements or a secondary-model rewrite reduce unusable replies, supporting usability checks alongside task success.Human annotators confirm the rewrite improvement.Source: §8 · p. 9

Limitations

The study uses scripted collaborations and a requester profile with field knowledge but no access to the agent’s work. Its sampling analysis covers one open-weight model.

Limitations

  • The collaborations are scripted, giving each model the same requests and recording private context for analyzing gap origins.Source: Limitations · p. 9
  • The reader represents a requester who knows the field but has not seen the agent’s work, and the gold set validates this profile.Source: Limitations · p. 9
  • The sampling analysis covers one open-weight model; the authors identify logged user sessions, other requester backgrounds, more models, and languages as next steps.Source: Limitations · p. 9

Ethics Statement

The paper describes its synthetic collaborations, requester-position reader, human annotation, and reply-writing conditions. It also reports how the reader’s judgments compare with annotators’ judgments.

Ethics Statement

  • All collaborations are synthetic and contain no personal data; participating researchers consented to use of their labels and received fair compensation.The annotations validate the reader and gap classifier and evaluate rewrites, but are not released.Source: Ethics Statement · p. 9

A Reader and Checker

  • The reader generates requester-relevant questions, searches the reply and requester context for answers, and marks unanswered questions as information gaps.A separate checker compares replies with the agent’s work record for errors, omissions, and downplayed problems.Source: A Reader and Checker · p. 11

B Agreement with Annotators

  • Annotators label replies as readable as-is, readable after added information, or not repairable or not answering the request; the latter two count as failures.Annotators saw requester-visible context, not the agent’s tool calls, outputs, or written documents.Source: B Agreement with Annotators · p. 12
  • The reader fails 171 of the 191 replies the annotator majority fails, and 34 of the 109 replies the majority passes.The reader’s gaps overlap with annotator-marked spans, and its no-answer verdict agrees with the majority on 290 of 300 replies.Source: B Agreement with Annotators · p. 12

C Collaborations

  • Collaborations use tool-equipped workspaces, multiple sessions, and project-memory documents; the agent may take up to 30 tool-calling turns per requester message.The writing conditions include a prompt requiring the reply to answer the latest request, state results and next steps, and define unfamiliar names.Source: C Collaborations · p. 13, D Writing Conditions · p. 14

E Rewriting

The rewriting experiment compares default replies with rewrites produced from the request, work record, and default reply. The paper also evaluates information-gap categories and the origins of coined labels.

E Rewriting

  • The rewriting model receives the request, work record, and default reply, then rewrites the whole message using only work-record facts.Its prompt asks for a reply that answers the latest message first and explains unfamiliar names.Source: E Rewriting · pp. 14–15
  • Human annotators’ failure share falls from 0.52 for default replies to 0.32 for rewrites, a difference of 0.201 [0.095, 0.321].The share is lower for rewrites in seven of eight models.Source: E Rewriting · p. 15

F Gap Categories

  • Among 480 replies, 258 had no information gap, 108 had one, 49 had two, and 65 had three or more, giving the reported distribution of information gaps across replies.The classifier agreed with the two annotators at Cohen’s κ of 0.69 and 0.67; the annotators agreed with each other at κ = 0.68. For gaps assigned the same category by both annotators, classifier agreement with that category was κ = 0.82.Source: F Gap Categories · p. 15
  • The study traces coined labels to their earliest occurrence in the agent’s context, using message type to identify their source.Labels without an earlier occurrence are counted as first used in the reply itself.Source: F Gap Categories · pp. 15–16

G Sampling Analysis

The sampling analysis draws 100 reply points from Qwen3.6-35B-A3B collaborations and uses pass@k to average the chance of finding a usable reply across points. Table 12 presents failure shares by kind and model family.

G Sampling Analysis

  • 100 reply points are sampled at random from collaborations run by Qwen3.6-35B-A3B locally with llama.cpp and thinking disabled.Source: G Sampling Analysis · p. 16
  • pass@k averages, across reply points, the probability that k samples drawn without replacement include at least one usable reply.The probability is defined using c usable replies among n samples for a point.Source: G Sampling Analysis · p. 16
  • Table 12 reports the share of failed replies by failure kind and model family.Source: Table 12 · p. 17

Related papers

Loading 2610.19659v1…