PaperlayerView exact revision on arXiv

Source-linked AI summary

EvalAgent: Discovering Implicit Evaluation Criteria from the Web

Manya Wadhwa, Zayne Sprague, Chaitanya Malaviya, Philippe Laban, Junyi Jessy Li, Greg Durrett

arXiv:2504.15219v2cs.CL

TL;DR

Open-ended writing tasks need nuanced, task-specific evaluation criteria, but human evaluation is difficult to scale. EVALAGENT mines instructional web guidance to generate criteria; experiments report that these criteria are actionable and more human-aligned than simply prompting an LLM.

  • Problem

    Evaluating LLM outputs for open-ended writing tasks is difficult to scale: automated methods are limited, while human evaluation is subjective and requires expertise.

    Source: §1 · pp. 1–2

  • Method

    EVALAGENT searches for instructional web documents, summarizes their advice, and synthesizes it into task-specific evaluation criteria.

    Source: §1 · p. 2

  • Results

    Across nine writing-task datasets, EVALAGENT criteria are reported as highly actionable and more human-aligned than criteria from simply prompting an LLM.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The framework produces implicit, less obvious, and actionable criteria that capture what humans think is important across writing tasks.

    Source: §8 · p. 9

    Web-grounded retrieval constrains criteria generation to the quality and scope of online information, which may be scarce or difficult to retrieve in niche domains.

    Source: §8 · p. 10

Abstract

from arXiv · show

Evaluation of language model outputs on structured writing tasks is typically conducted with a number of desirable criteria presented to human evaluators or large language models (LLMs). For instance, on a prompt like "Help me draft an academic talk on coffee intake vs research productivity", a model response may be evaluated for criteria like accuracy and coherence. However, high-quality responses should do more than just satisfy basic task requirements. An effective response to this query should include quintessential features of an academic talk, such as a compelling opening, clear research questions, and a takeaway. To help identify these implicit criteria, we introduce EvalAgent, a novel framework designed to automatically uncover nuanced and task-specific criteria. EvalAgent first mines expert-authored online guidance. It then uses this evidence to propose diverse, long-tail evaluation criteria that are grounded in reliable external sources. Our experiments demonstrate that the grounded criteria produced by EvalAgent are often implicit (not directly stated in the user's prompt), yet specific (high degree of lexical precision). Further, EvalAgent criteria are often not satisfied by initial responses but they are actionable, such that responses can be refined to satisfy them. Finally, we show that combining LLM-generated and EvalAgent criteria uncovers more human-valued criteria than using LLMs alone.

1 Introduction

Open-ended writing tasks make LLM outputs difficult to evaluate at scale, while human evaluation is subjective, expertise-intensive, and hard to scale. EVALAGENT grounds task-specific criteria in web-based expert advice, and its criteria are assessed for specificity, implicitness, and actionability.

1 Introduction

  • Open-ended LLM writing evaluation lacks robust automated methods, while human evaluation is subjective, requires expertise, and becomes infeasible at scale.Reported human-evaluation issues include lack of creativity, concept repetition, and logical inconsistencies.Source: §1 · p. 1
  • The framework addresses the challenge of identifying implicit criteria, which should reflect nuanced writing standards, domain knowledge, and appropriate guidelines.The paper distinguishes this from the separate challenge of ensuring evaluators can reliably interpret criteria.Source: §1 · p. 2
  • EVALAGENT generates task-specific evaluation criteria by searching for instructional web documents, extracting and summarizing their advice, then synthesizing it into criteria.Its three components are a Query Generator, Expert Retriever, and Criteria Generator.Source: §1 · p. 2
  • Across nine writing-task datasets, EVALAGENT criteria are evaluated for specificity, implicitness, and actionability, and are reported as highly actionable and more human-aligned than criteria from simply prompting an LLM.The paper also characterizes its criteria as more implicit, less obvious, and more actionable than LLM-generated criteria.Source: §1 · p. 2

2 Task Setting

The task is to identify criteria that are both specific to a prompt and desirable despite not being stated in it. Such criteria can inform evaluation and other response-related applications.

2 Task Setting

  • An implicit criterion is a prompt-unstated property that is still specific to the prompt and desirable for the response.The paper distinguishes implicit criteria from explicit criteria, including unstated but extremely obvious requirements.Source: §2 · p. 2
  • EVALAGENT focuses on implicit criteria that are specific, actionable, and closely aligned with human-authored criteria.Earlier work explored explicit criteria, while this work focuses on implicit ones.Source: §2 · p. 3
  • Generated criteria can be used in scoring functions that produce real-valued scores, binary judgments, or other formats, and their judgments can support pass rates, response refinement, or training rewards.The paper positions its criteria set as a shared starting point for applications such as refinement and evaluation.Source: §2 · p. 3
  • The work identifies criteria as a shared starting point across applications and leaves exploration of downstream applications to future work.The envisioned framework is flexible, but downstream application studies are outside this work's scope.Source: §2 · p. 3

3 EVALAGENT

EVALAGENT turns a prompt into an ordered set of evaluation criteria by generating targeted searches, retrieving and summarizing instructional advice, and synthesizing query-specific results. The framework can also combine web-derived criteria with criteria from an LLM baseline.

3 EVALAGENT

  • The Query Generator creates conceptual searches for instructional advice, including related concepts not directly stated in the prompt, rather than searching only the prompt itself.For an academic-talk prompt, examples include queries about key components and how to write an engaging talk.Source: §3 · p. 3
  • The Expert Retriever filters URLs for expertise and relevance, selects the top five, then uses an LLM to answer each query from URL content and summarize the answers into query-specific criteria.The summary step focuses on information relevant to the query and instruction.Source: §3 · pp. 3–4, Figure 2 · p. 4
  • The Criteria Generator aggregates common and unique properties across query-specific summaries, then rewrites criteria to align them with evaluation of the instruction.The rewriting step makes generic instructional advice specific to the user prompt.Source: §3 · p. 4, Figure 3 · p. 4
  • A ranking function orders criteria by relevance to the prompt, placing more relevant criteria first for applications that use only a subset.The method uses an LLM to score relevance.Source: §3 · p. 4
  • Web-derived criteria may not be exhaustive, particularly for criteria arising directly from the instruction, so the framework can merge them with other criteria such as explicit instruction-derived criteria.The merged list is ranked.Source: §3 · p. 4
  • EA-Web uses only web-derived criteria, whereas EA-Full combines them with an LLM-n baseline and reranks the combined criteria for prompt relevance.The framework's components use gpt-4o-mini-2024-07-18.Source: §3 · p. 4

4 Metrics for Assessing Generated Criteria

The paper introduces metrics for assessing generated criteria, using automatic measures for criterion properties and human judgments for perceived utility and obviousness.

4 Metrics for Assessing Generated Criteria

  • The paper introduces metrics to assess generated criteria, addressing a comparison that prior research had not systematically evaluated.The proposed properties include specificity, implicitness, and actionability.Source: §4 · p. 5

4.1 Automatic Metrics

  • Automatic metrics measure specificity by maximum normalized inverse word frequency, implicitness by one minus word overlap with the instruction, actionability by successful response revision, and recall by whether human criteria are entailed.Specificity and implicitness are aggregated across criteria; actionability is tested on 200 instructions, and recall uses human criteria as a proxy rather than a gold standard.Source: §4.1 · p. 5

4.2 Human Evaluation

  • Human evaluation rates each criterion's utility on a 1–3 scale and judges whether it is obviously implied by the instruction.Both measures are reported as averages across the human annotation set.Source: §4.2 · p. 5

5 Experimental Setup

The experiments use filtered long-form writing and planning instructions, add a dataset of user-issued instructions and critiques, and compare instruction-decomposition and LLM-prompted criteria-generation baselines.

5.1 Dataset Curation

  • The evaluation uses English, single-turn creative-writing and planning instructions requiring procedural, methodical, or creative long-form outputs, filtering examples from eight datasets.The dataset list includes BigGenBench, ChatBotArena, Art or Artifice, Dolomites, WritingBench, InfoBench, WildBench, and MT-Bench.Source: §5.1 · p. 6, Table 1 · p. 6
  • Ask-then-Critique adds user-issued instructions, model responses, and user-written critiques for responses to those users’ own instructions.Responses came from ChatGPT, Gemini, or LLaMa; other datasets’ human criteria were not provided by the instruction issuer.Source: §5.1 · p. 6
  • Dataset-specific evaluation criteria vary in quality and do not cover every evaluation property of an instruction.The criteria may be expert- or human-written, or LLM-generated and human-verified.Source: §5.1 · p. 6

5.2 Baselines

  • Instruction Decomposition derives criteria from the instruction, while LLM-Prompted asks an LLM for evaluation criteria and LLM-n overgenerates then ranks them.LLM-n is described as particularly effective when combined with EVALAGENT criteria.Source: §5.2 · p. 6

6 Results

EA-Full retrieves human-written criteria at a higher rate than the LLM baselines, while EA-Web criteria show greater specificity and implicitness and are actionable, though some valid criteria are out of scope for the models.

6.1 Properties of generated criteria

  • EA-Web produces more criteria with higher specificity and less instruction overlap than LLM-n, indicating greater implicitness.Human-written and human-verified criteria have flatter specificity and implicitness score distributions.Source: §6.1 · p. 7, Figure 4 · p. 8
  • Human annotators rate EVALAGENT criteria as low in obviousness while they maintain high utility; LLM-prompted criteria are mostly obvious and slightly lower in utility.Instruction Decomposition has the highest utility, as its criteria are decomposed constraints from the instruction.Source: §6.1 · p. 7
  • EA-Web achieves the highest actionability score through the largest post-editing pass-rate improvement and also has the lowest initial pass rates among systems.Lower pass rates are correlated with criteria rated less obvious by humans.Source: §6.1 · p. 7, Table 3 · p. 7

6.2 Alignment with human criteria

  • EA-Full recalls human criteria at a higher rate than LLM-60 and LLM-30+30, with gains after k = 10 on WritingBench and Ask-then-Critique and more pronounced gains on Art or Artifice.The reported recall comparison covers three datasets; the Art or Artifice trend indicates earlier capture of nuanced creative-writing criteria.Source: Table 4 · p. 8, §6.2 · pp. 8–9, Figure 5 · p. 9

6.1 Properties of generated criteria

  • Some EA-Web criteria are valid but out of scope, such as requesting charts for financial analysis when the evaluated models cannot generate charts.Source: §6.1 · p. 8

6.2 Alignment with human criteria

  • EA-Full matches human-written constraints even when specificity differs, including by recovering a broader completeness-and-detail constraint from several narrower criteria.The cited example connects outdoor activities, local cultural activities, and descriptive language to a broader constraint.Source: §6.2 · p. 9

7 Related Work

Related work covers criteria-generation benchmarks and LLM evaluators; EvalAgent focuses on implicit, grounded criteria that can complement evaluation systems.

Generating Criteria

  • Existing criteria benchmarks range from verifiable instruction constraints and instruction decomposition to LLM-generated, human-verified checklists and preference-derived criteria.EvalAgent instead centers on implicit, grounded criteria adaptable across feedback protocols and downstream applications.Source: §7 · p. 9

LLM-based Evaluators

  • LLM evaluators often use predecided or general overall-quality criteria, while EvalAgent generates criteria intended for integration with these systems.Source: §7 · p. 9

8 Conclusion

EvalAgent generates implicit, grounded criteria by searching instructional web documents, and its evaluation finds that the criteria capture what humans consider important. Its scope is bounded by online source availability, retrieved authors’ perspectives, and up-front retrieval and generation costs.

8 Conclusion

  • EvalAgent generates implicit, grounded criteria by searching the web for instructional documents; its evaluation finds these criteria capture what humans think is important.The framework is presented as a path toward scalable evaluation across writing tasks.Source: §8 · p. 9
  • The criteria available to EvalAgent are constrained by the quality and scope of online information, which can be scarce or difficult to retrieve for niche domains.In the nanoparticle example, searches returned scientific PDFs the system could not process or content unrelated to writing procedures.Source: §8 · p. 10
  • Web guidance centers retrieved authors’ perspectives; this study’s authors wrote in English, and their views may differ from those of other experts.The authors did not observe this as a first-order problem in their analysis and did not investigate it further.Source: §8 · p. 10
  • Retrieval and multi-stage prompting add latency and computational cost, but both retrieval and criteria generation can be precomputed and do not affect benchmark runtime.The authors expect overall system cost to decrease as capable-model inference becomes cheaper.Source: §8 · p. 10

A.1 Dataset Descriptions

  • The dataset descriptions cover benchmarks with human-written criteria, converted procedures, and filtered writing-task subsets, including BigGenBench and ChatbotArena.BigGenBench uses selected task categories and the provided rating-5 Likert description; ChatbotArena is filtered to first-turn English prompts tagged for creativity and real-world use and containing specified writing keywords.Source: §A.1 · pp. 13–14

A.2 Ask-then-Critique

  • The Ask-then-Critique study sought evaluations from instruction issuers; its final dataset contains 156 responses, 312 instructions, and 624 evaluations, with critiques rewritten as criteria.The study was added as an opt-in assignment in a graduate-level NLP class.Source: §A.2 · p. 14

A.3 Task categories

The appendix describes the breadth of task categories and the prompts and processing steps used to generate criteria from web guidance. It also reports that URL filtering accounts for most of EvalAgent’s computational cost.

A.3 Task categories

  • Datasets span high-level writing categories, including creative writing, documentation, communication, technical writing, and analysis and research.The categories were formed by clustering outputs from a prompted writing-category classifier.Source: §A.3 · p. 15

B EvalAgent Parameters and Prompts

  • EvalAgent decomposes each instruction into 2–3 actionable web-search queries, retrieves URLs for each query, and selects the top five using relevance and expertise ratings.The system retrieves 30 URLs per query; selected URLs averaged 3.77 for URL rating and 1.48 for relevance.Source: B EvalAgent Parameters and Prompts · p. 17
  • For each selected URL, the system answers the query from the document, combines advice across URLs and queries, rewrites it as concise yes/no criteria, filters irrelevant criteria, and ranks the remainder by relevance.The prompts ask for precise, instruction-aligned criteria and remove criteria that do not apply to a text-only response.Source: B EvalAgent Parameters and Prompts · pp. 17–19

B.1 URL Filtering

  • URL filtering rates content for expertise and reliability, then rates its relevance to the instruction and whether it offers useful writing advice.The content-quality scale describes expertise, reliability, clarity, and marketing; relevance is scored from 0 to 2.Source: §B.1 · pp. 19–20

C Computational Cost

  • URL filtering accounts for the bulk of EvalAgent’s computational cost.Table 6 reports tokens and associated costs for the system’s steps.Source: C Computational Cost · p. 20

D Baseline Methods

The baseline prompts generate evaluation checklists either by decomposing explicit instruction constraints or by asking for broader criteria. Both approaches specify constraints for concise, relevant, and precise checklist items.

D Baseline Methods

  • Instruction Decomposition asks an LLM to convert only explicitly stated instruction constraints into a numbered checklist.Each item is phrased as what the response should do.Source: D Baseline Methods · p. 20
  • The broader LLM-prompted baseline asks for relevant checklist questions that may cover either explicit or generally sensible criteria.The prompt asks for yes/no answerability, comprehensiveness without unnecessary entries, and precise evaluation of response aspects.Source: D Baseline Methods · pp. 20–21
  • A variant of the broader checklist prompt requests 30 factors, each starting with “the response should.”It retains the yes/no, concise, comprehensive, and precise criteria in a single prompt.Source: D Baseline Methods · p. 21

E Human Evaluation

Human annotators judged criteria for utility and obviousness, with agreement expected to vary because both judgments are subjective. Across methods and datasets, EvalAgent’s criteria have comparable utility and are consistently less obvious while retaining high utility.

E Human Evaluation

  • Annotators judged criteria for obviousness and relevance; relevance ranged from 1 (not relevant) to 3 (highly relevant).The study used two Upwork annotators and one annotator with a linguistics bachelor’s degree; they judged 700 criteria across 54 instructions.Source: E Human Evaluation · pp. 21–22

E.1 Annotator Agreement

  • Agreement was measured with Fleiss Kappa, and the authors expected lower agreement for implicit criteria because obviousness and utility can be subjective across annotators.Reported metric averages use majority vote.Source: §E.1 · p. 22

E.2 Average Utility and Obviousness Scores

  • EvalAgent consistently produces less obvious criteria with high utility, while utility is comparable across methods in Figure 8.Figure 7 reports average utility and obviousness scores; Figure 8 breaks average utility down by method and dataset.Source: §E.2 · p. 22, Figure 8 · p. 23, Figure 7 · p. 23

F.1 Lexical metrics: Specificity and Word Overlap

The section measures criterion specificity using a criterion’s most distinctive word and implicitness using its word overlap with the prompt. It also examines specificity relative to each method’s own pool of criteria.

F.1 Lexical metrics: Specificity and Word Overlap

  • Specificity is the maximum normalized inverse word frequency among a criterion’s words, with word frequency measured across the shared corpus of generated criteria.The section reports specificity scores for all datasets in Table 9.Source: §F.1 · pp. 23–24
  • Within its own criteria pool, ID tends to be most specific; natural-distribution LLM criteria are more specific than overgenerated LLM-n criteria.The passage attributes the latter pattern to overgeneration adding criteria shared across instructions.Source: §F.1 · p. 24
  • Implicitness is calculated as one minus the word overlap between a criterion and its prompt, using sets of non-stopword tokens.Implicitness scores are reported for all datasets in Table 10.Source: §F.1 · p. 24

F.2 Actionability

EvalAgent criteria were less obvious and less likely to be initially satisfied, while the recall analysis found a significant gap between the ranking method and oracle performance. The appendix also describes how actionability and recall were assessed.

F.2 Actionability

  • Actionability compares binary criterion satisfaction before editing with satisfaction after the response is edited to meet that criterion.The change is the post-edit satisfaction judgment minus the initial judgment.Source: §F.2 · pp. 24–25
  • The actionability experiment sampled 200 instructions, excluded ChatBotArena and MT-Bench, and assessed three models, using the same model for initial responses and editing.The three models were GPT-4o-2024-11-20, Claude-3-5-sonnet-20241022, and Llama-3.1-8B-Instruct.Source: §F.2 · p. 25
  • EvalAgent consistently generated criteria that were less obvious and less likely to be satisfied initially, indicating challenging and useful criteria.The analysis used a human-annotated subset, with initial responses from GPT-4o-2024-11-20 and obviousness judgments from human evaluators.Source: §F.2 · p. 25

F.3 Recall

  • The recall analysis checks whether human-generated or verified criteria are represented among generated criteria, then calculates Recall@k across instances.The matching prompt permits generalizing either criterion to assess whether one is entailed by the other.Source: §F.3 · pp. 25–26
  • The recall analysis found a significant gap between the ranking method and oracle performance, leaving ranking improvements for future work.Figure 12 reports oracle Recall@k results across three datasets.Source: §F.3 · p. 26, Figure 12 · p. 26
Loading 2504.15219v2…