PaperlayerView exact revision on arXiv

Source-linked AI summary

Edit Discovery in Human-AI Interactions

Alireza Mirrokni, Shayan Kiyani, George Pappas, Hamed Hassani

arXiv:2610.22661v1cs.LG

TL;DR

Human–AI interactions can contain correctable failures, but identifying them requires editor review that consumes limited resources. The paper formulates edit discovery as constrained selection and develops EDIT, an online method for learning from selected interactions. Across four reasoning and factual question-answering benchmarks, EDIT-REP reduces missed edits relative to direct LLM-based batch selection while maintaining reliable Type-I behavior and feasible editor cost.

  • Problem

    Human–AI interactions include both satisfactory responses and potentially useful failures, while reviewing every interaction can consume substantial editor resources.

    Source: §1 · pp. 1–2

  • Method

    The paper formulates Edit Discovery Optimization and develops EDIT, which uses an adaptive edit score, confirmation-control threshold, and cost-aware selection rule under partial feedback.

    Source: §1 · p. 2, §3.1 · p. 5

  • Results

    Across four reasoning and factual question-answering benchmarks, EDIT-REP reduces missed edits relative to direct LLM-based batch selection while maintaining reliable Type-I behavior and feasible editor cost.

    Source: §6 · p. 10

  • Takeaways & Limitations

    The results indicate the value of separating edit scoring, online control, and budget allocation in edit discovery.

    Source: §6 · p. 10

    Evaluation uses benchmark-derived edit labels with a limited 200-example human audit and tests edit discovery rather than downstream model improvements from incorporating corrections.

    Source: §6 · pp. 10–11

Abstract

from arXiv · show

Human-AI interactions are becoming a rich source of data for improving AI systems: while many model responses are already satisfactory, others expose failures that could become valuable supervision if identified and corrected. We study this setting through the notion of an editor: a more capable evaluation process that can inspect a response and either confirm it or modify it, but whose invocation is costly. The editor may be a human expert, a stronger AI system, or the same model equipped with additional computation, context, tools, or information. The key challenge is to decide which interactions to send to the editor before knowing whether an edit will actually be made, since invoking the editor itself consumes limited resources such as expert time, inference cost, latency, or tool-use overhead. We formulate this problem as Edit Discovery Optimization (EDO), which seeks to minimize edit opportunities missed among skipped interactions while controlling unnecessary confirmations and respecting an editor-cost budget. We characterize the population-optimal policy and show that it takes the form of a cost-adjusted threshold on the posterior probability of an edit. Building on this structure, we develop EDIT (Edit Discovery via Iterative Thresholding), an online algorithm for the partial-feedback setting in which edit labels are revealed only for selected interactions. EDIT combines an adaptive edit score, a confirmation-control threshold, and a cost-aware selection rule; it satisfies the editor budget at every round and controls the average confirmation rate up to a vanishing finite-horizon term, without distributional assumptions on the interaction stream. Across four reasoning and factual question-answering benchmarks, EDIT achieves a better trade-off between editor cost, Type-I error, and Type-II error than direct LLM-based batch selection.

1 Introduction

Human–AI interactions contain both satisfactory responses and failures that could provide corrective supervision, but reviewing them all is costly. The paper formulates selective review as Edit Discovery Optimization and develops EDIT to choose interactions under editor-cost and confirmation constraints.

1 Introduction

  • Human–AI interaction responses vary in usefulness: some are already satisfactory, while others expose mistakes, ambiguity, incompleteness, or misalignment that could become valuable supervision if corrected.Traditional sources of high-quality public data are increasingly saturated, and obtaining comparable new data is becoming more difficult.Source: §1 · p. 1
  • The central challenge is to find interactions likely to benefit from correction without spending scarce editor resources on responses that would simply be confirmed.Editor costs can include expert time, stronger-model inference, latency, and tool-use overhead.Source: §1 · p. 2
  • EDIT combines an adaptive edit score, an adaptive confirmation-control threshold, and a cost-aware selection rule for online selection with only partial feedback.The method is designed to satisfy the editor budget at every round and control average per-round Type-I error up to a finite-horizon term that vanishes with the number of rounds.Source: §1 · p. 2
  • Edit Discovery Optimization formalizes selection under limited editor resources by distinguishing confirmations among selected interactions from missed edits among skipped interactions and accounting for interaction-specific costs.Its population-optimal policy selects when posterior edit probability exceeds a cost-adjusted threshold.Source: §1 · p. 2
  • Across four reasoning and factual question-answering benchmarks, EDIT’s informative edit scores, particularly its learned representation score, reduce missed edits relative to direct LLM-based batch selection while maintaining reliable Type-I behavior and feasible editor cost.The introduction also reports experiments with Llama-3.3-70B-Instruct, with additional experiments using Qwen3-32B, response-length editor costs, and structured distribution shifts.Source: §1 · pp. 2–3

2 Related Work

The related work connects edit discovery to selective evaluation, human–AI collaboration, and calibrated selection or online control. These approaches allocate human or computational effort to improve decision quality or regulate selection under uncertainty.

Efficient evaluation and selective supervision

Human–AI collaborative procedures

  • Human–AI collaborative procedures narrow, prune, augment, or selectively acquire support to improve decision quality and recover overlooked answers.Examples include conformal prediction sets [Straitouri et al., 2023], collaborative uncertainty quantification [Noorani et al., 2025, 2026], and selective verification [Kiyani et al., 2026b].Source: §2 · p. 3

Calibrated selection and online control

  • Calibrated selection and online control use calibrated criteria or sequential feedback to regulate which instances or outputs are selected and which risks are controlled.The cited examples include conformal selection [Jin and Cand`es, 2023], Conformal Alignment [Gui et al., 2024], adaptive conformal inference [Gibbs and Candes, 2021], and conformal risk control.Source: §2 · pp. 3–4

3 Fundamentals of Edit Discovery

Edit discovery selects interactions to find likely corrections while limiting unnecessary confirmations and editor cost. The population analysis yields a cost-adjusted edit-probability threshold, which motivates an online policy that learns from feedback available only on selected interactions.

3 Fundamentals of Edit Discovery

  • Edit discovery treats editor selection as a trade-off between spending resources on responses that may be confirmed and skipping failures that would have yielded useful corrections.The formulation also allows interaction-specific editor costs, which may represent time, expense, latency, expertise, or other resources.Source: §3 · p. 4, §3.1 · pp. 4–5

3.1 Problem Formulation

  • The policy selects each prompt–response interaction with a probability, while the objective is to minimize missed edits among skipped items subject to controlling confirmations among selected items.The editor’s validated answer either modifies or confirms the AI response; edits can yield corrective data, while confirmations indicate acceptance without modification.Source: §3.1 · pp. 4–5

Edit Discovery Optimization (EDO)

  • EDO constrains the confirmation rate among selected interactions and total population editor cost, with the goal of minimizing Type-II error.The population analysis assumes the joint distribution of interactions and edit outcomes is known; the online method instead uses sequentially observed feedback.Source: Edit Discovery Optimization (EDO) · p. 5, §3.2 · p. 6

3.2 Optimal Policy in the Population Regime

  • The population-optimal policy selects when posterior edit probability exceeds a global threshold plus a cost-dependent adjustment, with possible randomization at the boundary.The online controller need not reconstruct the full joint distribution: it uses an adaptive edit score, updates the global threshold from observed feedback, and chooses a cost multiplier to enforce the budget.Source: §3.2 · p. 6

4 Online Edit Discovery

EDIT brings Edit Discovery Optimization into an online setting where edit labels are revealed only for selected interactions. It combines adaptive edit scoring and confirmation control with cost-aware selection, guaranteeing the per-round editor budget and controlling average Type-I error up to a vanishing finite-horizon term.

4 Online Edit Discovery

  • EDIT addresses online EDO when the interaction distribution and edit probabilities are unknown and labels become available only for selected responses.The online policy operationalizes EDO under partial feedback without assumptions about the incoming stream.Source: §4 · p. 6, Online interaction protocol · p. 6

Online interaction protocol

  • Each round brings a batch with predictable costs and hidden edit labels; only selected interactions reveal labels, which can update the policy before the next batch.Skipped interactions' Type-II error remains counterfactual in deployment, while selected labels and known costs support direct control of Type-I error and budget.Source: Online interaction protocol · pp. 6–7

4.1 EDIT Policy

  • EDIT replaces the population edit probability and threshold with an adaptive score and confirmation-control threshold, then uses a batchwise cost multiplier to meet the budget.The multiplier is zero when the unadjusted rule is feasible and otherwise increases only enough to satisfy the current budget.Source: Online interaction protocol · p. 7, §4.1 · pp. 7–8
  • When the score is learnable, EDIT updates its parameters by gradient descent on selected-example cross-entropy using revealed labels.The guarantee is agnostic to how the edit score is constructed.Source: §4.1 · p. 8
  • EDIT enforces the editor budget each round and controls average per-round Type-I error up to a term vanishing with T, without distributional assumptions on the stream.The guarantee applies to dependent, non-identically distributed, or nonstationary batches.Source: §4.1 · p. 8

5 Experiments

Experiments compare EDIT with LLM-based and score-variant baselines across four reasoning and factual question-answering benchmarks. EDIT-REP generally achieves lower Type-II error than LLM-Select while maintaining comparable or more stable Type-I behavior, and has the lowest final Type-II error on all four datasets.

5.1 Experimental Settings

  • The evaluation measures Type-I error, Type-II error, and editor cost on four benchmarks; primary runs use 250 rounds, batches of 10, unit costs, and a budget of 5.The benchmarks are MATH-500, MMLU-Pro, PopQA, and GPQA.Source: §5 · p. 8, §5.1 · p. 9
  • EDIT-REP maps each prompt–response pair to a frozen representation and learns a logistic linear probe; alternative scores use random values or LLM edit-probability estimates.Only the probe parameters are learned for EDIT-REP.Source: §5.1 · pp. 8–9
  • LLM-Select jointly ranks a full indexed batch given item costs and budget, while EDIT-RAND and EDIT-LLM provide training-free score variants.EDIT-LLM estimates the probability of a human edit from the prompt–response pair without a reference answer.Source: §5.1 · pp. 8–9

5.2 Results

  • Across four datasets, EDIT-REP generally has lower Type-II error than LLM-Select with comparable or more stable Type-I behavior, and achieves the lowest final Type-II error on each.Under the prescribed Type-I tolerance and budget, EDIT-REP is consistently strongest.Source: §5.2 · pp. 9–10, Figure 2 · p. 10
  • The main EDIT-REP score uses a 22.7M-parameter MiniLM encoder, while 4B Qwen representations achieve the lowest final Type-II error on all four datasets.The MiniLM encoder is over three orders of magnitude smaller than the 70B response model.Source: §5.1 · p. 9, §5.2 · p. 10

6 Conclusion and Limitations

EDIT combines edit scoring, adaptive confirmation control, and per-round budget enforcement, with a cost-adjusted threshold characterizing population-optimal selection. The conclusion reports improved edit discovery against direct LLM batch selection, while noting limits in human-edit and downstream-impact evaluation.

6 Conclusion and Limitations

  • Across four reasoning and factual QA benchmarks, EDIT-REP reduces missed edits versus direct LLM-based batch selection while maintaining reliable Type-I behavior.The paper also reports that score quality matters and that stronger encoders can further improve edit discovery.Source: §6 · p. 10
  • Experiments use benchmark-derived edit labels with a limited 200-example human audit, not large-scale human editing, and do not measure downstream model improvements from corrections.Future work includes richer workflows, heterogeneous costs, and realistic deployment settings with diverse users, domains, editor expertise, and resource constraints.Source: §6 · pp. 10–11

A Extended Related Work

  • Unlike aggregate evaluation and AI-assisted labeling, EDO targets individual interactions for correction, leaves skipped responses unlabeled, and learns from feedback only on selected interactions.Its objective also differs from selective prediction and deferral, which focus on prediction accuracy or utility rather than edit discovery and partial feedback.Source: A Extended Related Work · pp. 15–16
  • Conformal selection chooses outputs to accept or trust, whereas EDO sends likely-to-need-correction outputs for editing and couples its guarantee to per-round cost constraints.The paper notes that related methods generally require labeled calibration data or exchangeability-type conditions; its guarantee applies to arbitrary streams with labels revealed after editor feedback.Source: A Extended Related Work · p. 15
  • Human–AI collaboration work studies selective assistance and combined judgments, while EDO focuses on discovering editor modifications under confirmation and interaction-specific review-cost constraints.The related work includes prediction-set assistance, AI refinement of human-proposed sets, and policies for strong verification or beneficial support.Source: A Extended Related Work · p. 16

B.1 Proof of Theorem 1

  • The population-optimal policy selects when edit probability exceeds a cost-adjusted threshold, with possible randomization at the threshold boundary.This structure motivates EDIT’s decomposition into edit scoring, adaptive Type-I control, and budget enforcement.Source: §6 · p. 10, §B.1 · p. 18

B.2 Proof of Theorem 2

The proof establishes the online method’s budget and confirmation-control guarantees. The accompanying pseudocode gives its threshold selection rule, and the appendix tables document experiment tolerances, response-model accuracy, and shared hyperparameters.

B.2 Proof of Theorem 2

  • The proof verifies the budget constraint from the definition of βt and bounds αt by 1 + γ throughout the rounds.The proof’s stated goal is also to establish the average per-round Type-I constraint guarantee.Source: §B.2 · pp. 18–19

C EDIT Pseudocode

  • EDIT’s pseudocode selects an item when its estimated edit score exceeds αt + βt times its editor cost.The algorithm is identified as EDIT: Edit Discovery via Iterative Thresholding.Source: C EDIT Pseudocode · p. 19

D.1 Additional Tables and Figures

  • The appendix tables report model–dataset Type-I tolerances, benchmark response-model accuracies, and hyperparameters shared across the main online experiments.The same Type-I tolerance is used across EDIT score variants.Source: §D.1 · p. 19, Table 1 · p. 20, Table 2 · p. 20

D.2 Human Audit of Benchmark-Derived Edit Labels

The audit compares benchmark-derived edit labels with human judgments on 200 held-out responses, using GPT-5.6 Sol as an independent consistency check. Benchmark labels closely track human judgments across the aggregate and individual datasets.

D.2 Human Audit of Benchmark-Derived Edit Labels

  • The audit covered 200 held-out prompt–response pairs, 50 from each of MATH-500, MMLU-Pro, PopQA, and GPQA.A human annotator was the primary validation target, and GPT-5.6 Sol independently labeled the same examples as an additional consistency check.Source: §D.2 · p. 20
  • 96.5% benchmark–human agreement, κ = 0.930, and edit-class F1 = 0.963 support benchmark correctness as an edit indicator.The benchmark proxy disagreed with the human annotator on 7 of 200 interactions; dataset-level agreement ranged from 0.960 to 0.980, with κ > 0.916 for every dataset.Source: §D.2 · p. 20
  • GPT-5.6 Sol’s edit rate was 0.460, with agreement of 0.980 (κ = 0.960) against benchmark labels and 0.975 (κ = 0.950) against human labels.The benchmark and human edit rates were 0.470 and 0.465, respectively.Source: §D.2 · p. 20

D.3 Operating-Point Trade-offs and Complementary Metrics.

The section examines how operating-point choices affect EDIT-REP’s discovery and confirmation trade-off, then reports complementary discovery metrics and describes the prompts used in the experiments and audit.

D.3 Operating-Point Trade-offs and Complementary Metrics.

  • The operating-point sweep varies ε across 0.1–0.6 and per-round budget B across 2, 3, 4, 5, and 7, averaging five realized streams with 95% confidence intervals.The LLM-Select marker is its main B = 5 run and serves as a reference operating point.Source: §D.3 · p. 20
  • Across four datasets, broader selection reduces the edit rate among skipped interactions while increasing the confirmation rate among selected interactions.The relative advantage of EDIT-REP ranking is not tied to one ε or B; increasing the available budget generally increases edit recall, while ε determines how aggressively the budget is used.Source: §D.3 · pp. 20–21
  • EDIT-REP attains the highest edit recall and fewest missed edits on most dataset–model pairs, while EDIT-LLM achieves the strongest edit precision in several settings.Table 8 reports edit recall, missed-edit counts, and edit precision on the existing Llama-3.3-70B-Instruct and Qwen3-32B runs.Source: §D.3 · p. 21

D.4 List of LLM Prompts

  • Four prompt families generate candidate answers, select a budgeted batch, score per-pair edit probability, and independently evaluate benchmark-derived labels.The LLM-Select prompt uses question/problem and model answer, listed costs, and the total budget; it does not use the gold answer. The edit-scoring and audit prompts request binary/probability judgments, with the audit criterion grounded in correctness, reasoning, completeness, instruction following, relevance, and clarity.Source: §D.4 · p. 22, Common prompt · pp. 25–27

D.4.1 Candidate-Response Generation Prompts

  • The candidate-response generation prompt requires mathematical answers to end with a final line containing only #### followed by the answer.The user prompt supplies the math problem and repeats the required final-line format.Source: §D.4.1 · p. 22

D.5 Additional Experiments with Qwen3-32B

With Qwen3-32B, the main qualitative edit-discovery trends persist. A response-length-dependent cost analysis shows that costs vary by interaction and that EDIT remains effective under this cost structure.

D.5 Additional Experiments with Qwen3-32B

  • With Qwen3-32B, EDIT-REP generally has lower Type-II error than LLM-Select across four datasets, with clear improvements on MMLU-Pro and GPQA.EDIT-REP and EDIT-LLM substantially outperform EDIT-RAND on MATH-500, MMLU-Pro, and PopQA, while their relative ordering varies by dataset.Source: §D.5 · p. 28

D.6 Response-Length-Aware Editor Costs

  • On MATH-500, normalized response-length editor costs have a pronounced right tail from some long derivations; PopQA costs are more concentrated around unit cost, with fewer long-form outliers.Unit cost corresponds to the median response length within the relevant generation, and costs are clipped to [0.25, 4.0].Source: §D.6 · p. 29, Figure 7 · p. 29
  • Under response-length-dependent costs, both informed EDIT variants substantially reduce Type-II error versus random scores on MATH-500, and EDIT-REP has the strongest late-stage GPQA discovery while staying below the Type-I tolerance.LLM-Select often spends less but misses more edits; the analysis attributes a consequential role to accounting for editor burden when directing resources.Source: §D.6 · p. 29

D.7 Additional Experiments under Distribution Shift

The experiments evaluate nonstationary streams that move across subject groups or entity-popularity tiers without resetting the policy. EDIT-REP generally has lower Type-II error than LLM-Select while regulating Type-I error more reliably across phase changes.

D.7 Additional Experiments under Distribution Shift

  • Across phase changes, EDIT-REP generally attains lower Type-II error than LLM-Select while regulating Type-I error more reliably.The streams shift through STEM, life/health, humanities/social science, and law/other for MMLU-Pro, and from head to middle-popularity to long-tail entities for PopQA; all methods receive the same phases and shift boundaries.Source: §D.7 · p. 29
  • After shifts, informed EDIT scores remain separated from EDIT-RAND, while stable Type-I trajectories across score variants distinguish score ranking from threshold control.The passage interprets scores as determining which interactions receive review and the controller as determining selectivity.Source: §D.7 · p. 29

D.8 Effect of Embedding-Model Size

The experiment varies only the frozen representation model used by EDIT-REP, comparing three Qwen3 sizes while holding the response model and online setup fixed. Larger embeddings generally improve edit discovery, with Qwen3-4B lowest in final Type-II error across all four datasets.

D.8 Effect of Embedding-Model Size

  • The comparison replaces the default MiniLM encoder with three Qwen3 sizes while holding the response model, online probe, selection rule, tolerances, budget, stream order, and other hyperparameters fixed.Source: §D.8 · p. 30
  • Qwen3-1.7B consistently improves over Qwen3-0.6B and approaches Qwen3-4B on MATH-500 and MMLU-Pro.Source: §D.8 · p. 30
  • On MMLU-Pro and GPQA, all encoder sizes use essentially the same editor budget; on MATH-500 and PopQA, larger encoders still improve the Type-I–Type-II trade-off despite modestly different expenditure.Source: §D.8 · p. 30
  • Qwen3-4B achieves the lowest final Type-II error on all four datasets, with the clearest improvements on PopQA and GPQA.Source: §D.8 · p. 30, Figure 10 · p. 31
Loading 2610.22661v1…