PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond "AI Helps Humans": Decision-Targeted Evaluation Design for Human-Agent Teams in the Agentic Era

Hamed Khosravi, Xiaoming Huo

arXiv:2609.05527v1cs.AIcs.HC

TL;DR

The paper addresses how to decide whether a human–AI workflow beats both human-only and agent-only alternatives when those baseline outcomes are missing and costly to replay. TEAM-DESIGN allocates separate, task-specific replay probabilities using pilot estimates of prediction difficulty, decision difficulty, and replay cost, and provides validity guarantees for the resulting decision. It performs best when one comparison is clearly harder to settle, but can underperform variance-based allocation when the comparisons are similarly difficult.

  • Problem

    Deployment evaluation must determine whether the observed human–AI workflow beats both alternatives, even though replaying either missing baseline consumes expert time or compute.

  • Method

    TEAM-DESIGN learns baseline prediction difficulty and comparison gaps from a fully observed pilot, then assigns separate replay probabilities under a shared budget and evaluates both gains with selective-replay corrections.

  • Results

    TEAM-DESIGN has the lowest estimation error for the harder-baseline gain at every semi-synthetic budget below full replay and achieves 77.1% versus 65.1% for Neyman when one comparison is clearly harder.

  • Takeaways & Limitations

    Replay should target the comparison that is harder to establish, rather than treating the two missing baselines as equally difficult by default.

  • Takeaways & Limitations

    TEAM-DESIGN is not uniformly better: it needs a clearly harder comparison, and its first-order objective can misallocate replay when a noisy pilot places one comparison near its threshold.

Abstract

from arXiv · show

Wherever a coding agent works under engineer supervision, or a clinical model assists a radiologist, the deployment question is whether to keep the human-AI workflow or replace it with the human alone or the agent alone. The human-AI workflow is worth keeping only if it beats both of those alternatives. Yet once it is deployed, neither alternative outcome is observed: recovering one means replaying the task under that alternative, and every replay costs expert time or compute. Under a fixed replay budget, the design question is therefore which tasks should be more likely to receive a human-only replay, and which an agent-only replay. Existing methods do not directly target this decision. Agent benchmarks do not choose which missing baseline to measure, variance-based sampling ignores which of the two comparisons is closer to failing, and Bayesian information methods focus on learning model parameters instead of making the deployment decision. We propose TEAM-Design, a rule that gives every task two replay probabilities, one per baseline. It raises a probability where the missing baseline outcome is hard to predict from what is already known about the task and where that comparison is harder to establish, and lowers it where replay is expensive. We prove that the rule solves this budgeted design problem, and that drawing the replays at random from recorded probabilities still controls the chance of wrongly declaring that the workflow beats both. We reanalyze 6 clinical settings, where no human-AI workflow beats both alternatives, and a coding benchmark, where one does, then evaluate TEAM-Design on synthetic designs and on a semi-synthetic design built from a real chest X-ray reader study. TEAM-Design works best when one of the two comparisons is clearly harder to settle than the other, and can do worse than variance-based allocation when the two are similarly difficult.

1 Introduction

Deployment requires deciding whether to keep a human–AI workflow when the human-only and agent-only outcomes are unobserved and costly to recover. TEAM-DESIGN targets this decision by allocating separate replay probabilities across baselines and tasks.

  • The deployment question: The deployment decision is whether the human–AI workflow beats both the human-alone and agent-alone alternatives.The workflow is observed in production, but the alternative outcomes for the same tasks are not.
  • The missing baselines: Running both missing baselines on every task can exceed the evaluation’s value because replays consume expert time, model calls, compute, and tools.Human-only replays may require independent work by senior engineers or clinicians.
  • The proposed design: TEAM-DESIGN assigns each task separate human-only and agent-only replay probabilities rather than selecting one fixed subset of tasks.The two replay decisions are drawn separately, so a task may receive neither, one, or both replays.
  • The two settings: Clinical settings and coding provide contrasting evaluation contexts: clinical data support task-level comparisons, whereas some coding benchmarks report only separate overall scores.Separate aggregate scores do not provide three outcomes for each task, so selective replay cannot be tested on them.
  • Relation to prior work: The method targets a fixed-budget decision that existing agent benchmarks, variance-based allocation, and parameter-focused information methods do not directly address.Prior work provides related evaluation, sampling, and experimental-design components without choosing which missing baseline to measure on which task.

2 Problem formulation

The paper formulates deployment evaluation as estimating the workflow’s gains over two missing baselines under a shared replay budget. Both comparisons must clear their thresholds, so the harder comparison determines the evidence requirement.

  • Observed and missing outcomes: Each task has a workflow outcome T and two missing baseline outcomes, YH and YA, unless the corresponding baseline is replayed.Observed task information W can include tool calls, test failures, or runtime.
  • Deployment criterion: The workflow is retained only when its average gain over both baselines exceeds their required improvements.Setting δH = δA = 0 asks whether it beats both alternatives.
  • Replay design: A replay rule qj(W) gives baseline j a task-specific replay probability, with separate Bernoulli draws allowing neither, one, or both baselines to be replayed.The rules must respect minimum replay floors and a shared average budget B.
  • Variance under selective replay: Replay probability controls the variance of each gain estimate because baseline j is observed only when its replay occurs.Increasing qj(W) supplies more outcomes for comparison j and lowers its replay-dependent variance component.
  • Joint decision difficulty: The design objective prioritizes the comparison with greater difficulty, determined by noisy estimation and by how close its gain is to the required threshold.Improving an already easy comparison does not resolve the joint deployment decision.

3 TEAM-DESIGN

TEAM-DESIGN learns prediction difficulty, comparison gaps, and replay costs from a fully observed pilot, then assigns two budget-constrained replay rules. Its final estimates use selective-replay corrections and confidence bounds for both comparisons.

  • Pilot learning: A separate pilot runs all three configurations to estimate gains, baseline predictors, prediction difficulty, and variance components before evaluation replay is allocated.Pilot quantities plan the replay, while reported gains come from the separate evaluation pool.
  • Pilot learning: Prediction difficulty sj(W) is the expected squared error of the pilot predictor for baseline j at task information W.Larger difficulty means the baseline is harder to predict and replay reveals more information.
  • Evaluation and decision: Selective-replay scores remain unbiased because known replay probabilities inverse-weight observed corrections, although poor prediction increases variance.Averaging the scores yields estimated gains for both baselines without requiring every baseline outcome.
  • Replay allocation: TEAM-DESIGN combines variance reduction, stabilized distance from the comparison threshold, and replay cost to allocate each baseline’s probability across tasks.Replay rises where baseline outcomes are harder to predict, falls where replay costs more, and shifts toward the comparison needing more evidence.
  • Optimization: The square-root allocation with clipping is the exact solution of the constrained design problem, keeping probabilities within [qj,min, 1].The method searches over comparison weights and adjusts the budget multiplier until total expected replay equals B.
  • Evaluation and decision: The workflow is declared superior only when both one-sided lower confidence bounds clear their thresholds, and this joint requirement needs no multiplicity correction.The method records both replay probabilities and uses the evaluation pool to report the final decision.

4 Theoretical guarantees

TEAM-DESIGN is justified by three theoretical results: the harder comparison determines evaluation effort, the square-root replay rule optimally solves the budgeted design problem, and pilot-based random replay preserves false-claim control.

  • Evaluation effort: The harder of the two baseline comparisons determines the approximate number of evaluation tasks needed for joint testing power.The sample-size bound is derived by requiring each comparison to succeed with probability 1 − β/2, ensuring both succeed with probability at least 1 − β.
  • Evaluation effort: Minimizing the design objective minimizes the first-order sample-size requirement.
  • Optimal replay allocation: Optimal replay weights concentrate on comparisons attaining the maximum in the design objective, while both comparisons receive positive weight when the budget allows them to equalize.
  • Optimal replay allocation: The square-root replay rule is the exact solution to the cost-constrained replay problem, obtained from the convex program’s KKT conditions.Replacing unknown quantities with pilot estimates yields the TEAM-DESIGN rule in (10).
  • Valid inference: Pilot-based replay preserves asymptotic false-claim probability at most α when at least one true gain fails its threshold, conditionally and unconditionally.The guarantee requires an independent pilot, probabilities fixed before observing missing outcomes, positive replay-probability floors, and i.i.d. bounded evaluation outcomes.
  • Valid inference: The pilot affects efficiency rather than validity: a poor pilot widens intervals, but probability recording and selective-replay correction leave the error rate controlled.Because a false claim requires both bounds to clear while one comparison fails, the decision rule uses an intersection–union test without a multiplicity adjustment.

5 Results

The evaluation finds that human–AI workflows do not beat both alternatives in the six clinical settings, while TEAM-DESIGN improves decision rates when one comparison is substantially harder to establish. Its advantage reverses when comparisons are similarly difficult, and its real-structure validation shows lower error without guaranteeing more beats-both decisions.

  • 5.1 Do clinical human–AI workflows beat both alternatives?: Across 6 clinical settings and 8,723 reads, assisted clinicians beat clinicians alone but not models alone, so none beat both alternatives.The assisted clinician beat both alternatives in 0 of 6 settings.
  • 5.2 Does decision-targeted replay help when one comparison is harder?: TEAM-DESIGN concludes that the workflow beats both baselines in 77.1% of replications versus 65.1% for Neyman when one comparison is much closer to its threshold.It raises replay probabilities for the harder comparison and retains an advantage when replay costs differ five to one.
  • 5.3 Validation on real clinical structure: TEAM-DESIGN has the lowest root mean squared error for the harder-baseline gain below full replay, reaching 10.4 × 10^-3 versus 15.7 × 10^-3 for Neyman at one fifth of full replay.Lower error does not always increase the frequency with which both comparisons pass.
  • 5.4 Scope conditions: When the two comparisons are similarly difficult, TEAM-DESIGN trails Neyman, although a larger pilot narrows the gap.The method needs a clearly harder comparison and enough budget to shift measurements toward it.
  • 5.4 Scope conditions: A noisy pilot can allocate too much replay to a comparison near its threshold because the objective measures large-sample difficulty rather than exact fixed-budget success probability.The stabilizer τ constrains this behavior and works best on the scale of the pilot’s uncertainty.

6 Conclusion

TEAM-DESIGN allocates human-only and agent-only replays to the comparisons that need more evidence while accounting for prediction difficulty and replay cost. Its clinical reanalysis shows that beating the human baseline alone can produce the wrong deployment conclusion.

  • TEAM-DESIGN gives each task separate human-only and agent-only replay probabilities, directing more measurement toward the comparison needing more evidence while accounting for prediction difficulty and replay cost.
  • In the clinical data, assisted clinicians beat clinicians alone but not models alone, so evaluating only the human comparison can misclassify deployment.
  • The rule is most useful when one comparison is clearly harder, while its objective only bounds joint power to first order and does not maximize fixed-budget power.
  • The clinical evidence comes from matched reader studies rather than deployed workflows.

A Proofs

The proof derives the comparison difficulty from a variance decomposition and solves the resulting budgeted allocation through convex optimization. The optimal replay probabilities are characterized by KKT conditions and implemented using pilot estimates.

  • The variance decomposition separates prediction error from replay-induced variability, with the latter depending on the replay probability qj(W).The proof notes that no assumption relating Yj to T given W is required, allowing correlated outcomes.
  • Comparison j’s difficulty is bj + E[uj(W)/qj(W)], so replay allocation controls the qj-dependent term.
  • The allocation problem introduces comparison and budget multipliers, with stationarity requiring λH + λA = 1.
  • Convexity and KKT sufficiency yield an interior allocation rule, clipped to [qj,min, 1] by box constraints.
  • Replacing population quantities with pilot estimates produces the implementable rule (10).

A.4 Proof of Theorem 4.3

Theorem 4.3 establishes the asymptotic behavior of the randomized replay estimator conditional on the pilot. Its design-dependent and design-invariant components combine into a Gaussian limit, while the joint testing rule controls false declarations.

  • Replay indicators are conditionally independent across tasks and comparisons, with mean qj(Wi), so the correction term is centered at the conditional gain.
  • The proof isolates a design-dependent term controlled by replay probabilities from a mean-zero term that the design cannot affect.
  • Conditional Lindeberg conditions follow from bounded outcomes and replay probabilities bounded below by qj,min > 0.
  • The selected replay design converges to its population rule under pilot conditioning and uniqueness of the population minimizer.
  • The joint limit combines asymptotically independent Gaussian components, yielding N{0, Σ(q◦A)} with consistent paired-score covariance.
  • If any comparison satisfies ∆j ≤ δj, the probability of declaring success on both is asymptotically at most α.

B.1 Generating the evaluation data

The evaluation data combine constructed task-level scenarios with a semi-synthetic chest X-ray design to test unequal comparison difficulty, replay costs, correlation, and deployment-relevant assumptions. The experiments compare TEAM-DESIGN with several allocation rules under matched budgets.

  • Study construction: The constructed studies vary how unequal the two comparisons are because that imbalance is the feature TEAM-DESIGN is designed to exploit.
  • Data-generating setup: Each task contains a trace stratum, workflow outcome, human-only outcome, and agent-only outcome, with binary task success as the primary outcome.
  • Data-generating setup: The primary generator uses two strata with specified Bernoulli success probabilities, producing a small workflow-versus-human gain and a substantially larger workflow-versus-agent gain.These probabilities are design parameters rather than empirical findings.
  • Study construction: The balanced-comparison check makes the two population gains comparable to test whether a method built around one limiting comparison has an advantage.
  • Allocation comparisons: The unequal-cost scenario fixes expected total cost at one unit per task and compares TEAM-DESIGN, cost-aware Neyman, equal cost splitting, and other rules at matched cost.
  • Semi-synthetic design: The semi-synthetic chest X-ray experiment retains measured covariates, costs, reader structure, and correlations, but its case-level independence differs from the theorem’s read-level assumptions.
  • Results and protocol: In the agent-limited regime, full replay clears both bounds in only 9.0% of replications, so every rule performs alike.

B.2 Comparators and metrics

The study compares replay strategies under a shared budget and evaluates them by the beats-both decision, error control, and supporting estimation metrics.

  • TEAM-DESIGN increases replay where prediction is difficult, comparison thresholds are harder to establish, and replay costs are lower.Neyman uses prediction difficulty and cost but not threshold distance; uniform replay uses none of these inputs.
  • All methods receive the same pilot and replay budget, enabling direct comparison of allocation strategies.
  • The Gaussian comparators include uniform replay, Neyman allocation, parameter-EIG, decision-targeted Bayesian design, and a surrogate oracle.The oracle substitutes true gains and residual second moments for pilot estimates, isolating pilot-estimation costs.
  • The primary outcome is P(LH > 0 and LA > 0), with false-claim rate used under a boundary null.Additional measures include gain bias, gain standard error, replay counts, total cost, limiting-comparison budget share, and oracle gap.

B.3 Gaussian acquisition study

The Gaussian study tests acquisition rules under shared budgets and shows that TEAM-DESIGN performs best when the two comparisons differ in difficulty, while preserving nominal false-claim control.

  • 76.3% beats-both decision rate for TEAM-DESIGN exceeded 75.2% for Bayesian batch design, 59.5% for parameter-EIG, and 60.1% for equal allocation.All reported methods were within Monte Carlo error of the nominal false-claim rate.
  • At the widest comparison gap, TEAM-DESIGN increased power from 70.2% to 81.1%, versus 81.3% best attainable.Its true-input allocation moved from 0.962 to 0.893, near the joint-power optimum of 0.907.
  • At a 0.70 power target, TEAM-DESIGN required 20,000 measurements versus 26,000 for parameter-EIG in the most imbalanced cell.When gains were equal at 0.02, TEAM-DESIGN required 52,000 versus 40,000 for parameter-EIG.
  • The study compares methods at (∆H, ∆A) = (0.02, 0.10) under a shared pilot and common total budget, with size-corrected power calibrated to 5% false-claim rate.
  • Interleaving measurement and estimation scored higher but exceeded the nominal false-claim rate, so it was not comparable with methods committing before measurement.The reported fixed-sample bound becomes invalid when sample sizes are chosen from the data.

B.4 Sensitivity analyses

Sensitivity analyses show that TEAM-DESIGN’s advantage depends on replay-cost imbalance, evaluation size, minimum improvements, replay floors, and stabilizer choice, with important limits when comparisons are balanced or unresolved.

  • Raising the replay-cost ratio steadily narrows TEAM-DESIGN’s advantage, while correlation up to ρ = 0.75 preserves its pattern across unequal and balanced comparisons.Evaluation size determines whether any rule can resolve the comparisons; larger pilots lift TEAM-DESIGN, Neyman, and the oracle together.
  • At δH = δA = 0.02, TEAM-DESIGN achieved a 30.9% decision rate versus 25.4% for Neyman, a 5.5-point paired gain.
  • Raising the replay floor increased human-gain detection from 77.5% at 0.05 to 98.0% at 0.20 without increasing TEAM-DESIGN’s decision rate.Across 36 comparisons, TEAM-DESIGN led Neyman in 10 and trailed in 25; the one-sided sign test favored Neyman at p = 0.008.
  • Size-corrected power peaked at 75.3% when τ = 0.03, versus 59.5% at τ = 0.0025 and 70.0% at τ = 0.1.Small stabilizers track pilot noise, whereas large stabilizers erase bottleneck information.
  • At 20,000 measurements, TEAM-DESIGN reached 76.6% versus 58.8% for parameter-EIG, compared with 32.9% versus 31.3% at 8,000.Below the crossover, neither gain could be resolved and the first-order argument did not apply.
  • The design targets task success and may not establish suitability for safety, equity, workload, or distribution shifts; it also assumes both baseline replays are possible.
Loading 2609.05527v1…