Source-linked AI summary
The Algorithmic Automation Problem: Prediction, Triage, and Human Effort
Maithra Raghu, Katy Blumer, Greg Corrado, Jon Kleinberg, Ziad Obermeyer, Sendhil Mullainathan
TL;DR
The paper addresses the gap between algorithmic prediction and broader automation, which also requires deciding which instances algorithms should handle. It develops an optimization framework for prediction and triage, finding that selective assignment and better per-instance error estimates can improve automation performance.
Problem
Research discussions have equated prediction with automation, overlooking the decision of which instances to assign to algorithms.
Method
The paper formulates automation as jointly predicting outcomes, triaging instances, and allocating human effort across cases.
Results
Optimal triage reduces diabetic retinopathy error to roughly 3.5%, versus 4% with full automation and roughly 5.5% for human doctors.
Takeaways & Limitations
Effective automation depends on estimating human and algorithmic error per instance and using those estimates to allocate effort efficiently.
Takeaways & Limitations
Algorithmic error probabilities are difficult to estimate because deep neural networks are poorly calibrated, while human use of algorithmic assistance is difficult to model a priori.
Abstract
from arXiv · showhide
In a wide array of areas, algorithms are matching and surpassing the performance of human experts, leading to consideration of the roles of human judgment and algorithmic prediction in these domains. The discussion around these developments, however, has implicitly equated the specific task of prediction with the general task of automation. We argue here that automation is broader than just a comparison of human versus algorithmic performance on a task; it also involves the decision of which instances of the task to give to the algorithm in the first place. We develop a general framework that poses this latter decision as an optimization problem, and we show how basic heuristics for this optimization problem can lead to performance gains even on heavily-studied applications of AI in medicine. Our framework also serves to highlight how effective automation depends crucially on estimating both algorithmic and human error on an instance-by-instance basis, and our results show how improvements in these error estimation problems can yield significant gains for automation as well.
1 Introduction
The paper reframes automation as both prediction and instance-level triage, rather than prediction alone. It shows that selectively assigning cases to algorithms or humans can improve performance, including in diabetic retinopathy diagnosis.
- 1 Introduction: Automation includes deciding which instances to assign to an algorithm, not merely replacing human effort with predictions.The proposed algorithm should produce both a prediction and a triage judgment relative to human performance.
- 1 Introduction: Machine-learning systems typically solve prediction but not triage, implicitly assuming either full or zero automation.The paper argues that jointly optimizing prediction and triage can produce higher overall performance.
- 1 Introduction: 3.5% error is achieved with optimal triage, compared with 4% under full automation and roughly 5.5% for human doctors.The diabetic retinopathy example uses largely the same data and assigns only a fraction of instances to the algorithm.
- 1 Introduction: Algorithmic gains arise from heterogeneous per-instance performance and from reallocating freed human effort to remaining cases.The algorithm has zero errors on roughly 40% of instances, while difficult cases can be assigned to humans.
- 1 Introduction: Algorithmic triage remains underexplored, and improving triage may offer higher returns than further improving prediction.The paper argues that demonstrated gains likely do not exhaust the potential benefits of triage.
2 General Framework
The framework defines automation as jointly deciding which instances receive algorithmic predictions and how human effort is allocated to the remaining instances. It formalizes triage as an optimization problem and identifies instance-specific error estimation and effort allocation as sources of performance gains.
- Optimization formulation: Automation selects a subset S for algorithm-only processing and allocates human effort k_x across the remaining instances T under a total budget B.Loss is g(x) for algorithm-processed instances and f(x, k_x) for human-processed instances.
- Optimization formulation: Intermediate solutions between full automation and fully human processing constitute triage, with some instances assigned purely to the algorithm and others receiving human attention.Full automation is S = U, while the no-algorithm case is T = U.
- Information requirements: Effective automation requires estimates of algorithmic instance-specific error g(x), human error f(x, k), and improved allocations of human effort.The framework treats human error as dependent on the amount of effort allocated to each instance.
- Information requirements: Human outputs on algorithm-assisted instances may depend on how decision-makers incorporate the algorithmic prediction, making f(x, k_x) sensitive to assistance design.The framework's automation results do not require assumptions about this synthesis process.
- Implications: Simple heuristics can yield significant gains over both algorithmic and human effort, while stronger error and allocation approaches offer larger gains in principle.The paper uses predictors for algorithmic and human error and a stronger ground-truth benchmark to assess achievable improvements.
- Implications: Aggregate performance comparisons can hide heterogeneous instance difficulty, including sizeable subsets where the algorithm makes zero errors and should be automated.This heterogeneity can support automation even when preferences favor human effort overall.
3 Medical Preliminaries, Data and Experimental Setup
The study applies its framework to diabetic retinopathy screening using fundus photographs, a five-class CNN trained on doctor grades, and binary referable-DR decisions. Evaluation uses a smaller adjudicated dataset with multiple doctor grades and an adjudicated grade serving as a ground-truth proxy.
- Medical setting: The study uses fundus photographs to detect diabetic retinopathy, an eye disease involving blood-vessel damage caused by high blood sugar.Diabetic retinopathy can cause blindness but is treatable when detected early.
- Medical setting: Doctors grade each photograph on five DR severity classes, with grades 3 and above defining referable DR requiring immediate specialist attention.The study later treats triage and automation as binary referable versus non-referable classification.
- Data: The training data contain fundus photographs with multiple doctor grades, while the larger training set is more sparsely labelled than the evaluation data.The evaluation dataset is smaller but receives more extensive grading and adjudication.
- Algorithm: A convolutional neural network is trained for five-class DR classification using the empirical distribution of individual doctor grades as each image's target.The model outputs a distribution over the five DR grades.
- Algorithm: The algorithm sums CNN output mass for grades ≥3 into m(x_i), then labels an image referable when m(x_i) ≥ q_R.The threshold q_R is chosen so algorithmic and aggregated human referable-case counts match.
- Evaluation: Evaluation uses a gold-standard adjudicated dataset in which multiple doctor grades are supplemented by a jointly adjudicated grade used as a ground-truth proxy.The adjudicated grade supports evaluation of both individual doctors and the decision-making algorithm.
- Aggregation and thresholding: For aggregation, multiple binary grades are averaged and thresholded by R; the main text uses R = 0.5, corresponding to majority vote.Appendix results for R = 0.3 and R = 0.4 support the same conclusions.
4 The Triage Problem and Human Effort Reallocation
The section frames automation as choosing which instances receive algorithmic versus human effort, using instance-level error differences to triage cases. Experiments show that partial automation and human-effort reallocation outperform full automation and equal human coverage, while better triage prediction offers further gains.
- Per-instance error diversity: Average performance can conceal instances where human or algorithmic effort has lower error probability, and these differences are partially predictable.This predictability enables assigning instances to the party better suited to handle them.
- Per-instance error diversity: Human error probability is estimated from doctors’ average disagreement with the adjudicated grade, while algorithmic error probability requires calibration beyond the network’s raw output.The raw m(x_i) score is poorly calibrated and uses a different scale from human error estimates.
- Human effort reallocation: The optimization problem selects instances for algorithm-only handling and allocates a fixed human-effort budget across the remaining instances.The basic heuristic ranks instances by a triage score, automates an α fraction, and distributes human effort equally over the rest.
- Triage results: For both predicted-error and ground-truth triage, the best performance occurs at 0 < α < 1, outperforming full automation and equal human coverage.Ground-truth triage performs significantly better than triage using predicted error probabilities.
- Triage results: Simple uncertainty triage, m(x)(1 −m(x)), improves over pure algorithmic and pure human effort, though separate error predictors achieve larger gains.The uncertainty score provides a low-complexity proxy for algorithmic error without explicitly modeling human error.
- Differential costs and zero-error subsets: Triage identifies zero-error algorithmic subsets covering 35% of the data with simple uncertainty and 44% with separate error prediction.These results show why aggregate algorithmic performance can hide useful subsets for automation.
5 Related Work
Related work has largely compared algorithms with humans in prediction-only settings across medical imaging tasks. Some studies examine fixed human–algorithm interactions, but this section distinguishes those approaches from adaptive triage and effort allocation.
- Prediction-based comparisons: Prior medical-imaging studies commonly compare algorithm-only with human-only performance, including chest X-rays, Alzheimer’s detection, and fundus-photograph diagnosis.These comparisons implicitly treat automation as prediction success.
- Triage results: The paper’s triage analysis identifies subsets where algorithmic error is zero, including 35% with simple uncertainty and 44% with separate error prediction.These figures are reported as part of the paper’s analysis of triage rather than the prior-work comparisons.
- Fixed human–algorithm interactions: Other work studies fixed interactions, such as physician review of algorithmic outputs or predetermined combinations of physician and algorithmic judgments.These approaches do not establish the instance-level triage framework developed here.
6 Discussion
The discussion presents automation as a joint prediction-and-triage problem rather than an all-or-nothing replacement of human effort. It concludes that per-instance error estimation and efficient effort allocation can improve performance in diabetic-retinopathy diagnosis.
- Discussion: Stronger performance can result when algorithms both predict outcomes and triage instances between algorithmic and human handling.The framework uses estimates of human and algorithmic per-instance error to allocate effort.
- Discussion: The diabetic-retinopathy analysis shows performance gains for this framework in a well-studied medical-AI application.The discussion presents this as evidence that the broader automation formulation has practical value.
- Discussion: More accurate predictions of per-instance error have the potential to yield further automation benefits through improved triage and allocation of human and computational effort.The paper identifies algorithmic triage as an area for further study.
A Training Data and Models Details
The models are trained on fundus photographs with doctor-grade labels, using a five-class convolutional network and separate neural networks for algorithmic and human error prediction. Multiple doctor grades support empirical grade distributions and disagreement-based human-error targets.
- Training data: The training data contain fundus photographs labeled with five possible diabetic-retinopathy grades, with multi-doctor cases represented by empirical grade distributions.An image graded {2, 3, 3}, for example, is not represented as a one-hot label.
- Diagnostic model: The diagnostic model is an Inception-v3 convolutional neural network with an ImageNet-pretrained backbone and a new five-class classification head.The top of the network is retrained on images with at least two doctor grades for better calibration.
- Error prediction models: A separate fully connected network predicts whether the diagnostic model was correct from its prelogit embedding, producing an algorithmic error-probability estimate.This model is trained after the main diagnostic network.
- Error prediction models: The human-error predictor uses images with at least two doctor grades and learns binary targets derived from agreement or disagreement between split doctor-grade groups.The image embedding from the diagnostic network’s Prelogit layer serves as input.
B Computing Pr [Mi]
The experiments set C = 2000 for the method that calibrates error probabilities for convolutional-network outputs.
- C = 2000 in the experiments for calibrating the convolutional network’s output error probability.
C Triage and Allocation Algorithm
Triage ranks instances by score, automates the highest-scoring fraction, and distributes available human grading effort across the remainder.
- Instances are ordered by triage score, and the first αN are fully automated.
- The remaining (1 − α)N images receive the available cN doctor grades under an equal coverage protocol.Each remaining case receives cN/((1 − α)N) grades, with spare grades assigned to the hardest cases.
C.1 Results on other Thresholds
Results remain qualitatively stable across alternative aggregation thresholds, while error-prediction triage remains worse than ground-truth triage.
- The qualitative conclusion remains unchanged at thresholds 0.3 and 0.4: combining human and algorithmic effort beats full allocation and equal coverage.
- The calibration algorithm estimates Pr[Mi] by averaging instance errors over C repetitions.Each repetition samples doctor grades, assigns binary grades based on ranked model scores, and compares them with adjudicated grades.
- The results show a significant gap between triage using ground truth and triage using error predictions.
- Changing the aggregation threshold does not affect identification of zero-error subsets, although it can affect qR and Figure 6 results.
D Triage and Human Effort Reallocation with Model Grades
The effort-reallocation triage process assumes that algorithm decisions are unavailable for cases not automated, an assumption that depends on how algorithm outputs are used.
- The triage process assumes algorithm decisions are unavailable for the (1 − α)N cases left to human doctors.This may reflect computational cost or a deliberate choice to avoid biasing doctors when the algorithm is unsure.
- If algorithm decisions are also available for non-automated cases, the effort-reallocation scenario differs from the stated process.
E Results on Additional Holdout Dataset
On an additional holdout dataset without adjudicated grades, the experiments reproduce the main findings: instance-level differences are diverse, and triaging human and algorithmic effort outperforms full automation and equal coverage. Triaging also identifies zero-error subsets, although their fractions are slightly lower with noisier labels.
- Dataset and setup: The additional holdout dataset uses individual doctor grades without an adjudicated grade, with half used for proxy ground truth and the remainder for effort reallocation and evaluation.The doctor grades are slightly noisier than those in the adjudicated evaluation dataset.
- Instance-level variation: Around 10% of instances have human experts outperforming the algorithm, showing a diverse spread of instance-level error differences.The distribution is qualitatively identical to that observed with adjudicated data.
- Effort reallocation: Triaging to combine human expert effort with algorithmic decisions outperforms both full automation and equal coverage on the additional holdout dataset.This conclusion holds for triaging based on error prediction models and ground-truth scores, although a gap remains between them.
- Zero-error subsets: Triaging identifies zero-error subsets, but their fractions are slightly smaller on the holdout dataset, likely because its labels are noisier.The additional evaluation nevertheless supports the main findings.
- Zero-error subsets: Triaging by model uncertainty yields a slightly higher fraction of zero-error examples than a separate error prediction model in this evaluation.The comparison is averaged over three independent training repetitions, and one separate error model repetition makes two errors.