Source-linked AI summary
The Labor Economics of Paid Crowdsourcing
John Horton, Lydia Chilton
TL;DR
Paid crowdsourcing requires a model of how workers choose whether and how much to work, but conventional labor-supply evidence may not transfer directly to short, low-paying online tasks. The paper develops a reservation-wage model and estimation method, finding rational responses to incentives alongside target-earning behavior that limits the model’s completeness.
Problem
Paid crowdsourcing needs evidence and models explaining worker participation and output decisions under monetary incentives.
Method
The paper develops a rational labor-supply model and estimates worker reservation wages using a highly concave earnings function.
Results
Workers respond to prices, but some appear to pursue earnings targets, causing behavior that the rational model does not fully explain.
Takeaways & Limitations
Crowdsourcing incentive designers should consider workers’ propensity to work toward natural earnings targets, while recognizing that such schemes could backfire or be unethical.
Takeaways & Limitations
The calibrated model predicts the fraction of workers accepting an offer but not how many workers will see it, because search timing and worker availability also affect uptake.
Abstract
from arXiv · showhide
Crowdsourcing is a form of "peer production" in which work traditionally performed by an employee is outsourced to an "undefined, generally large group of people in the form of an open call." We present a model of workers supplying labor to paid crowdsourcing projects. We also introduce a novel method for estimating a worker's reservation wage--the smallest wage a worker is willing to accept for a task and the key parameter in our labor supply model. It shows that the reservation wages of a sample of workers from Amazon's Mechanical Turk (AMT) are approximately log normally distributed, with a median wage of $1.38/hour. At the median wage, the point elasticity of extensive labor supply is 0.43. We discuss how to use our calibrated model to make predictions in applied work. Two experimental tests of the model show that many workers respond rationally to offered incentives. However, a non-trivial fraction of subjects appear to set earnings targets. These "target earners" consider not just the offered wage--which is what the rational model predicts--but also their proximity to earnings goals. Interestingly, a number of workers clearly prefer earning total amounts evenly divisible by 5, presumably because these amounts make good targets.
1. INTRODUCTION
The paper develops a labor-supply framework for paid crowdsourcing and tests whether workers respond rationally to prices and task conditions. It finds price sensitivity alongside target-earning behavior that the rational model does not capture fully.
- Motivation: Paid crowdsourcing creates a practical need to understand how monetary incentives attract and motivate workers.Cash incentives are adjustable, unlike many task- or proposer-dependent non-monetary incentives.
- Motivation: Designers need a model predicting both participation and output conditional on participation.These correspond to extensive and intensive labor-supply margins.
- Contributions: The paper develops a rational labor-supply model and a novel reservation-wage estimation method using a highly concave earnings function.Reservation wage is the minimum compensation a worker accepts for performing a task.
- Findings: AMT experiments find workers clearly sensitive to price but insensitive to variations in task completion time.The experiments test responses to task difficulty and price.
- Findings: Some workers appear to target salient earnings goals rather than respond only to the offered wage.This behavior contrasts with the rational model’s prediction that workers consider the offered wage alone.
- Implications: The model misses important elements of reality but still makes reasonable predictions in some cases and can be calibrated for price/task scenarios.The paper demonstrates this applied use with pooled experimental data.
2. THEORY
The theory models workers choosing output by comparing payment benefits with time-based costs and uses concave payments to obtain informative interior output choices. It defines reservation wages through workers’ indifference between working and their next-best alternative.
- Reservation wages: A reservation wage is the economic value of a worker’s next-best alternative, such as another job, leisure, or renewed job search.A worker works when the net benefits of working exceed those of the alternative.
- Reservation wages: Estimating reservation wages is difficult because jobs combine monetary wages with non-monetary amenities and dis-amenities.Observing work reveals only that total benefits exceed total costs and the reservation wage.
- Worker choice: The model assumes workers choose continuous output y and maximize payment P(y) minus completion cost C(y).Payment is increasing and concave, while costs depend on task completion.
- Worker choice: The first-order condition p(y*) = c(y*) sets marginal payment equal to marginal cost.An interior optimum requires the net-benefit function to reach a global maximum, which occurs when it is concave.
- Cost curves and output: The experiments assume linear costs because completion times are essentially constant.Excluding the first task, completion times rise by less than 1 millisecond per task, although the second task is about 1.5 seconds faster than the first.
- Cost curves and output: With linear costs and a constant piece-rate, workers have no interior output solution and tend toward zero output or an employer-imposed cap.Strictly concave payments restore an interior solution by making the piece-rate fall with output.
3. EXPERIMENTAL PRELIMINARIES
The experiments test labor-supply predictions in an AMT clicking task with adjustable difficulty, prices, and concave payments. Subjects repeatedly choose whether to continue producing blocks while observing payment and performance information.
- Experimental design: The experiments test whether lower wages reduce output through greater required effort or lower payment for a fixed amount of work.Experiment A varies task difficulty, while Experiment B varies pay.
- Experimental setting: AMT lets workers choose among small human-intelligence tasks after viewing their attributes and usually a sample of the required work.Requesters set payments, qualifications, duration, repetition limits, and task interfaces.
- Task and interface: The clicking task alternates between green target and gray bars, requiring subjects to click back and forth across a specified distance.Missed clicks flash red, and workers can continue until the task ends.
- Task and interface: Each output block contains 10 back-and-forth clicks, and subjects may quit or continue after completing a block.The interface displays payment rates, totals, error rate, and average completion time before the next decision.
- Payment function: The payment schedule is concave, with later pieces paying less while remaining positive and total earnings approaching an upper limit.All experiments use a 10-task half-life for the schedule.
4. EXPERIMENT A: ∆DIFFICULTY
Experiment A increased completion time in HARD relative to EASY, but did not produce a discernible difference in output patterns. HARD workers nevertheless had relatively higher estimated reservation wages, although the effect was statistically weak.
- 4.2 Experiment design: 92 subjects were assigned to EASY or HARD, completing 18934 clicks under the same payment function.EASY used bars 100 pixels apart; HARD used bars 600 pixels apart.
- 4.1 Model prediction: y′(t) < 0: increasing unit completion time reduces a worker’s optimal output in the model.
- 4.2.2 Output: HARD subjects took about 11 seconds per block versus about 6 seconds in EASY, confirming the difficulty treatment affected completion time.The treatment effect was large and highly significant; HARD “hit” times were also right-shifted.
- 4.2.2 Output: Output patterns did not differ discernibly between groups despite HARD requiring more time per block.Mean output was nearly the same, and the larger number of HARD subjects quitting after one block was not statistically significant.
- 4.2.3 Reservation wages: The estimated reservation-wage distributions were similar across groups, with HARD showing a fat left tail.The geometric mean reservation wage was $0.89/hour in HARD and $1.5/hour in EASY.
- 4.3 Discussion: HARD workers had relatively higher reservation wages, but the effect was only weakly supported, with a t-statistic of 1.41.The result is described as theoretically ambiguous but internally consistent with HARD workers taking longer while producing similar output.
5. EXPERIMENT B: ∆PRICE
Experiment B varied only the payment ceiling while keeping the clicking task fixed. Lower pay reduced output, but it also produced lower imputed reservation wages, a pattern the authors attribute most compellingly to target-earning behavior.
- Experiment design: 198 subjects performed the same task under payment ceilings approaching either 10 cents in LOW or 30 cents in HIGH.Both groups used bars 100 pixels apart; each had 99 subjects.
- Output: Output was lower in LOW than HIGH, with the difference driven largely by more subjects quitting after small amounts of output.The output distribution was bimodal, including early quitters and workers completing more than 50 blocks.
- Output: The LOW group had a lower mean output, but the level estimate was imprecise because output was bimodal.The levels regression reports R2 = 0.0048 and N = 198.
- Reservation Wages: $1.56/hour in HIGH versus $0.71/hour in LOW was the estimated geometric mean reservation wage.Group assignment shifted both the central mass and the very-low-wage mass leftward in LOW.
- Discussion: Lower pay reduced output as predicted, but lower reservation wages in LOW imply that its output was not as low as the rational model predicts.Because the tasks were identical, differences in amenities cannot explain the reservation-wage gap.
- Discussion: Target earning is presented as the most compelling explanation for workers’ lower imputed reservation wages in LOW.At lower wages, target earners must produce more output to reach their earnings goals.
6. DEPARTURES FROM THE RATIONAL MODEL
The rational model cannot explain all workers: some appear to pursue earnings targets, producing focal-point earnings patterns that differ from rational labor-supply predictions.
- Target earning: Some workers appear to create earnings targets, so the rational model cannot explain all observed behavior.In the absence of income effects, treating past earnings as relevant creates a sunk cost fallacy in marginal work decisions.
- Target earning: Target earners may work less when wages are high, whereas rational workers are predicted to work more.The paper reports fairly unambiguous evidence of target earning in several experimental results, while evidence outside the experiments is mixed.
- Evidence of targeting: In every experiment, some subjects pursued maximum possible earnings despite the low wages associated with that strategy.The authors argue that a rational explanation would require a highly bimodal wage distribution, whereas targeting the full attainable amount is more plausible.
- Preferences for “focal point” earnings: Subjects often chose the minimum output needed to reach whole-cent earnings, quitting when they could not escape the 29-cent band.The whole-cent pattern may also reflect extreme risk aversion or beliefs that fractional-cent earnings could be withheld.
- Preferences for “focal point” earnings: Output spikes at 15, 20, and 25 cents indicate a preference for earnings amounts evenly divisible by 5.The smallest earnings amounts also attracted several subjects, but these early quitters presumably did not have or need a target.
- Testing modulo-5 targeting: 33 of 99 observations produced earnings divisible by 5, versus an expected per-trial probability of q = 0.22; observing at least this many by chance had probability 0.0027.The analysis compares observed divisible-by-5 earnings with the proportion of potentially realizable whole-cent amounts divisible by 5.
7. CONCLUSION
The paper offers a calibrated labor-supply model for predicting participation in paid crowdsourcing, while finding only partial agreement with rational-worker behavior. It also identifies target earning and outlines important assumptions and boundaries for applying the model.
- 7.1 Using the calibrated model: The calibrated model offers a useful approximation despite only partial agreement between its predictions and observed behavior.Its predictions rely on assumptions including a selected AMT sample, constant marginal costs, comparable task amenities, and fixed reservation wages.
- 7.1 Using the calibrated model: Workers with reservation wages above p0/t0 are predicted not to participate, so the model estimates participation from the reservation-wage distribution.The participation share is represented by F(p0/t0), where F is the cumulative distribution function.
- 7.1 Using the calibrated model: The pooled reservation-wage distribution is modeled as log normal, with extensive labor-supply elasticity computed from that approximation.The paper notes that the arithmetic mean exceeds the median because the log-normal distribution is right-skewed.
- 7.2 Future research: A major limitation is that the model predicts task acceptance but not how many workers will see an offer on AMT.Search timing and workers’ time-zone distribution may affect uptake, motivating further research on job search and offer visibility.
- 7. CONCLUSION: The model provides a price-theoretic foundation for crowdsourcing labor supply, addressing persistent labor-supply problems after moral hazard is controlled.It is presented as groundwork for a broader predictive theory of crowdsourcing.