Source-linked AI summary
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Kotaro Hara, Abi Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, Jeffrey Bigham
TL;DR
Crowd work is widely associated with low pay, but wage distributions and the factors linked to higher or lower earnings remain poorly understood. Using activity logs from Amazon Mechanical Turk workers, the paper finds that earnings are typically far below minimum wage, with only a small minority exceeding it.
Problem
Evidence on crowd workers’ wage distributions and earnings differences is limited despite concerns about low pay and unpaid completed work.
Method
The study analyzes logs from 2,676 workers performing 3.8 million Mechanical Turk tasks to measure hourly wages and characterize unpaid work, task attributes, and working patterns.
Results
96% of workers earned below the U.S. federal minimum wage, while requesters paid $11.58/h on average; dominant low-wage requesters pulled down the overall distribution.
Takeaways & Limitations
Worker-facing tools that show effective hourly wages and provide real-time task-selection strategies may help workers optimize earnings.
Takeaways & Limitations
Overlapping accepted tasks can make interval-based work-time estimates underestimate hourly wages because waiting time is counted as work.
Abstract
from arXiv · showhide
A growing number of people are working as part of on-line crowd work, which has been characterized by its low wages; yet, we know little about wage distribution and causes of low/high earnings. We recorded 2,676 workers performing 3.8 million tasks on Amazon Mechanical Turk. Our task-level analysis revealed that workers earned a median hourly wage of only ~\$2/h, and only 4% earned more than \$7.25/h. The average requester pays more than \$11/h, although lower-paying requesters post much more work. Our wage calculations are influenced by how unpaid work is included in our wage calculations, e.g., time spent searching for tasks, working on tasks that are rejected, and working on tasks that are ultimately not submitted. We further explore the characteristics of tasks and working patterns that yield higher hourly wages. Our analysis informs future platform design and worker tools to create a more positive future for crowd work.
DISCUSSION
The discussion attributes low AMT wages partly to unpaid work and emphasizes tools and platform changes that could help workers improve earnings. It also notes limitations in the dataset and analysis, including potential sample bias and the absence of causal evidence.
- Unpaid work: Unpaid work from task searching, qualification tasks, and returned or rejected tasks contributes to low hourly wages.The discussion suggests experienced workers may learn to minimize this unpaid work.
- Wage distribution: 96% of workers earn below the U.S. federal minimum wage, while requesters pay $11.58/h on average.Dominant requesters posting many low-wage HITs pull down the overall wage distribution.
- Worker tools: Real-time wage awareness and task-selection strategies could help workers optimize earnings despite the difficulty of measuring hourly wages.Visualized earnings information would provide feedback that AMT does not currently provide, but privacy concerns must be considered.
- Platform design: Improved task recommendations, requester communication, and higher minimum rewards could reduce unpaid work and help workers avoid unfair requesters.These approaches include automatically pushing tasks, estimating completion and acceptance likelihood, fixing broken HITs, and enabling collective bargaining.
- Limitations: The results may be biased toward experienced workers, do not establish causal relationships, and lack worker demographic information.The study also notes that estimated work intervals may not equal active HIT work and that script use may affect earnings.