PaperlayerView exact revision on arXiv

Source-linked AI summary

The Data-Production Dispositif

Milagros Miceli, Julian Posada

arXiv:2205.11963v1cs.HCcs.CYcs.LG

TL;DR

The paper examines how outsourced ML data work in Latin America organizes workers’ interpretation of data. Using dispositif analysis across Venezuelan platforms and an Argentine BPO, it finds that instructions reproduce requester-defined interpretations while poverty and dependence shape workers’ obedience.

  • Problem

    Outsourced data work depends on workers’ interpretation and classification, but these practices are embedded in profit-oriented industrial structures with ethical and political implications.

    Source: §1 · p. 1, §2.2 · p. 5

  • Method

    The authors apply dispositif analysis to instruction documents, interviews, and observations at three Venezuelan platforms and an Argentine BPO.

    Source: §1 · p. 2, §3.3.1 · p. 7

  • Results

    Requesters use instructions to impose predefined interpretations of data, while poverty and dependence leave workers little option but to obey.

    Source: §1 · p. 3, §5 · p. 28

  • Takeaways & Limitations

    The authors call for counteracting precarization, alienation, and surveillance and empowering data workers as assets in the pursuit of high-quality data.

    Source: §1 · p. 3

    The findings are bounded by the studied platforms, companies, individuals, geographical contexts, and the researchers’ positionality.

    Source: §5.2 · p. 32

Abstract

from arXiv · show

Machine learning (ML) depends on data to train and verify models. Very often, organizations outsource processes related to data work (i.e., generating and annotating data and evaluating outputs) through business process outsourcing (BPO) companies and crowdsourcing platforms. This paper investigates outsourced ML data work in Latin America by studying three platforms in Venezuela and a BPO in Argentina. We lean on the Foucauldian notion of dispositif to define the data-production dispositif as an ensemble of discourses, actions, and objects strategically disposed to (re)produce power/knowledge relations in data and labor. Our dispositif analysis comprises the examination of 210 data work instruction documents, 55 interviews with data workers, managers, and requesters, and participant observation. Our findings show that discourses encoded in instructions reproduce and normalize the worldviews of requesters. Precarious working conditions and economic dependency alienate workers, making them obedient to instructions. Furthermore, discourses and social contexts materialize in artifacts, such as interfaces and performance metrics, limiting workers' agency and normalizing specific ways of interpreting data. We conclude by stressing the importance of counteracting the data-production dispositif by fighting alienation and precarization, and empowering data workers to become assets in the quest for high-quality data.

1 INTRODUCTION

Outsourced data workers interpret and classify data within industrial structures shaped by profit-oriented interests. This study examines how instructions, work practices, and artifacts reproduce power/knowledge relations in outsourced ML data work.

1 INTRODUCTION

  • Workers’ data interpretation is embedded in industrial structures and naturalized profit-oriented interests, with ethical and political implications.Outsourced workers may be paid only a few cents per task, lack employment-linked social protection, and face control and surveillance.Source: §1 · pp. 1–2
  • The study investigates data work at three platforms in Venezuela and a BPO in Argentina using Foucault’s dispositif and dispositif analysis.The authors define data work as labor involved in collecting, curating, classifying, labeling, and verifying data.Source: §1 · p. 2
  • The analysis examines instruction discourses, workplace practices, and their materializations in objects such as platforms, interfaces, and surveillance tools.It draws on 210 instruction texts, 41 worker interviews, six manager interviews, eight requester interviews, and participant observation.Source: §1 · p. 2
  • Requesters use instructions to impose predefined, profit-oriented ways of interpreting, classifying, and sorting data, while poverty and economic dependence leave workers little option but to obey.Managers in BPOs and algorithms on platforms oversee the process; the study also identifies documents, tools, and interfaces as materializations of the dispositif.Source: §1 · p. 3

2 DEFINING KEY CONCEPTS

The paper uses dispositif to connect discourse, practice, artifacts, and power/knowledge relations, then situates outsourced ML data work within that framework. It focuses on data work outsourced through platforms and BPOs.

2.1 Dispositif and Other Foucauldian Concepts

  • Foucault’s framework treats discourse as knowledge-shaping and socially structuring, with normalized knowledge influencing what can be said, done, and thought.Power and knowledge are entangled, and dispositifs connect discourse with practices and artifacts.Source: §2.1 · p. 3
  • A dispositif is a network of discourses, practices, subjects, and objects whose relations shape knowledge, meaning, and power.The concept includes both what is said and what is left unsaid, as well as non-discursive practices and material objects.Source: §2.1 · pp. 3–4
  • Dispositif has not previously been applied to data production and data work in HCI and CSCW, motivating the paper’s methodological contribution.The authors present it as a comprehensive mode of analysis for approaching data work.Source: §2.1 · p. 4

2.2 Data Work for Machine Learning

  • The paper defines data work as labor for ML data production and narrows its focus to work outsourced through crowdsourcing platforms and specialized BPO companies.The broader term includes collection, curation, classification, labeling, and verification; the study treats other professionals and users as requesters rather than its focus.Source: §2.2 · p. 4
  • Platforms allocate fragmented online tasks and rely primarily on algorithms for hierarchy, while BPOs use more traditional management structures.BPOs often specialize in a service or domain and charge higher per-piece prices than platforms; companies may prefer them for perceived higher data quality.Source: §2.2 · pp. 4–5
  • Prior research has documented worker subjectivity and power asymmetries, alongside systems that privilege speed and scale over worker wellbeing.Studies have also addressed low wages and dependency in data work.Source: §2.2 · p. 5
  • The data-production dispositif addresses demand for cheaper, more profitable data and is too extensive to analyze in full, so the paper circumscribes its study to outsourced ML data work.The authors describe the dispositif as entangled with other discourse, action, and materialization networks.Source: §2.2 · p. 5

3 METHODOLOGY

The study adapts dispositif analysis to examine instruction texts, work practices, and artifacts, while making the researchers’ positionality explicit. It combines fieldwork, document analysis, interviews, and observations across Venezuelan platforms and an Argentine BPO.

3.1 Dispositif Analysis of Data Production

  • The analysis links instruction discourses, workplace practices, and artifacts to examine how power and knowledge shape meaning in data production.It follows Caborn’s four steps and Jäger and Maier’s three-dimensional framework, adapting them to data work.Source: §3.1 · pp. 5–6

3.2 Researcher Positionality

  • The authors disclose their backgrounds and acknowledge differences in class experience and the privilege of working and living in the Global North.They describe reflexivity about their position in relation to participants and the field of inquiry.Source: §3.2 · pp. 6–7

3.3 Data Collection and Analysis

  • The investigation combined several weeks of observation, 55 interviews, and 210 instruction documents collected during 2019–2021.Fieldwork covered three crowdsourcing platforms operating in Venezuela and a Buenos Aires BPO.Source: §3.3 · p. 7

3.3.1 Fieldwork.

  • Venezuelan platform fieldwork used virtual observation and convenience and snowball sampling, while Argentine BPO fieldwork included in-person and online work.The authors note that the platform sampling approach limits reproducibility and may bias participation toward similar social circles.Source: §3.3.1 · pp. 7–8

3.3.2 Instruction Documents.

  • Both authors analyzed 210 instruction documents through structural, detailed, and synoptic critical discourse analysis.The synoptic phase contrasted selected text fragments with interview and observation material to examine how work and managerial practices legitimize discourses.Source: §3.3.2 · p. 8

3.3.3 Interviews.

  • The 55 interviews explored workers’ experiences and organizational relations, using in-depth conversations with workers and expert interviews with managers and requesters.Interview analysis combined deductive categories from document analysis with inductive coding and cross-coding by both authors.Source: §3.3.3 · p. 10

3.3.4 Observations.

  • Observations covered interactions among workers, clients, platforms, and interfaces, with field notes combining real-time jottings, reflections, sketches, and permitted photos.The researchers examined instruction documents both as texts and as artifacts, focusing on their function, provenance, and the interactions they allow or constrain.Source: §3.3.4 · p. 11

4 FINDINGS

The findings section first situates the study in the tasks at its four fieldwork sites, then organizes its dispositif analysis around discourses, practices, and materializations.

4 FINDINGS

  • The findings analyze instruction taxonomies and warnings, worker obedience and precarious dependence, and materializations including interfaces and performance-measurement tools.The descriptive opening locates the analysis in particular tasks and ways of doing data work across the four fieldwork sites.Source: §4 · pp. 11–12

4.1 Different Tasks, Different Instructions

The study distinguishes outsourced data work by task type, from collecting and annotating data to checking algorithmic outputs and impersonating an AI service. The tasks vary in content, compensation, oversight, and worker exposure.

4.1 Different Tasks, Different Instructions

  • Outsourced AI work spans preparation, verification, and impersonation, with preparation divided into data collection and annotation.The study applies this task framework to both platforms and a BPO.Source: §4.1 · p. 12

4.1.1 Data Generation.

  • Platform data generation included low-paid web searches and better-paid photo, video, or audio capture, which workers found attractive despite privacy concerns.One BPO project used workers’ and family members’ IDs to create authentic and altered document images, with consent forms and a secured interface.Source: §4.1.1 · pp. 12–13

4.1.2 Data Annotation.

  • Annotation tasks ranged from computer-vision labeling and segmentation to classifying social-media text, audio, and video for hate speech or sexual content.BPO workers also labeled ID images as authentic or fake and marked the alterations.Source: §4.1.2 · p. 13

4.1.3 Algorithmic Verification.

  • Platform verification tasks asked workers to assess search-engine results, accompanying content, and adult content, sometimes by comparing results with a competitor.Source: §4.1.3 · p. 14
  • BPO verification involved reviewing marketplace content flagged by a moderation model, with workers confirming or correcting outputs and sometimes encountering disturbing material.Source: §4.1.3 · p. 14

4.1.4 AI Impersonation.

  • AI impersonation appeared only once, when workers on Clickrating replied to users as a social-media service using predefined facts and characteristics.The instructions also required short completion times and work in specified three-hour sessions.Source: §4.1.4 · p. 14

4.2 Linguistically-Performed Elements

Task instructions encode requester-defined ways of interpreting data, while binary labels and threats of lost work constrain workers’ ability to question those definitions. The section shows how these mechanisms normalize classifications and shape annotation practice.

4.2.1 Normalized Classifications.

  • Requester taxonomies can encode Global North assumptions and commercial priorities, as in US-centric racial categories and a “building awareness” label reserved for commercial posts.In road-labeling work, the request for “neutral Spanish” prompted disputes over which vehicle terms qualified.Source: §4.2.1 · pp. 15–16
  • Workers report that requester-defined judgments can override their contextual interpretations, including when a requester's hate-speech criteria conflict with how a Latin American worker reads a post.Source: §4.2.1 · p. 15

4.2.2 Binary and Residual Categories.

  • Binary labels can erase context, while “other” or “ambiguous” often absorbs errors rather than adding a meaningful class.Instructions sometimes direct workers to ignore ambiguity; task definitions have also been revised when omitted cases, such as pedestrians lying down, became apparent.Source: §4.2.2 · pp. 16–18

4.2.3 Warnings.

  • Warnings of bans, nonpayment, or dismissal reinforce the hierarchy; workers’ concerns are filtered through BPO management, and accuracy is defined by the client’s preferred truth value.The paper links these pressures to workers’ economic dependence and risk of losing their livelihood.Source: §4.2.3 · pp. 18–19

4.3 Non-Linguistic Practices and Social Context

Interviews show how poverty and employment insecurity shape outsourced data work, reinforcing obedience to requester instructions while workers also build networks of solidarity and organize for better conditions.

4.3.1 Poverty and Dependence.

  • Alamo recruits from poor communities and favors applicants with fewer alternatives, yet workers value its steady salaried jobs over precarious prior work.In 2019, Alamo’s hourly salary was equivalent to Argentina’s minimum legal wage.Source: §4.3.1 · pp. 19–20
  • Platform dependence limits task choice: workers rely on scarce online earnings, face withdrawal thresholds, and risk losing accumulated pay if banned.Workers reported earning $15–$60 per week, averaging about $20, against a reported Venezuelan minimum wage of about $1 per month.Source: §4.3.1 · pp. 20–21

4.3.2 Obedience to Instructions.

  • Briefings, repeated reviews, and fear of dismissal or nonpayment reinforce compliance with requester-defined interpretations of data.Requesters translate business requirements into labels, while mismatches with their guidelines are treated as “noise.”Source: §4.3.2 · pp. 21–22

4.3.3 Worker Solidarity and Organization.

  • Workers organize to improve conditions and share task advice, bug warnings, and Spanish-language guides that help peers interpret instructions.At Alamo, worker demands preceded fixed salaries and benefits; workers were permitted to work remotely in 2020.Source: §4.3.3 · pp. 22–23

4.4 Dispositif’s Materializations

Documents, interfaces, and performance metrics materialize requester priorities in daily data work, shaping what workers can understand, label, and have recognized as quality.

4.4.1 Documents as Artifacts.

  • Requester-facing project records preserve instruction changes and document delivered work, while English-language instructions and contracts can remain unintelligible to Spanish-speaking workers.Workers reported relying on translation tools and signing some documents without understanding the full scope of their contractual relationship.Source: §4.4.1 · pp. 24–25

4.4.2 Work Interfaces.

  • Interfaces embed requester-defined taxonomies in fixed labels, limiting workers’ ability to add categories; timed scores also privilege speed over reflection.Workers could not change predefined labels on the studied platforms, while only one observed Alamo project let workers co-create a taxonomy.Source: §4.4.2 · pp. 25–26

4.4.3 Tools to Assess Worker Performance.

  • Metrics quantify quality against requester expectations, intensifying surveillance and standardization; platform algorithms can ban workers and jeopardize their income.The studied platforms assess workers through repeated data, previously labeled answers, and comparisons with peers’ answers.Source: §4.4.3 · pp. 26–28

5 DISCUSSION

The data-production dispositif combines requester-defined interpretations, workers’ constrained labor, and artifacts that normalize those interpretations. The authors propose making the ML pipeline visible, improving labor conditions, and enabling workers to question and shape tasks, while noting the study’s bounded scope.

5 DISCUSSION

  • Workers’ data labor and subjectivities are embedded in industrial structures; instructions, work practices, and artifacts together sustain a dispositif that produces compliant workers.The authors describe its demand as historically and geographically contingent and tied to the AI industry’s need for more, cheaper, differentiated data.Source: §5 · pp. 28–30

5.1 Implications

  • Requesters’ instructions encode unilateral worldviews, including discriminatory classifications, while workers’ precarious position can limit their ability to challenge them.The authors argue that dismantling the dispositif requires making normalized discourses explicit and changing data workers’ material conditions.Source: §5.1 · p. 30

5.1.1 Fighting Alienation by Making the ML Pipeline Visible to Workers.

  • Keeping workers apart from the ML pipeline contributes to alienation; sharing pipeline context and recognizing their technical contributions could help them understand their work’s value.Workers interviewed at the BPO said more information about requesters and products could help them understand their work and perform better.Source: §5.1.1 · pp. 30–31

5.1.2 Fighting Precarization by Considering Data Workers Assets.

  • The study links outsourced data work to precarious conditions and argues that workers’ quality of life matters to fair and equitable ML.Practitioners interviewed said platforms were cheaper than BPOs, while BPO teams’ continuity and direct communication could support higher data quality.Source: §5.1.2 · p. 31

5.1.3 Fighting Workers’ Surveillance and Control by Encouraging Interrogation.

  • Surveillance and the risk of losing tasks can discourage workers from questioning instructions, even when they notice errors or harmful omissions.The authors propose worker feedback and agency as ways to treat workers as dataset co-producers rather than reproduction tools.Source: §5.1.3 · pp. 31–32

5.2 Limitations and Future Research

  • The findings are bounded by the platforms, organizations, people, and geographic contexts studied, as well as the researchers’ positionality.The paper also leaves relationships with wider economic, scientific, academic, and tech-industry dispositifs for future research.Source: §5.2 · p. 32

6 CONCLUSION

The paper examines outsourced ML data work in Venezuela and Argentina through an adapted dispositif analysis. It finds that instructions, economic dependency, and constrained worker agency shape data production, and proposes making encoded worldviews explicit, valuing workers, and empowering them to produce better data.

6 CONCLUSION

  • The study applies an adapted Foucauldian dispositif analysis to outsourced ML data work in Venezuela and Argentina, using instructions, interviews, and field observations.It defines the data-production dispositif through discourses, work practices, and materializations reproduced in and through ML data work.Source: §6 · p. 33
  • The authors find that requester instructions impose predefined interpretations of data, while poverty and dependency leave workers with little option but to obey.Source: §6 · p. 33
  • The authors propose making instruction worldviews explicit, treating workers as assets, and empowering them to produce better data; they also call for decommodification and documentation to support reflection, deliberation, and audit.Source: §6 · p. 33
Loading 2205.11963v1…