PaperlayerView exact revision on arXiv

Source-linked AI summary

The Fundamentals of Policy Crowdsourcing

John Prpic, Araz Taeihagh, James Melton

arXiv:1802.04143v1cs.CYcs.HCcs.SI

TL;DR

Research had yet to systematically investigate the state of crowdsourcing in policy making. The article reviews and organizes that literature using a framework combining policy-cycle and crowdsourcing typologies, finding a sparse, uneven body of research with open collaboration prominent. It identifies gaps in the literature as opportunities for further research and possible policy applications.

  • Problem

    Research had not yet systematically examined crowdsourcing and policy making together, despite its emerging use in policy contexts.

    Source: Introduction · p. 2

  • Method

    The article reviews policy-crowdsourcing research and combines typologies from crowdsourcing and policy research to organize it across techniques and policy stages.

    Source: Introduction · pp. 2–3

  • Results

    The review identifies 83 works, with research sparse and unevenly distributed across policy-cycle stages, predominantly focused on open collaboration while virtual labor markets and tournaments are relatively ignored.

    Source: Conclusion · pp. 14–15, Framework Analysis · p. 14

  • Takeaways & Limitations

    The identified research voids are useful opportunities for future study and may also offer policy-makers and researchers opportunities to pioneer crowdsourcing efforts.

    Source: Conclusion · p. 15

    The review excludes crowdsourcing used by politicians for vote-getting, while including public-administration uses such as soliciting ideas relevant to legislation.

    Source: Policy Crowdsourcing Framework · p. 10

Abstract

from arXiv · show

What is the state of the research on crowdsourcing for policy making? This article begins to answer this question by collecting, categorizing, and situating an extensive body of the extant research investigating policy crowdsourcing, within a new framework built on fundamental typologies from each field. We first define seven universal characteristics of the three general crowdsourcing techniques (virtual labor markets, tournament crowdsourcing, open collaboration), to examine the relative trade-offs of each modality. We then compare these three types of crowdsourcing to the different stages of the policy cycle, in order to situate the literature spanning both domains. We finally discuss research trends in crowdsourcing for public policy, and highlight the research gaps and overlaps in the literature. KEYWORDS: crowdsourcing, policy cycle, crowdsourcing trade-offs, policy processes, policy stages, virtual labor markets, tournament crowdsourcing, open collaboration

Introduction

Research on crowdsourcing has not yet systematically examined its application across policy making. The article addresses this gap by combining typologies from crowdsourcing and policy research into a framework for organizing the literature and identifying trends, gaps, overlaps, and trade-offs.

Introduction

  • Research had not yet systematically investigated crowdsourcing and policy making together, despite crowdsourcing’s growing use in non-commercial and policy contexts.Examples cited include transportation and urban planning.Source: Introduction · p. 2
  • The article merges fundamental typologies from the crowdsourcing and policy literatures into a framework for studying policy crowdsourcing.The framework is used to organize existing research, illustrate trends, identify gaps, and outline trade-offs.Source: Introduction · pp. 2–3
  • The framework situates an extensive body of policy-crowdsourcing literature and offers a systematic approach to investigating the topic.The article also identifies gaps and overlaps in the extant research.Source: Introduction · p. 3

Crowdsourcing

Crowdsourcing uses IT to draw on dispersed knowledge for problem solving, idea generation, and production. The article distinguishes virtual labor markets, tournament crowdsourcing, and open collaboration, which differ in how participants contribute and organize.

Crowdsourcing

  • Crowdsourcing uses IT to engage dispersed groups and individuals in problem solving, idea generation, and production for organizations.Its processes combine top-down efficiency and control with bottom-up innovation and creativity.Source: Crowdsourcing · p. 3

Virtual Labor Marketplaces (VLMs)

  • Virtual labor markets offer paid spot labor, typically assigning workers independent microtasks that can be completed rapidly at massive scale.Workers are generally anonymous, and microtasks often involve human computation such as tagging photos or transcribing audio.Source: Virtual Labor Marketplaces (VLMs) · pp. 3–4

Tournament Crowdsourcing (TC)

  • Tournament crowdsourcing solicits solutions through competitions, often drawing specialized crowds and offering fixed prizes for the best submissions.Some platforms require independent entries, while others allow or encourage teams.Source: Tournament Crowdsourcing (TC) · p. 4

Open Collaboration

  • Open collaboration invites voluntary public contributions through IT, with scale and participation depending on reach and engagement.Participants have greater scope to self-organize and may use their own networks to alter or amplify an organization’s agenda.Source: Open Collaboration (OCs) · p. 5

The Policy Cycle

The policy cycle treats policy making as an iterative sequence of stages, while public policy concerns interrelated decisions about goals and means. This staged framework builds on earlier efforts to analyze political systems and policy processes.

The Policy Cycle

  • Public policy is framed as interrelated decisions by political actors about selecting goals and the means to achieve them within their power.The article also describes policy as effective and acceptable courses of action directed toward explicit goals.Source: The Policy Cycle · p. 5
  • Lasswell’s staged analysis of political systems described inputs such as demands and support being converted into decisions and actions.This approach was later extended to policies.Source: The Policy Cycle · p. 5
  • The policy cycle comprises six iterative stages: agenda setting, problem definition, policy design, implementation, enforcement, and evaluation.The article’s cycle is based on efforts by Stone and Howlett and colleagues.Source: The Policy Cycle · p. 5, Figure 1 · p. 6

Comparison of Crowdsourcing Techniques

The article compares virtual labor markets, tournament crowdsourcing, and open collaboration across seven characteristics to describe their relative trade-offs. The comparison is a general synthesis rather than an exhaustive or definitive classification.

Comparison of Crowdsourcing Techniques

  • The comparison extends prior work from three characteristics to seven: cost, anonymity, crowd scale, IT structure, implementation time, task magnitude, and reliability.The authors describe the categories as a minimum general consensus, noting that applications may be hybrids.Source: Comparison of Crowdsourcing Techniques · p. 6, Table 1 · p. 10
  • Open collaboration is essentially free in direct IT costs, while tournaments generally have fixed prize costs and virtual labor markets incur variable per-task costs.Tournament and VLM use through intermediaries can also involve service-provider costs; anonymity ranges from generally linked identities in open collaboration to pseudonymous VLM identifiers.Source: Cost · p. 6, Anonymity · p. 7
  • Virtual labor markets generally offer millions of potential workers, tournament crowds are smaller, and open-collaboration scale varies by platform or organization.Crowd size sets a maximum on potential contributors and can constrain the quantity and speed of contributions; VLMs generally use episodic IT, open collaboration collaborative IT, and tournaments vary.Source: Scale of Crowd Size · p. 7, IT Structure · p. 8
  • Virtual labor markets can begin gathering contributions within minutes, while tournaments require contest design and a set duration.Open collaboration can be nearly immediate with an established audience or very slow to build, and voluntary contributions are not guaranteed.Source: Time to Implement the Form of… · p. 8
  • Virtual labor markets typically elicit microtasks, tournaments seek complete solutions, and open collaboration can span both task scales.All forms require pre-task preparation and post-task processing to generate value from contributions.Source: Task Magnitude · pp. 8–9
  • Tournament crowdsourcing is considered most reliable, virtual labor markets medium-reliability, and open collaboration variable by implementation.The authors characterize their comparison and assigned values as neither exhaustive nor definitive.Source: Reliability of the Crowd · p. 9, Summary · p. 9

Policy Crowdsourcing Framework

The authors combine crowdsourcing forms with policy-cycle stages to map existing policy-crowdsourcing research. The mapped literature is sparse and uneven, particularly across forms and stages.

Policy Crowdsourcing Framework

  • The framework combines three crowdsourcing forms with policy-cycle stages, and the team coded 83 selected works into one form and one stage.The resulting table situates the literature across both typologies.Source: Policy Crowdsourcing Framework · pp. 10, 12, Table 2 · p. 12
  • The review distinguishes policy crowdsourcing from electoral vote-getting, social media used only for information or service delivery, crisis mapping, and crowdfunding.Social media counts as crowdsourcing only when explicit resources are generated from the crowd through the IT used.Source: Policy Crowdsourcing Framework · pp. 10–11
  • The authors distinguish crowdsourcing from adjacent concepts by its IT-mediated focus, while noting differences in online/offline scope and level of analysis.They state that a detailed comparison of those concepts is beyond the article’s scope.Source: Policy Crowdsourcing Framework · p. 11
  • From 189 collected works, 83 remained after filtering; the search review stopped when three consecutive pages yielded no new relevant works.The research team reviewed the first 540 of 19,200 search results, including patents and citations.Source: Policy Crowdsourcing Framework · p. 12
  • Most reviewed policy-crowdsourcing research concerns open collaboration, while virtual labor markets and tournament crowdsourcing are relatively ignored.The authors note that open collaboration’s dominance merits further investigation, while reasons for the limited research on the other forms remain unclear.Source: Framework Analysis · p. 14
  • The authors suggest that VLMs’ limited research presence may reflect low adoption, low visibility, or ethics and national data-sovereignty concerns.They present these as possible explanations, not established causes.Source: Framework Analysis · p. 14
  • Case studies and other phenomenon-based methods dominate the reviewed literature, while experiments have been scarcely used.The authors describe experiments as a viable, relatively low-cost way to address identified research gaps.Source: Framework Analysis · p. 14

Conclusion

The review identifies 83 works on policy crowdsourcing and organizes them in a framework combining crowdsourcing forms and policy-cycle stages. It finds uneven coverage and unstudied areas, and sets out a broader basis for comparing crowdsourcing forms.

Conclusion

  • The review identifies 83 works and finds research on all three crowdsourcing forms, but coverage is far from evenly distributed across forms and policy-cycle stages.The authors describe the policy-crowdsourcing literature as a rapidly emerging interdisciplinary area.Source: Conclusion · pp. 14–15
  • The framework reveals numerous gaps where no research exists, though the authors say it is unclear whether these reflect research omissions or absent real-world applications.They describe the gaps as opportunities for researchers and policy-makers, while presenting that potential cautiously.Source: Conclusion · p. 15
  • The authors extend general crowdsourcing research by detailing seven dimensions for comparing its three generalized forms.They also bring crowdsourcing research into the policy domain as an emerging context for application.Source: Conclusion · p. 15
Loading 1802.04143v1…