Guiding Worker Self-Selection in Crowdsourcing Contests: An LLM-Augmented Algorithmic Approach

1 Introduction

• Workers choose contests and effort strategically, which can make the resulting effort distribution differ from the platform’s preferred distribution.Workers may also fail to reach a stable arrangement in which nobody wishes they had chosen differently.Source: §1 · p. 1
• The platform seeks overall effort while workers care about regret, the utility lost by entering a contest rather than their best alternative.The paper frames recommendations as a way to benefit both parties despite their differing interests.Source: §1 · pp. 1–2
• Earlier SSTC work studied special cases under restrictive assumptions, including identical worker attributes and participation in only one contest.The authors say these results do not provide a practical recommendation procedure.Source: §1 · p. 2
• GRAF orders workers by a score vector and iteratively assigns each worker to a contest and effort; in special SSTC cases it guarantees zero regret and platform optimality in polynomial time.Choosing the score vector is difficult when worker valuations, production efficiencies, and effort costs differ.Source: §1 · p. 2
• LLMScore evolves scoring algorithms for GRAF to balance platform utility and worker satisfaction despite costly regret evaluation, and produces human-readable code operators can inspect and adjust.It is trained on small instances of one setting and reused on larger, structurally different settings.Source: §1 · p. 2
• Across four settings, GRAF augmented by LLMScore consistently produces high-quality or near-optimal outcomes with low worker regret, including in settings it was never trained for.LLMScore was trained on small instances from one setting.Source: §1 · p. 2

2.1 Tullock Contest

• In the Tullock contest model, each contestant has a reward value and production function; winning probability depends on effort, and utility subtracts effort cost from expected reward.In the linear-production, linear-cost special case, the paper states a unique pure-strategy Nash equilibrium with effort ordered by contestant values.Source: §2.1 · pp. 2–3

2.2 Self-Selection in Tullock contests

• SSTC has workers first select contests and then compete; a pure self-selection strategy pairs a contest-selection vector with an effort vector.Effort is conditioned on the selection outcome, and regret measures the utility lost relative to the best alternative contest; zero regret implies a pure-strategy Nash equilibrium.Source: §2.2 · p. 3

2.3 Platform Objective

• The platform uses overall effort as a proxy for the quantity and quality of solutions, while recommendations respect workers’ contest demand.Without a closed-form equilibrium characterization, effort may require numerical computation.Source: §2.3 · p. 3, §2.2 · p. 3
• The platform’s effort objective is subject to utility constraints reflecting that contestants strategically choose where to participate.The paper also motivates overall effort as a measure of broad activity and many high-quality solutions, rather than only the winning solution.Source: §2.3 · pp. 3–4

3 GRAF: A Greedy Algorithmic Framework for SSTC

• GRAF orders contestants by a score vector, then commits each to a best-response contest until their participation demands are met.At each step, it computes a contestant’s best-response set given earlier contestants’ choices.Source: §3 · p. 4

3.1 Zero-Regret Contestants

• Under homogeneous contest type and rewards, single-contest participation, and linear production and effort costs, Algorithm 2 returns a PSNE in polynomial time.A PSNE is a no-regret self-selection outcome.Source: §3.1 · p. 4

3.2 Platform Optimality

• In uniform SSTC with n≥2m contestants, every PSNE achieves optimal overall effort (OE).This platform-optimality guarantee applies to uniform values and the stated contestant-to-contest ratio.Source: §3.2 · p. 4

4 LLMScore: An LLM-Based Framework to Augment GRAF

• Because contestants may have different values and contest parameters, choosing the order σ for GRAF is nontrivial; LLMScore is an LLM-based framework that automates σ’s design to augment GRAF.Source: §4 · p. 4
• LLMScore evolves a population of scoring algorithms that map SSTC instances to score vectors, seeking better trade-offs between overall effort and regret.The LLM generates initial algorithms and offspring through exploration or modification strategies; the population is updated by fitness.Source: §4 · p. 5
• Each scoring algorithm has a natural-language description and a code implementation formatted for integration into the evolutionary framework.The experiments implement the code as a Python function, though the paper says other programming languages could work.Source: §4 · p. 5
• When top-algorithm fitness stagnates, LLMScore asks the LLM for a new modification prompt strategy and retains high-scoring strategies.A strategy’s score is the average fitness of the top individuals it produced.Source: §4 · p. 5
• Fitness uses median overall effort and penalizes algorithms whose maximum regret exceeds a baseline’s in enough training instances.The median is used because extreme outliers occur frequently across instances; tied fitness values are sorted by median maximum regret.Source: §4 · pp. 5–6

5.1 Experimental Setups

• Scores trained only on 50 small SSTC 3 instances (n=10, m=3) transfer to larger and structurally different settings without retraining.The experiments test four settings, with 1,000 instances for each combination of contest and contestant counts.Source: §5.1 · p. 6, §5.2 · p. 6

5.2 Results

• Across nearly all characterized settings, GRAF[σLLM] beats handcrafted GRAF scores in median OE and median max regret, and exceeds IBR in OE in several larger settings.IBR performs best in SSTC 1, while its performance gap over GRAF-based methods narrows in SSTC 2.Source: §5.2 · p. 6, Table 1 · p. 7
• GRAF[σLLM] outperforms all baselines, including IBR, in both median OE and median max regret across SSTC 3 and SSTC 4 settings and instance sizes.Methods with identical results are condensed in Table 2.Source: §5.2 · p. 6, Table 2 · p. 7
• The learned scoring algorithms are substantially more sophisticated than the handcrafted scores used as baselines.The analysis examines examples from the best-OE and best-regret algorithms.Source: §5.2 · pp. 6–7
• On SSTC 4 instances, one IBR run takes at least 10× as long as GRAF[σID] or GRAF[σLLM], while their test-time costs are almost identical.LLMScore training is conducted offline and amortized over future test instances.Source: §5.2 · p. 7

Crowdsourcing and Worker Self-Selection

• The paper takes contests as given and studies recommendations for where heterogeneous workers compete, jointly considering platform value and worker satisfaction.Prior crowdsourcing work studies task recommendation and assignment, incentive design, and LLM use for performing crowdsourcing tasks.Source: §6 · pp. 7–8

Self-Selection in Tullock Contests

• Prior SSTC analyses offer partial equilibrium characterizations under restrictive assumptions, including like-minded contestants and limited contest selection.The cited comparison notes assumptions on production and cost functions; those prior works did not conduct empirical studies.Source: §6 · p. 8, Table 3 · p. 8

Learning Nash Equilibria

• Nash-equilibrium learning methods target mixed-strategy equilibria in games with at most five players and generally lack interpretability.Source: §6 · p. 8

LLMs for Algorithmic Design

• Prior LLM algorithm-design work includes evolving mathematical functions, heuristics, and agentic systems; this paper adapts the approach to jointly handle effort and regret and reuse learned algorithms across settings.The authors identify combining overall effort and regret, and training on small instances of one setting, as responses to their two challenges.Source: §6 · p. 8

7 Conclusion

• GRAF has guarantees of zero worker regret and platform optimality in special SSTC cases, while LLMScore yields high-quality or near-optimal outcomes with low dissatisfaction in general settings.LLMScore produces human-readable code that platform operators can inspect, audit, and adjust.Source: §7 · p. 8
• The evaluation uses synthetic utility-maximizing agents, so regret and overall effort are proxies rather than measurements of real worker dissatisfaction and solution quality.The authors call for calibrating instances against real platform data and validating recommendations with crowdworkers.Source: §7 · p. 8
• The effort solver may return local optima, general SSTC equilibrium existence remains open, and index-dependent scoring can change recommendations after relabeling.The authors suggest canonicalizing the raw contestant index as a possible route to permutation equivariance.Source: §7 · p. 8

Ethical Considerations

• All experiments use synthetic SSTC instances, with no human subjects, personal data, or live platform interactions.Source: Ethical Considerations · p. 9
• The authors frame low-regret recommendations as a way to reduce worker dissatisfaction, not to extract more effort against workers’ interests.Source: Ethical Considerations · p. 9
• Any deployment should respect worker autonomy: recommendations are advisory, constrained by each worker’s demand k_i, and should be validated against real behavior.Source: Ethical Considerations · p. 9

Listing S1: Code template

• The MN1 example adjusts valuation by player rank and applies a rank bonus when computing priority scores.Its prompt describes balancing incentives through rank-based utility adjustment; the code uses an assigned effort and winning-probability and cost terms.Source: Listing S1: Code template · p. 13
• The code template defines valuation, production-probability, and cost inputs, with production and cost functions expressed using contestant-specific parameters.Production is Q[i,j] times effort raised to R[i,j], while cost is C[i,j] times effort raised to D[i,j].Source: Listing S1: Code template · p. 13
• The M1 description simplifies priority scoring by considering maximum competing production and logarithmically scaling winning probability and cost.The associated passages define the competition effect using the maximum production of another contestant under an index-based effort assignment.Source: Listing S1: Code template · p. 14

• Algorithm 2 outputs a pure-strategy Nash equilibrium in polynomial time, using induction to show each added contestant preserves equilibrium.The proof checks that neither contestants already in the chosen contest nor contestants outside it want to deviate.Source: §A.1 · pp. 19–20
• In uniform SSTC with n ≥ 2m, every pure-strategy Nash equilibrium achieves optimal overall effort.Source: §A.1 · p. 20
• In uniform SSTC, equilibrium overall effort is zero when n ≤ m, and when n > m contest sizes differ by at most one.With n ≤ m, contestants can each select a different contest and obtain their highest utility without exerting effort.Source: §A.1 · pp. 20–21
• For 2 ≤ n ≤ 2m, an optimal allocation gives every contest at least two contestants; diminishing returns favor assigning contestants to less-populated contests.A singleton contest produces zero effort, while per-contest effort has diminishing returns as contestant count rises.Source: §A.1 · p. 21

• Experiments used GPT-4o mini on an Ubuntu 20.04 Linux virtual machine with an NVIDIA GeForce RTX 3050 Ti GPU and Intel i7-12700H CPU.The authors state that the full Python 3.10 implementation accompanies the submission as supplementary material.Source: §A.2 · p. 21
• LLMScore used three runs and averaged median overall effort and median maximum regret across 1,000 instances, with a population of 20 and a 180-second per-instance limit.The reported settings also include a 20-population evaluation resource and a 0.3 regret-penalty threshold.Source: §A.2 · p. 21
• For SSTC 3 and 4, approximate effort uses best-response dynamics with max_iter 100 and tolerance 10^-6, while Brent’s algorithm approximates individual effort.The effort procedure initializes efforts to ones and uses damping coefficient 1.Source: §A.2 · p. 22
• Iterated best response averages performance over 10 random starts and averages actions within a detected cycle; LLM-generated strategies are accepted only when at least one algorithm is valid.Validity excludes timeouts and coding errors.Source: §A.2 · pp. 21–22