PaperlayerView exact revision on arXiv

Source-linked AI summary

The Limits of Manipulation within Personalized Incentive Policies

Andrea Araldo, Alessa Mayer, Ravi Seshadri, Amir Brudner

arXiv:2610.13133v1cs.MA

TL;DR

Strategic manipulation is a critical vulnerability for personalized demand management, while existing AI work on strategic behavior is not behaviorally grounded for demand-side mitigation. The paper models repeated regulator-agent interaction with profile learning and finds that the manipulation-gain bound tightens when profiles are harder to distinguish.

  • Problem

    AI models of strategic behavior are not behaviorally grounded for demand-side mitigation, leaving manipulation in personalized demand management insufficiently addressed.

    Source: §1 · p. 1

  • Method

    The paper models a repeated Stackelberg game in which the regulator learns agent profiles from observed choices while agents may imitate another profile to obtain higher incentives.

    Source: §1 · p. 1, §3 · p. 3, §5 · p. 7

  • Results

    The manipulation-gain upper bound tightens as profiles become harder to distinguish, and travel-choice-calibrated simulations show the resulting deterrence improves policy effectiveness.

    Source: §7 · p. 7

  • Takeaways & Limitations

    The analysis can serve as a pre-deployment test to help regulators assess personalized policies’ robustness to manipulation.

    Source: §7 · p. 7

    The analysis assumes boundedly rational agents manipulate by consistently imitating another profile and focuses on agents whose true profile is 0.

    Source: §4.2 · p. 5

Abstract

from arXiv · show

Demand management policies aim to shift individual behavior toward sustainable choices (e.g., goods, transport modes). Personalized incentive policies tailor subsidies to agents' needs and constraints, but personalization also exposes the regulator to agents' strategic manipulative behavior. The extent to which agents can profitably manipulate such policies remains poorly understood, which limits their applicability. This paper establishes analytically the conditions under which manipulation benefits the agent. We formalize a Stackelberg game in which an agent may simulate a different profile to extract higher incentives from a regulator. Previous work typically assumes that the regulator would instantaneously classify agents into the simulated user profile. In contrast, we explicitly model the regulator's process of learning the agent profile, based on observed past behavior. This makes manipulation harder, since it forces the agent to consistently behave according to a simulated profile, thus renouncing following their true preferences. Manipulative agents thus incur a Cost of Manipulation, which has typically been neglected. By accounting for it, we get a counterintuitive finding: although classification uncertainty reduces personalization accuracy, it may benefit the regulator by deterring manipulation. We establish an upper bound on the agent's gain from manipulation, which becomes stricter as profiles become harder to distinguish for the regulator. Simulations on the London Passenger Mode Choice dataset illustrate this deterrence effect. The framework can therefore be used to stress-test the robustness of personalized incentive policies before deployment.

1 Introduction

Personalized demand management may make sustainable behavior policies more effective and acceptable, but agents can manipulate tailored incentives. This paper models repeated regulator-agent interaction and finds that preference elicitation can deter manipulation.

1 Introduction

  • Personalized policies may reduce inequitable impacts by targeting agents for whom behavioral shifts are less costly, improving effectiveness and acceptability.Negative signals can be unpopular and inequitable, while positive incentives may be costly or too weak to induce change.Source: §1 · p. 1
  • Personalized policies are vulnerable to agents concealing their preferences to obtain favorable signals, as illustrated by workers reducing labor supply to qualify for under-employment subsidies.The cited example is Firpo et al. 2014.Source: §1 · p. 1
  • The paper models a behaviorally grounded Stackelberg game in which a regulator learns agent profiles from observed choices while agents may simulate preferences for higher incentives.It models choices using Random Utility Theory and explicitly represents regulator preference elicitation rather than simple profile reporting.Source: §1 · p. 1
  • The analysis finds that realistic preference elicitation deters manipulation, countering prior results that agents could greatly manipulate regulators.The authors report that personalized incentives can consequently reduce social cost, with results validated in Monte Carlo simulations.Source: §1 · pp. 1–2

2 Related Work

Prior work often treats manipulation as easy strategic misreporting or permits behavior that is not economically and behaviorally grounded. This paper distinguishes its setting through learned profiles, constrained agent behavior, and demand-management incentives.

2 Related Work

  • Unlike prior strategic-reporting models, the regulator sets incentives before the agent chooses, making this interaction a Stackelberg game.In Adversarial Interaction Attacks, the defender and attacker interact through resource allocation and attack choices; here, the regulator leads by setting incentives for alternatives.Source: §2 · p. 2
  • Prior models let followers gain through strategic reporting, often by having the leader immediately accept a fake profile; Bayesian persuasion similarly allows agents to choose what information to report (Gan et al. 2019a,b; Nguyen 2019; Birmpas et al. 2020; Dawkins, Han, and Xu 2021; Kamenica and Gentzkow 2011; Jain and Perchet 2024).The paper characterizes this instantaneous preference elicitation assumption as unrealistic and too pessimistic for the leader.Source: §2 · p. 2
  • Instead of instant profile acceptance, the regulator learns profiles from observed choices; unlike screening contracts, alternatives are exogenous and stochastic choices do not perfectly reveal profiles (Chapman and Fisher 2025; Einav et al. 2022).The paper calls these settings realistic and instantaneous preference elicitation, respectively.Source: §2 · p. 2
  • Unlike Sessa et al. (2020), agents commit to one RUT-consistent fake profile and maximize utility rather than act adversarially, while incentives are deterministic rather than probabilistic.In Sessa et al., agents adversarially select a profile at each timeslot and the leader uses probabilistic policies.Source: §2 · p. 2
  • Matching work assigns manipulation to returning agents, and auction work to leaders or buyers; neither captures agents manipulating a regulator in demand-side management (Ionescu et al. 2023; Nedelec et al. 2020).In the auction setting, leaders can manipulate the seller, unlike the follower-agent manipulation studied here.Source: §2 · p. 2
  • Performative Prediction changes classifier parameters to reduce classification loss, unlike this regulator’s incentive-setting for social cost under transparent, fixed MLE classification rules.The paper argues that changing classification rules over time would conflict with public-sector transparency constraints.Source: §2 · p. 2
  • A VCG-like mechanism is unsuitable because it maximizes summed agent utility, whereas this regulator minimizes social cost tied to sustainability goals such as emissions.VCG aims to ensure truthful agent behavior while maximizing social welfare.Source: §2 · p. 2
  • The paper frames its contribution as addressing prior settings that are either too simplistic, allowing simple misreporting, or too pessimistic to ground manipulation economically and behaviorally.This gap motivates modeling realistic learning and constraints on manipulative behavior.Source: §2 · p. 3

3 Model

The model treats personalized incentives as a repeated regulator–agent game in which the regulator encourages lower-externality choices while learning the agent’s profile from past behavior. Agent choices follow a profile-dependent random-utility model, and the regulator updates its inferred profile and incentive over time.

3 Model

  • In the repeated Stackelberg game, the regulator proposes an incentive and the agent chooses among alternatives ordered from lower to higher externality.Applications include encouraging second-hand clothing, public transport, or renewable electricity over higher-externality alternatives.Source: §3 · p. 3

3.1 Random Utility Modeling

  • Choices follow a profile-dependent logit model, combining systematic utility with Gumbel random noise to represent unpredictable behavior and bounded rationality.Profiles encode preferences over alternative attributes; the difference ak − bk represents responsibility intensity, or inherent preference for the more sustainable alternative.Source: §3.1 · pp. 3–4

3.2 Interaction between Regulator and Agent

  • Each period, the regulator infers the agent’s profile from past incentives and choices, then selects an incentive to minimize estimated generalized social cost.The agent then chooses probabilistically, and receives the incentive only when choosing alternative 1; the regulator estimates social cost using an inferred utility function.Source: §3.2 · p. 4, §3.1 · p. 4

4 Analysis

The analysis derives optimal incentives and examines when a responsible agent benefits from imitating another profile. It bounds manipulation gains and gives a condition for deterring imitation.

4.1 Optimal Incentives

  • For an inferred profile, the optimal incentive is unique and non-increasing with responsibility intensity.The proposition also supports ordering the proposed incentives across profiles.Source: §4.1 · p. 4
  • The optimal incentive is upper-bounded; the bound increases with externality c¬1 and decreases with social cost c1.The paper links this comparative statics result to the social cost avoided by encouraging alternative 1.Source: §4.1 · p. 4

4.2 Agent’s Strategic Behavior

  • The analysis considers a responsible agent who may consistently imitate another profile, trading shifted incentive probabilities against loss of true utility.It defines gain from manipulation and cost of manipulation relative to honest behavior.Source: §4.2 · p. 5
  • Theorem 8 bounds manipulation gains, while the analysis tracks how imitation changes the inferred-profile and proposed-incentive distributions.The elicitation and personalization intensities measure changes to those distributions when an agent simulates another profile.Source: §4.2 · pp. 5–6
  • Manipulation potential rises with personalization intensity and interaction count, but its growth is at most linear; gains are also bounded.Theorem 9 states that increased personalization raises per-timeslot manipulation gains and that the growth in potential is at most linear with interactions.Source: §4.2 · p. 6
  • Corollary 10 gives a timeslot condition under which imitating profile r is deterred.Manipulation is beneficial only when its gain exceeds its cost.Source: §4.2 · p. 6

5 Impact of Preference Elicitation

Preference elicitation limits manipulation by making it harder to establish a simulated profile and reducing expected utility gains; under instantaneous elicitation, manipulation has no cost.

5 Impact of Preference Elicitation

  • Preference elicitation makes it harder to convince the regulator of an adopted profile r > 0 and substantially reduces expected utility gained from manipulation.Source: §5 · p. 7
  • Instantaneous Preference Elicitation immediately accepts the simulated utility profile, allowing the agent to maximize expected utility while behaving according to that profile.This model is implicitly assumed in previous AI works (Dawkins, Han, and Xu 2021; Birmpas et al. 2020; Gan et al. 2019a,b; Nguyen 2019).Source: §5 · p. 7
  • Realistic Preference Elicitation updates the regulator’s belief from observed choices over time, requiring repeated simulation that may incur costs and interim misclassification.The agent may have to choose alternatives they do not prefer to establish the simulated profile.Source: §5 · p. 7
  • Under InstPE, the cost of manipulation is null, and its gain is the difference between expected true utilities under the incentive associated with the true profile.Source: §5 · p. 7

6 Case Study: Travel-Mode Incentives

The London travel-mode case study compares realistic and instantaneous preference elicitation in Monte Carlo simulations. Under realistic elicitation, manipulation is deterred, while the instantaneous setting reverses the reported utility pattern.

6 Case Study: Travel-Mode Incentives

  • The simulations use London passenger mode-choice data and model stochastic agent choices, regulator profile inference, and incentive calculation across multiple seeds.The calibrated profiles corresponded to preferences for biking, public transport, and car, ordered by responsibility intensity.Source: §6 · p. 7
  • Under RealPE, manipulation costs exceed gains and honest behavior yields higher time-averaged true utility; under InstPE, the utility pattern is inverted.The study presents this contrast as evidence that realistic preference elicitation deters manipulation.Source: §6 · p. 7, Figure 2 · p. 7, Figure 1 · p. 7
  • The case study evaluates induced changes in cycling trips as evidence of the potential for personalized incentives to foster sustainable behavior.Figure 2b reports the change in the percentage of cycling trips, but the supplied passages give no numerical value or direction for that change.Source: §6 · p. 7, Figure 2 · p. 7

7 Conclusion · SUPPLEMENTAL MATERIAL:APPENDIX · A Table of notation (2 pages)

The conclusion presents an upper bound on gains from manipulation that tightens when profiles are harder to distinguish. Simulations calibrated on observed travel choices demonstrate a deterrence effect, and the analysis can help regulators test policy robustness before deployment.

7 Conclusion

  • The upper bound on manipulative agents’ gains tightens as the regulator’s ability to distinguish profiles decreases.The paper characterizes this as a deterrence effect.Source: §7 · p. 7
  • Simulations calibrated on observed travel choices show that the deterrence effect improves policy effectiveness.Source: §7 · p. 7
  • The analysis can serve as a pre-deployment test of a personalized incentive policy’s robustness to manipulation.Source: §7 · p. 7

A Table of notation (2 pages)

B Proofs · Proof of Lemma 1

The proof derives a utility representation under quasi-linearity with respect to income, then models the agent’s choice between alternative 1 and the best remaining alternative. It also explains why the utility term for alternatives i > 1 need not include income.

Proof of Lemma 1

  • Under quasi-linearity with respect to income, the utility function can be written in the representation stated by Lemma 1.Source: Proof of Lemma 1 · p. 11
  • For alternative 1, systematic utility is uk(1, y) = αk·xi+y, and thus uk(1, y) = ak+y.Source: Proof of Lemma 1 · p. 11
  • If the agent does not choose alternative 1, they choose the best among the remaining alternatives i > 1.Source: Proof of Lemma 1 · p. 11
  • For alternatives i > 1, the agent’s random utility is uk(i, y) + ϵi, with ϵi Gumbel-distributed at scale parameter θ.Source: Proof of Lemma 1 · p. 11
  • When choosing among alternatives i > 1, assuming the agent chooses the best among them is reasonable (Strzalecki 2025, §1.8).Source: Proof of Lemma 1 · p. 11
  • The last equality in the expected-utility derivation follows Strzalecki 2025, Prop. 3.6; bk need not specify y because alternatives i > 1 receive no incentive.Source: Proof of Lemma 1 · p. 11
  • The proof then uses θ to rewrite probability (1).Source: Proof of Lemma 1 · p. 11

Proof of Lemma 2 … D1 Dataset and Calibration Procedure

The appendix establishes conditions for unique optimal incentives and describes how the numerical experiments infer profiles from binary observations of whether the incentivized alternative was chosen.

Proof of Lemma 2

  • Expected externalities and incentives are computed using total expectation, since the incentive changes only the utility of alternative 1.The conditional distribution among alternatives i > 1 does not depend on the incentive.Source: Proof of Lemma 2 · p. 11
  • The estimated generalized social cost has a unique minimum: it is the positive root of gk when condition (40) holds, and zero otherwise.When the root is non-positive, the cost increases over the feasible nonnegative incentive range, so its minimum is at zero.Source: Proof of Lemma 2 · pp. 11–12

Proofs of Proposition 3 and Corollary 4

  • Each profile has a unique optimal incentive, characterized using the Lambert W-function.The supplied passages state the characterization but do not include the full formula.Source: Proofs of Proposition 3 and Corollary 4 · p. 12
  • The optimal incentive is non-increasing in ak − bk and becomes zero when that difference is sufficiently large.The passages link the zero-incentive case to condition (40), but do not specify its threshold.Source: Proofs of Proposition 3 and Corollary 4 · p. 12

Proof of Proposition 5

  • Proposition 5 states that the optimal incentive is upper-bounded.The supplied passage does not show the bound’s value.Source: Proof of Proposition 5 · p. 13

C Preference Elicitation via a Maximum-Likelihood Estimator

  • For online profile elicitation, the regulator observes whether alternative 1 was chosen, aggregating every other choice into a single binary outcome.The passages motivate this reduced observation as limiting data collection to what is needed to determine incentive eligibility.Source: C Preference Elicitation via a… · p. 13
  • The online likelihood uses Bernoulli observations conditional on each candidate profile; deterministic incentives add no likelihood factor.The passages distinguish this binary online likelihood from the multinomial likelihood used for offline calibration.Source: C Preference Elicitation via a… · p. 13
  • Among the profiles retained by the inference rule, the regulator selects one with the smallest optimal incentive, breaking ties by choosing the smallest profile index.The supplied passage describes this as the deterministic inference rule used in the numerical experiments.Source: C Preference Elicitation via a… · p. 13

D1.1 Data · D1.2 Profile-based choice model

The sections describe preparation of the London Passenger Mode Choice trip data and the profile-based choice model’s trip-specific utilities and choice probabilities.

D1.1 Data

  • The London Passenger Mode Choice dataset combines trip diaries from London.Source: D1.1 Data · p. 13
  • Recorded quantities include driving-licence status, household car ownership per person, trip distance, and bus and rail portions of PT in-vehicle time; trips missing required quantities are discarded.Total PT time sums access, bus, rail, and interchange time.Source: D1.1 Data · p. 14
  • Dcal contains 34,198 trips from 20,577 travelers, and Dsim contains 14,656 trips from 11,709 travelers after separate 70/30 splits by bike, PT, and car trips.The split preserves the proportions of all three modes in both sets.Source: D1.1 Data · p. 14

D1.2 Profile-based choice model

D1.3 Calibration procedure … D1.6 Optimal incentives and induced mode choices

The appendix calibrates profile-specific mode-choice behavior against observed trips, then uses the calibrated profiles and externality inputs to set the policy weight and optimal incentives.

D1.3 Calibration procedure

  • Calibration minimizes weighted average negative log-likelihood with a cross-profile dispersion penalty, while adjusting memberships to assign one third of travelers to each profile.The regularization weight is ρ = 0.01; equal profile sizes prevent calibration from favoring one profile.Source: D1.3 Calibration procedure · p. 15
  • The calibrated bike-oriented profile’s preference gap was £0.36, above the required boundary, which represents a 1.02-percentage-point preference over indifference.The boundary identifies a strictly bike-oriented profile without imposing a large behavioral separation.Source: D1.3 Calibration procedure · p. 16
  • The fitted profile model achieved 0.67 nat/trip versus 0.83 for the aggregate-share baseline and 1.10 for uniform choice, assigning 7.38 and 17.79 percentage points more probability, respectively, to observed choices.These results indicate that the profile-specific utilities capture systematic variation in mode choice beyond aggregate mode frequencies.Source: D1.3 Calibration procedure · p. 16

D1.4 Calibrated parameter values, profile construction and context parameters

  • Simulation profiles are constructed from membership-weighted representative trip attributes, with profile memberships inferred at higher confidence exerting greater influence; profiles are ordered by a0 − b0.The representative attributes include travel time, out-of-pocket cost, PT interchanges, driving-licence status, and household car ownership.Source: D1.4 Calibrated parameter values,… · pp. 16–17

D1.5 Externalities and calibration of the policy weight

  • Representative car externality includes carbon, congestion, and other road costs, while PT emissions use distance and a bus/rail factor weighted by the bus-time share.Bike is assigned zero direct operational emissions.Source: D1.5 Externalities and calibration of… · p. 17
  • The policy weight β is set by anchoring profile k = 2’s optimal incentive to a cycling field-experiment rate of 2.00 NOK per kilometre, applied to a mean trip distance of 5.41 km/trip.Source: D1.5 Externalities and calibration of… · p. 18

D1.6 Optimal incentives and induced mode choices

  • Table 8 reports zero-incentive mode probabilities, mode probabilities under each profile’s optimal incentive, and the resulting change in bike probability.Calibrated behavioral parameters, representative attributes, externalities, and β determine the optimal incentive for each profile.Source: D1.6 Optimal incentives and induced… · p. 18

D1.7 Monte Carlo simulation

The Monte Carlo simulation compares manipulation under instant and behavior-based profile learning over an 18-timeslot initial-learning horizon. At that horizon, finite-horizon estimates have the same signs as the analytic long-run averages.

D1.7 Monte Carlo simulation

  • The experiment uses 100 independent replications of 18 timeslots to examine initial learning, when the regulator has fewer than 20 choices for inferring the adopted profile.Source: D1.7 Monte Carlo simulation · p. 18
  • Using the same uniform random draw for honest and manipulative choices removes variation caused solely by different random numbers when comparing trajectories.Source: D1.7 Monte Carlo simulation · p. 18
  • Under InstPE, declaring a profile immediately sets its incentive while choices follow the true profile-0 distribution; under RealPE, choices follow the declared profile’s multinomial-logit probabilities.Source: D1.7 Monte Carlo simulation · pp. 18–19
  • For online inference, the regulator records bike versus non-bike choices, compares profile-specific Bernoulli log-likelihoods, and breaks ties by lowest optimal incentive, then lowest profile index.Source: D1.7 Monte Carlo simulation · p. 19
  • The reported manipulation advantage is the difference between the time-averaged gain from manipulation and its cost.Source: D1.7 Monte Carlo simulation · p. 19
  • At T = 18, analytic long-run advantages were 0.08 and 0.56 £/timeslot for InstPE (r = 1, 2) and −0.17 and −0.13 £/timeslot for RealPE (r = 1, 2); finite-horizon discrepancies were 0.00, 0.00, 0.00, and 0.03 £/timeslot.The estimates have the same sign as all four analytic averages; the largest discrepancy is for RealPE with r = 2.Source: Table 10 · p. 19, D1.7 Monte Carlo simulation · p. 19

D2 Social-Impact Relevance

The section presents calibrated simulation and theoretical bounds as a pre-deployment stress test for personalized incentive policies. It shows how regulators can assess manipulation risk and screen policies before exposing individuals or public budgets to a fragile intervention.

D2 Social-Impact Relevance

  • Regulators can calibrate behavioral profiles from observed choices, compute profile-based incentives, and test whether manipulation gains compensate for the Cost of Manipulation.Theoretical bounds provide an analytical robustness certificate, while simulation evaluates the mechanism under calibrated heterogeneity and stochastic choices.Source: D2 Social-Impact Relevance · p. 19
  • In the calibrated London case, the Cost of Manipulation exceeds the Gain from Manipulation even as the incentive policy increases sustainable choices.Explicitly modeling preference learning reverses the conclusion obtained under instantaneous preference elicitation.Source: D2 Social-Impact Relevance · p. 20
  • Policy screening makes assumptions explicit and auditable: incentives follow a deterministic, preannounced mapping, and elicitation can rely on binary information about incentivized choices.The framework supports assessment of strategic robustness across personalized sustainability policies, not only a single transport intervention.Source: D2 Social-Impact Relevance · p. 20
  • Deploy only when manipulation advantage remains non-positive over the relevant interaction horizon; otherwise, use the framework to identify profiles and horizons for redesign.The workflow calibrates the population’s behavioral model and evaluates manipulation incentives under realistic preference elicitation.Source: D2 Social-Impact Relevance · p. 20
Loading 2610.13133v1…