PaperlayerView exact revision on arXiv

Source-linked AI summary

Marrying Pricing and Advertising with LLMs

Alessandro Barro, Francesco Bacchiocchi, Francesco Emanuele Stradi, Alberto Marchesi

arXiv:2610.09985v1cs.GTcs.LG

TL;DR

The paper studies how a seller can jointly choose price and an LLM-generated advertisement when demand is unknown and feedback only reveals whether a purchase occurs. It proposes an online actor-critic method that adapts advertisement generation with LoRA and uses a demand model to guide prices and actor updates. Expected revenue gains are reported across three synthetic models and a marketplace simulator.

  • Problem

    Because price and advertisement both affect demand, the seller must learn their joint value from binary purchase feedback that reveals no response to alternatives.

    Source: §1 · pp. 1–2

  • Method

    An online actor-critic algorithm adapts a pretrained LLM with LoRA, uses a demand model to guide prices, and updates both components from purchase outcomes.

    Source: §1.1 · p. 2

  • Results

    Expected revenue rose 5.69%, 5.18%, and 55.96% over Reference across synthetic models, and average normalized expected revenue rose 5.81% under the marketplace simulator.

    Source: §1.1 · p. 2

  • Takeaways & Limitations

    The evaluation validates the approach on synthetic demand models and a marketplace simulator fitted to real-world data.

    Source: §5 · p. 8

    The formulation assumes product and environmental contexts are drawn from a fixed distribution across offers.

    Source: §2 · p. 3

Abstract

from arXiv · show

We study a sequential pricing problem in which a seller jointly posts a price and an advertisement generated by a large language model (LLM). The seller aims to maximize revenue under an unknown product demand that depends on both decisions, while observing only whether each offer leads to a purchase. We propose an online actor-critic algorithm that combines low-rank adaptation (LoRA) of a pretrained LLM with a demand model fitted to available data. At each round, the actor generates an advertisement, and the critic estimates purchase probabilities to guide price selection. Then, the resulting feedback is used to update both the actor and the critic, with the critic's revenue estimates providing a baseline for policy gradient updates of the actor. To evaluate our approach, we develop an evaluation framework with three synthetic demand models and a demand simulator built from real-world marketplace data. Finally, we compare our algorithm with benchmarks that do not jointly optimize price selection and advertisement generation, achieving expected revenue gains over the reference policy of 5.69%, 5.18% and 55.96% under the three synthetic demand models and 5.81% under the marketplace simulator.

1 Introduction

Pricing and advertising jointly shape demand, but prior work often studies them separately. The paper proposes an online actor-critic approach for joint optimization and evaluates it across synthetic demand models and marketplace data.

1 Introduction

  • Price and advertisement content both affect demand, so optimizing them separately can miss how an advertisement shifts the revenue-maximizing price.A loan-offer experiment found that showing one example loan rather than four had the same demand effect as lowering the monthly interest rate by 2 percentage points.Source: §1 · p. 1

1.1 Original contribution

  • The online actor-critic algorithm adapts a pretrained LLM with LoRA, uses a demand model to guide price selection, and updates both from purchase feedback.Actor updates balance expected revenue with a KL penalty for deviations from the fixed pretrained model.Source: §1.1 · p. 2
  • Expected revenue gains over Reference were 5.69%, 5.18%, and 55.96% under logit, probit, and price disclosure, respectively, and 5.81% under the marketplace simulator.The price-disclosure gain is fully explained by increased frequency of the rewarded content feature under that demand mechanism.Source: §1.1 · p. 2

1.2 Related Work

  • Prior LLM ad-feedback methods optimize advertisements without price, while Agrawal et al. (2023) jointly learn price and signaling schemes.The paper also situates its approach among work on LoRA, KL-regularized learning, and REINFORCE baselines.Source: §1.2 · p. 3

2 Problem formulation

The formulation models each offer as a price, advertisement, and observed context whose combination determines purchase probability. It constrains LLM adaptation to LoRA and regularizes expected revenue to limit drift from a fixed reference policy.

2 Problem formulation

  • Purchase probability is modeled as a demand function of price, advertisement, and product-environment context.The seller observes context before choosing an advertisement and price.Source: §2 · p. 3
  • Advertisements come from a pretrained LLM, with adaptation restricted to LoRA parameters while the reference model’s remaining parameters stay fixed.Selected weight matrices receive low-rank updates; unadapted parameters are unchanged.Source: §2 · p. 3
  • Across repeated rounds, contexts are drawn from a distribution, the seller chooses an advertisement and price, and feedback is a binary purchase outcome.The offer’s expected revenue is its price multiplied by the probability of purchase.Source: §2 · pp. 3–4
  • The objective penalizes divergence from a fixed reference model to balance expected revenue with proximity to the reference policy.The penalty coefficient controls this trade-off.Source: §2 · p. 4

3 Online actor-critic algorithm

The algorithm alternates between critic-guided price selection and learning from purchase feedback. A LoRA-adapted LLM generates advertisements, while a demand critic estimates purchase probabilities and supplies revenue estimates for actor updates.

3 Online actor-critic algorithm

  • The algorithm initializes the actor at the pretrained reference model and keeps actor and critic parameters fixed within each batch.The actor is LoRA-adapted; the critic estimates purchase probabilities from price, advertisement, and context.Source: §3 · p. 4
  • With probability ετ, the seller samples a price from an exploration distribution; otherwise, the critic guides price selection by estimated expected revenue.Each interaction yields a purchase outcome and realized revenue for subsequent updates.Source: §3 · p. 4
  • The actor update uses realized revenue minus a leave-one-out average of critic-predicted revenues, with a KL penalty to limit deviation from the reference model.The critic estimates rather than observed revenues form the baseline, aiming to reduce purchase-outcome noise.Source: §3 · p. 5
  • The critic is updated after every batch, while actor updates occur every K batches using interactions from the current batch.The critic loss combines binary cross-entropy on purchase outcomes with a penalty for deviations from its reference parameters.Source: §3 · p. 6

4 Experimental Results

The experiments compare Trained with Reference under a shared learned pricing rule and also assess online revenue against UCB. Synthetic and marketplace evaluations report revenue gains over Reference.

4.1 Synthetic Demand Models

  • Synthetic tests use logit and probit demand with an advertisement feature, plus price-disclosure demand built by interpolating observed purchase rates across prices.In logit and probit, demand decreases with price; in the disclosure model, the feature selects between the two demand curves.Source: §4.1 · pp. 6–7

4.1 Synthetic Demand Models; 4.2 Experiments Based on Real Data

  • Expected revenue rose 5.69%, 5.18%, and 55.96% over Reference under logit, probit, and price disclosure, and 5.81% in marketplace evaluation.The synthetic figures are reported in Table 1; the marketplace comparison reports normalized expected revenue.Source: Table 1 · p. 7, §4.1 · p. 7, §4.2 · p. 7

4.1 Synthetic Demand Models

  • Rewarded content became more frequent: durable rose from 82.14% to 97.11% in logit and 96.73% in probit, while price disclosure rose from 35.17% to 98.23%.Under the shared critic, increased disclosure frequency fully accounts for the price-disclosure revenue gain.Source: §4.1 · p. 7

4.2 Experiments Based on Real Data

  • Marketplace evaluation trained on one product per category and tested on 20 others using the same final critic; expected revenue was normalized by each product’s price scale.Each policy was evaluated with 100 advertisements and prices per test product.Source: §4.2 · p. 7
  • During marketplace online learning, Trained exceeded UCB in mean expected revenue over the final 100 batches in every category; UCB uses reference ads and reward-guided pricing.The comparison uses the same price grid and online interaction budget.Source: §4.2 · pp. 7–8

5 Conclusion and Future Work

The paper presents an online actor-critic approach for jointly selecting prices and LLM-generated advertisements, validated on synthetic and marketplace data. It identifies changing buyer preferences as a future challenge because they can alter both price sensitivity and advertising effects.

5 Conclusion and Future Work

  • The online actor-critic adapts advertisement generation through LoRA, while a demand model guides pricing and supplies a leave-one-out baseline for actor updates.The paper validates the algorithm on synthetic and real-world marketplace data.Source: §5 · p. 8
  • Changing buyer preferences may shift both price sensitivity and advertisement effects, requiring the critic to track a moving target from binary feedback as the actor adapts.Source: §5 · p. 8
  • Marketplace evaluation uses 100 fresh advertisements per policy and product with paired intervals; Figure 3 plots each test product’s mean revenue against equality.Source: Table 2 · p. 8, Figure 3 · p. 9

AI use statement

The authors report using generative AI tools to improve readability and refine the English exposition, and Codex to assist with code development.

AI use statement

  • The authors used generative AI to refine exposition and Codex to assist with code development, while retaining responsibility for the scientific content and final manuscript.Source: AI use statement · p. 10

A Unbiasedness of the Actor Gradient Estimator

The appendix establishes that the actor’s leave-one-out baseline has zero contribution to the conditional expected gradient, even if the critic is inaccurate or biased. It also introduces experimental protocols and additional results.

A Unbiasedness of the Actor Gradient Estimator

  • The proof uses conditional independence between a critic revenue estimate and another offer’s score, together with the zero-mean score identity.The critic is fixed before the batch, while advertisement draws and pricing randomizations are independent across interactions.Source: A Unbiasedness of the Actor Gradient… · pp. 12–13
  • The leave-one-out baseline contributes zero to the conditional expected actor gradient, even when the critic is inaccurate or biased.Critic quality can still affect selected prices and estimator variance.Source: A Unbiasedness of the Actor Gradient… · pp. 12–13
  • Unbiasedness holds at the sampling actor parameters with the batch’s pricing strategy held fixed.The actor, critic, exploration probability, and exploration distribution are fixed conditional on the pre-batch history.Source: A Unbiasedness of the Actor Gradient… · pp. 11–13

B Experimental Protocols and Additional Results

B.1 Shared Implementation and Evaluation

This section specifies shared actor-critic settings, revenue evaluation, uncertainty estimates, and the UCB pricing comparator.

B.1 Shared Implementation and Evaluation

  • The actor uses rank-eight LoRA on Qwen2.5-1.5B-Instruct, with 32-offer batches, actor updates every five batches, and 64-offer critic updates.The implementation uses 100 actor updates and 500 critic updates; disabling the adapters recovers Reference.Source: §B.1 · p. 13, Table 3 · p. 14
  • Expected revenue averages price times purchase probability across generated advertisements, excluding the optimization KL penalty.Trained and Reference use the same final critic, isolating text adaptation under shared learned pricing.Source: §B.1 · p. 13
  • Synthetic intervals use 2,000 bootstrap resamples, while marketplace intervals use paired product-level resamples and condition on fixed policies and demand models.The intervals exclude variation across training seeds and simulator fits.Source: §B.1 · p. 13
  • UCB uses one UCB1 arm per admissible grid price and selects prices independently of the current advertisement.It updates arm statistics from observed rewards after each offer; expected revenues are reserved for evaluation.Source: §B.1 · pp. 13–14

B.2 Synthetic Models, Calibration, and Reconstruction

The synthetic experiments model content-sensitive demand in three settings and compare Trained with Reference under shared learned pricing. Price disclosure produces the largest reported gain, which the authors attribute to increased disclosure frequency.

B.2 Synthetic Models, Calibration, and Reconstruction

  • Logit and probit demand use a binary durable-word feature, with illustrative coefficients and deterministic probit calibration to selected logit probabilities.The feature is permitted by both steel-bottle contexts; the cited credit-offer studies motivate the response families, not the numerical choices.Source: §B.2 · p. 14
  • Price-disclosure demand interpolates Plain Ad and Price Ad purchase probabilities separately across three observed dollar prices.This modeling assumption preserves probabilities within the observed range but does not reproduce the cited structural consumer-search model.Source: §B.2 · pp. 14–15, Table 4 · p. 15
  • Critic pretraining uses 1,024 unique reference advertisements for training and 256 disjoint advertisements for validation, with eight stratified prices per advertisement.The critic receives normalized price and the declared feature, not demand coefficients or exact probabilities.Source: §B.2 · p. 15
  • 55.96% revenue gain in price disclosure is fully explained by the increase in disclosure frequency under shared learned pricing.The reported test means cover 10,000 advertisements per policy and environment.Source: Table 5 · p. 15, §B.2 · p. 15
  • Under logit and probit, mean selected price and purchase probability both rise; under price disclosure, mean price falls while purchase probability and disclosure frequency rise.The shared critic selects the same price for both policies conditional on the disclosure feature.Source: §B.2 · p. 15
  • Synthetic UCB logs omit advertisements and purchase probabilities, so marginal training revenue is reconstructed from logged prices and offline reference feature frequency.The offline corpus is deduplicated, making its feature frequency an approximation to unrestricted online generations.Source: §B.2 · p. 15

B.3 Marketplace Data and Evaluation Protocol

The marketplace protocol builds product contexts and demand simulators from Avito listings, then evaluates trained policies on held-out products. Gains are broad across products but not universal.

B.3 Marketplace Data and Evaluation Protocol

  • The Avito dataset retains 20,826 listings across 393 products after filtering, deduplication, product grouping, and price trimming.Titles and descriptions are translated into English; images, timestamps, locations, and seller attributes are excluded from model inputs.Source: §B.3 · p. 16
  • Each product context contains a canonical name, category, and representative condition, while five fixed marketplace examples guide generation format and phrasing.Those examples’ product groups are excluded before the supervised and actor splits.Source: §B.3 · p. 16, Figure 4 · p. 16
  • The evaluation uses disjoint listing identities across splits, independently fitted demand networks, and frozen category-specific actors and critics.The critic and environment use frozen reference-model text embeddings and assume an available product price range; prices below historical minima are extrapolations.Source: §B.3 · pp. 16–17
  • Trained improves on 91 of 100 products, with positive category means and all nine product-level losses in Audio/Video or Mobile Phones.The reported product means are shown in Figure 5, and Table 6 reports revenue gains and improved-product counts.Source: Figure 5 · p. 17, §B.3 · p. 17, Table 6 · p. 18

B.4 Progressive Scarcity of Offline Data

The section tests how critic pretraining data scarcity affects revenue under a changing exploration schedule. Online learning continues with no supervised initialization, but the zero-data run ends below the full-data control.

B.4 Progressive Scarcity of Offline Data

  • Critic-training subsets are stratified by category and selected without demand targets, ranging from approximately 100% to 0% of 7,291 listings.Laptops are the category with the median number of critic-training listings; the actor-training product remains Asus X54H.Source: §B.4 · p. 18
  • Exploration increases as offline support decreases and declines as online feedback accumulates; at zero data, the critic starts from seeded random parameters with ρ = 0.Every run receives 16,000 online outcomes, and evaluation freezes the final actor and critic with price exploration disabled.Source: §B.4 · p. 18
  • Revenue is not monotone across intermediate data budgets, and one run per budget with overlapping product-bootstrap intervals does not show that removing offline data improves performance.Initial support and exploration vary together, so this comparison evaluates the complete missing-data learning procedure rather than the isolated effect of exploration.Source: §B.4 · p. 18, Figure 6 · p. 19
  • At zero initial data, expected revenue is 0.1670, 7.32% below the full-data control; with 25–100% of the data, it ranges from 0.1802 to 0.1895.Revenue is evaluated on the same 20 held-out Laptop products; Figure 6 intervals are conditional on one training run per data budget.Source: Table 7 · p. 18, §B.4 · p. 18, Figure 6 · p. 19

B.5 Critic Predictions in the Leave-One-Out Baseline

This diagnostic compares critic-based leave-one-out baselines with baselines using observed rewards. Critic estimates reduce baseline error after supervised fitting, but a randomly initialized critic performs worse than the observed-reward baseline.

B.5 Critic Predictions in the Leave-One-Out Baseline

  • The diagnostic compares critic error with Bernoulli purchase noise in an observed-reward baseline; ratios below one mean critic error is smaller than that noise.It averages conditional mean squared errors over sampled historical-offer batches and positions.Source: §B.5 · p. 19
  • At full data, baseline MSE ratios are 0.1734, 0.0151, 0.0337, 0.0095, and 0.0226; at zero data, critic error is approximately 70–115 times observed-baseline error.The ratios are reported in the category order of Table 2; Figure 7 compares critic-based and observed-reward estimators.Source: Figure 7 · p. 20, §B.5 · p. 20
  • For Laptops at full data, baseline MSE falls from 0.002256 to 0.000076, while revenue-advantage MSE falls from 0.072191 to 0.070011, a 3.02% reduction.Purchase noise from the offer being estimated remains under either baseline, limiting the effect of baseline improvement on the complete advantage.Source: §B.5 · p. 20, Figure 7 · p. 20
  • This diagnostic excludes the KL term and log-policy gradient, and does not establish lower full policy-gradient variance or higher final-policy revenue.Historical batches may also mix product contexts within a category.Source: §B.5 · p. 20
Loading 2610.09985v1…