PaperlayerView exact revision on arXiv

Source-linked AI summary

LLMAdBench: A Human Preference Benchmark for Advertising in LLM Responses

Rui Ai, Yuqing Liu, Sitao Qiu, Yun Qiao, Yuhan Wang, Jessica Xiwen Wang, Yiqi Yang, Lihong Huang, Ruiyao Sun, Kaifeng Zhang, Shengze Ding, Jiaqi He, Xinman Wang, Tianhao Gao, Jimmy Qin, Jianghao Lin, Chonghuan Wang

arXiv:2609.32533v1cs.AI

TL;DR

LLM advertising raises a practical question: where should matched ads appear without degrading the usefulness of answers, and how does disclosure affect preferences? LLMAdBench measures human choices across controlled placement comparisons and evaluates LLM judges. Zero-shot judges do not reliably reproduce human judgments, while preference-trained Qwen3-8B improves held-out prediction; human preferences also vary with disclosure and advertiser–user trade-offs.

  • Problem

    Evidence on ad insertion in LLM responses is limited, despite the need to understand placement and disclosure preferences when ads accompany decision-support answers.

    Source: §1 · p. 1

  • Method

    LLMAdBench compares response pairs that vary only in matched-ad placement, collecting human judgments across six dimensions and labeled or unlabeled disclosure conditions.

    Source: §1 · p. 2, §2.1 · p. 4, §2.2 · p. 4

  • Results

    Zero-shot frontier judges are unstable and differ from human placement preferences, while Qwen3-8B reaches 50.54 labeled and 55.76 unlabeled held-out accuracy after DPO, exceeding the strongest zero-shot judge by 2.40 and 5.44 points.

    Source: §4.2 · p. 7, §6.3 · p. 10, §7 · p. 11

  • Takeaways & Limitations

    Human preferences expose a visibility–user experience trade-off, and sponsorship disclosure systematically shifts preferred placement toward the response’s end.

    Source: §1.1 · p. 3

Abstract

from arXiv · show

Inserting advertisements (ads) into consumer-facing LLM output is emerging as a new business model, but there is little shared evidence on how such ad insertion should be evaluated or how it affects user preferences. We introduce LLMAdBench, a human-preference benchmark for studying advertising in LLM-generated content. The benchmark isolates a simple but practically important decision: given a user conversation, an LLM response, and a matched advertisement, where should the ad be placed? Our dataset compares pairs of responses that differ only in ad position while holding all other conditions fixed including the user query, base answer, advertisement, and disclosure condition. Human annotators evaluate each pair based on six criteria from both advertiser's and user's perspectives. The resulting benchmark contains more than 18000 human judgments across two disclosure conditions: explicitly labeling the ad as sponsored and merging it into the response without disclosure. We use LLMAdBench to evaluate eight frontier LLMs as preference judges and find that they are not reliable substitutes for human evaluation. Even the most stable models reverse roughly one quarter of their decisions when the presentation order is swapped, agreement across models is low, and their placement preferences differ systematically from those of human annotators. Moreover, LLMAdBench contains substantial learnable signal. In particular, a Qwen3-8B model fine-tuned on the human preferences improves substantially over its base model and outperforms all zero-shot frontier judges on the held-out prediction task. Beyond model evaluation, LLMAdBench provides quantitative evidence on the advertiser-user trade-off and shows that the sponsorship disclosure systematically changes users' preference over ad placement.

1 Introduction

LLMAdBench studies where matched ads should appear in LLM answers and whether they should be disclosed, addressing a gap in controlled human-preference evidence. It evaluates human preferences and LLM judges across that setting.

1 Introduction

  • Consumer-facing LLMs increasingly support everyday decisions and purchasing, creating an opportunity for advertising alongside a challenge of preserving answer usefulness.The paper frames matched advertising as an additional component while leaving the generated answer unchanged.Source: §1 · pp. 1–2

1.2 Related Work

  • Prior resources do not combine controlled placement variation with fixed answers and advertisements, human preferences, disclosure comparisons, and systematic LLM-judge evaluation.Existing evaluations often choose ad location during free-form generation or insertion, confounding placement with other content changes.Source: §1 · p. 2, §1.2 · p. 3

1.1 Contributions

  • LLMAdBench fixes the query, answer, advertisement, and disclosure condition while varying only placement in pairwise A/B/T comparisons across six dimensions.The benchmark contains 18,576 human judgments.Source: §1 · p. 2, §1.1 · p. 3
  • Eight frontier LLM judges are unreliable substitutes for humans: they show swap sensitivity, low inter-model agreement, and systematically different placement preferences.Even the most stable models reverse roughly one quarter of decisions when presentation order is swapped.Source: §1.1 · p. 3
  • Human preferences reveal a visibility–user experience trade-off, and sponsorship disclosure shifts preferred ad placement toward the response’s end.The preferred insertion position depends on whether the advertisement is disclosed.Source: §1.1 · p. 3

2 Task and Dataset

LLMAdBench compares ad placements around or within an unchanged answer, under labeled and unlabeled disclosure conditions. Its rubric and annotation design capture competing user and advertiser objectives across a reproducible human-preference dataset.

2.1 Task Definition

  • Each source instance yields four positions—front, early, late, and end—compared pairwise while the conversation, answer, ad message, and disclosure condition remain fixed.The insertion policy may make local wording adjustments to fit the ad at its position.Source: §2.1 · p. 4, Figure 1 · p. 4

2.2 Preference Rubric

  • Six rubric dimensions separate user-experience judgments from advertiser-relevant outcomes, with overall preference providing an all-things-considered assessment.Visibility is a neutral perceptual judgment; click likelihood measures stated intention rather than observed behavior.Source: §2.2 · p. 4

2.3 Dataset Construction

  • The benchmark uses 258 source conversations and one matched ad brief per conversation, producing 1,548 placement-pair comparisons per disclosure condition and 18,576 dimension-level judgments.The same placement pairs are evaluated with and without an explicit sponsorship label; ads have a median length of 25 words and responses 194 words.Source: §2.3 · pp. 4–5

2.4 Human Annotation

  • Annotators compare randomized candidate pairs using Version A, Version B, or Tie, and aggregate human placement preferences are reproducible under resampling.Split-half reliability for overall preference is 0.975 labeled and 0.966 unlabeled; these values are not a human performance ceiling.Source: §2.4 · p. 5

3 Evaluation Protocol

The evaluation protocol tests whether judges reproduce human pairwise labels and aggregate placement preferences. It also measures order sensitivity, judge agreement, and the stability of human placement estimates.

3.1 Prediction Task

  • Judges predict the retained human A/B/T label for a rubric dimension from the conversation, two rendered responses, and disclosure condition; ties remain a distinct class.The auxiliary placement-attention item is used only for quality control, not as a benchmark target.Source: §3.1 · pp. 5–6

3.2 Placement Preference Estimation

  • A Bradley–Terry model with Davidson’s tie extension and a presentation-order term converts pairwise labels into normalized placement scores for the four positions.Scores near 0.25 indicate no systematic preference; scores above 0.25 indicate a more-favored position under the fitted model.Source: §3.2 · p. 6

3.3 Reliability Diagnostics

  • Reliability diagnostics assess candidate-order swap consistency, chance-corrected agreement among model judges, and split-half stability of human aggregate placement scores.The split-half procedure re-estimates placement preferences on random source-instance halves and compares their normalized score vectors.Source: §3.3 · pp. 6–7

4 Benchmark Results

The benchmark compares eight zero-shot frontier judges with Qwen3-8B before and after preference training. On held-out A/B/T prediction, DPO moves Qwen3-8B above every zero-shot frontier judge in both disclosure conditions.

4.1 Models and Baselines

  • The eight zero-shot frontier LLM judges share a pairwise A/B/T prompt and predict human labels for individual rubric dimensions.The comparison also includes Qwen3-8B before and after DPO training on human comparisons from the training split.Source: §4.1 · p. 7

4.2 Main Leaderboard

  • After DPO, Qwen3-8B reaches 50.54 labeled and 55.76 unlabeled accuracy, exceeding the strongest zero-shot judge by 2.40 and 5.44 points, respectively.Paired McNemar tests with Holm correction confirm improvements over every frontier judge in both conditions.Source: §4.2 · p. 7, Table 3 · p. 7, Figure 2 · p. 8

4.3 What Preference Training Adds

  • DPO shifts Qwen3-8B from below the zero-shot frontier range to above it, indicating that human-comparison placement preferences are learnable but not reliably recovered by zero-shot prompting.The effect is larger in the unlabeled condition, where the judge must infer how placement changes the reading experience without an explicit disclosure marker.Source: §4.2 · p. 7, §4.3 · p. 8, Figure 2 · p. 8

5 Why Human Evaluation Is Needed

The benchmark treats human judgments as the preference target and tests whether LLM judges are stable and agree with one another. The zero-shot judges show substantial order sensitivity and limited pairwise agreement, while aggregated human placement signals are stable under resampling.

5 Why Human Evaluation Is Needed

  • The benchmark uses human judgments as its target because zero-shot LLM judgments are tested as automatic evaluators, not as label-defining substitutes.The evaluation checks invariance to a semantics-preserving A/B swap and chance-corrected agreement with other judges.Source: §5 · p. 8

5.1 Presentation-Swap Instability

  • Across six rubric dimensions, zero-shot judges’ mean swap error is 31.40%, with condition means of 28.93% labeled and 33.87% unlabeled.Even the lowest average swap error is 24.36%.Source: §5.1 · p. 8, Figure 3 · p. 9

5.2 Low Agreement Among LLM Judges

  • Mean pairwise Cohen’s κ is 0.402 labeled and 0.384 unlabeled, showing limited agreement across judges on the same original A/B presentation.The heatmap shows this limited agreement is not driven by a single outlier model.Source: §5.2 · p. 9, Figure 3 · p. 9
  • Majority voting across the eight zero-shot judges remains significantly less accurate than post-DPO Qwen3-8B in both disclosure conditions.Paired McNemar tests report p-values of 0.0037 labeled and 4.7 × 10^-8 unlabeled.Source: §5.2 · p. 9

5.3 Human Signal Stability

  • Human placement-signal split-half reliability ranges from 0.975 to 0.998 labeled and 0.952 to 0.999 unlabeled across six dimensions.Overall-preference reliability is 0.975 labeled and 0.966 unlabeled, supporting aggregated human judgments as the benchmark target.Source: §5.3 · p. 9

6 What LLMAdBench Reveals

Human placement preferences depend on disclosure and distinguish overall preference from click likelihood. Zero-shot judges often favor end placement regardless of disclosure, unlike human judgments.

6 What LLMAdBench Reveals

  • Placement scores are normalized across four positions, making 0.25 the equal-share reference and values above it above-baseline estimated preference mass.Figure 4 plots overall preference against click likelihood and defines Pareto efficiency using both scores.Source: §6 · p. 10, §6.2 · p. 10

6.1 Disclosure Changes the Preferred Placement

  • Disclosure reverses the preferred non-front placement: late scores highest unlabeled at 0.352, but end scores highest labeled at 0.364.Front placement is disfavored in both conditions, scoring 0.132 without disclosure and 0.176 with disclosure.Source: §6.1 · p. 10

6.2 Overall Preference and Click Likelihood Diverge

  • Without disclosure, late placement leads overall preference (0.352) while end leads click likelihood (0.325); with disclosure, end leads both at 0.364 and 0.369.Front and early placements are Pareto dominated in the unlabeled condition.Source: §6.2 · p. 10

6.3 LLM Judges Collapse Toward End Placement

  • All but Mistral-Large-3 in the unlabeled condition favor end placement in both disclosure conditions; Claude-Opus-4.6 assigns it 0.688 and 0.846 preference mass.Human end-placement scores are 0.281 unlabeled and 0.364 labeled, and the discrepancy remains in aggregate placement scores.Source: §6.3 · p. 10

7 Conclusion

LLMAdBench makes ad placement measurable through controlled human preferences and shows that disclosure changes placement preferences, while zero-shot judges differ from humans. The conclusion positions it as a testbed for ad-insertion policies and model judges.

7 Conclusion

  • The controlled benchmark fixes query, answer, ad, and disclosure while varying placement; its human labels expose judge instability, end-placement bias, and disclosure-dependent preferences.A fine-tuned Qwen3-8B judge outperforms zero-shot frontier judges on held-out human-label prediction, while visibility, overall preference, and click likelihood show distinct patterns.Source: §7 · p. 11
  • The benchmark can be extended to richer ad formats, longer interaction histories, additional disclosure interfaces, and submitted judge models.Its stated goal is to make trade-offs among user preference, click-oriented outcomes, and platform design measurable.Source: §7 · p. 11

A.1 Related Work

  • Auction research spans token-level bids, response-level preferences, and retrieval-based segment auctions; these approaches trade contextual richness against repeated inference or coarse relevance signals.Later approaches seek lower computational costs while preserving allocation quality, including learned allocation-generation mechanisms and embedding filtering with LLM-logit reranking (Duetting et al., 2024; Zhao et al., 2025; Sun et al., 2026).Source: §A.1 · p. 14
  • Ad-generation studies address standalone ad copy and embedded sponsored content, including PILA, which integrates sponsored content by rewriting an existing response (Zhang et al., 2026b).The cited work reports improved ad effectiveness while preserving response quality.Source: §A.1 · p. 14
  • Prior evaluation and perception studies offer evaluators, benchmarks, and user evidence, but do not isolate how the same ad's position interacts with explicit sponsorship disclosure (Zhang et al., 2026a; Turner-Smith et al., 2026; Tang et al., 2025).Tang et al. (2025) reports that users often fail to recognize personalized embedded ads and sometimes prefer undisclosed advertising responses.Source: §A.1 · p. 14

B.1 Data Sources and Matching

The appendix details how LLMAdBench sources and matches conversation-ad pairs, constructs and renders placement candidates, collects human judgments, and evaluates aggregate reliability and model predictions.

B.1 Data Sources and Matching

  • Conversation contexts come from WildChat and ad material from Webis Generated Native Ads 2024; matching ranks candidates by embedding cosine similarity, thresholds at 0.65, then filters relevance with Qwen3-8B.Retrieval retains up to five candidates per turn; human pairwise preferences, not the relevance filter, supply benchmark labels.Source: §B.1 · p. 15

B.2 Position Selection

  • Four candidates place the ad at the response beginning, end, or one of two selected internal sentence boundaries.The two internal boundaries with the highest local-continuity scores are ordered by their position in the answer; higher scores indicate smaller disruption.Source: §B.2 · p. 16

B.3 Candidate Rendering

  • Qwen3-8B rewrites only the ad for each selected local context while preserving its core message and advertised entity; paired candidates share the conversation, answer, and ad message.Recovered multi-turn context is used for annotator materials, not retrieval, position selection, or rendering.Source: §B.3 · pp. 16–17

B.4 Annotation Interface and Quality Control

  • Twelve English-proficient students and researchers evaluate six unordered placement pairs per instance; pair order and task order are randomized, with two attention checks and replacement annotation for failures.Annotators see the conversation context and two candidate responses, then answer six rubric questions using A, B, or Tie.Source: §B.4 · p. 17

C.2 Model-Judge Prompt

  • Model judges answer six A/B/T questions about flow, visibility, helpfulness, click likelihood, appropriateness, and overall preference in one JSON object.A Tie is specified for equal, minor, subjective, unclear, or uncertain differences.Source: §B.4 · p. 17, §C.2 · p. 21

B.5 Aggregate Human Signal Reliability

  • The split-half procedure partitions source instances, re-estimates normalized preference vectors, and repeats across 100 random splits with a Spearman-Brown correction.The reliability unit is the aggregate preference vector over positions, matching the target used to evaluate model judges.Source: §B.5 · pp. 17, 19
  • The final human aggregate placement signal is highly reliable: corrected split-half reliability ranges from 0.952 to 0.999 unlabeled, and overall preference reliability is 0.975 labeled.The labeled-condition reliabilities for the first five dimensions are 0.983, 0.998, 0.991, 0.992, and 0.985.Source: §B.5 · p. 19

C.1 A/B/T Prediction

  • Each prediction target is a retained human A/B/T label for a placement pair and rubric dimension; exact accuracy requires matching that label, and source instances stay within one train-test partition.The auxiliary placement-attention item is excluded from prediction accuracy.Source: §C.1 · pp. 19–20

C.3 Presentation-Swap Error

The appendix specifies presentation-swap consistency, model-judge agreement, and paired system comparisons, along with the held-out accuracy reporting and statistical tests.

C.3 Presentation-Swap Error

  • The presentation-swap test checks whether a judge's verdict remains equivalent after exchanging A and B, mapping A to B, B to A, and Tie to Tie.A prediction is swap-consistent only after this label reversal; Tie must remain Tie.Source: §C.3 · p. 22

C.4 Agreement Among Model Judges

  • Judge agreement is compared using chance-corrected pairwise agreement and Fleiss’ κ for the full panel; human aggregate reliability is assessed separately by split-half testing.The pairwise statistic is summarized across judge pairs.Source: §C.4 · p. 23

C.5 Paired Statistical Comparisons

  • Paired systems are compared with continuity-corrected McNemar tests based on discordant correctness counts, with Holm adjustment across comparison families.The tests assess whether accuracy advantages remain reliable after sample variation and multiple comparisons.Source: §C.5 · p. 23

D Omitted Details in Section 4

  • The detailed experiments use the same held-out test rows and scoring rules, reporting the fixed epoch-5 DPO Qwen3-8B checkpoint in both disclosure conditions.Its prompt receives only superficial modifications to match the training template.Source: D Omitted Details in Section 4 · p. 24

D.1 Accuracy with Confidence Intervals

  • Table 5 reports held-out A/B/T accuracy with 95% bootstrap confidence intervals over rubric-question targets, using the final eight-model comparison and fixed epoch-5 DPO checkpoint.Each held-out placement pair contributes six prediction targets.Source: Table 5 · p. 24, §D.1 · p. 24

D.2 Paired Statistical Comparisons

  • Paired McNemar tests compare the fixed epoch-5 DPO checkpoint with each of eight zero-shot frontier judges, applying Holm correction separately by disclosure condition.The reported tests use the paired held-out evaluation rows.Source: §D.2 · p. 24, D Omitted Details in Section 4 · p. 24

D.3 DPO Training Setup and Checkpoint Accuracy

DPO training improves Qwen3-8B’s held-out A/B/T accuracy, with most checkpoint gains appearing in the first two or three epochs. The training setup uses augmented human preference comparisons and a source-question-level split.

D.3 DPO Training Setup and Checkpoint Accuracy

  • Training examples encode clear preferences and ties differently, augment comparisons by swapping A/B order, and balance added tie-versus-version comparisons.The split is made before augmentation, preventing variants of the same comparison from crossing between training and test sets.Source: §D.3 · pp. 24–25
  • Post-DPO Qwen3-8B beats all eight zero-shot baselines in both disclosure conditions; its advantage is large on unlabeled comparisons and modest for the two smallest labeled margins.The comparison uses paired predictions on the full evaluation sets, with Holm-corrected p-values across the eight comparisons in each regime.Source: Table 6 · p. 25
  • The preference data use an 80/20 split at the original question level, yielding 23,530 labeled and 21,381 unlabeled training rows.The corresponding test sets contain 3,714 labeled and 3,716 unlabeled rows.Source: §D.3 · p. 25
  • Qwen3-8B is adapted with LoRA and sigmoid-objective DPO for five epochs using AdamW, cosine decay, and a 5 × 10^-6 learning rate.Training uses β = 0.1, a 0.03 warmup ratio, and one NVIDIA H200 GPU.Source: §D.3 · p. 25, Table 7 · p. 25
  • At epoch five, accuracy is 50.54 in the labeled condition and 55.76 in the unlabeled condition; gains largely saturate by the second or third epoch, with only small later changes.Source: §D.3 · p. 26

D.4 Error Breakdown by Label and Rubric Dimension

The error breakdown shows that DPO improves Qwen3-8B’s recovery of underpredicted labels and raises accuracy across rubric dimensions. Gains are especially pronounced for unlabeled comparisons, while labeled results remain mixed against zero-shot judges.

D.4 Error Breakdown by Label and Rubric Dimension

  • The main leaderboard averages accuracy over all A/B/T prediction targets, while the appendix separates results by gold label and rubric dimension.Tables 8 and 9 provide those two breakdowns.Source: §D.4 · p. 26, Table 8 · p. 26, Table 9 · p. 27
  • 20.22 to 49.36 labeled Version-A recall and 5.82 to 20.50 Tie recall show DPO improves recovery of both underpredicted labels.Version-B recall remains high at 67.07 in the labeled condition.Source: §D.4 · p. 26, Table 8 · p. 26
  • 12.21 to 49.55 unlabeled Version-A recall and 22.33 to 52.52 Tie recall mark stronger recovery of these labels after DPO.These recall changes are reported in the gold-label breakdown.Source: Table 8 · p. 26, §D.4 · p. 27
  • The recall gains are not merely a shift toward one directional answer; DPO recovers preference cases the base model largely misses.This interpretation accompanies the improvements in Version-A and Tie recall.Source: §D.4 · p. 27
  • Post-DPO Qwen3-8B improves on every rubric dimension in both disclosure conditions, with unlabeled gains exceeding twenty points on Q2, Q5, and Q6.In the labeled condition, it leads zero-shot judges on Q2, Q4, and Q5, while some judges remain competitive on Q1 and Q6.Source: §D.4 · p. 27, Table 9 · p. 27
  • Labeled-condition comparisons remain mixed against frontier judges: post-DPO Qwen3-8B leads on Q2, Q4, and Q5, but not on every dimension.Some zero-shot judges remain competitive on Q1 and Q6.Source: §D.4 · p. 27, Table 9 · p. 27

E.1 Presentation Order and Swap Diagnostics

Presentation-order diagnostics show residual order preferences among most zero-shot judges, and judge agreement is structured rather than uniform. The section also describes how human aggregate reliability and annotator response profiles are assessed.

E.1 Presentation Order and Swap Diagnostics

  • Swap error measures whether judges preserve their A/B/T verdict after Version A and Version B are exchanged; the section reports these errors by judge, condition, and rubric dimension.The table’s error percentages indicate invariance to presentation-order swaps.Source: §E.1 · p. 27, Table 10 · p. 28
  • Most judges show residual preference for Version B after placement is accounted for; Llama-4-Maverick instead favors Version A, while GPT-5.4’s intervals include zero in both conditions.The order-term diagnostic complements swap error, which tests whether a verdict is preserved when the two versions exchange places.Source: §E.1 · p. 27, Table 10 · p. 28, Table 11 · p. 29

E.2 Agreement Among LLM Judges

  • Agreement clusters among Grok-4.1, GPT-5.4, Kimi-K3, Gemini-3.1-Pro, and Claude-Opus-4.6 more strongly than with three other judges.The reported pairwise κ pattern suggests additional judges do not necessarily provide independent measurements of the same preference target.Source: §E.2 · p. 28, Table 13 · p. 30
  • Majority voting by eight zero-shot judges remains below post-DPO Qwen3-8B, with significant paired McNemar results in both conditions.The tests report χ2 = 8.4, p = 0.0037 for labeled and χ2 = 29.8, p = 4.7 × 10^-8 for unlabeled comparisons.Source: §E.2 · p. 28, Table 14 · p. 30

E.3 Human Aggregate Reliability

  • Human aggregate reliability is estimated by fitting placement models on two source-instance halves and correlating their normalized four-position preference vectors across 100 random splits.The section also reports annotator Version-A shares and Tie rates by disclosure condition, with between-annotator variation.Source: §E.3 · pp. 30–31, Table 16 · p. 31

F.1 Human Placement Scores Across Dimensions

Human placement profiles vary by rubric dimension, and labeling shifts preferences across positions. Zero-shot judges concentrate overall preference on end placement, while human results show a visibility–preference trade-off and link click likelihood to both dimensions.

F.1 Human Placement Scores Across Dimensions

  • Human naturalness, helpfulness, and appropriateness share the overall-preference top-position ordering, while visibility and click likelihood show different placement patterns.Front is most salient for visibility in both conditions; click likelihood concentrates on edge positions, especially end.Source: §F.1 · p. 31, Table 17 · p. 31
  • Labeling shifts human overall preference from late toward end placement, increases end score mass for helpfulness and appropriateness by more than 0.15, and makes interior ads easier to notice.For visibility, front score falls from 0.802 to 0.605 as early and late positions gain score mass.Source: §F.1 · p. 31, Table 18 · p. 32

F.2 LLM-Judge Placement and Dimension Profiles

  • Except for Mistral-Large-3 in the unlabeled condition, every retained zero-shot judge makes end placement modal in both conditions; four judges assign it over 0.75 of labeled overall-preference score mass.The human modal position instead changes from late to end after labeling.Source: §F.2 · p. 32, Table 19 · p. 32
  • Human visibility preferences usually oppose overall preference, experience-oriented dimensions align closely with it, and click-likelihood alignment is less consistent across zero-shot judges.The same-winner comparison excludes ties on either question.Source: §F.2 · pp. 32–33, Table 20 · p. 33

F.3 Human Trade-off and Click-Likelihood Model

  • Moving an ad from end to front raises visibility by 0.666 and lowers overall preference by 0.150 unlabeled, versus a 0.514 visibility gain and 0.188 preference loss labeled.These point-estimate changes quantify the human placement trade-off across disclosure conditions.Source: §F.3 · p. 33, Table 21 · p. 34
  • Visibility and overall preference are both positively associated with stated click likelihood, and each adds information beyond the other in both disclosure conditions.The overall-preference coefficient is larger, so stated click likelihood is not simply a salience judgment.Source: §F.3 · pp. 33–34, Table 22 · p. 34

Related papers

Loading 2609.32533v1…