Source-linked AI summary
"If I Had to Buy Just ONE: Galaxy S26 Ultra": Auditing AI-Generated Product Recommendations
Lucas G. Uberti-Bona Marin, Thales Bertaglia, Giovanni Astante, Bram Rijsbosch, Gijs van Dijck, Anikó Hannák, Gerasimos Spanakis, Konrad Kollnig
TL;DR
AI assistants increasingly advise consumers on purchases, raising questions about the impartiality and reliability of commercial recommendations. This audit analyzes real product queries across chatbot interfaces, APIs, and Google AI Overviews. It finds substantial differences in framing, products, and displayed sources, and concludes that audits must account for repeated responses and consumer-facing conditions.
Problem
Commercial advice from AI assistants raises questions about impartiality amid potential commercial interests, while evidence on commercial queries and consumer-facing systems remains limited.
Source: Introduction · p. 1Method
The authors audit real commercial-advice queries across ChatGPT and Gemini interfaces and APIs and Google AI Overviews, examining recommendations and observable source information.
Source: Data and Methodology · p. 2, Response and Source Extraction · p. 4, Conclusion · p. 10Results
ChatGPT expresses first-person preference in 79% of product-recommending responses versus 7% for Gemini and 2% for AI Overviews, while displayed domains and API-interface outputs differ substantially.
Source: Conclusion · p. 10Takeaways & Limitations
Commercial advice cannot be understood through isolated responses or API observations alone; audits should account for repeated observations, delivery conditions, and the source information observed.
Source: Conclusion · p. 10The audit covers 117 physical-product queries repeated three times, excluding highly specific and geographically constrained requests, so it captures only part of commercial advice and not the full range of possible responses.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Consumers increasingly use AI chatbots for advice on what to buy. With companies like OpenAI and Google monetising their AI through advertising, this raises difficult questions about the bias and impartiality of such advice. In response, we conduct an AI audit of popular chatbots using real commercial-advice queries. First, we curate a dataset of 2,528 real commercial-advice queries (ConsumerQ). Then, we evaluate 1,536 responses to product queries from popular AI chatbots: ChatGPT (chatbot and API), Google Gemini (chatbot and API), and Google Search (AI Overviews). We find that ChatGPT expresses a first-person product preference in 79% of product-recommending responses, compared with 7% for Gemini and 2% for AI Overviews, while the products recommended often change across repeated requests. Displayed sources vary strongly: for the same query, the ChatGPT and Gemini interfaces share only 5.4% of domains on average, with no domain in common in 76.7% of comparisons. APIs provide a different view from their corresponding interfaces, with mean domain overlaps of 12.0% for ChatGPT and 14.8% for Gemini, and also differ in the types and layers of source information they expose. Our findings show that neither isolated responses nor API observations can be assumed to represent the commercial advice consumers encounter. Independent audits of AI-mediated commercial advice should therefore account for repeated responses, consumer-facing conditions, and the source layer being observed.
Introduction
AI assistants increasingly mediate product choices by selecting products, synthesising sources, and sometimes presenting personal recommendations. This audit examines how recommendations and sources vary across systems and repeated queries, and whether APIs reflect consumer interfaces.
Introduction
- Conversational and anthropomorphic design may affect trust and purchase intentions, while users may over-rely on LLM advice even when it is wrong.Repeated requests can also return different products or sources.Source: Introduction · p. 1
- Commercial advice raises impartiality questions amid potential advertising interests, data sharing, and publisher licensing arrangements.The paper also situates these questions within EU regulatory scrutiny and consumer-protection enforcement.Source: Introduction · pp. 1–2
- Prior audits focus largely on general queries and cited-source properties, while API audits using researcher-written templates may not capture commercial advice seen by consumers.The authors argue that commercial-advice audits should use consumer-facing interfaces.Source: Introduction · p. 2
- The paper asks how recommendations vary across providers and repeated queries, which sources they cite, and whether APIs can support audits of consumer-facing chatbots.Source: Introduction · p. 2
- ChatGPT expresses a first-person preference in 79% of responses, versus 7% for Gemini and 2% for AI Overviews; ChatGPT and Gemini share only 5.4% of displayed domains on average.For 76.7% of comparisons, they share no domain. The ChatGPT interface and API overlap by 12.0% on average, and Gemini’s by 14.8%.Source: Introduction · p. 2
Data and Methodology
The authors audit consumer-facing chatbots, APIs, and Google AI Overviews using real commercial-advice queries and compare answer and source information across conditions. They construct a product-query sample from CONSUMERQ and examine observable sources without treating APIs as proxies for interfaces.
Data and Methodology
- Consumer interfaces best capture recommendations presented to users, while APIs offer more controlled requests; neither provides full system observability, so the audit treats them as distinct conditions.The study covers ChatGPT and Gemini interfaces and APIs, plus Google AI Overviews; annotation tasks use calibrated codebooks and adjudicated gold standards.Source: Data and Methodology · p. 2
CONSUMERQ dataset
- CONSUMERQ contains 2,528 confirmed commercial-advice queries from LMSYS-Chat-1M and WildChat-1M; its users skew toward technology, so it supplies query wording rather than prevalence estimates.The annotated pool had 89.0% filter precision; the authors could not estimate recall because non-flagged queries were not annotated.Source: CONSUMERQ dataset · p. 3
Physical product recommendations
- The product-recommendation sample comprises 117 manually validated physical-product queries, filtered to exclude overly specific or geographically constrained requests.Queries were submitted verbatim; the sampled questions had a median length of eight words.Source: Physical product recommendations · p. 3
Audit Methodology
- Each query was submitted three times across five conditions, yielding 1,755 observations and 1,536 responses; interface requests were paired with identical API requests moments apart.The 702 interface–API pairs had a median delay of 0.17 seconds, reducing the chance that short-term search changes influenced responses.Source: Audit Methodology · pp. 3–4
Response and Source Extraction
- The audit separates generated answers, displayed sources, and additional source information, which differs by condition and may not represent all sources used internally.For example, the ChatGPT API reports search-returned pages and answer citations, while the Gemini API reports grounding sources but not the complete retrieved set; analyses concern observable source information.Source: Response and Source Extraction · p. 4
RQ1: How do product recommendations vary across AI providers?
Across consumer interfaces, recommendation framing differs markedly, and repeated queries can yield different product recommendations. The section compares framing and consistency across providers, while noting a naming caveat for product-overlap measures.
RQ1: How do product recommendations vary across AI providers?
- The RQ1 analysis uses 836 interface responses; 700 recommending a product or brand were further annotated for product order, presentation, best labels, first-person preference, and clarifying questions.Source: RQ1: How do product recommendations… · p. 4
How are product recommendations presented?
- 79% of ChatGPT product-recommending responses express first-person preference, versus 7% for Gemini and 2% for AI Overviews; “best” labels appear in 73%, 43%, and 48%, respectively.The figure compares measured framing properties among answers naming a product, with 95% bootstrap confidence intervals over queries.Source: How are product recommendations… · pp. 4–5, Figure 2 · p. 5
- Gemini and AI Overviews ask clarifying questions in 94% and 97% of responses, versus 82% for ChatGPT; product grouping by category occurs in 71%, 65%, and 44%, respectively.Source: How are product recommendations… · p. 5
How consistent are the recommended products?
- Across repeated queries, the presence of a product changes in 11–13% of queries, while first-person framing varies in 5% of AI Overview, 15% of Gemini, and 21% of ChatGPT queries.Best-label variation ranges from 27% to 40%; variation in product naming, best labels, and category presentation is similar across interfaces despite different base rates.Source: How consistent are the recommended… · p. 5
- Mean product Jaccard overlap across repetitions is 0.421 for AI Overview, 0.287 for Gemini, and 0.178 for ChatGPT; inconsistent naming makes reported overlaps lower bounds.ChatGPT’s overlap does not differ significantly from the overlaps between Gemini and AI Overview or between ChatGPT and AI Overview responses.Source: How consistent are the recommended… · p. 5
RQ2: What sources are displayed with AI
Displayed sources differ sharply across consumer-facing systems and change across repeated requests. The source set an audit observes also depends on which visible citation layer it records.
RQ2: What sources are displayed with AI
- The audit compares displayed sources at domain and exact-page levels, without inferring whether or how they contributed to recommendations.A single response captures only part of the source set encountered across repeated requests.Source: product recommendations? · pp. 5, 7
- 5.4% mean domain overlap between ChatGPT and Gemini, with no shared domain in 76.7% of comparisons; exact-page overlap was only 2.4%–6.5%.Among source-displaying responses, ChatGPT and Gemini each showed a median of three domains, compared with five for AI Overviews.Source: product recommendations? · pp. 5–6
- 26.0% for ChatGPT, 29.8% for Gemini, and 45.9% for AI Overviews were the mean domain overlaps across repeats; three requests increased ChatGPT’s distinct domains from 2.65 to 5.56.After three requests, distinct sources reached 4.37 domains and 4.54 pages for Gemini, and 7.52 domains and 9.38 pages for AI Overviews.Source: product recommendations? · p. 6
- Editorial and product-review sources made up 56.7% of displayed domains for ChatGPT and 45.2% for Gemini, compared with 25.4% for AI Overviews.AI Overviews also included user-generated, social, and community sources (22.2%), retailers and marketplaces (17.8%), and manufacturers and brands (16.1%).Source: product recommendations? · pp. 6–7
- AI Overviews combined three or more source types in 61.7% of responses, versus 18.8% for ChatGPT and 19.0% for Gemini.Single-type responses were 39.5% for ChatGPT, 32.5% for Gemini, and 12.5% for AI Overviews.Source: product recommendations? · p. 7
- AI Overview panels contained 5.79 domains and 7.17 pages on average, versus 4.92 domains and 5.83 pages inline; inline citations were contained in the panel.The two layers had mean Jaccard similarities of 88.5% for domains and 86.7% for pages; Gemini citation spans covered a median of 39.0% of response characters, which does not establish source support.Source: product recommendations? · p. 7
RQ3: Can APIs be used to audit AI product
APIs do not reproduce the sources shown in their corresponding interfaces: source incidence, domains, pages, and source composition differ. These are comparisons between provider-controlled conditions, not effects attributable to access method alone.
RQ3: Can APIs be used to audit AI product
- Because model configuration and other provider-controlled components also differ, these comparisons measure whether conditions reproduce one another, not an access-method effect.An API audit characterizes its specific API condition and observed source layer, not the consumer’s source environment.Source: recommendations? · pp. 7–8
- ChatGPT interface–API domain overlap was 12.0% and Gemini’s was 14.8%; the pairs shared no domain in 60.9% and 43.4% of cases, respectively.APIs displayed sources more often by 9.7 percentage points for ChatGPT and 8.1 points for Gemini, but this did not yield similar source sets.Source: recommendations? · p. 7, Figure 5 · p. 8
- Domain overlap across repeats was higher for APIs than interfaces for ChatGPT (37.9% versus 26.0%) and Gemini (43.0% versus 29.8%), while page-level patterns split by provider.Page overlap fell from 18.1% to 13.3% for ChatGPT but rose from 28.4% to 34.1% for Gemini.Source: recommendations? · p. 7
- Matched domain display differences included greater API exposure for manufacturer domains on ChatGPT and for YouTube and Reddit on Gemini, by 44.4 and 29.1 percentage points.Technology publishers appeared more often through the ChatGPT interface; the figure plots matched display-probability differences, with positive values indicating greater API exposure.Source: recommendations? · p. 7, Figure 6 · p. 8
- Gemini’s API displayed 661 distinct domains versus 376 through its interface, while its API source mix included three or more types in 69.9% of responses versus 19.0% in the interface.ChatGPT’s single-type responses increased from 39.5% in the interface to 52.3% in the API, unlike Gemini’s drop from 32.5% to 4.2%.Source: recommendations? · p. 8, Figure 7 · p. 9
- ChatGPT search returned 13.48 domains and 37.38 pages per observation on average, while final citations contained 2.62 domains and 3.22 pages; Gemini exposes no separate search-returned set.The returned sources are not established as contributors to the answer, and the two APIs expose different process stages.Source: recommendations? · p. 8
Auditing AI-mediated Commercial Advice
Recommendations, their framing, and displayed sources vary across systems and repeated requests, while APIs do not reproduce consumer interfaces. The audit is limited by its query, geographic, temporal, and system coverage, and cannot explain interface–API differences.
Auditing AI-mediated Commercial Advice
- Among product-recommending responses, ChatGPT expressed a first-person preference in 79%, versus 7% for Gemini and 2% for AI Overviews; systems also differed in recommendation decisiveness and presentation.Source: Auditing AI-mediated Commercial Advice · p. 9
- ChatGPT and Gemini shared only 5.4% of displayed domains for the same query, while repeated requests increased ChatGPT’s mean observed domains from 2.65 to 5.56.A single response cannot characterize a system’s recommendations or sources.Source: Auditing AI-mediated Commercial Advice · p. 9
- Interface and API responses differed: ChatGPT shared only 12.0% of displayed domains, and the source layers exposed across conditions do not alone establish which sources contributed to recommendations.The ChatGPT API returned 37.38 pages per observation through web search on average, while 3.22 appeared in final citations.Source: Auditing AI-mediated Commercial Advice · p. 9
Limitations
- The audit covers 117 manually annotated physical-product queries repeated three times, excluding highly specific and geographically constrained requests.Three repetitions cannot capture the full range of responses a condition may produce.Source: Limitations · p. 9
- The queries are predominantly in English and responses were collected in the Netherlands, so results should not be read as representative of Dutch consumer behaviour.The query sources are not representative samples of all consumers.Source: Limitations · p. 9
- The audit excludes assistants such as Claude, and AI Overviews appeared in only 134 of 351 searches; results for them apply only where an overview appeared and was recovered.The systems were captured at one point in time, while their models, search components, and interfaces continue to change.Source: Limitations · pp. 9–10
- Interface–API comparisons show whether conditions produce similar recommendations and sources, but not why they differ; model configuration and other provider-controlled components may also differ.The observed differences cannot be attributed to access method alone.Source: Limitations · p. 10
- Measuring source visibility could inform commercial efforts to optimise content for AI advice, potentially advantaging actors with resources to influence these systems.The authors report these patterns because understanding how commercial information reaches consumers is necessary for independent auditing and consumer protection.Source: Limitations · p. 10
Related Work
Prior work examines shopping-assistant recommendations and generative-search properties, but much relies on researcher-designed queries and APIs. Commercial advice across major providers and consumer interfaces remains insufficiently studied.
Related Work
- Shopping-assistant research has studied popularity and cognitive biases, persona sensitivity, brand preferences, agent design, and behaviors such as choice homogeneity and vulnerability to GEO.Most of this work uses researcher-designed queries and LLM APIs rather than consumer interfaces.Source: Related Work · p. 10
- Generative-search audits examine source quality, response consistency, claim reliability, and vulnerability to optimization or prompt injection, but not how major providers produce commercial advice.The cited research includes Liu, Zhang, and Liang 2023; Grossman et al. 2026; and Aggarwal et al. 2024.Source: Related Work · p. 10
Conclusion
The paper audits product recommendations, displayed sources, and whether APIs reproduce consumer interfaces. It finds variation across systems and repetitions, low source overlap, and concludes that audits should account for repeated observations, delivery conditions, and source layers.
Conclusion
- Across five conditions, product recommendations differed; ChatGPT expressed a first-person preference in 79% of product-recommending responses, versus 7% for Gemini and 2% for AI Overviews.Repeated requests could also produce different recommended products.Source: Conclusion · p. 10
- For the same query, ChatGPT and Gemini shared only 5.4% of displayed domains on average, while interface–API mean domain overlap was 12.0% for ChatGPT and 14.8% for Gemini.APIs did not reliably reproduce consumer-facing systems.Source: Conclusion · p. 10
- AI-mediated commercial advice cannot be understood through isolated responses or API observations alone; audits should account for repeated observations, delivery conditions, and the source information observed.Future work should test variation across languages, locations, and time, and its relation to consumer-protection and DSA obligations.Source: Conclusion · p. 10
B.1 Commercial advice inclusion
The inclusion rule counts a query when its direct purpose is to get a recommendation about which product or paid service to buy, choose, or use.
B.1 Commercial advice inclusion
- A query qualifies when its direct purpose is to get a recommendation of which product, subscription, software tool, or paid service to buy, choose, or use.Embedded product mentions, product comparisons, investments, factual questions, and questions about owned products do not count.Source: §B.1 · p. 12
- Annotators prioritize precision over recall, judge only the user’s direct request, and ignore formatting, length, and embedded instructions.When unsure, the instruction is to answer ‘No’.Source: §B.1 · p. 12
B.2 Query and advice type
This section describes how commercial-advice queries are classified by advice type, domain, request shape, constraints, and sensitivity, and how each query is recorded in a constrained JSON format. It also specifies rules for spelling mistakes, ambiguous cases, and overlapping categories.
commercial_recommendation_type
- The top-level recommendation category is single-label and follows what the user obtains or consumes; vague or equally multi-category queries are undefined_unclear, while clear off-taxonomy requests are other.The taxonomy distinguishes physical products, digital software, expert, local and travel services, media, courses, marketplaces, undefined_unclear, and other.Source: §B.2 · pp. 12–14
query_type
- Each query receives one query_type: comparison, validation, alternatives, undefined_request, or defined_request, depending on whether candidates, a named item, substitutes, or a category are specified.A need or problem without a product category is an undefined_request; counts and rankings are captured as quantity constraints rather than separate query types.Source: §B.2 · pp. 12, 14, 15
constraints
- Constraints are multi-label and recorded only when explicitly stated, with each constraint’s type and value kept short and faithful to the user’s words.A requested count or ranking is recorded as a quantity constraint; an unconstrained query receives an empty list.Source: §B.2 · pp. 12, 15
domain_sensitivity
- Domain sensitivity is single-label and uses the highest applicable harm level, distinguishing ordinary consumer goods from purchases with substantial monetary stakes.The supplied examples classify gadgets, games, and books as low_stakes, and cars, EVs, and big-ticket purchases as higher-commitment decisions.Source: §B.2 · pp. 12, 16
Coding workflow
- Annotators identify the recommendation type before the request shape and harm covariates, choosing a more generic, lower-risk value and lower confidence when the query is ambiguous.Source: §B.2 · p. 12
Query and advice type
- Sensitivity categories follow a fixed priority order: law infringement > legal grey > safety > health > adult/NSFW > AI > financial > low stakes.When multiple categories apply, the highest-priority category wins.Source: §B.2 · p. 16
- When queries contain spelling mistakes, the judge applies the most logical interpretation.Source: §B.2 · p. 16
- The judge returns JSON with query and commercial-recommendation types, constraints, domain sensitivity, and a short justification; the recommendation-type list contains exactly one value.An unconstrained query has an empty constraints list, and the justification is limited to 25 words.Source: §B.2 · p. 16
B.3 Broad physical product recommendation inclusion
The codebook sets inclusion rules for broad physical-product recommendation queries and specifies how responses and displayed-source domains are annotated. It also defines how publisher-type assignments and their rationale are recorded.
B.3 Broad physical product recommendation inclusion
- Queries qualify when they ask for help choosing a readable English consumer product, exclude specific products and other specified cases, and meet the codebook’s openness and constraint rules.The rules exclude services, business products, free-product requests, specific prices, places, or dates, as well as requests for lists or specific numbers of recommendations.Source: Part A: Question type · p. 16, Part B: Product type · p. 17, Part C: Question openness · p. 17
B.4 RQ1 codebook
- The RQ1 codebook records whether responses mention specific products, brands, or retailers, and distinguishes safety, knowledge, and preference refusals.It also tracks avoid or don’t-buy mentions and proceeds to further questions when a product, brand, or retailer is mentioned.Source: §B.4 · p. 17
2. Recommendation list
- Recommendation lists are classified by whether they contain one option, a main recommendation with alternatives, category-based options, a ranked list, or an unranked list.The codebook also records recommendation strength and confidence limitations, product prices and review claims, and clarifying questions.Source: §2 · p. 18
B.5 Source type
- For source-type annotation, each domain is supplied with its registered domain, displayed-page titles and URLs, audit conditions, and an index.The titles and URLs are truncated, and the pooled page evidence is de-duplicated and ordered by first appearance.Source: §B.5 · p. 19
1. Publisher type
- Publisher type is an identity classification, not a quality rating, and each domain receives exactly one of ten publisher categories.A Netherlands-oriented label applies only when the publisher or service primarily targets the Netherlands; a .nl suffix alone is insufficient.Source: §1 · p. 19
3. Ambiguous
- The output schema includes a field indicating whether the model considers its publisher-type assignment uncertain, but the instruction does not further define that field.Source: §3 · p. 19
4. Rationale
- The rationale instruction asks for the model’s instruction verbatim, with domain batches sent as JSON and responses constrained to the specified fields.The domain-classification instruction requires one allowed publisher type, a Netherlands-orientation label under the stated rule, and preserved indices.Source: §4 · p. 19