PaperlayerView exact revision on arXiv

Source-linked AI summary

Shopping by algorithm: How agentic AI deploys human heuristics as a surrogate consumer

Davood Wadi, Yu Ma

arXiv:2609.28372v1econ.GNcs.AI

TL;DR

Consumers increasingly delegate purchasing decisions to LLMs, but whether human-targeted pricing cues influence AI shopping agents and what governs that influence remain open questions. Using Tool-Lab to trace information acquisition across eight deployed LLMs, the paper finds that costly attribute access under vague goals leads most evaluated models to omit diagnostic information and make choices resembling human heuristics.

  • Problem

    It remains unclear whether pricing cues designed for human bounded rationality influence autonomous AI shopping agents and what governs when they do.

    Source: Introduction · p. 1, General discussion · p. 15

  • Method

    The paper introduces Tool-Lab, which traces LLMs’ pre-choice information acquisition while varying attribute acquisition costs and goal-prompt specificity.

    Source: Introduction · p. 2

  • Results

    With costly attribute acquisition and vague goals, most evaluated LLMs omitted diagnostic attributes, and their resulting choices resembled human heuristic decisions.

    Source: Introduction · p. 2, General discussion · p. 15

  • Takeaways & Limitations

    Pricing-cue influence in delegated AI shopping depends chiefly on the information agents hold at choice, rather than on cue weighting alone.

    Source: General discussion · p. 15

    Choice optimality measures selection of the lowest-unit-price alternative but excludes search costs, so it does not capture consumer welfare.

    Source: Limitations and future research · p. 18

Abstract

from arXiv · show

Consumers increasingly delegate purchasing decisions to Large Language Models (LLMs) acting as surrogate consumers. Using "Tool-Lab," an adaptation of information-board process tracing that places product attributes behind costly tool calls, we examine how marketing pricing cues (i.e., just-below pricing and promotional framing) influence AI shopping agents. Across eight commercially deployed LLMs from three providers, we trace pre-choice information acquisition. Under zero cost, pricing cues rarely mislead. Imposing acquisition costs under a vague goal prompt leads LLMs to omit diagnostic attributes required to compute unit price and choose suboptimal choices resembling human heuristics. Relative to a specific goal prompt that mainly preserves diagnostic search and choice optimality, a vague goal prompt under constraints creates a search-mediated vulnerability. This research demonstrates that marketing heuristics in delegated AI shopping are governed by storefront information architecture, not necessarily immutable LLM flaws.

Introduction

In delegated AI shopping, vulnerability to marketing cues depends on the pre-choice information state shaped by retrieval costs and prompt specificity, not simply on fixed LLM flaws. Tool-Lab makes this search pathway observable and shows how missing diagnostic attributes can produce heuristic-like, suboptimal choices.

Introduction

  • Consumers increasingly delegate purchasing to autonomous LLM surrogate consumers, leaving open whether human-targeted just-below pricing and promotional framing still influence choices.Prior pricing strategies were designed to exploit human bounded rationality.Source: Introduction · p. 1
  • Retailer-controlled retrieval costs and consumer prompt specificity jointly shape agents’ pre-choice information; costly retrieval plus vague goals can truncate search and undermine choice.Vague prompts leave unclear which attributes are diagnostic, including cents or product weight needed to calculate unit prices.Source: Introduction · pp. 1–2
  • Tool-Lab hides product attributes behind costly tool calls and records pre-choice acquisitions, enabling direct measurement of search depth and composition.Across 16,400 sessions, the studies evaluate eight deployed LLMs from Google, OpenAI, and Anthropic, contrasting zero-cost with $10.00 or $50.00 acquisition costs and vague with specific goals.Source: Introduction · p. 2
  • Under full information and specific goals, LLM choices largely align with normative optima, but acquisition costs under vague goals can remove diagnostic attributes and produce suboptimal choices.Validation experiments (N = 5,200) indicate that for most LLMs, suboptimality stems from missing diagnostic attributes rather than arithmetic failures.Source: Introduction · p. 2
  • The paper challenges the view that LLM heuristic-like behavior is solely inherited from training, identifies delegation instructions as a determinant, and introduces process tracing of AI information acquisition.Source: Introduction · p. 2
  • Retailers should make diagnostic attributes available, consumers should state explicit criteria, developers should clarify vague goals and audit search truncation, and regulators should consider AI-facing unit-price disclosures.Source: Introduction · p. 2

Literature review

Marketing research increasingly considers LLMs as autonomous surrogate consumers, but their shopping behavior combines analytical capabilities with heuristic susceptibility. The literature leaves open whether heuristic-like choices are innate or shaped by storefront information and delegated prompts.

AI agents as surrogate consumers

  • LLMs are increasingly used as autonomous shopping assistants, extending surrogate consumption from human delegates to agents that can evaluate, select, and acquire goods (Schick et al., 2023; Wang et al., 2024).Prior behavioral research more often treated LLMs as passive synthetic respondents for emulating survey responses (Joerling, 2025; Sarstedt et al., 2024; Arora et al., 2025).Source: AI agents as surrogate consumers · pp. 2–3

The rationality paradox

Marketing pricing cues

Storefront information architecture

  • Heuristic-like AI choices may arise from storefront information architecture and delegating prompts, rather than solely from innate training defects.The innate-defect account is unverified, and terminal choices cannot show whether diagnostic calculations were attempted (Johnson et al., 2008).Source: Storefront information architecture · p. 4

Conceptual development

The conceptual framework defines unit-price minimization as the normative objective for packaged-goods choices and treats heuristic choice as a resource-conserving departure from full evaluation. It proposes that acquisition costs and vague goals can reduce diagnostic information acquisition, undermining choice optimality.

Normative and heuristic objectives

  • The normative objective is to minimize unit price; heuristics conserve resources by substituting a simpler objective or evaluating the normative one with fewer attributes.Diagnostic attributes enter the unit-price equation, including price components, discount percentages, and weight; other attributes, such as packaging or calories, are non-diagnostic. Promotional discount evaluation can also invite transaction utility, such as maximizing the discount percentage or minimizing discounted list price.Source: Normative and heuristic objectives · p. 5

Acquisition Costs

  • LLM acquisition costs include external constraints from storefront or prompt architecture and internal constraints on computational resources and search.The framework uses acquisition cost to cover both kinds of constraint; exhaustive inspection through unbounded tool calls could make task execution prohibitively slow and expensive.Source: Acquisition Costs · p. 5

Goal specificity

  • Under high acquisition costs, vague goals can reduce diagnostic acquisition by prompting objective substitution or omission of costly attributes, making choices suboptimal when omitted information affects rankings.The proposed pathway is that vague prompts negatively moderate the effect of acquisition cost on diagnostic acquisition, diagnostic acquisition supports choice optimality, and prompt specificity moderates the indirect effect of acquisition cost on optimality.Source: Goal specificity · pp. 5–6

The Tool-Lab paradigm

Tool-Lab adapts process tracing to delegated LLM purchasing by hiding product attributes behind costly tool calls, making pre-choice information acquisition observable.

The Tool-Lab paradigm

  • Tool-Lab embeds LLMs in a stateful environment where attributes are hidden behind tool calls that incur consumer-paid costs, while choice submission is free.Agents reveal attributes with inspect_cell calls and end the session by submitting a choice.Source: The Tool-Lab paradigm · p. 6
  • Recording pre-choice acquisitions quantifies total and diagnostic information search and distinguishes suboptimal choices under complete information from those under incomplete information.When LLM reasoning capabilities and provider access permit, Tool-Lab can also record reasoning traces between tool calls.Source: The Tool-Lab paradigm · p. 7
  • Because identical prompts can produce heterogeneous search trajectories and decisions, Tool-Lab evaluates each experimental cell across multiple independent sessions per LLM.Sessions have no state retention between trials; option and attribute order is randomized, and tool responses are validated against strict JSON schemas.Source: The Tool-Lab paradigm · p. 7
  • The experiments test eight commercially deployed LLMs from Google, OpenAI, and Anthropic.The models are Gemini 3.5 Flash Lite, Gemini 3.8 Flash, Gemini 3.1 Pro, GPT 5.6 Luna, GPT 5.6 Terra, GPT 5.6 Sol, Claude Sonnet 5, and Opus 5.Source: The Tool-Lab paradigm · p. 7

Study 1

Under costly information acquisition, vague goals lead LLMs to omit diagnostic attributes and make less optimal choices, while specific goals largely preserve diagnostic search and choice quality. The section also tests whether these patterns depend on prompt wording, price presentation, or purchase volume.

Baseline

  • With full attribute information, cent-to-dollar sensitivity ranged from 0.55 to 1.12, and seven of eight LLMs’ 95% confidence intervals contained 1.00.Flash discounted cents relative to dollars; for the other LLMs, there was no evidence of systematic discounting or favoring of the cent component.Source: Baseline · p. 8

Tool-Lab

  • Tool-Lab tests how acquisition costs and constrained information affect LLM choices, which the full-information baseline could not trace.The setup places product attributes behind costly tool calls to trace pre-choice information acquisition.Source: Tool-Lab · p. 8

Stimuli and sample

  • The coffee choice task contrasted $4.99 for 10 oz (0.499 $/oz) with $5.00 for 11 oz (0.455 $/oz), requiring six diagnostic attribute inspections to compute unit price.The design crossed $0 versus $10 per inspection with vague versus specific goals across eight LLMs and 3,200 simulated sessions.Source: Stimuli and sample · p. 8

Procedure and measures

  • The prompts let LLMs inspect attributes or submit a choice, and the study measured choice optimality and how many of six diagnostic attributes were acquired.The potential surplus was scaled to five purchased bags so that identifying the optimal product could mathematically justify cumulative tool costs.Source: Procedure and measures · p. 8

Results

  • With $10 costs, vague prompts yielded diagnostic acquisition M = 3.81 (SD = 1.30) and choice optimality M = 0.29 (SD = 0.28), versus specific-prompt M = 5.72 and M = 0.83.Under vague prompts, costs reduced acquisitions of cents (M = 0.77, SD = 0.78) and weight (M = 1.10, SD = 0.64), while dollars remained high (M = 1.94, SD = 0.09). Diagnostic acquisition positively predicted choice optimality (b1 = 0.263, SD = 0.034, 95% CI [0.196, 0.329]); the indirect effect was negative under vague prompts (indirect effect = −0.522, SD = 0.132, 95% CI [−0.783, −0.256]) but overlapped zero under specific prompts (indirect effect = −0.070, SD = 0.057, 95% CI [−0.184, 0.044]).Source: Results · p. 9

Prompt variation

  • Prompt variations replicated the moderated pathways for Pro and Flash, while sentence reordering removed the cost-related acquisition truncation and attenuated the indirect effect for Flash Lite.For Flash Lite under sentence reordering, a3 = −0.120 (95% CI [−0.375, 0.140]) and IMM = −0.029 (95% CI [−0.093, 0.034]); diagnostic acquisition still predicted choice optimality in every variation.Source: Prompt variation · p. 10

Price partitioning

  • With unified price attributes, costly search under vague prompts still reduced diagnostic acquisition and optimality, shifting omitted information from cents to weight.Weight acquisition fell to M = 0.56 to 1.71 while price-cell acquisitions remained M = 1.96 to 2.00; the specific prompt showed no cost effect on diagnostic acquisition or optimality.Source: Price partitioning · p. 11

Sensitivity to purchase volume

  • A purchase-volume experiment tested whether Pro’s incomplete diagnostic search under vague goals reflected low-value attribute truncation or substitution of the normative objective.The design varied purchase volume across 1, 5, 50, and 250 bags for Flash Lite, Flash, and Pro under vague goals and zero acquisition cost; no results are provided in the supplied passages.Source: Sensitivity to purchase volume · p. 11, §4 · p. 11

4 (Purchase volume:

Increasing purchase volume reduced diagnostic information acquisition and choice optimality for Flash Lite and Pro, while Pro increasingly adopted an absolute-price objective. Within every purchase-volume condition, Pro sessions using that objective searched less diagnostically and were less likely to choose optimally than unit-price-objective sessions.

Purchase volume

  • As purchase volume rose from 1 to 250 bags, Pro’s diagnostic acquisitions fell from M = 5.55 to 4.22 and optimality from .80 to .35; Flash Lite’s optimality fell from .62 to .37.Flash remained at ceiling (M = 6.00; optimality = 1.00). Flash Lite retained cents acquisition (M ≈2.00), while weight acquisition fell from M = 1.82 to 1.30 and diagnostic acquisition from M = 5.77 to 5.30.Source: §4 · p. 11
  • Human coding validated the judge LLM’s classification of reasoning objectives with 89.2% agreement and Cohen’s κ = .78.A human evaluator blindly coded a stratified subsample of 120 traces.Source: §4 · p. 12
  • Flash adopted a unit-price objective in 90.8% to 94.0% of sessions, while Pro’s absolute-price objective rose from 38.0% at 1 bag to 83.0% at 50 bags and 84.0% at 250 bags.Under an absolute-price objective, cents and product weight were functionally non-diagnostic because Coffee A’s whole-dollar price was lower than Coffee B’s.Source: §4 · p. 12
  • Across every volume condition, Pro sessions with an absolute-price objective acquired 1.10 to 1.95 fewer diagnostic cells and were 42 to 64 percentage points less likely to choose optimally than unit-price sessions.All 95% bootstrap confidence intervals for the diagnostic-acquisition differences excluded zero.Source: §4 · p. 12

Study 2

Study 2 finds that promotional discount cues are usually integrated into choices when information is free, but acquisition costs under a vague goal prompt reduce diagnostic search and choice optimality. The moderated mediation results link this vulnerability to omitted diagnostic attributes rather than a direct cost penalty.

Study 2

  • Evaluating a promotional discount requires applying it to list price to calculate net expenditure, then dividing by product weight to derive unit value.Source: Study 2 · p. 12

Baseline

  • With freely available information, all eight LLMs used list price negatively, discount amount positively, and weight positively in choice.The coefficient ranges were βLP = −1.06 to −3.08, βDA = 0.76 to 3.08, and βW = 0.78 to 1.79; six models valued discount savings about equivalently to an equal list-price reduction (ratio = 0.99 to 1.03), while Opus over-weighted them (ratio = 1.15) and Flash Lite under-weighted them (ratio = 0.72).Source: Baseline · p. 13

Tool-Lab

  • Tool-Lab compares zero versus $50.00 inspection costs and vague versus specific prompts across eight LLMs in 3,200 sessions.Its three alternatives represented lowest price, lowest price per weight, or discounted choice; the 21-cell board contained nine diagnostic and 12 non-diagnostic cells.Source: Tool-Lab · p. 13

Results

  • With $50.00 acquisition costs, vague prompts yielded lower diagnostic acquisition (M = 4.82, SD = 3.49) and choice optimality (M = 0.48, SD = 0.29) than specific prompts (M = 8.26, SD = 1.03; M = 0.88, SD = 0.23).Under the vague prompt, discount percentage (M = 1.58, SD = 1.23) and weight (M = 1.31, SD = 1.14) were inspected less than list price (M = 1.94, SD = 1.24); under the specific prompt, the cost-related decline was driven primarily by discount percentage (M = 2.50, SD = 0.59).Source: Results · pp. 13–14
  • Moderated mediation supports a search-mediated effect of acquisition costs on choice optimality, with prompt specificity moderating the indirect effect.Under vague prompts, costs reduced diagnostic acquisition (aVague = −3.576, SD = 1.218, 95% CI [−5.925, −1.062]); under specific prompts, the reduction overlapped zero (a1 = −0.674, SD = 0.462, 95% CI [−1.582, 0.260]). Diagnostic acquisition predicted optimality (b1 = 0.078, SD = 0.031, 95% CI [0.016, 0.141]); the Index of Moderated Mediation was credibly negative (IMM = −0.228, SD = 0.132, 95% CI [−0.521, −0.009]), while the direct effect was uninformative (c′ = −0.006, SD = 0.020, 95% CI [−0.045, 0.035]).Source: Results · p. 14

Validation studies

The validation studies show that most LLMs choose optimally when given complete diagnostic information and that choices under partial information generally follow the normative policy for the information available. They also show that suboptimal choices cannot be strictly attributed to inability to calculate the normative objective, though the arithmetic test may alter choice.

Choice under exogenously assigned information states

  • With complete diagnostic information and zero cost, seven of eight LLMs were optimal in most sessions: pooled optimality was 99.7% in Study 1 and 98.6% in Study 2.Flash Lite was the exception, choosing optimally in 4.0% of Study 1 sessions and 0.0% of Study 2 sessions.Source: Choice under exogenously assigned… · pp. 14–15
  • Choice degradation under partial information was primarily driven by limited diagnostic information, motivating tests that fixed the information state to distinguish acquisition from utilization failures.The design also included matched-count conditions varying diagnosticity while holding the number of revealed cells constant.Source: Choice under exogenously assigned… · p. 14
  • When diagnostic attributes were withheld, seven models matched the information-conditional Expectimax policy in 99.3% of sessions (2,434 of 2,450 truncated trials).Choosing the lower whole-dollar or list price under partial information was normatively consistent in the tested settings.Source: Choice under exogenously assigned… · pp. 14–15

Reported valuations

  • Arithmetic failure was 0.0% across all eight LLMs, both study environments, and all cost conditions; hallucinations for uninspected cells ranged from 0.0% to 8.5% of sessions.Whenever diagnostic attributes were inspected, every LLM computed total prices and unit values without mathematical error, and uninspected cells were correctly reported as unknown when search was truncated.Source: Reported valuations · p. 15
  • The valuation test shows suboptimal choices cannot be strictly attributed to inability to compute the normative objective, but its concurrent calculation-and-reporting requirement may alter choice.Unlike the main experiments, the validation task required reporting price metrics alongside the choice, creating an intermediate scratchpad.Source: Reported valuations · p. 15

General discussion

Across eight commercially deployed LLMs, pricing cues mainly affected delegated choices through the attributes agents acquired, not how they weighted the cues. Under costly search and vague goals, most models omitted diagnostic information and made choices resembling human heuristic decisions.

General discussion

  • With costly acquisition and vague goals, most evaluated LLMs omitted diagnostic attributes, and choices reflected the information acquired rather than cue weighting.The resulting choices resembled human heuristic decisions.Source: General discussion · p. 15
  • With all attributes visible, seven of eight LLMs weighted cents like whole dollars and six of eight treated a dollar saved through a discount like a dollar spent through list price.With partial information, seven of eight LLMs selected the alternative implied by the normative policy in 99.3% of sessions; no LLM made an arithmetic error when reporting unit values.Source: General discussion · p. 15
  • Under vague goals, cost reduced weight acquisition (Study 1: M = 1.10 of 2; Study 2: M = 1.31 of 3), although weight is diagnostic for unit-price but not absolute-price objectives.Under specific goals, cost mainly reduced acquisition of cents or discount percentage; when price was consolidated into one attribute, truncation shifted to weight, and the pathway replicated for the three Gemini LLMs tested.Source: General discussion · p. 16
  • As volume rose from one to 250 bags, Pro’s absolute-price reasoning rose from 38% to 84%, diagnostic acquisition fell from 5.55 to 4.22, and choice optimality fell from .80 to .35.Holding volume constant, sessions articulating an absolute-price objective acquired fewer diagnostic attributes and were 42 to 64 percentage points less likely to select the optimal alternative.Source: General discussion · p. 16
  • All three hypotheses were supported for seven of eight LLMs in Study 1 and six of eight in Study 2 across three providers, with replication across prompt phrasings and consolidated-price presentation.Luna maintained diagnostic acquisition under cost, while Flash Lite made suboptimal choices even with complete information.Source: General discussion · p. 16
  • The pathway may extend beyond tested models because capability cannot evaluate attributes an LLM did not acquire, while retailers set acquisition costs and consumers set goal specificity.The authors note that benchmarks also find cognitive biases in LLMs are not resolved by increasing capability (Echterhoff et al., 2024; Knipper et al., 2025; Malberg et al., 2025), and Tool-Lab can be rerun on newer models.Source: General discussion · p. 16

Theoretical implications

The paper argues that AI agents’ susceptibility to marketing cues depends on both the information architecture they shop through and the principal’s delegation instruction. Tool-Lab separates failures of information acquisition from failures of information use when evaluating agent choices.

Theoretical implications

  • Heuristic-like behavior in delegated shopping emerges from the interaction between the AI agent and its information architecture, shifting analysis beyond the LLM in isolation.Training may shape behavior, but does not alone determine whether marketing cues influence delegated choices.Source: Theoretical implications · p. 16
  • Under costly acquisition, delegation-instruction specificity and retailer information architecture jointly determine whether marketing cues influence an AI agent’s choice.The principal’s instruction shapes the surrogate’s cue susceptibility even though the consumer never encounters the cue.Source: Theoretical implications · p. 17
  • Tool-Lab shows that most suboptimal choices arose from incomplete information acquisition, enabling separate evaluation against the information held and the full attribute set.It observes acquisition and choice, not neural-network internals, and offers a reusable method for studying agents that must acquire information before choosing.Source: Theoretical implications · p. 17

Managerial implications

The findings imply that outcomes depend on whether AI agents can access diagnostic product attributes and understand the consumer’s objective. Retailers, consumers, developers, and regulators can reduce suboptimal choices by improving attribute access, clarifying objectives, and checking agents’ information acquisition.

Retailers

  • Retailers should not rely on human-oriented pricing cues to influence agents with complete information, but should make diagnostic attributes readily available to prevent suboptimal choices.Source: Managerial implications · p. 17

Consumers

  • Consumers should state the criterion that defines a good choice, such as lowest price per unit, because vague prompts can lead agents to pursue simpler objectives and make suboptimal purchases under constraints.Source: Managerial implications · p. 17

AI agent developers

  • Developers should clarify vague objectives, verify that agents acquire the attributes required by consumers’ objectives, and audit each deployed LLM because vulnerability varies substantially across models.Source: Managerial implications · p. 17

Regulators

  • Regulators should consider requiring diagnostic attributes to be easily accessible to AI agents, since disclosure rules for human shoppers may not ensure agents can acquire value-relevant information.Source: Managerial implications · p. 17

Limitations and future research

The findings are qualified by limits in what Tool-Lab measures, the cost and choice settings studied, and the outcomes used; the authors propose research to test mechanisms, scope, welfare, and strategic interactions.

Mechanisms and measurement

  • Tool-Lab traces observable tool calls and verbal reasoning reports, not neural mechanisms, so it shows when truncation yields heuristic-like choices but not why truncation occurs.Reasoning traces were available only from providers that expose them and were classified only for three Gemini LLMs in the purchase-volume study, making evidence about objective substitution versus value-based omission suggestive. Future work could use activation patching and vary attribute value while holding the stated objective constant.Source: Limitations and future research · pp. 17–18

LLM-level differences

  • Flash Lite chose suboptimally in Study 2 even with complete diagnostic information, while undisclosed, co-varying LLM characteristics prevent explaining individual differences.Future research could identify moderators of acquisition and choice pathways, including by manipulating reasoning effort within an LLM.Source: Limitations and future research · p. 18

Acquisition costs

  • Tool-Lab imposes explicit monetary inspection costs, which are measurable but differ from agents’ internal deployment costs.Future studies could test whether live e-commerce costs produce similar diagnostic truncation and whether internal costs, such as reasoning-effort settings, have comparable effects.Source: Limitations and future research · p. 18

Outcome measurement

  • Choice optimality measures selection of the lowest-unit-price alternative but excludes search costs, so optimal choices can still incur costs exceeding the purchase cost.The authors propose welfare-inclusive measures combining product value, search costs, and time to assess when agent choices are economically defensible or normatively suboptimal.Source: Limitations and future research · p. 18

Scope and generalizability

  • The studies used one product category with an unambiguous unit-price objective and one stimulus configuration per study, limiting generalization to broader choice settings.The specific goal prompt named the normative objective exactly, likely an upper bound on goal specificity’s benefit; future work could test larger choice sets, varied price advantages, naturalistic multi-attribute or affective requests, and clarifying questions.Source: Limitations and future research · p. 18

Strategic interactions

  • The studies placed one shopping agent in a static environment, leaving untested strategic interactions in which seller agents adapt product information to buyer searches.Future research could examine whether seller agents learn to exploit the omission patterns documented in the studies.Source: Limitations and future research · p. 18

Statements and declarations

The authors report no human-subject involvement or relevant competing interests, state that all authors contributed and approved the manuscript, and make the experimental datasets available through an anonymized OSF repository.

Statements and declarations

Tables and figures

In Study 2, higher acquisition costs under a vague goal prompt were associated with fewer diagnostic acquisitions and lower choice optimality, with a negative conditional indirect effect through diagnostic acquisition.

Study 1 Tool-Lab

  • Figure 1 presents the conceptual framework, while Figure 2 depicts the Study 1 Tool-Lab setup with a $10 cost per inspect_cell call.Source: Figure 1 · p. 23, Figure 2 · p. 24

Studies 1–2 Tool-Lab

  • The tables report diagnostic-acquisition counts and choice optimality across eight LLMs, with 3,200 sessions per study; Study 1 acquisition counts combine price dollars, price cents, and weight across alternatives.Study 1 diagnostic acquisitions have a maximum of 6; Study 2 counts list price, discount percentage, and weight across three alternatives, with a maximum of 9.Source: Table 1 · p. 23, Tables and figures · pp. 23, 25, Table 3 · p. 25

Study 2 Tool-Lab

  • Study 2 defines the normatively optimal alternative as Coffee E: $10.00 list price, 0% discount, 10.9 oz, and $0.917/oz.Choice optimality is the proportion of sessions selecting that alternative.Source: Table 3 · p. 25, Tables and figures · p. 25
  • In Study 2, acquisition cost under the vague goal prompt (aVague) was -3.576* 1.218 [-5.925, -1.062], while diagnostic acquisition (b1) was 0.078* 0.031 [0.016, 0.141].The first-stage path is in cell counts and the second-stage path is in probability units.Source: Table 4 · p. 25, Tables and figures · p. 26
  • For the vague goal prompt, the conditional indirect effect was -0.280* 0.154 [-0.625, -0.026], and the index of moderated mediation (IMM c) was -0.228* 0.132 [-0.521, -0.009].The specific-prompt conditional indirect effect was -0.053 0.044 [-0.156, 0.021], whose interval includes zero.Source: Table 4 · p. 25, Tables and figures · p. 26

Web appendix A: Supplementary figures and tables · Study 1 - Baseline

Study 1’s baseline appendix reports conditional-logit estimates with all attributes visible and cent-to-dollar price-sensitivity ratios; GPT-5.6 Terra and Sol make identical cost-minimizing choices across all 1,014 choice sets.

Study 1 - Baseline

  • Table W1 reports conditional-logit estimates for Study 1’s baseline when all attributes are visible.Source: Table W1 · p. 27
  • Table W1 reports standard errors in parentheses and marks significance at p < .05, p < .01, and p < .001.Source: Study 1 - Baseline · p. 27
  • Table W2 reports the ratio of cent to dollar price sensitivity in the baseline.Source: Table W2 · p. 27
  • GPT-5.6 Terra and Sol selected the same cost-minimizing alternative in all 1,014 choice sets.Each LLM had 1,014 choice sets (4,056 observations).Source: Study 1 - Baseline · p. 27
  • A ratio of one means a cent is weighted at exactly one hundredth of a dollar; ratios below one indicate that the cent component is discounted relative to the dollar component.Confidence intervals are exact (Fieller).Source: Study 1 - Baseline · p. 27
  • Fractional cents are scaled to dollars, so both coefficients express marginal disutility from a one-dollar increase in that price component.Source: Study 1 - Baseline · p. 27

Study 1 - Tool-Lab … Web appendix J: Software details

This section documents the Tool-Lab experiments’ choice-set designs, prompt and analysis procedures, model implementations, and replication details. Its appendices examine information acquisition, choice and calculation behavior, stated shopping objectives, and valuation reporting under varying information and acquisition-cost conditions.

Study 1 - Tool-Lab

  • Study 1 counted diagnostic acquisition across price dollars, price cents, and weight (maximum six) and defined the optimal choice as Coffee B, $5.00 for 11 oz.The table note describes 3,200 sessions across eight LLMs, with 100 independent Monte Carlo sessions per cell.Source: Study 1 - Tool-Lab · pp. 28–29

Study 2 - Baseline

  • Study 2 Baseline defined discount amount as Discount % × List Price and used βDiscount Amt/|βList Price| to compare savings sensitivity with list-price sensitivity.A ratio of one indicates equal weighting of a dollar saved and a dollar spent; values above one indicate promotional savings are overweighted relative to base cost.Source: Study 2 - Baseline · pp. 33–34

Study 2 - Tool-Lab

  • In Study 2 Tool-Lab, Cost × Deal was −7.624 [−8.026, −7.222] for Terra, −0.600 [−1.023, −0.192] for Sonnet, and −2.178 [−2.587, −1.770] for Opus.Diagnostic Search estimates were 0.067 [0.056, 0.076] for Terra, 0.200 [0.167, 0.232] for Sonnet, and 0.143 [0.127, 0.160] for Opus. Entries are posterior means with 95% equal-tailed credible intervals.Source: Table W19 · p. 34, Study 2 - Tool-Lab · p. 35

Design balance and randomization checks

Delegating prompt and structured output schema

Econometric model and utility specification

Choice set generation and Pareto filtering

  • The Study 1 promotional design retained 1,149 non-dominated four-option sets, while the full-information cent design retained 1,014 non-dominated four-option sets.In Study 1, discounts were assigned to two alternatives per set and net prices were withheld; the cent design crossed whole-dollar prices and fractional cents with brands and package weights.Source: Choice set generation and Pareto… · pp. 35, 37

Design balance and randomization checks

  • The 1,149 choice sets balanced discounts across four brands and had a maximum VIF of 1.51; a negligible display-position imbalance was controlled with fixed effects.The design check covered 4,596 alternative profiles; discount assignment had χ2(3) = 2.42, p = .491, and position variation had Cramér’s V = 0.041.Source: Design balance and randomization checks · p. 38

Delegating prompt and structured output schema

  • The Study 2 prompt populated product attributes and discounts dynamically, counterbalanced option positions, and constrained responses to validated JSON using the same schema across providers.The prompt template and option formatting are shown in Table W23; non-discounted alternatives had blank discount fields.Source: Delegating prompt and structured… · p. 38, Table W23 · p. 38

Econometric model and utility specification

  • The conditional logit models utility from weight, list price, discount amount, brand, and position, and assess promotional sensitivity by comparing the marginal utility of savings with that of base-price reductions.A βDA/|βLP| ratio above 1.00 indicates over-weighting promotional savings; parameter estimates and Fieller ratio tests are reported in Tables W15 and W16.Source: Econometric model and utility… · p. 38, Test of promotional sensitivity · p. 39

Model formulation and structural equations

  • The moderated mediation framework specifies that prompt framing moderates how monetary tool costs affect pre-choice information acquisition, while diagnostic information contributes additively to choice accuracy.The second-stage cost-by-framing interaction is excluded because costs are theorized to affect choice quality solely through information deprivation. Under the LPM, out-of-bounds fitted probabilities occurred in less than 2.8% of posterior draws across all 3,200 Study 1 sessions and less than 3.1% of Study 2 draws; logistic sensitivity analyses retained identical directional conclusions and statistical significance.Source: Model formulation and structural… · pp. 40–41, Justification of the linear… · p. 41

Prior elicitation and Bayesian inference

  • Bayesian models used regularizing priors and four MCMC chains, yielding 12,000 post-warmup draws with no divergent transitions; pathway estimates were derived draw by draw.Convergence checks found all potential scale reduction factors below 1.005, bulk effective sample sizes above 2,500, and tail effective sample sizes above 2,000; estimates include credible intervals and directional posterior probabilities.Source: Prior elicitation and Bayesian inference · pp. 41–42, Posterior pathway derivations · p. 42

Classification task

  • Objective classification used each session’s first pre-acquisition reasoning line to assign price-per-weight, price, or other, with the judge blind to subsequent tool use and choice.Gemini 3.5 Flash Lite could not be classified because it emits no reasoning before acquisition; the primary classifications used the Nemotron judge, with two independent judge LLMs as checks.Source: Classification task · p. 42, Judge LLMs · p. 43

Validation against human coding

  • The Nemotron judge had the highest agreement with human coding, but substantive estimates were similar across judges; disagreements centered on “other,” which was excluded from focal contrasts.A human rater and the judges independently classified 120 pre-acquisition traces; the margin in agreement was small relative to what that sample could resolve.Source: Validation against human coding · pp. 43–44

Objective shift

  • Across three independent judge LLMs, pooled contrasts by stated optimization objective were −1.38 to −1.56 diagnostic cells and −.49 to −.55 in optimality, with overlapping confidence intervals throughout.The experiment held acquisition fees at zero (c = $0.00), and reasoning-trace extraction was confined to the Google Gemini family.Source: Table W28 · p. 44, Objective shift · p. 44

Web appendix F: Exogenously assigned information states

  • Exogenously assigning visible attributes at zero cost addresses the ambiguity between acquisition and utilization failures in Tool-Lab’s endogenous search process.Unassigned attributes were explicitly identified as existing but unavailable, so choices were made under stated missingness rather than assumed absence or irrelevance.Source: Web appendix F: Exogenously assigned… · pp. 44–45, Design and Procedure · p. 45

Design and Procedure

  • The design tested nine information states across the two studies, varying diagnostic and non-diagnostic attributes, including matched-count conditions that held information volume constant while changing diagnostic content.Eight LLMs completed 1,600 Study 1 and 2,000 Study 2 sessions; choice optimality and consistency with the conditional Expectimax benchmark were measured separately.Source: Design and Procedure · p. 45

Results

  • With complete diagnostic information, seven of eight LLMs chose optimally 92%-100% of the time; pooled optimality was 99.7% in Study 1 and 98.6% in Study 2.This indicates that, in these tasks, suboptimal choices under acquisition costs are more plausibly explained by omitted diagnostic attributes than inability to evaluate them when available.Source: Results · p. 46, Table W30 · p. 47, Table W31 · p. 47
  • Across truncated states, seven LLMs selected the Expectimax-prescribed alternative in 2,434 of 2,450 Study 1 trials (99.3%) and 3,128 of 3,150 Study 2 trials (99.3%).Thus, choosing a superficially lower whole-dollar or list price under partial information could be normatively consistent rather than a utilization failure.Source: Results · p. 46
  • Gemini 3.5 Flash Lite chose optimally in 4% of complete-information Study 1 sessions and 0% of Study 2 sessions, indicating difficulty using the provided numbers to compute unit prices.The complete diagnostic attributes were visible without acquisition cost.Source: Results · p. 46

Web appendix G: Reported valuations

  • A concurrent valuation-reporting condition required LLMs to provide intermediate attribute calculations for every alternative when submitting their choice.This addresses whether reported values reflect the information state at choice, rather than retrospective calculations after a decision.Source: Web appendix G: Reported valuations · p. 49

Design

  • The valuation condition compared zero-cost sessions with sessions charging $10 per inspection in Study 1 and $50 per inspection in Study 2, under the vague goal prompt.Each study included 800 sessions across the same eight LLMs.Source: Design · p. 49

Methodological considerations

  • Concurrent reporting avoids retrospective rationalization but may prime diagnostic search or act as a scratchpad that reduces suboptimal choices.Because both concurrent schema modification and post-choice elicitation can affect behavior, this condition tests arithmetic ability, not behavior under resource constraints.Source: Methodological considerations · pp. 49–50

Results

  • Arithmetic failure and Incorrect (unknown) were 0.0% across all eight LLMs, both studies, and all cost conditions when reporting valuations.When search truncated, LLMs generally acknowledged uninspected cells as unknown; exceptions were Claude Sonnet 5 and Claude Opus 5 in costly Study 1 inspection (0.8%) and Claude Opus 5 under Study 2’s $50 cost (8.5%).Source: Results · pp. 51–52
  • Gemini 3.5 Flash Lite computed unit values with 100.0% accuracy yet chose suboptimally in Study 1 and selected Study 2’s optimal alternative in only 24.0% of zero-cost and 34.0% of costly sessions.In Study 1, it selected Coffee A in 22.0% of zero-cost and 20.0% of costly sessions despite Coffee B having the better unit value.Source: Results · p. 52

Web appendix H: Expectimax normative benchmarking

  • Expectimax benchmarks search by recursively evaluating each tool call’s expected value of information against compounding financial search costs under resource scarcity.It approximates a decision-theoretic solution to a partially observable Markov decision process.Source: Web appendix H: Expectimax normative… · p. 53

The shopping task

  • The task delegates a single choice for a five-bag purchase, and tool fees are included with purchase costs when evaluating effective cost per ounce.The study varies cell-inspection costs across five levels in each environment, while submitting a final choice is free.Source: The shopping task · p. 53, Objective function · pp. 53–54

Optimization

  • The benchmark uses empirical instant-coffee attribute priors and compares myopic search with two-step lookahead, which captures complementary value across attributes.Chance outcomes are quantized to 10 representative percentiles when an empirical domain exceeds the enumeration limit.Source: Priors · p. 54, Optimization · p. 55

Normative benchmark - Study 1

  • In Study 1, both Expectimax horizons acquired all six diagnostic cells and chose optimally at $0.00 and $0.01; at $10.00, the two-step planner stopped after three calls and still chose Coffee B.In Study 2, rising costs first truncated discount-percentage acquisition, then list-price and weight acquisition.Source: Normative benchmark - Study 1 · p. 56, Normative benchmark - Study 2 · p. 57

Normative benchmark - assigned information states

  • At three of four Study 1 states and three of five Study 2 states, the normative choice given revealed cells differed from the alternative with the best unit price at true values.A choice can be state-consistent yet economically inferior when attributes such as cents or weight remain withheld.Source: Normative benchmark - assigned… · p. 58

Experimental parameters and default decoding configuration

  • The studies retain provider-default decoding settings to measure LLM behavior as deployed, noting that Google and Anthropic had deprecated custom sampling parameters.The passage dates Google’s deprecation to July 2026.Source: Experimental parameters and default… · p. 59

Application programming interfaces and client libraries

Model identifiers, sampling parameters, and reasoning efforts

  • Gemini 3.5 Flash Lite, 3.8 Flash, and 3.1 Pro used default T = 1.0 and top-p = 0.95, with minimal, medium, and high thinking, respectively.Their API identifiers were gemini-3.5-flash-lite, gemini-3.8-flash, and gemini-3.1-pro-preview.Source: Model identifiers, sampling… · p. 59
  • GPT-5.6 Luna, Terra, and Sol used T = 1.0, top-p = 1.0, null presence and frequency penalties, and default medium reasoning effort.Their API identifiers were gpt-5.6-luna, gpt-5.6-terra, and gpt-5.6-sol.Source: Model identifiers, sampling… · p. 59
  • Claude Sonnet 5 and Opus 5 used T = 1.0, top-p = 1.0, adaptive thinking, and default high reasoning effort.Their API identifiers were claude-sonnet-5 and claude-opus-5.Source: Model identifiers, sampling… · p. 59

Output structure and replication controls

Web appendix J: Software details

  • Parallel mediation models used R 4.6.1 with brms 2.23.0, cmdstanr, and Stan 2.39.0; per-LLM mediations used lavaan 0.6.21.The brms reference is Bürkner (2017).Source: Web appendix J: Software details · p. 60
  • Parallel mediations used four chains of 4,000 iterations, including 1,000 warmup iterations, retaining 12,000 posterior draws, with adapt_delta = .99 and maximum treedepth 12.Source: Web appendix J: Software details · p. 60

Related papers

Loading 2609.28372v1…