PaperlayerView exact revision on arXiv

Source-linked AI summary

Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context

Jingru Jia, Zehua Yuan, Junhao Pan, Paul E. McNamara, Deming Chen

arXiv:2406.05972v2cs.AIcs.CYcs.HCcs.LGecon.TH

TL;DR

LLM decision-making under uncertainty is widely used, but its behavioral tendencies and sensitivity to demographic context remain insufficiently understood. This paper develops a behavioral-economics framework using multiple-choice lottery experiments to estimate three parameters in three commercial LLMs. The models generally show human-like risk aversion, loss aversion, and small-probability overweighting, while demographic embedding produces model-specific behavioral changes and raises ethical concerns.

  • Problem

    Existing studies leave LLMs’ intrinsic decision-making tendencies and their responses to socio-demographic context insufficiently understood.

    Source: §1 · p. 1

  • Method

    The paper uses behavioral-economic multiple-choice-list experiments to estimate risk preference, probability weighting, and loss aversion for three commercial LLMs.

    Source: §1 · p. 2, §4 · p. 4

  • Results

    The three LLMs generally exhibit human-like risk aversion, loss aversion, and small-probability overweighting, with substantial variation across models and demographic contexts.

    Source: §1 · p. 2, Result Comparisons with Context-free… · p. 7

  • Takeaways & Limitations

    Demographic context can significantly alter LLM decision behavior, requiring attention to ethical implications and potential deployment biases.

    Source: §1 · p. 2, §7 · p. 10

    Direct human comparison is difficult for sensitive demographic features, and financial experiments may not capture behavior in other domains.

    Source: §6 · p. 10

Abstract

from arXiv · show

When making decisions under uncertainty, individuals often deviate from rational behavior, which can be evaluated across three dimensions: risk preference, probability weighting, and loss aversion. Given the widespread use of large language models (LLMs) in decision-making processes, it is crucial to assess whether their behavior aligns with human norms and ethical expectations or exhibits potential biases. Several empirical studies have investigated the rationality and social behavior performance of LLMs, yet their internal decision-making tendencies and capabilities remain inadequately understood. This paper proposes a framework, grounded in behavioral economics, to evaluate the decision-making behaviors of LLMs. Through a multiple-choice-list experiment, we estimate the degree of risk preference, probability weighting, and loss aversion in a context-free setting for three commercial LLMs: ChatGPT-4.0-Turbo, Claude-3-Opus, and Gemini-1.0-pro. Our results reveal that LLMs generally exhibit patterns similar to humans, such as risk aversion and loss aversion, with a tendency to overweight small probabilities. However, there are significant variations in the degree to which these behaviors are expressed across different LLMs. We also explore their behavior when embedded with socio-demographic features, uncovering significant disparities. For instance, when modeled with attributes of sexual minority groups or physical disabilities, Claude-3-Opus displays increased risk aversion, leading to more conservative choices. These findings underscore the need for careful consideration of the ethical implications and potential biases in deploying LLMs in decision-making scenarios. Therefore, this study advocates for developing standards and guidelines to ensure that LLMs operate within ethical boundaries while enhancing their utility in complex decision-making environments.

1 Introduction

The paper examines whether LLM decision-making under uncertainty resembles human behavior and whether demographic context introduces bias. It proposes a behavioral-economics framework to evaluate risk preference, probability weighting, and loss aversion across three commercial LLMs.

  • The study asks how LLMs handle risk and uncertainty relative to humans and whether demographic features affect fairness and ethical decision-making.Source: §1 · pp. 1–2
  • The framework evaluates risk preference, probability weighting, and loss aversion using behavioral-economic theory and three experiment sets.Source: §1 · p. 2
  • Embedding socio-demographic features produces distinctive model-dependent behavior patterns, including increased risk aversion in some contexts.Source: §1 · p. 2
  • In context-free evaluations, the three LLMs generally show human-like risk aversion, loss aversion, and overweighting of small probabilities, with model-specific differences.Source: §1 · p. 2
  • The findings motivate careful ethical evaluation and standards for fair and ethical LLM-supported decision-making.Source: §1 · p. 2

2 Background and Related Work

Prior work finds human-like economic and social behaviors in LLMs but leaves their intrinsic decision-making tendencies and demographic sensitivity insufficiently understood. Because EUT and PT rely on assumptions that may not fit LLMs, the paper adapts the TCN model to estimate three behavioral parameters.

2.1 LLMs’ Behavior Study

  • Studies report that LLMs display economic behaviors, human-like moral judgments, and rationality in financial and game-theoretic settings.Source: §2.1 · p. 2
  • Existing frameworks often assume that LLM preferences align with humans, leaving demographic-feature effects and intrinsic decision tendencies unresolved.Source: §2.1 · pp. 2–3
  • Prior evidence of bias and degraded performance for minority-group information motivates quantitative evaluation of LLM behavior under uncertainty.Source: §2.1 · p. 3

2.2 Expected Utility Theory and Prospect Theory

  • Prospect Theory models risk aversion, loss aversion, and probability weighting, whereas Expected Utility Theory treats risk aversion as the sole utility-curvature determinant.Source: §2.2 · p. 3
  • Neither EUT nor PT is sufficient because LLM behaviors can deviate from EUT assumptions, while applying PT without verifying LLM baselines risks circular reasoning [22].Source: §2.2 · p. 3
  • The adapted TCN model combines EUT and PT to estimate risk preference, loss aversion, and nonlinear probability weighting without presupposing LLM tendencies.Source: §2.2 · p. 3

3 Preliminary

The preliminary framework represents decisions through revealed preference and a TCN utility model combining value curvature, probability weighting, and loss aversion. Its parameters are interpreted behaviorally to characterize choices under uncertainty.

  • Revealed-preference analysis represents choices as selected bundles and tests whether a utility function rationalizes them under budget constraints.Source: §3 · pp. 3–4
  • The axiom of revealed preference and GARP test consistency by requiring preference relations without reciprocal cycles.Source: §3 · p. 4
  • The study integrates revealed-preference principles with a TCN experimental design combining Expected Utility Theory and Prospect Theory.Source: §3 · p. 4
  • The model uses σ for value-function curvature, λ for loss aversion, and α for probability weighting; σ > 0 indicates risk aversion and α < 1 indicates overweighting small probabilities.Source: §3 · p. 4

4 Framework and Design

The framework elicits LLM preferences through multiple-choice lottery experiments, identifies switching points, and converts them into bounded estimates of σ, α, and λ. It then evaluates both context-free and demographic-feature-embedded models.

4 Framework and Design

  • The resulting parameters evaluate context-free and demographic-feature-embedded LLMs through behavioral analysis and regression models.Source: §4 · pp. 4–6
  • Three multiple-choice lottery series elicit preferences: positive-outcome series estimate σ and α, while negative-outcome trials estimate λ.Source: §4 · pp. 4–5
  • Each LLM query is treated like an individual subject, with repeated interactions used to estimate broader behavioral tendencies.Source: §4 · p. 5
  • A switching point is the smallest lottery index where preference changes between consecutive options, identified through their comparative utilities.Source: §4 · p. 5
  • The value and probability-weighting functions are evaluated at switching points to derive inequalities that bound σ, α, and λ.Source: §4 · p. 5
  • Parameter estimation narrows intervals iteratively until all inequalities are satisfied, then uses interval midpoints as final estimates.Source: §4 · p. 5

5 Evaluation and Results

The evaluation applies behavioral-economic measures to three commercial LLMs and finds shared risk aversion alongside model-specific probability weighting, loss aversion, and demographic sensitivity.

5.1.1 Results and Key Findings

  • All three LLMs exhibit risk-averse decision-making in context-free lottery experiments, although their probability weighting and loss aversion profiles differ.The study evaluates ChatGPT-4-Turbo, Claude-3-Opus, and Gemini-1.0-Pro using repeated lottery decisions and parameters σ, α, and λ.Source: §5.1.1 · p. 6, §5 · p. 6, §5.1 · p. 6

Result Comparisons with Context-free Evaluations

  • Embedding demographic features changes LLM decision-making behavior, with no significant difference between random and real-world demographic distributions.Figure 4 compares the three context settings within each model.Source: Result Comparisons with Context-free… · p. 7, Figure 4 · p. 7, §5.2 · p. 7
  • ChatGPT becomes riskier and shifts from small-probability underweighting to overweighting, Claude reduces loss aversion, and Gemini becomes more conservative with greater loss aversion.Gemini instead shifts from small-probability overweighting to underweighting, with α slightly above 1.Source: Result Comparisons with Context-free… · p. 7

Models’ Sensitivity to Demographic Features

  • The foundational-feature analysis uses OLS regression to estimate how demographic variables relate to σ, α, and λ.Categorical demographic features are binary encoded, with coefficients representing feature associations with decision parameters.Source: Models’ Sensitivity to Demographic… · p. 8, Table 3 · p. 8
  • Claude shows broader sensitivity to foundational demographic variables, while ChatGPT and Gemini display more limited but model-specific sensitivities.Claude is particularly responsive to youth, rural residence, education, and marital status; ChatGPT shows gender differences in risk preference, and Gemini shows age- and education-related loss-aversion effects.Source: Models’ Sensitivity to Demographic… · p. 8, Figure 5 · p. 8
  • Advanced demographic features reveal distinct model patterns, including Claude’s broad responsiveness and targeted sensitivities in ChatGPT and Gemini.Claude responds to sexual orientation, ethnicity, disability, religion, and political beliefs; ChatGPT is sensitive to political beliefs and physical disability, while Gemini is sensitive to ethnicity.Source: Models’ Sensitivity to Demographic… · p. 9, Figure 6 · p. 9, Table 4 · p. 9

6 Discussion

The discussion emphasizes that demographic prompts can produce model-specific biases and unintended effects, raising questions about ethical alignment and the interpretation of LLM behavior relative to humans.

  • Demographic features can significantly affect LLM decision-making, creating potential biases and unintended consequences for customized models.The paper warns that minority users may receive advice shaped by generalized stereotypes, including in financial decisions.Source: §6 · pp. 9–10
  • Ethical deployment requires balancing human-like behavior with standards that remediate bias and support fairer decision-making.The discussion considers structured reasoning, targeted fine-tuning, and RLHF adjustments as possible routes toward greater fairness.Source: §6 · p. 10
  • The study reports discrepancies between human and LLM behavior, including different risk-preference patterns for sexually minority groups.The authors attribute these discrepancies either to limitations in human studies or to biased understandings learned by LLMs, and call for further research.Source: §6 · p. 10
  • Demographic prompting produces behavior that differs from context-free baselines, leaving the appropriate intrinsic behavior of LLMs unresolved.The authors frame this as a question about the boundary between closed-loop answering systems and artificial intelligence.Source: §6 · p. 10
  • Direct comparison with humans is constrained by the sensitivity and complexity of demographic features, while the financial setting limits behavioral coverage.The authors recommend experiments in other domains and comparisons across more models to assess robustness.Source: §6 · p. 10

7 Conclusion

The paper establishes a behavioral-economics framework for evaluating LLM decision-making across risk, probability, and loss, while showing model-specific deviations and demographic sensitivity.

  • The study finds that all three evaluated LLMs show human-like general tendencies but differ in the extent of their deviations.The conclusion evaluates decision-making through risk, probability, and loss perspectives.Source: §7 · p. 10
  • Socio-demographic features significantly affect LLM decision-making, revealing variability and potential biases in model outputs.The conclusion calls for continued scrutiny and refinement to avoid perpetuating or worsening societal biases.Source: §7 · p. 10
  • The paper frames future work around balancing realistic human-like behavior with ethical responsibility, both with and without demographic embeddings.This question is presented as a boundary between closed-loop answering systems and genuine artificial intelligence.Source: §7 · p. 10

A Tables of Raw Data from Experiments

This appendix section organizes raw experimental outputs, parameter comparisons, and regression analyses for foundational and advanced demographic features.

B Multiple Choice List Design

The framework uses three lottery-game series to estimate decision-making parameters and compares context-free behavior with behavior elicited after demographic prompts.

B Multiple Choice List Design

  • Three multiple-choice lottery series contain 35 rows and target risk preference, probability weighting, and loss aversion.The first two series assess risk and probability weighting, while the third introduces monetary losses to assess loss aversion.Source: B Multiple Choice List Design · p. 15
  • Series 1, Series 2, and Series 3 are provided as the experiment’s multiple-choice lists.Source: Table 9 · p. 15, Table 10 · p. 15, Table 11 · p. 15

C Prompt Design

  • The experiments compare context-free LLM behavior with behavior after demographic features are embedded in the prompt.The context-free condition uses no preset human knowledge, whereas the demographic-embedded condition adds specified demographic attributes.Source: C Prompt Design · p. 16

C.1 Context-free Experiment Prompt

The context-free evaluation uses repeatedly administered prompts and preserves response history to simulate a human-like lottery-question interaction. The prompt list is documented in Table 12.

  • Prompts are repeatedly sent through the LLMs’ APIs for the context-free evaluation.Source: §C.1 · p. 16
  • Response history is collected and aggregated into subsequent inputs, preserving the interaction context.Source: §C.1 · p. 16
  • Table 12 lists the prompts used for the context-free behavior evaluation.Source: Table 12 · p. 16, §C.1 · p. 16

C.2 Demographic-feature Embedded Experiment Prompt

The demographic-feature experiment augments context-free questions with demographic components and maintains session history so models retain prior lottery and feature information. Its prompt materials are provided in Tables 13 and 14.

  • Each demographic-feature prompt combines the context-free question body with a demographic component from the Table 13 template.Source: §C.2 · p. 16, Table 13 · p. 17
  • Demographic features are added before each prompt to help preserve feature memory during long-term conversations.Source: §C.2 · p. 16
  • Session history is maintained so participating LLMs retain memory of the previous lottery.Source: §C.2 · p. 16
  • Table 14 provides the complete list of prompts for the demographic-feature embedded evaluation.Source: §C.2 · p. 16, Table 14 · p. 17
Loading 2406.05972v2…