PaperlayerView exact revision on arXiv

Source-linked AI summary

STEER: Assessing the Economic Rationality of Large Language Models

Narun Raman, Taylor Lundy, Samuel Amouyal, Yoav Levine, Kevin Leyton-Brown, Moshe Tennenholtz

arXiv:2402.09552v2cs.CLecon.GNq-fin.EC

TL;DR

Broad assessment of LLM decision-making calls for a principled account of economic rationality beyond restricted task-specific tests. The paper turns that account into a taxonomy and benchmark-based STEER report card, finding strong performance differences by model size and question difficulty.

  • Problem

    Existing LLM decision-making evaluations cover restricted tasks, limited economic settings, or open-world games, while broader assessment requires a holistic account of good decision-making and its testable components.

    Source: §1 · p. 2

  • Method

    The paper taxonomizes 64 elements of economic rationality, builds a benchmark distribution with generated questions, and combines performance with a user-provided rubric in STEER report cards.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    Across the benchmark, performance fell as grade level increased; models below 40 billion parameters were not reliably above random guessing, and GPT-4 Turbo led across metrics and elements.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The benchmark can highlight model strengths and limitations to help users judge where models can be relied upon and where they need more work.

    Source: §6 · p. 21

    The benchmark barely scratches the surface of real-world complexity, where models may need to combine multiple elements and handle more context than the tests typically include.

    Source: §8.2 · p. 22

Abstract

from arXiv · show

There is increasing interest in using LLMs as decision-making "agents." Doing so includes many degrees of freedom: which model should be used; how should it be prompted; should it be asked to introspect, conduct chain-of-thought reasoning, etc? Settling these questions -- and more broadly, determining whether an LLM agent is reliable enough to be trusted -- requires a methodology for assessing such an agent's economic rationality. In this paper, we provide one. We begin by surveying the economic literature on rational decision making, taxonomizing a large set of fine-grained "elements" that an agent should exhibit, along with dependencies between them. We then propose a benchmark distribution that quantitatively scores an LLMs performance on these elements and, combined with a user-provided rubric, produces a "STEER report card." Finally, we describe the results of a large-scale empirical experiment with 14 different LLMs, characterizing the both current state of the art and the impact of different model sizes on models' ability to exhibit rational behavior.

1 Introduction

The paper argues that broad assessment of LLM decision-making needs a principled account of rational behavior, not only collections of ad hoc tasks. It builds a taxonomy and benchmark around economic rationality, then reports model and prompting results across that benchmark.

1 Introduction

  • Existing assessments cover narrow tasks, limited economic settings, or open-world games, motivating a holistic way to describe and test decision-making components.LLMs are being configured as economic agents and as components of broader systems.Source: §1 · pp. 1–2
  • The taxonomy evaluates adherence to first principles of decision-making, including utility maximization and distinctions among deterministic, single-agent, and multi-agent settings.Figure 1 organizes decision-making into settings and conceptually similar modules.Source: §1 · p. 2, Figure 1 · p. 3
  • The authors identify 64 rationality elements with well-defined notions of a right answer and derive a fine-grained benchmark distribution for STEER report cards; prompts generated 24,500 multiple-choice questions for 49 elements.They manually validated 2,450 generations and evaluated 14 language models, from Llama 7B to GPT-4 Turbo, on 134,750 test questions.Source: §1 · p. 2
  • Models below 40 billion parameters were not reliably above random guessing; performance fell with grade level, while GPT-4 Turbo led across metrics and elements.Self-explanation generally helped, and few-shot prompting helped with up to three examples but hurt beyond that point.Source: §1 · p. 3

2 STEER: Systematic and Tuneable Evaluation of Economic Rationality

STEER turns a taxonomy of economic rationality into a configurable benchmark and report card for evaluating LLM decision-making. Questions are generated across domains and difficulty levels, validated, and scored for accuracy and robustness.

2 STEER: Systematic and Tuneable Evaluation of Economic Rationality

  • STEER encodes decision scenarios as multiple-choice questions with exactly one correct choice, organized by a taxonomy of rationality elements.The benchmark supports user-selected subsets through its report cards.Source: §2 · p. 3, §2.2 · p. 4

2.1 Generating and Validating Questions

  • GPT-4 Turbo generates questions from templates that specify question criteria and formatting and include a gold-standard example.The researchers programmatically remove formatting errors and spot-check samples; two validators judged 98.54% of checked questions valid, with a 97% minimum for Avoidance of the Endowment Effect.Source: §2.1 · pp. 3–4
  • Templates vary subject matter across domains and, for most elements, difficulty across grade levels 1–13.The authors describe grade-level choices as somewhat arbitrary and aimed for similar difficulty across templates within a grade.Source: §2.1 · pp. 3–4
  • A test is a set of questions for one rationality element at a particular domain and grade level; each test uses five templates.Across an element, the authors use 10–40 templates.Source: §2.1 · p. 4

2.2 STEER Report Card

  • STEER report cards let users select grade ranges, domains, metrics, and models, then group performance summaries by settings and modules.Default subsets also cover settings, models, and use cases.Source: §2.2 · pp. 4–5, Figure 4 · p. 5

2.2.2 Accuracy

  • STEER offers exact-match accuracy, normalized accuracy relative to random guessing, and expected calibration error to assess answer accuracy and confidence calibration.Normalized accuracy adjusts for tests with different numbers of choices; calibration uses 10 uniformly spaced probability bins.Source: §2.2.2 · pp. 5–6

2.2.3 Robustness

  • Robustness scores include each element’s minimum exact-match accuracy across domains and a dependency measure based on performance gaps among conceptual prerequisites.The dependency graph represents relationships among simpler and more advanced rationality elements; dependency robustness compares an element’s random-guessing gap with those of its prerequisite elements.Source: §2.2.3 · pp. 6–7

3 Applying our Benchmark: Setup

The setup evaluates 14 LLMs of varying parameter sizes as black boxes and tests self-explanation and few-shot prompting as ways to tailor model responses.

3 Applying our Benchmark: Setup

  • The study evaluates 14 LLMs of varying parameter sizes, including two API models and 12 open-source models run on an A100 GPU.All models were decoded with temperature 0.Source: §3 · p. 7
  • The authors treat models as black boxes and test self-explanation through separate calls or a single call combining explanation and answer selection.They measure effects on accuracy and confidence, represented by log probabilities when available.Source: §3 · p. 7
  • Few-shot prompting tests 0, 1, 2, 4, or 5 examples drawn from the question’s domain and grade level.Its effect is measured as a difference in accuracy and confidence for each element.Source: §3 · p. 7

4 Elements of Economic Rationality

This section organizes economic rationality into measurable elements across foundational and single-agent decisions, multi-agent games, and decisions made on behalf of others. It covers equilibrium reasoning, preference aggregation, truthful participation, allocation, and auction selection, using von Neumann–Morgenstern utility axioms as a reference for canonical answers, sometimes supplemented by linear utility for money.

4.1 SETTING 1: FOUNDATIONS

  • Foundations cover arithmetic, optimization, probability, logic, and theory of mind as core skills for valuing outcomes, reasoning about likelihoods, choosing well, and modeling others’ beliefs.The framework includes these capabilities to standardize previously well-studied NLP tasks and connect them to its dependency graph.Source: §4.1 · p. 8, §4.1.5 · p. 10

4.2.1 MODULE 2.1: AXIOMS OF UTILITY IN DETERMINISTIC ENVIRONMENTS

  • Deterministic single-agent rationality tests preference transitivity and independence, while its bias module asks agents to disregard sunk costs and avoid the endowment effect.The endowment-effect criterion equates willingness to pay to acquire an object with willingness to accept for selling it when owned.Source: §4.2.1 · p. 11, §4.2.2 · p. 11

4.2.3 MODULE 2.3: AXIOMS OF UTILITY IN STOCHASTIC ENVIRONMENTS

  • For stochastic choices, the taxonomy tests consistent preferences over lotteries and expected-utility computation and maximization under linear utility for money.It also includes avoiding loss aversion by evaluating all potential outcomes objectively rather than overvaluing potential losses.Source: §4.2.3 · pp. 11–12, §4.2.4 · p. 12

4.2.5 MODULE 2.5: AVOIDANCE OF COGNITIVE BIASES IN STOCHASTIC SETTINGS

  • Stochastic-bias elements assess avoiding the gambler’s fallacy, maintaining consistent risk preferences across positive and negative payoffs, and treating known and ambiguous risks consistently across framing.These elements are framed as opposites of behaviors associated with cognitive biases.Source: §4.2.5 · p. 13

4.3.1 MODULE 3.1: NORMAL FORM GAMES

  • Multi-agent decisions add strategic reasoning because other agents pursue their own utilities based on beliefs, including beliefs about the focal agent’s behavior.Normal-form elements include interpreting payoffs, finding best responses and dominant strategies, eliminating dominated strategies, and identifying pure Nash equilibria.Source: §4.3 · p. 13, §4.3.1 · p. 14

4.3.2 MODULE 3.2: EXTENSIVE FORM GAMES

  • Extensive-form games test backward induction and subgame-perfect Nash equilibrium, which requires equilibrium strategies at every decision point, not only for the game as a whole.The section describes backward induction as the analysis used to find subgame-perfect Nash equilibrium.Source: §4.3.2 · p. 15

4.3.3 MODULE 3.3: IMPERFECT INFORMATION IN EXTENSIVE FORM GAMES

  • For imperfect-information extensive games, the taxonomy tests whether an agent can compute and select a sequential-equilibrium strategy.This setting includes situations where agents cannot distinguish some of their own action nodes, reflecting limited knowledge or memory.Source: §4.3.3 · p. 15

4.3.4 MODULE 3.4: INFINITELY REPEATED GAMES

  • Infinitely repeated games require tools beyond backward induction, with elements covering feasible and enforceable equilibrium payoffs and selection of trigger strategies.The module considers discounted utility and strategies such as grim trigger, tit-for-tat, and tit-for-two-tat.Source: §4.3.4 · p. 15

4.3.5 MODULE 3.5: BAYESIAN GAMES

  • Bayesian-game elements require best responses and belief updates, while subgame-perfect Bayes–Nash strategies must remain equilibria in every subgame.Strategies maximize expected utility given beliefs about other agents’ types and strategies.Source: §4.3.5 · p. 16

4.4 SETTING 4: DECISIONS ON BEHALF OF OTHER AGENTS

  • A decision-maker acts on behalf of others, either aggregating known preferences into a social preference or choosing from a decision set, with objectives including social good or own utility.When preferences are not known, the decision-maker may need to elicit potentially dishonest reports; the paper distinguishes social choice from mechanism design.Source: §4.4 · p. 16

4.4.1 MODULE 4.1: AXIOMS OF SOCIAL CHOICE

  • Social-choice axioms include Pareto efficiency, monotonicity, transitivity, and non-dictatorship for welfare functions aggregating individual preferences.The setting assumes the decision-maker knows agents’ preferences.Source: §4.4.1 · p. 16

4.4.2 MODULE 4.2: SOCIAL CHOICE

  • Social-choice elements cover selecting plurality, Borda-count, and Copeland winners, as well as choosing a fair-division algorithm suited to the context.Borda assigns m − i points to an alternative ranked i’th; Copeland awards one point per victory and half a point per tie in pairwise contests.Source: §4.4.2 · p. 17

4.4.3 MODULE 4.3: DESIRABLE PROPERTIES IN MECHANISM DESIGN

  • Mechanism-design elements include truthful reporting as a dominant strategy, voluntary participation, and balancing the money charged to and rewarded to agents.The module notes that inducing truthful reports is generally impossible without additional ingredients, such as monetary charges or rewards.Source: §4.4.3 · p. 18

4.4.4 MODULE 4.4: MECHANISM DESIGN

  • Mechanism-design elements include running the top trading cycles algorithm to find a stable allocation.Source: §4.4.4 · p. 18
  • Auction elements select revenue-maximizing formats by bidder characteristics: risk-seeking favors English or second-price auctions, risk aversion favors first-price, and affiliated values favor English over second-price over first-price.The section also describes English ascending bids, first-price payment of the winner’s bid, and second-price payment of the second-highest bid.Source: §4.4.4 · p. 18

5 STEER Report Card

The report cards compare models overall and across grades, behavioral biases, domains, and dependencies, then examine self-explanation and few-shot prompting. Results show performance differences by model size and task difficulty, alongside adaptation effects.

Whole-Benchmark SRC

  • Performance closely tracked model size, and GPT-4 Turbo won nearly all head-to-head matchups but closed only a third of the gap between random guessing and perfect answers.GPT-3.5 Turbo ranked second, followed by Llama-2 70b; normalized-accuracy and exact-match orderings were similar among models that beat random guessing.Source: §5 · pp. 18–19, Table 2 · p. 19

Grade-Specific SRCs

  • GPT-4 Turbo's performance fell from closing three quarters of the guessing-to-perfect gap on Grade 0 questions to roughly random guessing from Grade 9 onward.GPT-3.5 Turbo fell to random guessing from Grade 7 onward; GPT-4 Turbo and GPT-3.5 Turbo scored −0.121 and −0.214 normalized accuracy, respectively, on Best Response.Source: §5 · p. 19, Figure 21 · p. 20

Cognitive Bias SRC

  • 0.575 versus 0.377 normalized accuracy for GPT-4 Turbo in deterministic versus stochastic environments; both GPT models were susceptible to framing effects in stochastic settings.For GPT-3.5 Turbo, the corresponding scores were 0.307 and 0.173; both GPT models performed worse on Avoidance of the Certainty Effect than on Avoidance of the Reflection Effect.Source: §5 · pp. 19–20

Domain Robustness SRC

  • Performance varied substantially across domains even when average accuracy was high, with the largest variation on Avoidance of Sunk Cost Fallacy.GPT-3.5 Turbo showed the strongest such bias in medical-project investment questions; GPT-4 Turbo's Maximize Expected Utility performance was most variable across medical-treatment domains, without a consistent risk-seeking or risk-averse preference.Source: §5 · p. 20

Dependency Robustness SRC

  • Higher-order elements were almost always harder than their prerequisites, except Pure Nash Equilibrium tests, where models did worse on its prerequisite, Iterated Removal of Dominated Strategies.The authors attribute this exception to a weakness in the Pure Nash Equilibrium test, which did not adequately represent dominance-solvable games.Source: §5 · p. 20

Self-Explanation

  • Self-explanation improved all models overall, with especially large GPT-4 Turbo gains on Avoidance of the Certainty Effect (0.079 →0.508) and Avoidance of the Reflection Effect (0.390 →0.865).Gains were not uniform across modules and were largest on low-grade-level questions and cognitive biases in stochastic environments.Source: §5 · p. 20

Few-Shot Prompting

  • All models improved from zero-shot to three-shot prompting, while performance deteriorated at four or more examples.GPT-4 Turbo plateaued at three examples; some models that scored below random guessing in zero-shot improved with examples.Source: §5 · p. 20

6 Discussion and Conclusions

The authors present STEER as a benchmark for assessing economically rational behavior and discuss its uses and remaining limits. They treat current experimental scores as lower bounds while calling for broader coverage and human baselines.

Discussion and Conclusions

  • STEER assesses economically rational behavior that draws on logic, probability, optimization, reasoning about others, economic principles, and contextual judgment.Source: §6 · p. 21
  • The benchmark can highlight model strengths and limitations and support decisions about where models may need fine-tuning, new datasets, or specialized architectures.The authors also identify market analysis, policy simulation, and consumer-behavior understanding as potential areas of impact.Source: §6 · p. 21
  • Current experimental results are lower bounds because different prompting and adaptation strategies may achieve much better performance; the authors also call for broader benchmark coverage and human baselines.Source: §6 · p. 21

7 Social Impact Statement

The paper discusses potential benefits and harms from using LLMs as agents or as substitutes in human-subject experiments. It emphasizes the need to validate reliability and to confirm research findings with real human subjects.

7 Social Impact Statement

  • Unpredictable or unreliable LLM agents could cause harms ranging from economic losses to the inadvertent perpetuation of biases.The authors say reliability validation is critical and describe their work as only a first step toward helping developers assess whether systems are safe and reliable enough to deploy.Source: §7 · p. 21
  • Delegating tasks to AI could lower costs and free people from tedious work, while also displacing paid human labor and potentially disadvantaging workers.Source: §7 · p. 21
  • Using LLMs in place of human subjects could make experiments more frequent and statistically significant without risking harm to participants, but could replicate model biases in findings.The authors recommend using LLMs only for exploratory analyses, followed by validation with real human subjects.Source: §7 · p. 21

8 Limitations

The limitations concern comparison across user-configured report cards, dependence on prompting and adaptations, and the gap between benchmark tasks and real-world complexity.

8.1 Benchmark

  • User-defined report cards can make comparisons difficult: there is no necessarily correct global model ranking, and aggregating scores across different tasks is inherently difficult.The benchmark offers several scoring functions for different scenarios, while allowing groups with different use cases to form different preferences and rankings.Source: §8.1 · pp. 21–22

8.2 Results

  • Results depend on prompting and adaptations, and the benchmark’s breadth made it infeasible to create different adaptations for every element.The authors suggest that carefully constructed prompts for a specific use case could increase performance.Source: §8.2 · p. 22
  • The benchmark’s domain coverage only scratches the surface of real-world complexity, where tasks may combine elements or require more context than the tests typically include.Testing combinations of elements and more complex contexts is identified as a future addition.Source: §8.2 · p. 22

A.1.1 MODULE 1.1: ARITHMETIC

The arithmetic module illustrates questions about calculating sums, accumulated obligations, and expected values.

A.1.1 MODULE 1.1: ARITHMETIC

  • One arithmetic example asks for the sum of 10 and 5.Source: §A.1.1 · p. 26
  • Another asks how much is owed after a friend paid for five dinners costing $30 each.Source: §A.1.1 · p. 26
  • A rainfall scenario asks for the expected number of inches given a 20% chance of 0.5 inches of rain.Source: §A.1.1 · p. 26

A.1.2 MODULE 1.2: OPTIMIZATION

The optimization module illustrates comparison of monetary amounts and choices that maximize volume or profit under constraints.

A.1.2 MODULE 1.2: OPTIMIZATION

  • A basic comparison asks which is greater: $10 or $50.Source: §A.1.2 · p. 26
  • A box-making problem asks which corner-cut size maximizes volume when forming an open box from 15-inch-square cardboard.Source: §A.1.2 · p. 26
  • A factory problem asks for the profit-maximizing mix of two widget types given their labor requirements and 15 available labor hours.Source: §A.1.2 · p. 26

A.1.3 MODULE 1.3: PROBABILITY

The probability module uses short scenarios to test probabilistic reasoning, from basic event probabilities to updating beliefs after observing an outcome.

A.1.3 MODULE 1.3: PROBABILITY

  • A marble-draw question tests calculating the chance of drawing a red marble from a jar with 5 red, 3 blue, and 2 green marbles.Source: §A.1.3 · p. 27
  • A fishing scenario asks how likely a carp catch is when there is an 85% chance of catching a trout or nothing.Source: §A.1.3 · p. 27
  • A rugby scenario asks for the probability that Tom played a lower-ranked team given that he scored, using the stated scoring and matchup probabilities.The setup gives a 55% overall scoring chance, a 79% scoring chance against lower-ranked teams, and a 27% chance of facing such teams.Source: §A.1.3 · p. 27

A.1.4 MODULE 1.4: LOGIC

The logic module presents questions about deriving conclusions from stated premises and recognizing equivalent conditional statements.

A.1.4 MODULE 1.4: LOGIC

  • A syllogism asks whether the conclusion that dolphins give live birth follows from the premises that dolphins are mammals and mammals give live birth.Source: §A.1.4 · p. 27
  • A conditional-reasoning question asks whether Karen’s being smart implies she will enter a good college, given premises linking smartness to good grades and good grades to college admission.Source: §A.1.4 · p. 27
  • A prompt asks which statement is logically equivalent to the claim that regular retirement-fund contributions lead to accumulated savings.Source: §A.1.4 · p. 27

A.1.5 MODULE 1.5: THEORY OF MIND

The appendix provides scenario-based questions spanning individual choice, strategic interaction, social choice, and mechanism design. The examples range from preference and utility calculations to equilibrium and allocation problems.

A.2.1–A.2.2: DETERMINISTIC UTILITY AND BIASES

  • Deterministic-choice examples test utility axioms and cognitive biases, including preference consistency, phone selection when a third option is added, sunk costs, and willingness to trade after losing an item.Source: §A.2.1 · p. 27, §A.2.2 · pp. 27–28

A.2.3–A.2.5: STOCHASTIC UTILITY AND BIASES

  • Stochastic-choice questions cover preference axioms, risk-neutral expected-utility calculations, and biases such as the gambler’s fallacy, framing across gains and losses, and ambiguity aversion.Examples include a treatment with uncertain outcomes, a fair coin after five heads, and urn gambles with unknown proportions.Source: §A.2.3 · p. 28, §A.2.4 · p. 28, §A.2.5 · pp. 28–29

A.3.1–A.3.2: NORMAL AND EXTENSIVE FORM GAMES

  • Game-theory examples ask about payoffs, best responses, dominated actions, and Nash equilibrium in normal-form games, as well as backward-looking choices and bargaining in extensive-form games.Source: §A.3.1 · p. 29, §A.3.2 · p. 30

A.3.3–A.3.4: IMPERFECT INFORMATION AND REPEATED GAMES

  • Imperfect-information examples pose a sequential decision with uncertain alien types, while repeated-game questions ask about feasible or enforceable payoffs and whether a trigger strategy sustains cooperation.Source: §A.3.3 · p. 30, §A.3.4 · pp. 30–31

A.3.5: BAYESIAN GAMES

  • Bayesian-game examples ask a buyer to make an offer under uncertainty about a painting’s quality and a firm to choose an entry strategy without knowing the incumbent’s response.Source: §A.3.5 · p. 31

A.4.1–A.4.2: SOCIAL CHOICE

  • Social-choice questions test axioms such as Pareto efficiency, monotonicity, transitivity, and non-dictatorship, alongside plurality, Borda-count, and Copeland election procedures.Source: §A.4.1 · p. 31, §A.4.2 · p. 31

A.4.3–A.4.4: MECHANISM DESIGN

  • Mechanism-design questions ask whether auction and public-good mechanisms satisfy incentive compatibility, individual rationality, or budget balance, and include allocation and auction-format problems.Source: §A.4.3 · p. 32, §A.4.4 · p. 32

B Web Application Extras

The web application offers visual navigation of the rationality framework and a curriculum view filtered by grade range. Users can download a test dataset for the selected range of elements.

Web application

  • The hierarchical visualization expands settings and modules for a selected element, while the file-tree view provides navigation and displays the selected element’s description and an example task.The illustrated selection is loss aversion.Source: Figure 22 · p. 33
  • The curriculum view filters elements and settings by grade range; the grade 7–10 example includes only some elements and tasks, not the complete framework.Source: B Web Application Extras · p. 33, Figure 23 · p. 34
  • Users can download a test dataset containing benchmark elements within the selected and filtered range.Source: B Web Application Extras · p. 33

Related papers

Loading 2402.09552v2…