PaperlayerView exact revision on arXiv

Source-linked AI summary

Measuring Human-Like Bias in LLMs? A Critique of Human-Derived Bias Constructs in LLM Evaluation

Antonela Tommasel, Markus Schedl

arXiv:2610.00070v1cs.CL

TL;DR

Human-derived bias constructs are increasingly applied to LLMs, but psychological measures and model-side observables do not straightforwardly support the same interpretations. The paper maps construct, subject, and evaluation-context mismatches to a framework for distinguishing warranted claims, while noting that it is conceptual rather than an empirical evaluation.

  • Problem

    Human-derived bias constructs are increasingly used to evaluate LLMs, but their translation from measures of human cognition to model outputs leaves the relation between evidence and interpretation indirect.

    Source: §1 · p. 1

  • Method

    The paper develops an analytical framework relating human-derived constructs and model-side observables to interpretations including distributional association, observable behaviour, normative harm, and psychological analogy.

    Source: §1 · p. 2, §4 · pp. 4–5

  • Results

    The paper distinguishes distributional association from observable behaviour, normative harm, and psychological analogy, arguing that model-side associations alone do not establish behaviour, harm, or a human-like mechanism.

    Source: §4 · pp. 4–5, §5 · p. 5

  • Takeaways & Limitations

    Bias evaluations can use human-derived constructs, but claims about model bias should be stated more precisely in relation to the measured evidence and its interpretation.

    Source: §5 · p. 5

    The contribution is conceptual and literature-based, not an empirical test of whether specific probes are stable across models, prompts, languages, or deployment settings.

    Source: Limitations · p. 6

Abstract

from arXiv · show

Researchers increasingly use human-derived bias constructs to study Large Language Models (LLMs), including social-cognitive constructs such as implicit bias and stereotype activation, and cognitive biases such as anchoring, framing effects, and confirmation bias. Such approaches offer alternatives to overt bias probes, particularly when direct questioning may obscure bias or when model behaviour appears normatively acceptable. However, adapting human bias constructs to LLMs introduces an inferential gap. Psychological instruments were developed to study human cognition and social behaviour, whereas LLM evaluations rely on probabilities, text completions, rankings, or simulated decisions. This paper critiques human-centered bias evaluation in LLMs. We show how this gap arises from mismatches pertaining to human-derived constructs, human-model differences, and evaluation contexts, which can blur distinct interpretations of model bias. We then introduce a framework providing an analytical lens for relating these elements to warranted interpretations, with attention to target constructs, operationalizations, scope of inference, and limits of human analogy.

1 Introduction

LLM bias evaluations increasingly adapt human-derived constructs to model outputs, extending beyond overt bias probes and aggregate metrics. The paper argues that this transfer creates an inferential gap and offers a framework for clarifying which claims model-side evidence can support.

1 Introduction

  • Human-derived bias constructs are increasingly used to study LLMs, including stereotype associations, implicit-bias-like patterns, anchoring, framing, and confirmation bias.These approaches complement direct attitude queries and aggregate performance metrics, which may obscure variation across conditions.Source: §1 · p. 1
  • Psychological instruments target human cognition and behaviour, whereas LLM evaluations measure probabilities, completions, rankings, refusals, prompt-driven changes, or simulated decisions.The relationship between the invoked construct and the model behaviour measured is therefore not straightforward.Source: §1 · p. 1
  • The framework relates human-derived constructs and model-side observables to warranted interpretations, clarifying what evaluation designs support without prescribing a new protocol.The paper notes that an association such as higher man–career probability does not by itself establish biased recommendations, harm in hiring, or human-like implicit bias.Source: §1 · pp. 1–2

2 From Evaluation of Human Bias Constructs to LLM Observables

LLM bias probes adapt two broad families of human-derived constructs: social-cognitive bias and cognitive bias in judgment and reasoning. Both translate a construct into model-side observables, making them useful probes while leaving an inferential gap.

Social-cognitive bias

  • Social-cognitive probes adapt prejudice and fairness concepts to measure socially meaningful associations or disparities, including stereotype-consistent pairings and demographic-conditioned outputs.IAT-inspired work compares response-time differences in humans with model likelihoods, generations, or outputs.Source: §2 · p. 2

Cognitive bias in judgment and reasoning

  • Cognitive-bias evaluations test whether LLMs show patterns analogous to human judgment and reasoning biases under controlled tasks, including anchoring, framing, confirmation, overconfidence, and recency effects.Anchoring probes measure shifts toward cues, while framing probes compare answers to logically equivalent prompts with different wording (Echterhoff et al., 2024).Source: §2 · p. 2
  • Both families translate human-derived constructs into model-side observables, but differences in learning processes, mechanisms, and response modalities mean the same construct label may denote different evidence.The translation makes psychological paradigms useful for probing LLMs while raising questions about which interpretations their measurements support.Source: §2 · p. 2

3 The Inferential Gap in Human-Centered Bias Evaluation

The inferential gap arises from construct, subject, and evaluation-context mismatches when human-derived bias constructs are transferred to LLMs. These mismatches shape what model-side measurements can warrant, without making the constructs inappropriate for evaluation.

3 The Inferential Gap in Human-Centered Bias Evaluation

  • Construct mismatch concerns whether a model-side measure captures the intended theoretical construct; a likelihood difference may show stronger associations without establishing human-like implicit bias.Interpreting model probability as analogous to an indirect measure of automatic association requires justification.Source: §3 · p. 3
  • Subject mismatch arises because LLMs are trained artifacts shaped by data, fine-tuning, instruction following, safety interventions, decoding, and prompting rather than human participants with embodied social experience.Behavioural similarity alone does not establish shared underlying reasoning or explanation.Source: §3 · p. 3
  • Evaluation-context mismatch reflects the effects of task setting, prompt design, and response modality on measured behaviour, making model behaviour and evaluation choices hard to disentangle.Prompt templates, demographic labels, examples, decoding settings, and response formats can alter measured associations.Source: §3 · p. 3
  • These mismatches do not make human-derived constructs inappropriate, but claims from likelihood differences, biased generations, or answer shifts depend on how construct, observable behaviour, and context are connected.A measurement may be meaningful evidence while supporting different interpretations depending on those relations.Source: §3 · p. 3

4 An Analytical Framework for Interpreting Bias Claims

The framework connects human-derived constructs, model-side evidence, and evaluation context to four distinct interpretations of bias findings. It clarifies what each interpretation can support and what further evidence stronger claims require.

4 An Analytical Framework for Interpreting Bias Claims

  • Construct, subject, and evaluation-context mismatches each caution against interpreting model evidence more strongly than it warrants.These mismatches concern treating measurements as human constructs, inferring human-like mechanisms from similar behaviour, and generalizing beyond prompt-conditioned outputs.Source: §4 · p. 4
  • The framework distinguishes distributional association, observable behaviour, normative harm, and psychological analogy as separate inferential endpoints.Table 1 links interpretations to typical model-side evidence, relevant mismatches, and the additional support needed for each claim.Source: §4 · pp. 4–5, Table 1 · p. 4
  • Harm claims require a specified setting, affected groups, and plausible consequences, while psychological analogy requires justification of which human-construct features are preserved, absent, or transformed.The framework treats both as stronger interpretations than simply identifying an output pattern.Source: §4 · pp. 4–5
  • Likelihood contrasts, embedding similarities, and association scores most directly support distributional-association claims, not claims about interaction behaviour, harm, or human-like mechanisms.The measured association still needs justification as a meaningful proxy for the intended construct.Source: §4 · pp. 4–5
  • Generated outputs, rankings, recommendations, classifications, and answer shifts support context-bound behavioural interpretations; stronger claims require robustness across relevant evaluation conditions.Outputs depend on prompts, model versions, decoding choices, refusal behaviour, and interaction context.Source: §4 · pp. 4–5
  • In the IAT-inspired example, association differences support a distributional-association reading; behaviour, harm, or implicit-bias claims need additional support.Behaviour claims need evidence that associations shape outputs or task behaviour; harm claims need a use context and plausible consequences, while human-like implicit bias needs a psychological analogy.Source: §4 · p. 5

5 Conclusions

The conclusion argues that human-derived constructs can inform LLM bias evaluation, but do not transfer unchanged. The framework distinguishes what different evidence supports, and Appendix C illustrates the same logic with an anchoring-style evaluation.

5 Conclusions

  • Human-derived constructs remain useful for LLM evaluation, but their transfer creates construct, subject, and evaluation-context mismatches.The conclusion calls for more precise claims about what bias probes can and cannot reveal.Source: §5 · p. 5
  • The framework treats LLM bias evaluation as both measurement and scientific interpretation, distinguishing association, behaviour, harm, and psychological analogy.The conclusion argues that NLP research needs clearer accounts of what probes can and cannot reveal as LLMs enter daily-life tasks.Source: §5 · p. 5
  • Appendix C illustrates the framework’s interpretive logic with an anchoring-style evaluation.This example accompanies the conclusion’s call for clearer accounts of probe limits.Source: §5 · p. 5

Limitations

The paper is a conceptual, literature-based contribution rather than an empirical evaluation, and its framework does not cover every psychological construct or exhaust all interpretations of model bias. It also cautions against misleading equivalences between human cognition and model behaviour when communicating bias findings.

Limitations

  • The paper does not test whether bias probes produce stable results across models, prompts, languages, or deployment settings.It focuses on social-cognitive constructs and cognitive biases in judgment and reasoning; extending the framework to other constructs would require additional work.Source: Limitations · p. 6
  • The four proposed interpretations—distributional association, observable behaviour, normative harm, and psychological analogy—are recurring distinctions, not an exhaustive taxonomy of model bias.Normative harm also depends on social context, affected communities, and deployment conditions that an abstract framework cannot fully specify.Source: Limitations · p. 6

Ethical Considerations

  • Imprecise bias claims may anthropomorphize models, obscure how harms arise, or overstate what an evaluation demonstrates.Conversely, overly narrow interpretations may understate risks when biased outputs are likely to affect users, groups, or society in deployment.Source: Ethical Considerations · p. 6
  • Distinguishing model associations, observable behaviour, normative harm, and psychological analogy can support more responsible communication of bias findings.The intended audiences include researchers, developers, policymakers, and affected communities.Source: Ethical Considerations · p. 6

A Examples of the Inferential Gap in Existing Bias Evaluations

Examples of existing evaluations can yield informative evidence about model responses while leaving stronger claims about beliefs, personality, or human-like bias dependent on additional construct justification. The studies illustrate this inferential gap across political and personality questionnaires, social desirability, and simulated negotiation.

Political bias and self-perception

  • Rutinowski et al. (2024) use political and personality instruments to measure ChatGPT responses under questionnaire conditions, but claims about political views or stable personality depend on their validity for LLMs.Such interpretations transfer human constructs involving social, experiential, and psychological dimensions to a model.Source: A Examples of the Inferential Gap in… · p. 8

Social desirability bias

  • Salecha et al. (2024) find that LLM responses shift toward socially desirable trait profiles when the evaluation context becomes identifiable.Interpreting this as human-like social desirability requires specifying which parts of the human construct are preserved or transformed; similar patterns may reflect training data, instruction following, alignment procedures, or learned conventions.Source: A Examples of the Inferential Gap in… · p. 8

Anchoring bias

A Examples of the Inferential Gap in Existing Bias Evaluations

  • Across these examples, evaluations provide informative findings, but translating them into claims about beliefs, personality, bias, harm, or human-like psychological processes requires care.The evidence may concern responses under particular evaluation conditions without establishing the stronger interpretation.Source: A Examples of the Inferential Gap in… · p. 9

B Example Distinguishing Two Families of Bias Probes

Social-cognitive and cognitive-bias probes translate different human-derived constructs into different model-side observables, so their direct interpretations differ. Stronger psychological or harm-based claims require support beyond those initial measurements.

Social-cognitive bias probe

  • An IAT-inspired social-cognitive probe tests whether an LLM produces stronger associations between social groups and stereotyped attributes, using likelihood contrasts, embedding similarity, ranking, or generated continuations.Its most direct interpretation is distributional association unless further evidence links it to outputs, harms in a use context, or human implicit bias.Source: B Example Distinguishing Two Families… · p. 9

Cognitive-bias probe

  • An anchoring-style cognitive probe tests whether an LLM’s answer shifts toward an irrelevant numerical cue across prompt variants.The most direct interpretation is observable behaviour under the tested prompt conditions; a human-like anchoring claim requires a task-irrelevant anchor, a meaningful target or baseline, and systematic shifts across controlled variants.Source: B Example Distinguishing Two Families… · p. 9

B Example Distinguishing Two Families of Bias Probes

  • The central evidential issue differs: social-cognitive probes must justify association as a proxy, while anchoring probes must distinguish a cognitive-bias pattern from general prompt sensitivity.In either case, claims about harm or psychological analogy require additional support beyond the initial model-side measurement.Source: B Example Distinguishing Two Families… · p. 9

C Applying the Framework to an Anchoring-Style Evaluation

For an anchoring-style evaluation, an output shift under controlled prompt variation supports a claim about context-sensitive behaviour. Stronger claims about human-like anchoring or harm require additional evidence and justification.

C Applying the Framework to an Anchoring-Style Evaluation

  • A higher estimate after exposure to an arbitrary numerical anchor supports the claim that the model is sensitive to an anchoring-style cue under the tested conditions.The direct finding is an output shift under controlled prompt variation.Source: C Applying the Framework to an… · p. 9
  • A human-like anchoring interpretation requires specifying the construct, justifying the anchor as task-irrelevant, identifying a meaningful target or baseline, and showing systematic shifts across controlled variants.Without this mapping, the result is better described as context-sensitive model behaviour.Source: C Applying the Framework to an… · p. 9
  • A harm-based interpretation additionally requires connecting the output shift to a concrete use setting, affected users or groups, and plausible consequences.Examples include distorted salary estimates in compensation support, biased evaluations in financial decision support, or unfair outcomes in online marketplaces.Source: C Applying the Framework to an… · p. 9
Loading 2610.00070v1…