PaperlayerView exact revision on arXiv

Source-linked AI summary

Humans and LLMs Diverge on Probabilistic Inferences

Gaurav Kamath, Sreenath Madathil, Sebastian Schuster, Marie-Catherine de Marneffe, Siva Reddy

arXiv:2602.23546v1cs.CLcs.AI

TL;DR

The paper asks how humans and reasoning LLMs handle probabilistic inferences that are not strictly entailed and remain largely unexplored in open-ended settings. It introduces PROBCOPA and compares human and model likelihood judgments, finding persistent differences in distributions and response variation. The authors also identify common patterns in LLM reasoning chains while noting limitations in likelihood elicitation and item provenance.

  • Problem

    Evidence is limited on how reasoning LLMs behave on open-ended, non-deterministic probabilistic inferences rather than logical or mathematical tasks.

  • Method

    The paper introduces PROBCOPA and compares human likelihood judgments with responses from reasoning LLMs using verbalized scores and distributional metrics including Wasserstein distance.

  • Results

    Reasoning LLMs fail to match human judgment distributions and produce less response variation, especially for inferences involving uncertainty.

  • Takeaways & Limitations

    The findings support evaluating reasoning beyond deterministic settings and provide initial insight into model behavior in open-ended, non-deterministic contexts.

  • Takeaways & Limitations

    The dataset’s COPA-derived items may have appeared in most models’ training data, and verbalized likelihood scores may not faithfully reflect model uncertainty.

Abstract

from arXiv · show

Human reasoning often involves working over limited information to arrive at probabilistic conclusions. In its simplest form, this involves making an inference that is not strictly entailed by a premise, but rather only likely given the premise. While reasoning LLMs have demonstrated strong performance on logical and mathematical tasks, their behavior on such open-ended, non-deterministic inferences remains largely unexplored. We introduce ProbCOPA, a dataset of 210 handcrafted probabilistic inferences in English, each annotated for inference likelihood by 25--30 human participants. We find that human responses are graded and varied, revealing probabilistic judgments of the inferences in our dataset. Comparing these judgments with responses from eight state-of-the-art reasoning LLMs, we show that models consistently fail to produce human-like distributions. Finally, analyzing LLM reasoning chains, we find evidence of a common reasoning pattern used to evaluate such inferences. Our findings reveal persistent differences between humans and LLMs, and underscore the need to evaluate reasoning beyond deterministic settings.

1 Introduction

The paper studies how humans and LLMs make probabilistic inferences from partial information, introducing PROBCOPA to compare their likelihood judgments. It finds persistent differences between human and model judgments and identifies patterns in LLM reasoning chains.

  • Motivation: Humans use partial information to reach conclusions that are likely rather than strictly entailed.The paper calls these judgments probabilistic reasoning and the individual cases probabilistic inferences.
  • Contributions: The study compares human and LLM judgments of commonsense probabilistic inferences.
  • Findings: Models generally align with humans on highly likely or highly unlikely inferences but struggle with uncertain cases and show little sampled-response variation.
  • Contributions: PROBCOPA contains 210 handcrafted English probabilistic inferences with at least 25 human annotations per item.
  • Contributions: The paper identifies patterns in LLM reasoning chains that help explain how models arrive at final responses.

2 The PROBCOPA Dataset

PROBCOPA reframes COPA commonsense alternatives as separate probabilistic inference items, then collects numerical likelihood judgments from human annotators. The resulting dataset has 25–30 annotations per item and highly reproducible aggregate responses.

  • Dataset construction: PROBCOPA targets inferences that vary in likelihood given a premise rather than being logically entailed.
  • Dataset construction: COPA items pair a premise with two plausible alternatives, one more plausible than the other.
  • Dataset construction: Each COPA item is split into two NLI-style premise–hypothesis items, producing 210 probabilistic inferences from 105 sampled test items.
  • Human annotation: 328 native English speakers annotated up to 30 items each, with attention checks and randomized stimulus order.
  • Human annotation: Participants rated each hypothesis on a 0–100 sliding scale as the likelihood of the hypothesis given the premise.
  • Validation: Each item retained 25–30 unique-participant annotations, with a median of 28 annotations per item.
  • Validation: Mean item ratings were highly reproducible across two validation rounds, with Spearman’s ρ = 0.98 and 0.97.

3 Analysis of Human Responses

Human responses in PROBCOPA are graded, item-level distributions are usually unimodal, and uncertainty is associated with greater disagreement. Items with greater disagreement also tend to take longer to answer.

  • Methodology: Raw likelihood scores are retained to preserve meaningful differences in how annotators use the event-likelihood scale.
  • Methodology: Differential entropy measures response dispersion, with higher values indicating greater dispersion and lower values indicating greater concentration.
  • Human response distributions: Human likelihood scores across the dataset have three modes, with many responses between them indicating graded judgments.
  • Human response distributions: Human responses for individual items are almost always unimodal, unlike the dataset-wide tri-modal distribution.
  • Agreement and difficulty: No items show close annotator agreement that a hypothesis has medium likelihood.
  • Agreement and difficulty: Spearman’s ρ = 0.31 (p = 6.45e − 06) links higher response entropy with longer response times.The authors interpret this association as evidence that higher variation relates to item difficulty rather than noise.

4 Comparison with Responses from Reasoning LLMs

The paper compares eight reasoning LLMs with human probabilistic-likelihood judgments using verbalized numerical estimates and distributional metrics. Models align better on extreme-likelihood items but diverge in uncertain cases, show less response variation, and remain unlike humans despite ensembling.

  • Method: Wasserstein distance measures similarity between human and model likelihood-score distributions without requiring matching support.Lower values indicate higher distributional similarity, while higher values indicate lower similarity.
  • Method: Eight contemporary reasoning LLMs are evaluated against human likelihood judgments using verbalized scores from 0 to 100.Each model reasons about a premise-hypothesis pair and returns a numerical likelihood estimate, repeated across runs.
  • Results: Models rarely return medium-likelihood scores, instead favoring strong judgments across the numerical scale.This tendency is least extreme for GPT-5 but remains present across the reported models.
  • Results: Model-human alignment is strongest for highly likely or unlikely items and breaks down for middle-range items where human judgments show less consensus.The same pattern appears for both median scores and complete response distributions, whereas human-human similarity remains roughly consistent across items.
  • Results: Models show less response variation than humans for nearly every item, while temperature and persona prompting fail to produce human-level variability.Human baseline participants exhibit roughly similar variation across annotation rounds.
  • Results: Higher reasoning effort does not produce a statistically significant change in median likelihood scores.This result contrasts with prior findings that increased reasoning yields more overconfidence.
  • Results: Ensembling model outputs improves distributional alignment but remains worse than the human-human baseline.Model-human distributional differences are substantially larger than differences between independent human groups.

5 Analyzing LLM Reasoning Chains

The paper analyzes reasoning chains to connect model reasoning behavior with human uncertainty and to identify recurring evaluation strategies. Longer chains tend to occur on items humans judge less consistently, while models frequently consider alternative scenarios explicitly.

  • 5 Analyzing LLM Reasoning Chains: Reasoning-chain analysis examines common patterns in how LLMs evaluate probabilistic inferences.The analysis focuses on intermediate reasoning tokens produced before final likelihood responses.
  • 5 Analyzing LLM Reasoning Chains: Ensembling outputs improves model-human distributional alignment but still falls short of the human-human baseline.This provides a distribution-level comparison alongside the reasoning-chain analysis.
  • 5 Analyzing LLM Reasoning Chains: Reasoning-chain length correlates much more weakly with human response time than with human differential entropy.The highest reported correlation with human response time is lower than Gemini-3’s 0.50 correlation with human entropy.
  • 5 Analyzing LLM Reasoning Chains: 90 of 100 sampled reasoning chains explicitly consider alternative scenarios when forming likelihood judgments.The alternatives are used to frame the model’s final response across different tested models.

6 Discussion

The study finds that models remain overconfident on inherently uncertain inferences and diverge from human probabilistic judgments. Human responses are graded, while model reasoning chains commonly consider alternative scenarios.

  • Models rarely indicate medium likelihood, instead favoring more extreme judgments even for inherently uncertain inferences.This reflects the study’s connection between model overconfidence and open-ended probabilistic reasoning.
  • Human participants produce graded probabilistic judgments, and their response distributions are generally unimodal.The authors contrast this pattern with significant, often bimodal, variation reported for NLI data.

7 Related Work

Prior work has examined human inference variability, NLI, reasoning LLMs, probability-related abilities, and uncertainty quantification. This paper distinguishes its focus on everyday uncertain events from formal probability tasks and deterministic reasoning benchmarks.

  • Reasoning in Humans: Human reasoning research reports recurrent logical fallacies and motivates understanding human reasoning in terms of probabilistic beliefs.Oaksford and Chater’s account provides the motivation operationalized by this study.
  • Natural Language Inference: NLI traditionally classifies premise–hypothesis pairs as entailment, contradiction, or neutral, but human judgments show variation beyond annotation noise.Related work finds substantial judgment variation in datasets including SNLI and MNLI.
  • Reasoning LLMs: Reasoning LLMs generate intermediate reasoning chains before final responses and have shown strong capabilities on reasoning tasks.The paper notes that these models are commonly studied using mathematical and coding benchmarks.
  • Research Gap: Reasoning LLMs remain comparatively underexplored in open-ended, non-deterministic reasoning contexts.The paper identifies this gap against the field’s emphasis on automatically verifiable math and coding tasks.
  • Probabilistic Reasoning: Existing probability research often tests formal probability application or explicit statistical distributions, whereas this study examines everyday uncertain events without requiring probability theory.This distinction defines the paper’s setting within work on LLM probabilistic reasoning.
  • Uncertainty Quantification for LLMs: Uncertainty quantification studies how confident models are relative to how confident they should be, while this paper focuses on the likelihood assigned to probabilistic inferences.The paper places its task adjacent to, but distinct from, conventional LLM uncertainty quantification.

8 Conclusion

The paper evaluates probabilistic reasoning in humans and LLMs with PROBCOPA, finds persistent differences in judgments and output variation, and identifies recurring patterns in model reasoning chains.

  • PROBCOPA contains 210 English probabilistic inferences, each annotated by at least 25 human participants.The dataset supports comparison of human and model judgments in open-ended, non-deterministic settings.
  • Models fail to match human judgment distributions or produce human-level output variation on probabilistic inferences.The conclusion reports significant differences between human and reasoning-LLM judgments.
  • Analysis of model reasoning chains identifies common reasoning patterns, but correlations with human behavior are mixed.The authors frame these findings as initial evidence for studying reasoning beyond logical or deductive tasks.

Limitations

The study is limited by its English-only scope, possible training-data exposure from COPA-derived items, and uncertainty about the faithfulness of verbalized likelihood scores.

  • The study is limited to English, with additional limitations discussed by the authors.
  • Verbalized likelihood scores may not faithfully represent reasoning models’ likelihood judgments.The authors call for alternative likelihood-elicitation methods suited to reasoning models.
  • COPA-derived sentences may have appeared in models’ training data and could affect their behavior on these items.Although the reframed task yields judgments different from original COPA labels, prior exposure remains possible.

B Model Inference Details

The study specifies the reasoning models, inference providers, prompt format, and likelihood-rating scale used to compare model outputs with human annotations. Persona prompting varied demographic or psychological descriptions across sampled responses.

  • Table 2 lists the exact model versions evaluated in the study.
  • Inference used provider APIs and batch processing, with Grok-4.1 Fast accessed through OpenRouter.The other models were accessed through Gemini, OpenAI, Anthropic, or Together AI APIs.
  • Models received a premise and hypothesis, then returned a likelihood value from 0 to 100 after internal reasoning.The prompt required the final number to be enclosed in <answer> tags.
  • The prompt supplied verbal descriptions mapping numerical ranges from absolutely no chance through absolutely certain.
  • Persona prompting appended demographic or psychological descriptions to sampled responses to test whether modeled variation became more human-like.Each PROBCOPA item received 30 sampled responses using different persona descriptions.

E Claude Opus-4.6

Claude Opus-4.6 produced nearly invariant likelihood scores and usually no reasoning-chain summaries, especially under low and medium reasoning-effort settings. The authors offer a speculative API-related explanation but lack transparency to verify it.

  • Claude Opus-4.6 returned almost completely invariant likelihood scores across 30 sampled responses for each tested item.The study also found almost no reasoning-chain summaries in these responses.
  • Output length was exactly 10 tokens under medium and low reasoning effort, and often remained the same under high effort.
  • The authors speculate that an adaptive thinking budget may classify these inputs as not requiring meaningful intermediate reasoning.They emphasize that this explanation remains speculative because the model or API is not transparent.

F Extended Results Figures

Extended figures document human–model differences in likelihood distributions, item-level alignment, response entropy, reasoning-chain relationships, and prompting manipulations. Additional materials also describe annotation interfaces and validation procedures.

  • Human responses form a tri-modal overall likelihood distribution, whereas model responses are bi-modal with few middle-range scores.
  • Reasoning-chain length correlates more consistently with human likelihood-score entropy than with human response time.Grok-4.1 Fast is excluded because it does not return reasoning-chain information; Claude Sonnet-4.5 uses output tokens as a proxy.
  • Model–human Wasserstein distances are highest for items with middle-range human median scores, unlike human-to-human comparisons.
  • Human baseline annotations resemble the original annotations in differential entropy and correlate closely with their median likelihood scores.
  • Increasing temperature produces more diverse model responses, while increasing reasoning effort does not significantly change models’ likelihood scores.
  • Persona prompting does not yield human-level variation or human-like response distributions.
  • Instructional examples used slider-based likelihood ratings with automatic feedback before the main phase stated that responses had no right or wrong answers.
  • A validation experiment varied prompt wording without producing different response distributions from the original annotations.
Loading 2602.23546v1…