PaperlayerView exact revision on arXiv

Source-linked AI summary

When Does Defendant Statement Matter? A Study of Bias and Persuasion in LLM-Simulated Jurors

Cho-Ying Wu

arXiv:2609.09887v1cs.CLcs.CY

TL;DR

LLMs had not been systematically studied as common-law jurors, despite the importance of persuasion and bias in jury decision-making. This paper introduces JuryBench and evaluates LLM-simulated jurors, finding that background fit outweighs emotional contagion and remorse and that severity judgments favor matched-background defendants.

  • Problem

    Prior research had not systematically examined LLMs as common-law jurors, including how defendant statements, juror ideology, and juror–defendant background relate to decisions.

    Source: §1 · p. 2

  • Method

    JuryBench combines 500 controversial U.S. criminal cases, varied defendant backgrounds and statements, and 12 ideological juror profiles evaluated across 20 frontier LLMs.

    Source: §1 · p. 2

  • Results

    Background fit was more significant than emotional contagion and remorse, and jurors generally showed greater leniency toward defendants with similar backgrounds.

    Source: §4.2 · p. 6, §4.3 · p. 7

  • Takeaways & Limitations

    JuryBench provides a controlled testbed for studying LLM-jury behavior and potential support for mock-jury analysis.

    Source: §5 · p. 9

    The simulation combines opening statements and prosecution evidence into one stage and limits testimony and cross-examination to the defendant’s speech.

    Source: Limitations · p. 9

Abstract

from arXiv · show

LLMs have been used to simulate human decision-making in professional settings, yet their behaviors in common-law jury trials remain unexplored. We study when and how a defendant's courtroom statement affects LLM-simulated jurors, focusing on persuasion, ideological bias, and background-based affinity. To support the analysis, we introduce JuryBench, a benchmark containing controversial criminal cases in U.S. criminal law. In each case, a defendant can claim various plausible justifications to support acquittal or reduced liability. We fix the base case and design defendants of different backgrounds, who give courtroom statements with varying emotional appeal or rebuttal. Jurors with diverse ideological profiles across the spectrum are simulated. We examine 20 frontier LLMs, resulting in a total of 432K decisions and rationales, and quantify changes in verdict severity. Our findings show that LLM-jury simulation echoes many human-jury findings. First, emotional persuasion can be detrimental, since jurors may perceive it as evidence of guilt or inconsistency. Next, we show that background fit between jurors and defendants is a stronger and significant factor than other isolated factors, and that jurors are in general harsher toward opposite-background defendants and lenient toward same-background ones. Finally, we find that juror ideology also strongly shapes severity judgments. These findings highlight both the promise and risks of using LLMs to model jury reasoning and call for careful evaluation. The data and code are available at https://github.com/choyingw/JuryBench

1 Introduction

The introduction presents the first systematic study of LLM-simulated jurors in common-law criminal trials, focusing on emotional persuasion and juror bias. It introduces JuryBench to examine how defendant statements and backgrounds interact with jurors’ decisions.

1 Introduction

  • Common-law jurors evaluate facts, evidence, witness credibility, and justifications using life experience and moral conviction, with verdicts requiring unanimous agreement.Juries consist of twelve people without legal training; they determine guilt, while prosecutors and judges decide charges and sentences.Source: §1 · pp. 1–2
  • Emotional persuasion is central to common-law courtroom storytelling: greater emotional contagion may persuade jurors, but excessive rhetoric or performative appeals can backfire.The paper identifies this tension as a reason to study how defendant statements affect juror decisions.Source: §1 · p. 2
  • Lawyers use jury background checks, focus groups, and mock trials to anticipate bias and shape trial strategies, and some have begun using LLM juries for mock trials.The introduction notes that lawyers may use mock trials to revise strategies and reshape statements intended to move jurors emotionally.Source: §1 · p. 2
  • The study addresses an unexplored role for LLMs: how simulated jurors respond to persuasive defendant statements and how ideology and background affect their decisions.The authors describe this as the first systematic study of LLM-jury behavior and analyze decisions and rationales.Source: §1 · p. 2
  • 500 controversial U.S. criminal cases in JuryBench vary defendants’ backgrounds and plausible justifications, with twelve ideological jurors evaluated across 20 frontier LLMs.The evaluation produced 432K juror decisions and reasons, and the benchmark includes around 400 potential charges and long-form defendant statements.Source: §1 · p. 2

2 Related Work

Prior legal-AI work addresses individual tasks or courtroom simulations that differ from common-law jury evaluation, while human jury-bias research and LLM persona simulation motivate examining persona effects on trial outcomes.

2.1 AI for the Juristic Domain

2.2 Human Jury Bias

  • Human-jury research describes early first-impression bias, possible leniency toward similar-background defendants, harsher judgments of value-violating ingroup members, and a conservative tilt toward guilty verdicts.Bias may begin during opening remarks or when jurors learn a defendant’s background, and jurors may reinforce their initial story. Background affinity can favor similar defendants and disadvantage outsiders, though black-sheep effects can reverse this pattern when an ingroup defendant violates core values. Studies also associate conservative attitudes with greater support for guilty verdicts than liberal profiles.Source: §2.2 · p. 3

2.3 LLM for Persona Simulation

  • LLM persona simulation has been studied in general conversation and domains such as healthcare, but not to examine human-like bias in legal LLM-jury outcomes.Persona instructions implicitly guide responses without specifying exactly what to say or how to react, and may not explicitly reveal a persona’s background (Wang et al., 2025b; Hu and Collier, 2024; Ni et al., 2026; Wang et al., 2025c; Kyung et al., 2025; Wang et al., 2025a). The work analyzes whether different personas affect trial outcomes.Source: §2.3 · p. 3

3 Simulation Framework

The framework uses LLMs and human experts to create trial-like materials unavailable in public records, then simulates ideologically diverse jurors deciding cases with or without defendant statements. It measures verdict severity across varied defendant backgrounds, statements, and charges.

3 Simulation Framework

  • Because public judgment documents omit detailed backgrounds and juror profiles and present post-verdict facts conclusively, LLMs and human experts collaboratively craft the trial materials.Criminal-case information may also be sealed or incomplete for privacy and safety reasons.Source: §3 · pp. 3–4

3.1 Case Generation

  • GPT-5.4 generates 500 controversial, arguable criminal cases across about 400 potential charges, with defendant backgrounds, case backgrounds, evidence, and emotionally persuasive statements.Backgrounds include stereotype-aligned Group A and stereotype-subverting Group B; a practicing lawyer reviews case feasibility and statement fit, while experts assess emotional contagion, remorse, and affinity scores.Source: §3.1 · p. 4

3.2 Juror Simulation

  • GPT-5.4 generates juror profiles with ideology scores uniformly distributed from −5, most conservative, to +5, most liberal.The simulation follows prior work on ideology simulation.Source: §3.2 · p. 4

3.3 Case Decision

  • Jurors receive profiles as system prompts and decide every case across defendant-group and statement conditions, giving brief reasons that are checked for verdict consistency.The prompt directs jurors to consider the available evidence and charges, and to return not guilty when evidence cannot support a guilty decision.Source: §3.3 · p. 4
  • The benchmark covers multiple charges, mens rea distinctions, and attempted offenses, and asks jurors to report the most severe crime for which a defendant is guilty.Mens rea categories include intentional, reckless, and negligent conduct.Source: §3.3 · p. 4
  • Severity is scored from 0 for acquittal or valid justification to 15 for the heaviest charge, with scores above 5 classified as felonies.A human expert labels each potential charge; scores of 5 or below are misdemeanors, while scores above 5 typically carry sentences exceeding one year.Source: §3.3 · pp. 4–5

4 Analysis and Findings

Across LLM-simulated jurors, defendant–juror background fit and juror ideology shaped verdict severity, while statements often left verdicts unchanged and could either mitigate or intensify judgments.

4 Analysis and Findings

  • Across 500 crafted cases, average mode share was 0.645, the histogram peaked at 50%-60%, and only three cases had unanimous decisions, indicating substantial controversy.Mode share was calculated from the most frequent charge among 960 predictions per case.Source: §4 · p. 5

4.1 How do statements affect jury decisions?

  • About 70%-80% of statement cases showed no verdict change, indicating that simulated jurors often retained their initial judgments.The authors connect this decision inertia to empirical findings on jurors’ early case impressions.Source: Table 1 · p. 5, §4.1 · p. 5
  • Statements could reduce severity when jurors accepted a defense or related to remorse, but intense, inconsistent, or remorseful statements could also reinforce guilt judgments.Examples include statements interpreted as weakening credibility or as implicit admissions of responsibility.Source: §4.1 · pp. 5–6

4.2 Which latent factors affect jury decisions?

  • Background fit was more influential than emotional contagion or remorse: most models had significant positive coefficients, and none had significant negative coefficients.Jurors often reduced severity when they related to a defendant’s background, while opposite-background statements could increase severity.Source: Figure 2 · p. 6, §4.2 · pp. 6–7
  • Emotional contagion reduced severity for a modest majority of models, while remorse reduced severity for a sizable minority, had the opposite effect in some models, and was neutral for half.Some jurors instead treated rhetoric as a reason for distrust or interpreted remorse as reinforcing guilt.Source: §4.2 · p. 6

4.3 Do Group-A and Group-B differ in verdict severity?

  • Nearly all models showed lower severity for Group-A in same-direction cases, but lower severity for Group-B in opposite-direction cases, consistent with matched-background leniency and mismatch penalties.The analysis compared cases with same-direction versus opposite-direction background fit.Source: §4.3 · p. 7
  • Statement-related shifts in the Group-A/Group-B severity gap had roughly evenly divided directions across models, while opposite-direction cases often had larger shifts.This suggests stronger post-statement movement in the gap for opposite-direction cases, without a consistent overall tendency to enlarge or close it.Source: §4.3 · p. 8

4.4 How does ideology affect jury’s decisions?

  • Conservative jurors assigned higher severity than neutral jurors, who were harsher than liberal jurors; stronger ideological leaning generally meant more extreme severity judgments.About 60% of models showed the same direction of statement-related severity change across ideologies, and about 70% showed larger changes for conservative than liberal jurors.Source: §4.4 · p. 8

5 Conclusion

This work introduces JuryBench for systematic study of LLM jurors in common-law cases and finds that simulated decisions echo several legal-psychology findings. The benchmark may support mock-jury analysis and trial-strategy revision while informing more transparent and accountable legal simulations.

5 Conclusion

  • JuryBench enables fine-grained analysis of LLM-jury behavior through controversial criminal cases, diverse juror profiles, controlled defendant backgrounds, and emotionally persuasive statements.The paper presents this as the first systematic study of LLM-jury behavior in the common-law setting.Source: §5 · p. 9
  • LLM-jury decisions echo legal-psychology findings about sticky early impressions, double-edged defendant statements, and ideology- or affinity-driven judgments.Source: §5 · p. 9
  • JuryBench offers a controlled testbed for mock-jury analysis and lawyers’ mock trials with focus groups to revise trial strategies, while supporting transparent and accountable legal simulations.Source: §5 · p. 9

Limitations

The study’s findings are bounded by a simplified, text-only simulation of individual jurors in U.S. common-law criminal trials and by its choice of 20 frontier LLMs. These limits exclude important courtroom and jury processes and may constrain how well results generalize.

Limitations

  • The simulation combines trial stages, simplifies testimony and cross-examination to defendant speech, and omits jury discussion and unanimous consensus.The authors focus on individual jurors’ responses to emotional persuasion; full cross-examination and group deliberation are outside the study’s scope.Source: Limitations · p. 9
  • The study primarily examines U.S. common-law criminal trials, although the implicit factors shaping LLM-jury decisions are not limited to a country or case type.The current scope primarily draws on U.S. criminal charges.Source: Limitations · p. 9
  • The text-only courtroom simulation cannot capture how body language, facial expressions, and other real-courtroom cues may amplify emotional contagion or remorse.Source: Limitations · p. 9
  • The study evaluates 20 frontier LLMs, and their relative behaviors may change as providers update newer versions’ training data, safety policies, and inference systems.Source: Limitations · p. 9

Ethical Considerations

The tool is intended to help legal professionals simulate jurors for mock-trial strategy, not automate legal judgment or interfere with real cases.

Ethical Considerations

  • Although cases, defendants, and jurors are synthetic, profiles contain sensitive social attributes and criminal allegations; the release will include intended-use restrictions, documentation, and warnings about intentionally introduced stereotypes.Source: Ethical Considerations · p. 9

A Human Evaluation

In a small human evaluation, defendant statements tended to increase verdict severity, though participants also sometimes recognized justifications and empathized with defendants. The evaluation was intended as evidence of similarity between human and LLM reasoning, not as a representative jury benchmark.

A Human Evaluation

  • Twelve U.S. lay participants without legal training judged 25 selected cases before and after seeing each defendant’s statement, with plain-English explanations of potential charges.Participants were instructed not to revise their initial decision retroactively and briefly explained changed decisions.Source: A Human Evaluation · pp. 12–13
  • When severity decreased, participants most often recognized grounds for justification, sometimes empathizing through emotional contagion; some statements combined genuine remorse with valid justifications.The authors present this evaluation as evidence of similarity between LLM and human reasoning, not as a large, representative human-jury benchmark.Source: A Human Evaluation · p. 13
  • In the 25-case evaluation, humans leaned toward greater severity after defendant statements (negative SE), often treating remorse as evidence of guilt or statements as performative.These reasons for increased severity were also described in the paper’s account of LLM reasoning.Source: Table 2 · p. 13, A Human Evaluation · p. 13
  • Participants’ mostly neutral self-described ideologies and difficulty perceiving defendant backgrounds in text limited bias analysis, so the regression omitted background fit; coefficients were 0.173 for p and -0.168 for q.Both coefficient p-values were below 0.05 but above 0.01.Source: A Human Evaluation · p. 13

B Joint-Effect Regression

The interaction model shows that LLM jurors associate reduced post-statement severity with defendant-juror affinity and coherent emotional-remorseful appeals, while isolated cues can backfire. It complements the additive model by explaining how cue effects depend on their combination.

Joint-Effect Regression

  • In the full interaction model, coefficients are conditional rather than independent causal effects, and each factor’s marginal effect depends on the other terms.Unlike the isolated additive terms, the full model captures joint effects, making its coefficients more complex to interpret.Source: B Joint-Effect Regression · p. 13
  • Positive M, pq, and pqM terms associate with reduced post-statement severity, indicating matched affinity and coherent emotional-remorseful appeals—especially together—favor leniency.The positive pqM interaction indicates that affinity strengthens the mitigating effect when emotionality and remorse appear together.Source: B Joint-Effect Regression · pp. 13–14
  • Isolated emotionality or remorse can backfire, and their mitigating effects weaken under stronger affinity, although affinity alone is associated with lower severity.The authors suggest isolated cues may seem performative, insufficient, or like implicit evidence of responsibility; their reduced effect under affinity may reflect saturation or credibility discounting.Source: B Joint-Effect Regression · pp. 13–14
  • The additive model profiles each LLM’s marginal sensitivity to emotional contagion, remorse, and background fit, while the interaction model explains how those sensitivities combine.The additive results identify M as the most robust standalone predictor, while p and q are weaker and model-dependent.Source: B Joint-Effect Regression · p. 14

C Prompts in Use · D Instructions for Legal Expert and Manual Check

The appendix documents prompts for generating cases, juror profiles, and verdicts, alongside expert instructions and manual checks of generated materials.

C Prompts in Use

D Instructions for Legal Expert and Manual Check

E Common-Law and Civil-Law Settings

Civil-law systems center codified statutes and legal professionals, while common-law criminal trials emphasize adversarial argument and lay jurors’ evaluation of evidence and persuasion.

Civil-law settings

  • Civil-law systems center written statutes and codes, with legal professionals interpreting provisions, examining case materials, and producing judgments consistent with codified rules.Representative jurisdictions include France, Germany, Italy, Spain, Japan, South Korea, China, and countries in continental Europe and Latin America.Source: E Common-Law and Civil-Law Settings · p. 14

Common-law settings

  • Common-law criminal trials rely on adversarial argument and trial fact-finding; U.S. lay jurors assess evidence, credibility, defendant statements, and proof beyond a reasonable doubt.Attorneys translate legal claims into persuasive narratives appealing to commonsense, life experience, moral judgment, credibility, and emotion.Source: E Common-Law and Civil-Law Settings · pp. 14–15

Related papers

Loading 2609.09887v1…