PaperlayerView exact revision on arXiv

Source-linked AI summary

Which Answer Did the Seventeen-Year-Old Write? A Behavioral Case Study of Author-Age Misattribution, Miscalibrated Confidence, and Non-Corrective Self-Explanation in Six Frontier Language Models

Brianne Lee

arXiv:2610.31946v1cs.CL

TL;DR

Language models may mistake writing style for demographic identity, even when the authors’ ages are known and the style-age pairing is counter-stereotypical. This case study tests six models on real handwritten answers under varied conditions and finds confident misattribution, especially for the youngest writer’s formal answer. The findings are limited to a small, single-topic, single-language stimulus set.

  • Problem

    The study asks whether models treat linguistic register as an author’s demographic identity rather than as a feature of how the text was written.

    Source: §1 · p. 1

  • Method

    The authors test six models on real handwritten answers with recorded ages, varying framing and attribution format across experimental conditions.

    Source: §3.1 · p. 4, §3.3 · p. 5

  • Results

    Several models treat register as an unacknowledged demographic proxy, while stated confidence and post-hoc explanation quality do not reliably track correctness.

    Source: §5 · pp. 11–12, §8 · pp. 13–14

  • Takeaways & Limitations

    The findings are directly relevant to systems that infer, personalize, or moderate based on demographic characteristics inferred from writing.

    Source: §8 · p. 14

    The three-item, single-topic, single-language case study demonstrates a mechanism but does not estimate population-level misattribution rates or support extrapolating its accuracy figures to other texts or languages.

    Source: §6 · p. 12

Abstract

from arXiv · show

Three students answered the same exam prompt by hand: "What is happiness? Are you happy?" Their ages, recorded at collection, are 17, 28, and 45. The 17-year-old's answer is hedged and formally phrased. Asked to match each answer to an age and gender label, most frontier language models confidently assign it to an adult. This paper reports a structured case study of that failure across six models: Claude Fable 5, Gemini 3.1 Pro, GPT-5.6 Sol, Grok 4.5, GPT-4o (2024), and DeepSeek V4. We ran 1,081 API sessions and scored 3,225 responses across six experimental conditions that manipulate framing, disclosure format, and audience-awareness instruction. Three of six models perform at or near chance (1/3) on the matching task while reporting 73-88% mean confidence. The youngest writer's answer is misattributed to an adult in up to 90% of trials. Removing the multiple-choice scaffold collapses accuracy further: three models never place within three years of the true age once open-ended estimation is required. Institutional framing ("a community-college biology final") significantly degrades the best-performing model's accuracy; a neutral label does not. A single audience-awareness instruction significantly improves one model (+21 points) and moves another to ceiling, with no detectable effect on the three weakest models. Stated confidence is uncorrelated or inversely correlated with correctness for five of six models. After ground truth is revealed, one model names its own stereotyping in 68 of 114 explanations, yet shows the lowest accuracy and highest confidence of any model under blind presentation: articulate retrospective diagnosis that does not track the quality of its judgments. This is a three-item, single-topic, single-language case study meant to isolate a mechanism, not to estimate a population bias rate. Data, code, and raw model outputs are publicly released.

1 Introduction

The paper argues that language models mistake writing style for demographic identity: formality and hedging can read as adulthood even when the writer is a teenager. It uses real, counter-stereotypical answers with recorded ages to examine this mechanism, not to estimate its prevalence across writing.

1 Introduction

  • A formally phrased, hedged answer that most tested models assign to an adult was written by the 17-year-old.The three handwritten answers came from writers aged 17, 28, and 45, with ages recorded at collection.Source: §1 · p. 2
  • The study uses real, human-written answers whose recorded authorship runs against the register-age cue, allowing the mismatch to be examined directly.The authors contrast this design with scraped or synthetic text whose demographic labels are self-reported or inferred.Source: §1 · p. 2
  • The paper examines six dimensions, including framing, forced-choice versus open-ended attribution, confidence as a correctness signal, and post-reveal explanations.It also reports AI-assisted tooling errors caught through human verification.Source: §1 · p. 2
  • With three items on one topic in one language, the study demonstrates a mechanism rather than estimating how often it occurs across writing.The authors describe the failure as real and severe where it occurs, while limiting claims about prevalence.Source: §1 · p. 2

2 Related Work

The related work situates the study at the intersection of demographic inference, register as an authorship signal, confidence calibration, self-correction, and evaluation format. Its distinctive design tests these questions using counter-stereotypical texts with ages recorded at collection.

2.1 Demographic inference from text

  • Prior demographic-inference studies use scraped or synthetic text with self-reported or inferred labels, whereas this study uses recorded ages and deliberately counter-stereotypical register.Unfiltered natural data may contain genuine correlations between formal register and older authorship.Source: §2.1 · p. 3

2.2 Register, not demography

  • Register research treats stylistic variation as an authorial signature; this paper asks whether models read that signal as identity rather than writing style.The paper reports that several models conflate the two, with three performing at or below chance.Source: §2.2 · p. 3

2.3 Confidence, calibration, and overconfidence

  • Rather than measuring only aggregate calibration, the study tests whether confidence predicts correctness within each model; for five of six, it does not or runs in the wrong direction.Mean confidence differences between models are large and highly reliable, so stated confidence may indicate which model was asked more than whether its answer is correct.Source: §2.3 · p. 3

2.4 Self-correction and its limits

  • Earlier self-correction work measures correction within one interaction, while this study compares post-error explanations with the same model’s accuracy and confidence in independent sessions.The paper reports that these measures can dissociate.Source: §2.4 · p. 4

2.5 Forced-choice evaluation

  • The paper compares forced-choice matching and open-ended estimation on identical stimuli, avoiding differences in item difficulty across evaluation formats.This directly tests whether multiple-choice accuracy overstates performance relative to open-ended generation.Source: §2.5 · p. 4

3 Method

The study uses three fixed, handwritten responses with ages and genders recorded at collection, then tests six models under varied framing and response conditions. It scores accuracy and confidence with session-level methods that account for dependent item assignments.

3.1 Stimuli

  • Three handwritten answers to the same happiness prompt form the fixed stimulus set, with author ages and genders recorded at collection.The texts are deidentified in released materials and reproduced verbatim.Source: §3.1 · p. 4

3.2 Models

  • Six API-accessed systems were tested, including five frontier or near-frontier models at the time and GPT-4o (2024) as an earlier-generation comparison.The paper records exact API model strings and access dates in the released repository.Source: §3.2 · p. 5

3.3 Experimental conditions

  • Conditions varied blind, institutional, neutral, and audience-aware framing, open-ended age estimates, and self-reported probability distributions.Matching conditions required a one-to-one assignment among labels; open-ended estimates were scored against the 17-year-old’s true age using a range or ±3-year criterion.Source: §3.3 · p. 5

3.4 Scale and data quality

  • After excluding 18 duplicated-response rows, the analysis used 3,225 item-level rows from 1,075 sessions.The full grid initially produced 1,081 API sessions and 3,243 scored rows.Source: §3.4 · p. 6

3.5 Metrics

  • Accuracy, misattribution, confidence–correctness gap, session scores, and confidence–correctness correlation quantify performance; session-level tests account for dependent item rows.Post-reveal explanations were lexically coded, and the authors note the coding was not independently multi-rater validated.Source: §3.5 · p. 6

4 Results

The results show substantial differences across models, including confident performance at or below chance for several systems. The 17-year-old’s formal answer is often read as adult-written, and open-ended age estimation performs worse than matching.

4.1 F1: Misattribution is current, severe, and in several cases at or below chance

  • Table 1 shows three models averaging 0.69 to 1.05 correct per session against a chance mean of 1, despite 73–88% mean confidence; the other three perform above chance or well above it.Grok 4.5 is indistinguishable from chance, DeepSeek V4’s below-chance mean is suggestive rather than established, and GPT-4o’s session distribution departs from chance.Source: Table 1 · p. 7, §4.1 · p. 7

4.2 F2: Same text, opposite readings

  • Claude Fable 5 correctly assigns the 17-year-old’s answer in 33 of 36 trials, while the three lowest-accuracy models predominantly assign it to 28F.The reported cues include emotional maturity and life-stage framing, which ran in the wrong direction for this writer.Source: §4.2 · p. 7, Table 2 · p. 8

4.3 F3: Open-ended attribution is far worse than matching

  • Under open-ended estimation, Grok 4.5, GPT-4o (2024), and DeepSeek V4 each score 0.00 on the teen’s item; even Claude Fable 5 scores only 0.26.Gemini 3.1 Pro scores 0.33, while GPT-5.6 Sol scores 0.17; scoring allows a range or a point estimate within ±3 years of age 17.Source: §4.3 · p. 8

4.4 F4/F5: Context inversion and a capability-gated fix

Institutional framing significantly harms Claude Fable 5, while an audience-awareness instruction improves GPT-5.6 Sol; confidence–correctness relationships vary by model and are often weak or negative.

4.4 F4/F5: Context inversion and a capability-gated fix

  • Claude Fable 5 falls from 0.94 to 0.69 under institutional framing (p< 0.001), while audience awareness raises GPT-5.6 Sol from 0.63 to 0.84 (p= 0.002).Audience awareness also brings Claude Fable 5 to 72 of 72, but its change is not statistically detectable (p= 0.082); a neutral label carries no such cost for Claude.Source: §4.4 · pp. 8–9, Table 3 · p. 8
  • The three lowest-accuracy models show no detectable change under either manipulation, which the authors interpret as a floor effect.These models were already at or near chance, leaving little room for detectable movement at this sample size.Source: §4.4 · p. 9

4.5 F6: Confidence is a per-model trait, not a per-item signal

  • Confidence tracks correctness positively only for Claude Fable 5, negatively for Gemini 3.1 Pro, Grok 4.5, and GPT-4o, and not detectably for GPT-5.6 Sol or DeepSeek V4.DeepSeek's null correlation is not evidence of calibration: its confidence range is narrow (SD 5.2; mean 88.0).Source: §4.5 · p. 9

4.6 F7: Confession without correction

After ground truth is revealed, DeepSeek V4 often names stereotyping in its explanations despite having the lowest blind accuracy and highest confidence. A comparison of GPT-4o and GPT-5.6 Sol shows improvements across the reported metrics, but the pattern is not general across labs.

4.6 F7: Confession without correction

  • DeepSeek V4's post-reveal explanations contain the stem stereotyp- in 68 of 114 sessions, despite its lowest blind accuracy and highest stated confidence.Across 603 reveal sessions, external blame was rare: 4 total, including 3 from DeepSeek V4.Source: §4.6 · p. 10, Table 5 · p. 10
  • The authors conclude that self-diagnosis quality does not track judgment quality; stateless sessions prevent claims about learning between explanations.They characterize post-hoc self-explanation as plausible text generation rather than a readout of the process producing judgments.Source: §4.6 · p. 10

4.7 F8: A generational comparison

  • GPT-4o (2024) to GPT-5.6 Sol shows accuracy rising from 0.25 to 0.63, confidence falling from 0.86 to 0.64, and the confidence–correctness gap falling from +0.60 to +0.01.The authors interpret this as consistent with a deliberate training-time shift, but note that newer DeepSeek V4 performs poorly and never abstains; progress appears lab-specific, not automatic with recency or scale.Source: §4.7 · pp. 10–11

4.8 F9: A meta-finding on tool fallibility

The paper reports both AI-assisted analysis errors caught through manual review and findings from a probability-distribution condition. In that condition, expressing uncertainty as a distribution did not improve weak models' accuracy or reduce strong models' accuracy.

4.8 F9: A meta-finding on tool fallibility

  • AI-assisted coding and analysis produced several confident errors, all caught by manual human verification before inclusion in the released data or paper.The authors recommend budgeting for adversarial human review and reporting what it catches.Source: §4.8 · p. 11

4.9 Condition P: self-reported distributions

  • In Condition P, probability assigned to the true label ranges from 25.0% for DeepSeek V4 to 61.4% for Gemini 3.1 Pro, while mean entropy stays between 0.98 and 1.34 bits.Expressing uncertainty as a distribution did not make weak models more accurate or strong models less so.Source: §4.9 · p. 11, Table 6 · p. 11

5 Discussion

Across the study’s conditions, several models treat linguistic register as evidence of demographic identity, and the weakest judgments can be delivered with high confidence. Confidence often fails to track correctness, while prompting and retrospective explanations show distinct limits as indicators of model capability.

5 Discussion

  • Several models treat linguistic register as a demographic proxy rather than a feature of how a passage was written.This error is clearest on the deliberately counter-stereotypical item.Source: §5 · p. 11
  • 73–88% confidence accompanies the models’ most severe errors, rather than hedged uncertainty.The paper notes that this failure mode could operate silently in systems that infer or personalize from writer demographics.Source: §5 · p. 12
  • For five of six models, per-item confidence is uncorrelated with correctness or associated with correctness in the wrong direction.For Gemini, GPT-4o, and Grok, routing low-confidence answers to review while trusting high-confidence answers would misroute in the wrong direction on this task.Source: §5 · p. 12
  • Audience-awareness prompting appears to elicit existing calibration in some models, but cannot create it where it is absent.It significantly improves GPT-5.6 Sol, moves Claude Fable 5 from 0.94 to ceiling without a statistically detectable change, and has no measurable effect on the three weakest models.Source: §5 · p. 12
  • A model’s fluent retrospective self-diagnosis does not track its judgment quality in this study.The model that most extensively diagnosed its error was also least accurate and most confident under blind presentation, cautioning against using self-correction language as a reliability proxy.Source: §5 · p. 12

6 Limitations

The study is a deliberately narrow case study, not an estimate of how often demographic misattribution occurs generally. Its session structure, limited power, coding procedures, and data collection conditions also constrain interpretation.

6 Limitations

  • Three items on one topic in English demonstrate a mechanism, but do not support population-level error rates or extrapolation to other topics, styles, or languages.Item responses within a session are dependent, so chance tests use session scores; some other analyses pool item rows and should be read with clustering in mind.Source: §6 · p. 12
  • At this sample size, tests cannot distinguish a true null from a real-but-small context effect for the three lowest-accuracy models.Absence of significance for those models is not evidence that the phenomenon is absent.Source: §6 · p. 12
  • Post-reveal coding uses prespecified lexical matches rather than independently validated multi-rater qualitative coding.Inter-rater reliability was not computed, and the stereotyp- count does not distinguish self-blame from other uses of the word.Source: §6 · pp. 12–13
  • Post-reveal text is unavailable for condition E, and Gemini and Claude have smaller, unevenly distributed sample sizes that widen their confidence intervals.Reveal text is available for conditions A, B, and D only; API timeouts also removed some Gemini sessions.Source: §6 · p. 13
  • The study has no human-rater baseline, and consumer products may not reproduce results from direct API access with fixed system framing.The authors also disclose AI-assisted processing errors caught by manual verification; released outputs and scripts permit independent checks.Source: §6 · p. 13

7 Ethics Statement

The study uses three handwritten exam answers collected in 2025, with age and gender recorded at collection. The authors describe the consent and review status and release the data and code to support verification.

7 Ethics Statement

  • The three handwritten answers came from a 2025 community-college biology final, and each writer recorded age and gender at collection.Released materials are deidentified and retain no personally identifying information beyond age and gender; these recorded categories are not proposed as a general demographic taxonomy.Source: §7 · p. 13
  • The answers were signed for a class activity, but the signed originals were later lost and no research-specific consent documentation or formal ethics review exists.Retrospective written consent is being sought, and any text whose writer declines will be removed from the release and subsequent paper versions.Source: §7 · p. 13
  • The study analyzes three existing texts without further interaction with their writers, and releases data, code, and raw outputs for independent verification.Data/text are released under CC BY 4.0 and code under MIT.Source: §7 · p. 13

8 Conclusion

The conclusion reports that several tested systems misread linguistic register as a demographic cue, with confidence and post-hoc explanation failing to reliably indicate judgment quality. These findings come from a small, deliberately constructed case study and are relevant within that scope to systems that infer, personalize, or moderate based on writing.

8 Conclusion

  • Most of six tested models confidently misattributed the deliberately counter-stereotypical answer, and one model’s fluent error diagnosis coexisted with the lowest blind accuracy and highest confidence.The study asked models to match three real writers to their short texts and recorded age and gender labels.Source: §8 · p. 13
  • Within this case study, register acts as an unacknowledged demographic proxy for several systems, while stated confidence and post-hoc self-explanation do not track judgment quality.The findings are relevant to deployed systems that infer, personalize, or moderate based on a text’s demographic characteristics.Source: §8 · p. 14
  • The findings come from a small, deliberately constructed case study and should be interpreted at that scope.The study does not establish how often these patterns occur across writing generally.Source: §8 · p. 14
Loading 2610.31946v1…