PaperlayerView exact revision on arXiv

Source-linked AI summary

Examining Social Attribution in LLM Reasoning: A Theory-Guided Probing Methodology

Zhaoxin Yu, Qingchao Kong, Dajun Zeng, Wenji Mao

arXiv:2610.12022v1cs.CLcs.AIcs.LG

TL;DR

LLM social attribution remains underexplored, despite its role in sociotechnical judgments of responsibility and blame. The paper evaluates LLM judgments and probes their internal attribution-dimension representations using a theory-guided benchmark and methodology. It finds measurable but incomplete human agreement, with model-size association, and reports decodable dimensions whose judgment influences align with Attribution Theory.

  • Problem

    Social attribution is well studied in psychology but remains underexplored in AI, particularly whether LLM responsibility and blame judgments align with humans and use theory-derived attribution dimensions.

    Source: §1 · p. 2

  • Method

    The authors build a two-subset benchmark of human responsibility/blame judgments and probe LLM hidden states for five attribution dimensions and their effects on judgments.

    Source: §1 · p. 3, §4.1 · p. 5

  • Results

    Across 32 LLMs, agreement with human responsibility and blame judgments is measurable but incomplete and positively associated with model size; attribution dimensions are decodable and their judgment influences align with Attribution Theory.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    The findings indicate that LLM responsibility and blame judgments exhibit theory-consistent dimension representations and influences, alongside incomplete agreement with human judgments.

    Source: §1 · p. 3, §6 · p. 10

    Differences in model families and training mean the observed association between agreement and model size cannot be attributed to parameter count alone.

    Source: §5.2 · p. 8

Abstract

from arXiv · show

Large language models (LLMs) are increasingly deployed in sociotechnical systems where social attribution, the reasoning process attributing external events to the causes and reasons of agents' social behaviors, plays a critical role. These processes involve judgments of social cause, responsibility, and blame/credit to agents. Although attributional models are well-studied in social psychology and cognition through Attribution Theory, social attribution remains underexplored in AI, particularly LLM social reasoning. This paper provides the first systematic exploration of LLM social attribution. Our work focuses on responsibility and blame attributions, examining current LLMs' judgments and their underlying internal mechanisms. Guided by attribution theory, we construct a social attribution benchmark consisting of a Vignette subset based on classic scenarios from attribution theory research and a Reality subset based on real-world social narratives, yielding 7,639 responsibility/blame judgment questions. On this basis, we evaluate 32 representative LLMs and 5 basic non-LLM baselines. To further explore the internal mechanisms underlying the LLM judgment process, we develop a probing-based methodology to investigate the latent-space representations of 5 key attribution dimensions and the consistency of their influences on LLM judgments compared to those in human social attribution. Our research findings reveal that current LLMs exhibit measurable but incomplete agreement with human responsibility and blame judgments, and meanwhile, this agreement is positively correlated with model size. Some attribution dimensions are systematically decodable from specific positions in LLM hidden states, and their influences on the final judgment are consistent with those indicated by human Attribution Theory. The dataset and associated code are available at https://github.com/Yuzhaoxin946/SAB-Bench.

1 INTRODUCTION

Social attribution underlies consequential judgments in sociotechnical settings, yet it remains underexplored in LLM research. This study evaluates LLM responsibility and blame judgments against human judgments and probes whether their internal representations and influences align with Attribution Theory.

1 INTRODUCTION

  • The paper investigates whether LLM responsibility and blame judgments match human judgments and whether theory-derived attribution dimensions are represented internally and influence judgments in human-theory-consistent ways.The questions focus on judgment accuracy and the internal mechanisms underlying LLM social attribution.Source: §1 · p. 2
  • The authors construct a bilingual benchmark with controlled classic vignettes and realistic social narratives, then evaluate 32 LLMs against human judgments and five non-LLM baselines.The benchmark contains 7,639 responsibility/blame questions and measures both exact agreement and relative rankings.Source: §1 · pp. 2–3
  • The study uses attribution-theory-guided probes and directional hidden-state interventions to examine dimension representations and their influence on final judgments.Linear probes quantify decodability; interventions test whether dimension effects match patterns predicted by human attribution theory.Source: §1 · pp. 2–3
  • Specific attribution dimensions are represented in LLM latent spaces, and their influences on final judgments align with human social attribution theory.The introduction states these findings without specifying their strength or the models in which they occur.Source: §1 · p. 3

2 RELATED WORK

Prior computational work has addressed normative or legal judgments and other LLM-probing targets, but responsibility and blame attribution dimensions have not been specifically probed. This paper takes a descriptive approach, comparing LLM judgments with human patterns.

2 RELATED WORK

  • Unlike normative or prescriptive studies, this work descriptively evaluates systematic patterns in LLM responsibility and blame judgments against human judgments without assuming human judgments follow universal rationality principles.The benchmark also annotates five theory-derived attribution dimensions for studying intermediate variables.Source: §2 · p. 3
  • Prior LLM social-context benchmarks mainly assess social-norm judgments or legal judgments, rather than descriptive alignment of responsibility and blame judgments with humans.Social-norm tasks assess norm-consistent explanations or recommendations; legal tasks assess statute- and precedent-grounded reasoning.Source: §2 · p. 3
  • Earlier probing work studies diverse knowledge, instruction-following, truth, emotion, and cognitive constructs, but prior work had not specifically probed dimensions underlying responsibility and blame judgments.Theory-guided probing and interventions in this paper are motivated by related studies of cognitive appraisal and other latent constructs.Source: §2 · p. 3

3 THEORETICAL FOUNDATION

Attribution Theory treats responsibility and blame as structured judgments shaped by agents’ causal involvement, mental states, and situational constraints. The paper focuses on five dimensions for examining those judgments.

3 THEORETICAL FOUNDATION

  • Responsibility concerns accountability for an event, while blame additionally concerns whether an agent deserves moral criticism; responsibility does not necessarily entail blameworthiness.Responsibility considers causal involvement, controllability, and mitigating factors; blame also considers justifications and excuses.Source: §3 · p. 3
  • Attribution dimensions capture causal involvement, mental states, and situational constraints in structured responsibility and blame judgments.The paper names intention, voluntariness, foreknowledge, controllability, and obligation as its five dimensions.Source: §3 · pp. 3–4
  • Intention concerns deliberate action, voluntariness concerns freedom from pressure, foreknowledge concerns awareness or foreseeability, controllability concerns capacity to prevent outcomes, and obligation concerns duty to prevent them.The text notes that foreknowledge typically increases responsibility and blame, while obligation is key for unintentional outcomes.Source: §3 · p. 4

4 METHODOLOGY

The methodology constructs a human-annotated benchmark for responsibility and blame judgments, then probes theory-derived dimensions in LLM hidden states. It tests both their decodability and whether interventions along their representations shift judgments in theory-predicted directions.

4.1 DATA CONSTRUCTION

  • The Vignette subset controls attribution dimensions across classic scenarios, while the Reality subset reconstructs real-user narratives; expert agreement filtering yields 7,639 questions.The benchmark is bilingual, and each question concerns one agent, scenario, language, and responsibility or blame task, with labels no, low, medium, or high.Source: §4.1 · pp. 4–5
  • LLM-human agreement is measured by exact-label accuracy and by Spearman correlation between predicted and human agent rankings within each Vignette scenario.Invalid predictions are excluded from accuracy, and scenarios containing them are excluded from the correlation calculation.Source: §4.1 · p. 5

4.2.1 DECODABILITY OF ATTRIBUTION DIMENSIONS

  • Layerwise logistic probes test whether five theory-derived dimensions can be decoded from hidden states at the judgment-label prediction step.Vignette evaluation holds out scenario types, Reality uses random cross-validation, and balanced accuracy of 50% represents chance.Source: §4.2 · p. 5, §4.2.1 · p. 5

4.2.2 INFLUENCE OF ATTRIBUTION DIMENSIONS ON JUDGMENT

  • Directional interventions vary hidden states along dimension representations, then test whether judgment shifts align with theory-derived expectations.Controls use random or shuffled-label directions; ties and shifts opposite to any tested sign count as failures, and tests pass only when all three comparisons meet the stated criterion.Source: §4.2.2 · p. 6

5 EXPERIMENTS

The experiments find measurable but incomplete agreement between LLMs and human responsibility and blame judgments, with agreement associated with model size. Probes recover attribution-dimension information from hidden states, and interventions show theory-aligned influence most consistently for intention and obligation.

5.1 EXPERIMENTAL SETUP

  • The evaluation covers 32 LLMs under four prompt strategies and compares them with five non-LLM baselines; mechanistic analyses use five LLM backbones with CoT prompts.The appendices provide additional model and non-LLM results, valid-output rates, breakdowns, sanity checks, earlier-token analyses, and intervention-strength sensitivity results.Source: §5.1 · p. 7, §5.5 · p. 10

5.2 PERFORMANCE EVALUATION ON LLM SOCIAL ATTRIBUTION

  • Both accuracy and ranking agreement increase with log-scale parameter count across the 23 open-source models, but family and training differences prevent attributing the trend to parameter count alone.The fitted slopes are positive for both metrics (p < 0.001), with R2 = 0.766 for accuracy and R2 = 0.806 for ρ.Source: §5.2 · pp. 7–8, Figure 4 · p. 8
  • 58.00% accuracy and ρ = 0.627 make Grok-4.3 the highest-scoring model on both metrics averaged across responsibility and blame.Across most models, accuracy is higher for responsibility, while Spearman’s ρ is higher for blame; mBERT accuracy is comparable to selected 4–8B LLMs.Source: §5.2 · p. 7, Figure 3 · p. 8

5.3 DECODABILITY OF ATTRIBUTION DIMENSIONS IN LLM HIDDEN STATES

  • Attribution dimensions are consistently decodable from hidden states, with most middle and later layers above chance; intention and obligation approach 90% in Llama-3.1-8B and Qwen models.Foreknowledge is weaker in DeepSeek-R1-1.5B and Llama-3.2-1B, while voluntariness and controllability vary more across models; decodability often peaks before the final layer. Original probes outperform shuffled-label controls across all five models.Source: §5.3 · pp. 8–9, Figure 5 · p. 9

5.4 INFLUENCE OF ATTRIBUTION DIMENSIONS ON JUDGMENT

  • Interventions support theory-aligned effects most consistently for intention and obligation, which pass directional tests in all five models and both tasks at 60% and 80% relative depth.Across all settings, 33 of 50 pass at 60% depth, compared with six at 20% and nine at the final layer; other dimensions vary by model, task, and layer.Source: §5.4 · p. 9, Figure 6 · p. 10

5.5 SUPPLEMENTARY EXPERIMENTAL RESULTS

  • Supplementary experiments report model and non-LLM results, valid-output rates, breakdowns, sanity checks, earlier-token analyses, and intervention-strength sensitivity.These results are reported in Appendices E.1–E.4, F.1–F.2, and G.1–G.2.Source: §5.5 · p. 10

6 CONCLUSION

The section summarizes an investigation of LLM responsibility and blame judgments against human judgments and attribution theory, using controlled vignettes and reconstructed real-world narratives, and probing theory-derived dimensions in model hidden states. It also describes responsibility and blame judgments separately and examines expert agreement and how agreement-based filtering changes the evaluation sample.

6 CONCLUSION

  • Across 32 LLMs, closed-source models generally outperform open-source models, human-judgment agreement rises with model size, and intention and obligation are especially decodable across five probed models.Interventions on these dimensions produce the most consistent theory-predicted shifts, especially in middle and later layers; other dimensions' effects vary across models, tasks, and layers.Source: §6 · p. 10

A.1 CLASSICAL ATTRIBUTION THEORIES

  • Classical accounts distinguish causal involvement, responsibility, and blame, with control, mental states, coercion, and mitigation shaping judgments; PMoB routes intentional cases through reasons and unintentional cases through preventability.These distinctions motivate separate responsibility and blame tasks and attention to intermediate attribution dimensions.Source: §A.1 · p. 14

A.2 OPERATIONAL DEFINITIONS OF ATTRIBUTION DIMENSIONS

  • The benchmark operationalizes intention, voluntariness, foreknowledge, controllability, and obligation as distinct factors associated with an agent's actions, constraints, capacities, and duties.Intention concerns deliberate action rather than necessarily intending the outcome; practical control is separated from foreknowledge, while obligation means a duty to prevent harm.Source: §A.2 · p. 15

A.3 THEORETICAL POSITIONING AND DIRECTIONAL EXPECTATIONS

  • Attribution theory guides variable selection and directional hypotheses, but the study does not assume LLMs instantiate human cognitive processes.The directional tests are aggregate conditional tendencies, not universal monotonic rules for every scenario, because intentions, excuses, and relationships can alter a factor's relevance.Source: §A.3 · p. 16

B DATASET CONSTRUCTION AND ANNOTATION

  • Vignette varies attribution factors in coherent controlled scenarios, while Reality adds reconstructed everyday conflicts with less explicit and more ambiguous attribution information.Together, the subsets support evaluation against human judgments and probing of dimensions underlying those judgments.Source: B DATASET CONSTRUCTION AND ANNOTATION · p. 17, §B.2 · p. 17

B.1 VIGNETTE CONSTRUCTION

  • The Vignette subset uses 11 role-constrained templates across eight scenario types and three relation categories, with scenario types defining held-out probing groups.Templates constrain meaningful dimension combinations according to agent roles rather than varying all five dimensions independently for every agent.Source: §B.1 · p. 17

B.2 REALITY PREPROCESSING / B.3 PROMPTS FOR LLM-ASSISTED DATA CONSTRUCTION

  • Reality narratives are reconstructed from AITA posts to retain core negative-outcome events while removing irrelevant or harmful content; LLMs clean and inspect stories, but humans assign final judgments.Each selected narrative has at least two relevant agents and a target agent queried separately for responsibility and blame.Source: §B.2 · p. 18, §B.3 · p. 19

B.4 HUMAN ANNOTATION PROTOCOL AND GUIDELINES

  • Before agreement filtering, annotation comprised 5,344 Vignette and 3,456 Reality questions, with two experts independently assigning four-level judgments.Reality annotators also labeled the five dimensions; matching responsibility and blame judgments were retained, and judgment annotators did not see dimension labels or model outputs.Source: §B.4 · pp. 19–20
  • The annotation protocol separates responsibility from blame, noting that limited intention, foreknowledge, voluntariness, control, or obligation can reduce blame despite responsibility.Blame labels range from high to no blame, with medium and low levels reflecting mitigating factors or limits on action, knowledge, control, or obligation.Source: §B.4 · p. 21

B.5 INTER-ANNOTATOR AGREEMENT

  • 86.8% exact agreement across 8,800 questions before filtering, with mean unweighted κ of 0.826 across four task–language combinations; all reported coefficients were significant (p < 0.001).Agreement was lower in several vicarious-blame and Reality conditions than in general vignettes.Source: §B.5 · p. 21, Table 2 · p. 22

B.6 AGREEMENT-BASED FILTERING AND DISTRIBUTION CHANGES

  • Filtering removed 1,161 of 8,800 questions; the Vignette share rose from 60.7% to 62.5%, while relation shares shifted at most 1.1 and dimension-value shares at most 2.4 percentage points, though judgment-label frequencies changed.The retained sample represents expert-agreed cases and excludes disagreements; the distribution checks do not recover those excluded judgments.Source: §B.6 · p. 22

B.7 FINAL DATASET STATISTICS

The final benchmark contains 7,639 responsibility and blame questions across Vignette and Reality subsets, in English and Chinese. The evaluation protocol applies four prompting strategies to 32 LLMs and specifies how outputs are converted into judgments.

B.7 FINAL DATASET STATISTICS

  • 7,639 questions comprise 4,776 Vignette and 2,863 Reality items, split into 3,837 English and 3,802 Chinese questions.Tables 7–9 report judgment, dimension, and retained Vignette-question distributions; Figures 8–9 visualize frequent content words in the English and Chinese Reality narratives.Source: §B.7 · p. 23, Table 7 · p. 23, Table 8 · p. 24

C BENCHMARK EVALUATION PROTOCOL

  • The benchmark evaluates agreement with human judgments at the individual-agent label level and by relative agent ordering within scenarios.The protocol specifies models, elicitation strategies, answer extraction, metrics, and comparison methods.Source: C BENCHMARK EVALUATION PROTOCOL · p. 26

C.1 EVALUATED MODELS AND INFERENCE SETTINGS

  • 32 LLMs are evaluated, including nine accessed through provider APIs and 23 locally deployed open-source models spanning 0.5–120B parameters.The experiments group models by access mode; the open-source group consists of instruction-tuned or chat models.Source: §C.1 · p. 26
  • Each model is evaluated in both languages and judgment tasks using Direct, CoT, Self-reflection, and Theory-guided prompting.Theory-guided prompts name the five attribution dimensions but do not supply their human-annotated values.Source: §C.1 · p. 26

C.2 PROMPT TEMPLATES

  • Prompt templates request one of four labels—high, medium, low, or no—with different instructions for direct answers, reasoning, reflection, or dimension analysis.The scenario, task question, and target agent are inserted into each template.Source: §C.2 · pp. 26–27

C.3 ANSWER EXTRACTION AND INVALID OUTPUTS

  • A rule-based parser selects the last valid boxed label, falls back to the last standalone label, and marks outputs without an extractable label invalid.Accuracy excludes invalid individual predictions, while ranking evaluation excludes any scenario containing an invalid prediction.Source: §C.3 · p. 27

C.4 EVALUATION METRICS AND STATISTICAL REPORTING

The paper defines complementary measures for exact judgment agreement and within-scenario ranking, and details baseline comparisons and hidden-state probing. Its reporting specifies model-size regressions, probe controls, and how probe scores are aggregated.

C.4 EVALUATION METRICS AND STATISTICAL REPORTING

  • Exact agreement measures whether a valid prediction matches the human-assigned level, while Spearman correlation measures whether models rank agents similarly within Vignette scenarios.Ranking uses ordinal scores for no, low, medium, and high; it is unavailable for Reality scenarios, which have one target agent.Source: §C.4 · pp. 27–28
  • Model-size regressions relate task-averaged agreement to log10 parameter count across 23 open-source models, with 95% confidence intervals based on scenario-group bootstrap resampling.The regressions summarize an association and do not isolate parameter count from model-family, training, architecture, or post-training differences; aggregate figures can conceal per-strategy variation.Source: §C.4 · p. 28

C.5 NON-LLM BASELINES

  • Five non-LLM baselines span label-frequency, surface-cue, explicit attribution-rule, and supervised text-classification approaches with different information sources.The Attribution-Theory Rule receives annotated dimensions, unlike text-based methods; mBERT is trained separately for responsibility and blame.Source: §C.5 · pp. 28–29

D PROBING AND INTERVENTION DETAILS

  • Representation experiments test dimension decodability and theory-aligned influence separately, using five frozen backbones prompted with CoT.Probes examine every layer; the label-token prediction state follows scenario context and generated reasoning, while the last input token tests an earlier position before generated reasoning.Source: D PROBING AND INTERVENTION DETAILS · p. 30, §D.1 · p. 30, Table 11 · p. 30

D.2 DATA SPLITS AND RESULT AGGREGATION

  • Probe evaluation uses separate language–subset runs, holding out entire Vignette scenario types and using random cross-validation for Reality.Dimension-specific samples exclude missing or inapplicable labels; combined decodability scores are computed from pooled held-out predictions, not averaged run scores, and balanced accuracy weights both binary values equally.Source: §D.2 · pp. 30–31

D.3 PROBE TRAINING AND DECODABILITY CONTROLS

The section specifies probe training, controls, and matched interventions, then reports full benchmark results across models and prompting strategies. It distinguishes judgment-category accuracy from agent-ranking agreement and describes the statistical criteria used to assess intervention effects.

D.3 PROBE TRAINING AND DECODABILITY CONTROLS

  • Probe decodability is measured with held-out balanced accuracy; shuffled labels test label–representation association, random projection tests 64-feature compression, and lexical features test surface-text prediction.The design also varies vignette phrasing and character identities and holds out scenario types to reduce shortcut opportunities.Source: §D.3 · pp. 31–32

D.4 INTERVENTION IMPLEMENTATION AND HYPERPARAMETERS

  • Intervention directions are derived from probe weights and residualized against other dimension directions, then tested at the label-prediction step against matched random-direction and shuffled-label controls.The residualization reduces shared linear components but does not establish complete independence among attribution factors; the earlier-token experiment edits the last input token instead.Source: §D.4 · p. 32

D.5 STATISTICAL TESTING AND REPORTING DETAILS

  • A setting passes the directional tests only when N > 0 and all three comparisons have p < 0.01; a gray, non-passing result does not establish that a dimension or influence is absent.The binomial tests assume independent question-level indicators, and reported p-values do not account for dependence among questions from the same scenario.Source: §D.5 · p. 33

E.1 FULL LLM RESULTS ACROSS PROMPTING STRATEGIES

  • Grok-4.3 leads on blame accuracy (60.03%) and blame rank agreement (ρ = 0.702), while responsibility accuracy exceeds blame accuracy across models (43.10% versus 39.82%).Responsibility rank agreement shows the reverse pattern (0.348 versus 0.400), separating exact-category agreement from agent ranking.Source: Table 13 · p. 34, §E.1 · p. 34, Table 14 · p. 35
  • Across models and tasks, CoT has the highest mean accuracy (43.13%) and correlation (0.397), while Self-reflection does not improve average agreement over Direct prompting.Theory-guided prompting improves aggregate scores over Direct but does not guarantee the best judgment; individual model–strategy exceptions remain.Source: §E.1 · p. 35
  • The closed-source/API group exceeds the open-source group in responsibility and blame accuracy, while family comparisons show model size alone is insufficient to explain performance.Within Qwen, accuracy rises across Qwen2-7B, Qwen2.5-7B, and Qwen3-8B from 37.77% to 42.26% to 48.43% for responsibility and 33.31% to 37.04% to 41.56% for blame.Source: §E.1 · p. 35

E.2 FULL NON-LLM BASELINE RESULTS

The five non-LLM baselines probe agreement from label frequencies, surface cues, attribution dimensions, and supervised text classification. The section reports baseline performance and the validity rate of LLM outputs.

E.2 FULL NON-LLM BASELINE RESULTS

  • The five baselines test agreement from label frequencies, surface textual cues, explicit attribution dimensions, or supervised text classification.The Attribution-Theory Rule has the highest responsibility rank agreement (0.303), while the strongest LLMs exceed the baselines overall.Source: §E.2 · p. 36
  • The Attribution-Theory Rule leads the baselines on blame accuracy (46.91%) and correlation (0.368), while mBERT has the highest responsibility accuracy (43.47%).The rule receives annotated dimension values, whereas mBERT and LLMs must infer relevant information from text, so the comparison does not isolate architecture or establish a necessary reasoning mechanism.Source: §E.2 · p. 36, Table 15 · p. 36

E.3 VALID-OUTPUT RATES

  • 98.49% of evaluated LLM outputs are valid, while 1.51% are invalid and excluded from metric calculations.Validity means the evaluator can extract one canonical answer label from {no, low, medium, high}.Source: §E.3 · p. 37, Table 16 · p. 37

E.4 PERFORMANCE BREAKDOWNS

Performance varies across agent relations, dataset subsets, and attribution-dimension values, so aggregate scores do not capture uniform model behavior. The decodability checks compare hidden-state probes with shuffled-label, compressed-representation, and lexical controls.

E.4 PERFORMANCE BREAKDOWNS

  • For GPT-5-mini, responsibility accuracy is highest on Reality (61.89%), but blame accuracy is highest on vicarious-blame scenarios (66.60%).Its scores differ from command-chain scenarios, and the breakdowns support retaining complementary groups rather than treating aggregate agreement as uniform across social relations.Source: §E.4 · p. 38, Table 17 · p. 38
  • Attribution-dimension group differences are not universal: GPT-5-mini responsibility accuracy is higher when intention is present (62.82% versus 42.01%), whereas Grok-4.3 blame accuracy is higher when intention is absent (61.89% versus 59.23%).These observational differences may reflect scenario composition and human-label distributions; they do not measure a dimension’s causal effect.Source: §E.4 · p. 40, Table 19 · p. 40

F.1 DECODABILITY SANITY CHECKS

  • Original probes consistently outperform shuffled-label controls and generally outperform 64-dimensional random projections; for three named backbones, they also exceed lexical controls across most layers.The lexical comparison is not uniformly favorable across every model and layer, and the controls do not imply all lexical or contextual correlations were removed.Source: Figure 10 · p. 41, §F.1 · p. 42, Figure 11 · p. 42

F.2 DECODABILITY AT THE LAST INPUT TOKEN

The section tests whether attribution dimensions are decodable before response generation, not only at the label-prediction step. Decodability is strong at both positions, with differences across models and dimensions.

F.2 DECODABILITY AT THE LAST INPUT TOKEN

  • Attribution dimensions are probed at the last input token, before response generation, and compared with the label-token prediction step.The same type of probes is trained at the earlier position.Source: §F.2 · p. 46, Figure 15 · p. 47
  • Balanced accuracy averages 73.8% at the last input token and 71.1% at the label-token prediction step; four of five models decode better at the input position.Qwen3-8B is the main exception, with later-position gains for voluntariness, foreknowledge, and controllability; Llama-3.2-1B also improves later for voluntariness.Source: §F.2 · p. 46, Figure 15 · p. 47
  • At both positions, decodability generally rises from early layers and peaks in middle or later layers, with intention and obligation among the most decodable dimensions.The comparison does not show that reasoning uniformly increases decodability or that both positions implement the same computation; separate interventions test whether earlier representations influence judgments.Source: §F.2 · p. 46, G ADDITIONAL INTERVENTION ANALYSES · p. 48

G.1 INTERVENTION AT THE LAST INPUT TOKEN

Earlier-token interventions show theory-aligned influence on judgments, though effects vary across models and layers. Average signed changes generally grow with intervention strength, but cross-model sensitivity comparisons have limits.

G.1 INTERVENTION AT THE LAST INPUT TOKEN

  • The earlier-token comparison moves probe training and intervention from the label-token prediction step to the input prompt’s last token, testing effects that must propagate through generation.The same feasible signs, directional hypotheses, and random-direction and shuffled-label controls are used; success requires strict alignment for every tested sign.Source: §G.1 · p. 48
  • At α = 8, theory-aligned effects appear in several models and intermediate layers, especially Llama, but remain uneven across models, dimensions, and layers.Earlier-position edits can affect eventual judgments, but the model- and layer-dependent pattern does not establish a broadly stable mechanism.Source: Figure 16 · p. 48, §G.1 · p. 48

G.2 SENSITIVITY TO INTERVENTION STRENGTH

  • The sensitivity measure averages feasible signed judgment changes across questions and layer–dimension settings, complementing binary tests that do not measure change magnitude.A positive average is not a directional-success rate above 1/2 and does not establish that the directional tests pass.Source: §G.2 · p. 49
  • Average directional change generally increases from α = 1 to 16 for responsibility and blame, with the largest increases in Llama-3.2-1B and Qwen3-4B.The 8B models and DeepSeek-R1-1.5B show smaller positive increases; these curves do not show that directional-success tests pass at every strength.Source: Figure 17 · p. 49, §G.2 · p. 49
  • Cross-model sensitivity differences are specific to this intervention setup because the same α need not represent the same relative hidden-state change across backbones.The observed pattern alone does not establish that smaller models have simpler attribution mechanisms or that larger models are generally more robust.Source: §G.2 · p. 49

H.1 DATASET COVERAGE AND HUMAN-JUDGMENT REFERENCE

The benchmark combines controlled vignettes with reconstructed real-world narratives, while its reference judgments and probing results have defined scope. The paper cautions against generalizing those results to all settings or using benchmark agreement for consequential decisions about individuals.

H.1 DATASET COVERAGE AND HUMAN-JUDGMENT REFERENCE

  • Vignettes and Reality narratives provide complementary evidence, but the benchmark covers negative outcomes and responsibility/blame, not positive credit or all social explanations.Reality narratives come from a particular online community, so translated versions do not broaden the underlying cultural or platform coverage.Source: §H.1 · p. 50
  • Human reference labels are judgments from two psychology experts, not populationwide consensus or uniquely correct moral verdicts; matching-label filtering may retain clearer cases.Accuracy therefore means agreement with this defined reference within the dataset’s coverage.Source: §H.1 · p. 50

H.2 INTERPRETATION OF PROBING AND INTERVENTION EVIDENCE

  • Decodability shows that a classifier can access attribution information, not that a model uses it as a human reasoner would.Scenario-held-out evaluation and matched controls test alternative explanations for probe and intervention results, but leave some representational explanations untested.Source: §H.2 · p. 50
  • Residualizing probe weights removes only shared linear components captured by those probes; it does not establish construct independence or isolate every feature affected by an edit.The approach also leaves dimension interactions, nonlinear representations, and the full judgment process incompletely characterized.Source: §H.2 · p. 50
  • Internal analyses cover five backbones and specific prompts, layers, token positions, and strengths, so their patterns should not be generalized to all models or inference settings.Differences among backbones confound scale, family, and training; failing a directional test does not prove a model ignores that dimension.Source: §H.2 · p. 50

H.3 DATA ETHICS AND RESPONSIBLE USE

  • Reconstructed posts are anonymized and use fictional names, but benchmark agreement is research evidence, not a basis for consequential decisions about real individuals.The benchmark cannot fully represent the social, moral, and sometimes legal considerations involved in responsibility and blame.Source: §H.3 · p. 51
Loading 2610.12022v1…