PaperlayerView exact revision on arXiv

Source-linked AI summary

The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models

Chonghan Qin, Xiachong Feng, Ziyun Song, Xiaocheng Feng, Jing Xiong, Lingpeng Kong

arXiv:2605.06196v1cs.AIcs.CL

TL;DR

It is unclear whether LLMs internally encode the social granularity of prompted roles, from individual experience to institutional or macro-level reasoning. The paper constructs and analyzes a contrast-based Granularity Axis across ordered roles, finding a dominant, transferable representation direction whose steering shifts response granularity, with model-dependent controllability.

  • Problem

    The paper asks whether LLMs internally distinguish the granularity of prompted social roles rather than realizing them through a shared role-playing template.

  • Method

    The authors construct 75 roles across five granularity levels, derive a macro-minus-micro hidden-state contrast axis from role-conditioned responses, and evaluate its geometry and steering effects.

  • Results

    The axis is the dominant geometric direction in Qwen3-8B, aligns with PC1 at cosine 0.972, explains 52.6% of role-space variance, and shifts output granularity under steering across models.

  • Takeaways & Limitations

    Social role granularity is represented as an ordered, transferable latent direction with behavioral consequences, rather than merely a stylistic surface feature.

  • Takeaways & Limitations

    The single contrast axis captures the chosen endpoint-separating direction and is bounded here by two 8B instruction-tuned models, a manual 75-role taxonomy, and LLM-based judging.

Abstract

from arXiv · show

Large language models (LLMs) are routinely prompted to take on social roles ranging from individuals to institutions, yet it remains unclear whether their internal representations encode the granularity of such roles, from micro-level individual experience to macro-level organizational, institutional, or national reasoning. We show that they do. We define a contrast-based Granularity Axis as the difference between mean macro- and micro-role hidden states. In Qwen3-8B, this axis aligns with the principal axis (PC1) of the role representation space at cosine 0.972 and accounts for 52.6% of its variance, indicating that granularity is the dominant geometric axis organizing prompted social roles. We construct 75 social roles across five granularity levels and collect 91,200 role-conditioned responses over shared questions and prompt variants, then extract role-level hidden states and project them onto the axis. Role projections increase monotonically across all five levels, remain stable across layers, prompt variants, endpoint definitions, held-out splits, and score-filtered subsets, and transfer to Llama-3.1-8B-Instruct. The axis is also causally relevant: activation steering along it shifts response granularity in the predicted direction, with Llama moving from 2.00 to 3.17 on a five-point macro scale under positive steering on prompts that admit local responses. The two models differ in controllability, suggesting that steering depends on each model's default operating regime. Overall, our findings suggest that social role granularity is not merely a stylistic surface feature, but a structured, ordered, and causally manipulable latent direction in role-conditioned language model behavior.

1 Introduction

The paper asks whether LLMs internally distinguish the social granularity of prompted roles and tests this with a contrast-based Granularity Axis spanning micro- to macro-level roles. It finds that granularity organizes role representations geometrically and has behavioral consequences under activation steering.

  • Motivation: LLMs may represent social roles at different granularities rather than through a shared role-playing template.The distinction matters because micro roles emphasize personal experience and immediate concerns, whereas macro roles involve coordination, institutional constraints, and long-horizon strategy.
  • Motivation: Granularity confusion occurs when a model reasons at a social scale mismatched to the context.This can make nominally multi-stakeholder simulations appear multiperspective in text while collapsing toward a single perspective in representation.
  • Method: The study constructs 75 ordered social roles across five granularity levels and collects shared-question responses under multiple prompt variants.Role-conditioned responses are converted into averaged role-level hidden-state vectors for analysis.
  • Method: The Granularity Axis is defined as the difference between mean macro-role and micro-role representations and is tested against the dominant geometry of role space.The pipeline also evaluates whether steering along this direction changes behavioral granularity.
  • Findings: In Qwen3-8B, the axis aligns with PC1 at cosine 0.972 and explains 52.6% of role-space variance, while projections increase monotonically across five levels.The structure remains robust across layers, prompt and question splits, endpoint definitions, score-filtered subsets, and transfer to Llama-3.1-8B-Instruct.
  • Findings: Activation steering shifts output granularity in the predicted direction across both models, although controllability depends on the model.The findings frame granularity as an internally represented and behaviorally manipulable property rather than only a stylistic surface feature.

2 The Granularity Axis

The Granularity Axis is a contrast-based direction in role-conditioned activations, constructed from ordered social roles and evaluated for geometric alignment and behavioral steering. Role projections are expected to increase from micro to macro levels, while intervention along the axis should shift response granularity.

  • 2 The Granularity Axis: The pipeline constructs and validates a contrast-based Granularity Axis from role-conditioned hidden activations.It collects role-conditioned responses, forms role-level representations, compares the axis with principal geometry, and tests activation steering.
  • 2 The Granularity Axis: 75 social roles span five ordered levels from Individual (Micro) through Group/Community, Organization (Meso), Institution (Systemic), and Nation/Super-Actor (Macro).The taxonomy contains 15 roles per level and includes examples such as Worried Parent, Community Organizer, Hospital Administrator, Central Bank Governor, and World Bank President.
  • 2 The Granularity Axis: Five prompt variants and shared questions produce role-conditioned responses while testing robustness to instruction style.The variants alter identity assignment, role-play instruction, worldview, scale and time horizon, and practical constraints; they are treated as a robustness factor rather than distinct tasks.
  • 2 The Granularity Axis: Role representations are obtained by mean-pooling assistant-turn activations and averaging them over each role's response set.The resulting role-level vectors are extracted at every layer before geometric analysis.
  • 2 The Granularity Axis: The axis averages the shift from micro roles to macro roles, with intermediate-level projections expected to rise monotonically from Level 1 to Level 5.Micro roles use levels 1–2 and macro roles use levels 4–5; PCA tests alignment with the dominant role-space direction.
  • 2 The Granularity Axis: Positive steering strength pushes outputs toward institutional, systemic, and strategic reasoning, whereas negative strength pushes them toward individual, local, experience-centered reasoning.The intervention adds the axis to each generated-token activation at a selected layer and leaves prompt encoding unchanged.

3 Experiments

The experiments construct and evaluate a five-level role-granularity scale across two language models, testing its geometric structure, steering effects, robustness, and human alignment. The Granularity Axis organizes role representations monotonically and produces directionally consistent but model-dependent behavioral shifts.

  • Experimental Setup: 75 social roles were organized across five granularity levels, with 91,200 responses collected from shared questions and five prompt variants.The study used Qwen3-8B and Llama-3.1-8B-Instruct on the same pipeline.
  • Representation Results: At the target layer, the Granularity Axis aligned with PC1 at cosine 0.9720 and explained 52.57% of role-space variance in Qwen3-8B.In Llama-3.1-8B-Instruct, the corresponding values were 0.9596 and 42.46%.
  • Representation Results: Mean projections increased monotonically from L1 to L5 in both models, with saturation between the two macro-most levels.The shared rise-then-saturate pattern persisted despite different absolute projection scales.
  • Steering Results: Positive steering shifted outputs toward the macro end, but effects depended on model and prompt set.Llama moved from 2.0000 to 3.1667 on micro-targeted prompts, while Qwen showed a +0.5000 shift under α = +4; Llama also showed 0.425 degeneration under α = −4 on generic prompts.
  • Steering Results: The steering direction was specific to the Granularity Axis, while baseline and random directions did not reproduce the micro–macro movement.The authors therefore characterize steering as a partial, model-dependent causal probe rather than uniform control.
  • Robustness and Controls: The representation ordering remained stable across layers, endpoint definitions, prompt variants, held-out prompt/question splits, and score-filtered subsets.Role holdout was highly correlated but slightly fragile in Qwen, and high-stakes domains remained softer controls.
  • Human Evaluation: Human annotators selected the positively steered outputs as more macro above chance in all four evaluated cells.Likert re-rating produced human–judge Spearman correlations between 0.58 and 0.79.
  • Conclusion: Together, the experiments support internal differentiation of social roles by granularity rather than realization through a shared role-playing template.This conclusion combines the geometric, causal, robustness, and human-evaluation criteria tested in the section.

4 Analysis and Limitations

Held-out intermediate levels validate that the contrast axis captures graded social scale rather than only its endpoint contrast. Steering effects depend on baseline saturation, coefficient choice, and model-specific stability.

  • Analysis: Monotonic ordering across five levels in both models validates the endpoint-defined axis on held-out intermediate roles.This supports a graded latent property rather than memorization of the micro and macro endpoints.
  • Analysis: Qwen’s generic-prompt baselines exceed 4.9 on the 1–5 scale, leaving little headroom for positive steering.Micro-targeted prompts avoid this saturation and make behavioral effects more visible.
  • Analysis: Qwen is conservative but stable, whereas Llama is more responsive but less stable, reaching a 0.425 degeneration rate under α = −4.A larger behavioral shift therefore does not necessarily indicate cleaner control.
  • Analysis: Interpretable steering requires baseline position, coefficient sweeps, and per-condition degeneration rates.These diagnostics distinguish controlled movement from saturation or collapse.
  • Limitations: A single contrast axis captures the dominant endpoint-separating direction, while correlated dimensions may collapse into one axis.Residual PCA can test whether persistent orthogonal structure supports a multi-axis account; claims are bounded to two 8B models, a manual 75-role taxonomy, and LLM judging.

5 Related Work

Prior work studies LLM social simulation, role conditioning, and low-dimensional behavioral directions, while sociological micro–macro theory motivates testing social scale as an internal representation.

  • LLMs as Social Simulators and Role-Conditioned Agents: LLMs are studied as social simulators and role-conditioned agents, alongside documented validity gaps from real human samples.Related work also examines persona fidelity, consistency, personalization, and persona-induced behavioral shifts.
  • Activation Steering and Representation Engineering: Interpretability research finds that high-level concepts can occupy low-dimensional activation directions used for activation steering and representation engineering.The Assistant Axis is the closest precedent for linking persona behavior to an interpretable latent direction.
  • Micro–Macro Theory and Social Scale: Micro–macro sociology motivates the hypothesis that prompted roles may organize along social scale without imposing a fixed ontology on LLMs.The cited traditions connect individual action, collective behavior, agency, structure, and nested ecological systems.

6 Conclusion

The paper presents the Granularity Axis as a contrast-based direction organizing prompted roles from individual to macro-level reasoning. It reports robust representation-level structure but only partial, model-dependent behavioral control.

  • Conclusion: The Granularity Axis organizes prompted social roles from individual to macro-level reasoning along a contrast-based latent direction.The conclusion frames this organization as continuous rather than a discrete persona library.
  • Conclusion: The axis aligns with dominant role-space geometry, extrapolates to intermediate levels, transfers across model families, and shifts output granularity under intervention.Representation recovery is robust, whereas behavioral control remains partial and shaped by each model’s default operating regime.
  • Conclusion: The pipeline may support auditing multi-agent simulations, deployment-time social-scale control, and extensions to formality, time horizon, and risk aversion.These are presented as directions for generalizing the contrast-and-project approach.

A Qualitative Examples

Qualitative coefficient sweeps show movement between individual advice and institutional policy framing, while large steering magnitudes can produce repetition or nonsensical outputs.

  • Qualitative comparison: Llama responses include both household-oriented advice and broader institutional, systemic, and international frames on the housing prompt.The examples range from renting a room and talking to a landlord to regulatory reform and international cooperation.
  • Setup: Table 6 compares both models on the same housing-cost prompt across coefficients α ∈ {±8, ±12, ±16, ±20, ±24, ±28, ±32, ±36}.The prompt was chosen because it permits both individual coping advice and institutional policy responses.
  • Qwen3-8B: Qwen’s negative coefficients replace system-level policy language with increasingly individual or household-level advice.Examples include talking to landlords, finding roommates, saving money, moving neighborhoods, or seeking local help.
  • Qwen3-8B: Qwen’s positive coefficients emphasize policy categories, institutional coordination, data systems, international comparison, and climate resilience.The sweep shows a relatively smooth semantic movement, though the most negative setting begins repetitive low-information advice.
  • Degeneration: At α = −8, Llama moves toward concrete household advice with low-quality repetition; from α = −12 or about α = +16, outputs become increasingly repetitive or nonsensical.These settings are treated as evidence of directional sensitivity and instability rather than successful high-magnitude control.

B Steering Results

Steering along the Granularity Axis produces measurable directional shifts in response granularity, but stronger interventions can trigger degeneration and model-dependent instability.

  • Steering Results: The steering diagnostics test whether movement along the Granularity Axis changes behavioral granularity and when that effect becomes unstable.Scores use granularity_overall ratings on a 1–5 scale, with higher values indicating more macro-level reasoning.
  • Steering Results: The strongest negative Llama-3.1-8B-Instruct shift remains after degeneration filtering, but only 23/40 responses remain.The authors therefore interpret this result as behavioral sensitivity rather than stable control.
  • Steering Results: Llama-3.1-8B-Instruct is highly responsive but becomes unstable under stronger interventions, whereas Qwen3-8B remains more stable.Qwen3-8B generic-prompt scores are often saturated near the macro end, limiting observable movement.
  • Steering Results: Response-length and degeneration diagnostics distinguish granularity shifts from simple length or degeneration artifacts.The main caveat is Llama-3.1-8B-Instruct at α = −4 on generic prompts, where both response length and degeneration increase substantially.

C Robustness and Control Analyses

Layer-wise analyses show that the Granularity Axis is a stable middle- and later-layer phenomenon in both models rather than a single-layer artifact.

  • Layer-wise Stability: Granularity ordering in Qwen3-8B first appears at Layer 3, briefly breaks at Layers 5–7, and then holds consistently from Layers 8–35.At Layer 18, cosine alignment with PC1 is 0.9720, with Spearman correlation 0.9472 and Pearson correlation 0.9414.
  • Layer-wise Stability: Across layers, contrast–PC1 cosine remains approximately 0.972–0.986 in Qwen3-8B and 0.956–0.987 in Llama-3.1-8B-Instruct.Projection-level Spearman correlations remain approximately 0.928–0.957 and 0.912–0.954, respectively.
  • Layer-wise Stability: Llama-3.1-8B-Instruct shows monotonicity from Layers 6–31 after brief breaks at Layers 4–5.At Layer 18, cosine alignment is 0.9596, with Spearman correlation 0.9459 and Pearson correlation 0.9373.

C.2 Endpoint Ablations

Alternative endpoint definitions preserve the Granularity Axis’s alignment with PC1, projection correlations, and monotonic ordering in both models.

  • Endpoint Ablations: Qwen3-8B endpoint variants retain cosine alignment of 0.9452–0.9989 and Spearman correlations of 0.9359–0.9598, with monotonic ordering preserved.The result is therefore not specific to one endpoint choice.
  • Endpoint Ablations: Llama-3.1-8B-Instruct endpoint variants retain cosine values of 0.9279–0.9988 and Spearman correlations of 0.9276–0.9646, with all variants monotonic.Endpoint changes do not remove the recovered ordering.

C.3 Held-out Robustness

Held-out, prompt-variant, filtering, confound-control, and subgroup analyses largely preserve the granularity signal, while role-holdout and some semantic domains remain harder cases.

  • Held-out Robustness: Held-out prompt and question splits remain monotonic in Qwen3-8B, while role holdout retains high correlation but does not perfectly preserve monotonic ordering.The role-holdout correlations are Spearman = 0.9335, Pearson = 0.9404, and cosine = 0.9821.
  • Prompt Variants: All five prompt variants remain monotonic in both models, with projection-level Spearman correlations ranging from 0.9381 to 0.9568 in Qwen3-8B and 0.9342 to 0.9585 in Llama-3.1-8B-Instruct.Macro–micro gaps range from 15.0941 to 18.9780 in Qwen3-8B and 2.6218 to 3.6250 in Llama-3.1-8B-Instruct.
  • Score Filtering: Score filtering preserves the projection-level signal, with Spearman correlations reaching 0.9564 in Qwen3-8B and 0.9503 in Llama-3.1-8B-Instruct under score ≥3.The signal becomes slightly cleaner after lower-quality role-play responses are removed.
  • Generic versus Specific Role Controls: Generic and specific role buckets remain monotonic, but their controls are not fully level-matched and provide only supportive rather than definitive evidence against title-based explanations.The control analysis therefore does not isolate granularity from correlated role properties completely.
  • Domain and Family Controls: Most semantic domains and family ladders remain monotonic, while health and economy contain the principal failures.Health is least stable across domains; economy fails monotonicity in both models, and health also fails within Llama’s family ladders.

E Reproducibility, Assets, and Societal Impact

The paper documents its reproducibility setup, including shared and steering-specific prompt sets, prompt templates, evaluation procedures, model assets, compute, and broader-impact considerations.

  • Reproducibility: 91,200 role-conditioned responses and hidden-state extraction required approximately 20–30 GPU hours on NVIDIA A100 GPUs with 80GB memory.LLM-as-judge evaluation used API-based models.
  • Prompt sets: The main representation experiments reuse 240 shared extraction questions across all roles and prompt variants to limit question-distribution confounding.The steering evaluation instead uses generic and micro-targeted prompt sets.
  • Prompt sets: Steering evaluation uses 40 broad social-policy and coordination prompts plus 12 micro-targeted prompts designed to admit local, personal, and short-horizon responses.The micro-targeted set includes household, caregiving, housing, health, education, work, migration, food, transport, finance, safety, and community scenarios.
  • Assets and societal impact: The study uses Qwen3-8B under Apache 2.0 and Llama-3.1-8B-Instruct under the Llama 3.1 Community License Agreement, while noting benefits and misuse risks for social simulation.The authors frame the work as representation-first and emphasize limitations to mitigate overclaiming.
  • Prompt templates: The pipeline instantiates five role-conditioned prompt variants from each role's name, description, and granularity level, then pairs them with a user question.The variants use different phrasings while preserving the intended role perspective.
  • Evaluation: Role-play quality is scored from 0 to 3 by a judge given the target role, granularity level, user question, and generated answer.A separate steering judge rates six granularity-related dimensions, an overall granularity score, and degeneration.
Loading 2605.06196v1…