PaperlayerView exact revision on arXiv

Source-linked AI summary

Interpreting and Steering LLM Agents for Social Simulations

Jiayue Gaveal Fan, Arul Murugan, Shreyas Krishnan, Abhishek Nagaraj

arXiv:2609.16436v1cs.LGcs.AIcs.CL

TL;DR

LLM agents are promising tools for social simulation, but their black-box nature limits interpretation and steerability. The paper compares prompting, SAE feature steering, and probe direction steering across preference and capability tasks, finding that SAE- and probe-based methods often outperform basic prompts, though the advantage depends on prompting strategy.

  • Problem

    LLM agents’ black-box behavior makes it difficult to interpret the mechanisms behind their actions or steer them to test specific mechanisms in social simulations.

    Source: §1 · p. 2, §2.1 · pp. 5–6

  • Method

    The paper compares prompt-based interventions, SAE feature steering, and probe-based direction steering across preference and capability tasks.

    Source: §1 · p. 4, §3.3 · p. 13, Synthetic Data Generation · p. 14

  • Results

    SAE- and probe-based steering often outperform basic prompting across the tasks, but their advantage narrows against stronger prompting strategies.

    Source: §1 · p. 3, §5 · p. 27

  • Takeaways & Limitations

    SAEs can support inductive discovery of behavioral mechanisms, while probes provide finer control for experiments on predefined measurable traits.

    Source: §5 · p. 28

    Creativity probe control is partial: achieved-versus-target slope is below unity, and creativity may resist compression into a single linear direction; excessive SAE steering can also cause hallucinations or off-target shifts.

    Source: §4.2.2 · p. 26, §5 · p. 29

Abstract

from arXiv · show

Simulations based on large language models (LLMs) have proven to be powerful for understanding human behavior, making them valuable additions to the social scientific toolkit. However, LLMs are ultimately black boxes based on deep neural networks which limits their value for social science. This is because of a lack of (i) interpretability: i.e. the ability to assign clear mechanisms driving observed behavior; and a lack of (ii) steerability: i.e. the ability to mute or amplify specific theoretically meaningful mechanisms of action to drive specific model behavior. Here, we demonstrate how the black box could be opened up to further enrich LLM-based simulations. Specifically, we compare three types of methods: (1) prompt-based manipulation, (2) SAE-derived feature steering, and (3) probe-based direction steering and examine their utility for LLM-based social scientific simulations. We do so by interpreting and steering two foundational components of human behaviors, namely preferences (risk attitudes, altruism) and capabilities (divergent creativity, product innovation), operationalized using four classic economic and creative tasks implemented as natural-language interactions. Overall, our results show that SAE- and probe-based techniques often outperform basic prompt-based methods for steering LLM agents, although this advantage depends on the specific prompting strategy involved. Together, SAEs and probes constitute an effective pipeline for social scientists seeking to interpret and steer agents in social simulations: SAEs decompose agents' internal representations into human-readable features, after which probes can reliably shift agents' behaviors in specified directions. We discuss implications of these methods for future work using LLM agents for social scientific simulations.

1 Introduction

LLM agents offer social scientists a scalable way to simulate behavior, but their black-box nature limits interpretation of mechanisms and steering of behavior. This paper compares prompt-based, SAE-based, and probe-based approaches across preference and capability tasks, finding that SAE and probe steering often outperform basic prompting, with the advantage narrowing against stronger prompts.

1 Introduction

  • Black-box LLMs limit social science because researchers cannot reliably interpret the mechanisms behind behavior or steer agents to test predicted mechanisms.Interpretability supports inductive mechanism discovery, while steerability supports deductive tests of hypotheses.Source: §1 · p. 2
  • The paper compares prompt manipulation, SAE feature steering, and probe direction steering across lottery, ultimatum, divergent creativity, and product innovation tasks.The tasks represent preferences and capabilities, respectively.Source: §1 · p. 3
  • SAE- and probe-based steering often outperform basic prompting across the four tasks, although their advantage narrows against few-shot and chain-of-thought prompting.SAEs provide better understanding and control than basic prompts; probes more precisely steer predefined traits, while SAEs can explore and steer previously unknown traits.Source: §1 · pp. 3–4
  • The paper presents a comparative framework for interpreting and steering LLM agents and validates it as an early proof of concept across classic preference and capability tasks.Its stated contributions include fine-grained behavioral control beyond prompting alone.Source: §1 · p. 4

2 Related Work

Generative agents offer low-cost, accessible simulations, but their opacity limits researchers’ ability to understand and manipulate the mechanisms behind behavior. Probes and SAEs provide ways to inspect internal representations, yet their systematic adaptation and comparison for social-scientific simulation remains an open area addressed by this paper.

2.1 Using Generative Agents in Social Science

  • Generative agents can support low-cost, accessible simulations with contextual nuance, but their black-box behavior limits interpretation and reliable steering.Opacity can leave researchers unable to isolate or manipulate causal processes, while post-hoc explanations may misrepresent model reasoning.Source: §2.1 · pp. 5–6

2.2 Opening the Black Box

  • LLMs generate text autoregressively from token embeddings through Transformer layers, whose hidden states encode contextual information before being projected to next-token probabilities.The paper situates probes and SAEs at these internal activations, which are not directly human-readable.Source: §2.2 · pp. 6–7
  • Probes decode predefined concepts from hidden representations, whereas SAEs use sparse latent features to identify patterns associated with behavior.Both can analyze or modify internal representations before next-token prediction; neither family had been systematically adapted to social-scientific simulation, to the authors’ knowledge.Source: §2.2 · p. 7

3 Methodology

The methodology compares prompting, SAE feature interventions, and probe-based steering on individual-agent preferences and capabilities, using tasks grounded in human experiments. It pairs SAE feature inspection with trait-specific probe training and uses separate evaluation procedures for preference and creativity tasks.

3 Methodology

  • The study compares prompting, SAE steering, and probe steering on individual-agent preferences and capabilities using established human-experiment tasks.The selected tasks target risk and altruism in lottery and ultimatum games, plus divergent creativity and product innovation.Source: §3 · p. 8
  • Preference tasks use baseline, persona prompting, SAE steering, and probe steering; capability tasks add a high-temperature condition motivated by a proposed randomness–creativity relationship.The simulations use 40 agents in four scenarios for each preference task.Source: §3 · pp. 8–9

3.1 Prompting Setting

  • Prompting baselines include persona, chain-of-thought, few-shot, and combined few-shot-plus-CoT conditions, with the same task instruction and no changes to weights or decoding parameters.Few-shot exemplars are omitted for stapler innovation because the Ocsai corpus is specific to brick uses.Source: §3.1 · pp. 9–11

3.2 Sparse Autoencoders Setting

  • SAEs encode layer activations into sparse latent features for inspecting behavioral concepts, then modify selected features and decode them back into activation space for steering.The authors inspect top-activated features; positive steering strength amplifies a feature and negative strength dampens it.Source: §3.2 · pp. 11–12

3.3 Probes Setting

  • The probe pipeline trains a logistic-regression probe on intermediate activations to score trait dimensions and steer behavior along a normalized probe direction.Layer 48 was selected by a cross-validated sweep, and steering strength and sign set direction and magnitude.Source: §3.3 · pp. 12–13

Synthetic Data Generation

  • Preference probe data use controlled synthetic trials and choice-based binary labels, while creativity labels come from an independent judge’s median split across four Torrance dimensions.For lottery and ultimatum tasks, switching points and acceptance thresholds are retained as steering-calibration targets rather than training labels.Source: Synthetic Data Generation · p. 14

Probe Training Procedure

  • The logistic-regression probes use final-token activations, z-score normalization, cross-validated regularization, and the layer with the best held-out performance.Layer 48 achieved the highest cross-validation score in the reported Llama-3.3-70B-Instruct runs; held-out preference-trial classification accuracy was 82%.Source: Probe Training Procedure · p. 15

3.4 Evaluation Protocol for Creativity Tasks

  • Creativity responses are scored by five independent LLM judges under a blind, length-controlled rubric, with mean scores reported across judges.Mean pairwise judge Spearman correlation was 0.77 across 680 unique responses, and condition ordering was preserved across all five judges.Source: §3.4 · pp. 15–16

4 Results

SAE features reveal task-linked and sometimes unexpected behavioral codes, while steering effectiveness varies by construct and method. Probes offer precise preference control, but stronger prompting can outperform SAE steering on creativity tasks.

4 Results

  • The evaluation compares baseline, persona prompting, SAE feature steering, and capability-only high-temperature scenarios across 40 agents per scenario.Preference outcomes track behavior changes as rewards increase; capability outcomes use average creativity scores and standard deviations.Source: §4 · pp. 16–17, Table 2 · p. 16

4.1.1 Preference Tasks

  • Lottery features rank probability-based comparisons and option selection highest, while ultimatum features reflect economic tradeoffs and negotiation; lower-ranked features also expose ethics and task-procedure codes.The authors interpret these features as capturing both task mechanisms and agents’ broader behavioral codes.Source: §4.1.1 · pp. 17–18, Figure 3 · p. 18

4.1.2 Capability Tasks

  • Capability-task features are generic at baseline, noisy under high temperature, more relevant under prompting, and most targeted under steering.Unexpected features can also support inductive theorizing, but their activation strength and underlying logic require careful evaluation.Source: §4.1.2 · pp. 19–20, Figure 4 · p. 20

4.2.1 Preference Tasks

  • Lottery SAE steering produces dose-dependent transitions around 75 tokens at moderate activation and 65–70 tokens at stronger activation, unlike prompting’s abrupt, unpredictable effects.The baseline switches around 125 tokens, while the “slightly risky” prompt produces risk-seeking at 20–25 tokens.Source: §4.2.1 · pp. 21–22, Figure 6 · p. 22
  • Prompting does not produce graded lottery risk-seeking: switching stays at 97–101 tokens across dose levels, while CoT shifts it to approximately 140; ultimatum few-shot CoT shifts acceptance thresholds from roughly 10 to 40 tokens.The advantage of SAE/probe steering is sharpest for risk and narrows for altruism, where strong prompting is competitive.Source: §4.2.1 · pp. 22–23, Figure 7 · p. 23
  • In the lottery, probe steering spans switching points of approximately 30–200 tokens with about 2-token mean absolute error; ultimatum probe targets range from 30–60 tokens.Higher positive λ increases risk-seeking, while ultimatum acceptance thresholds shift systematically and saturate at extreme λ values.Source: §4.2.1 · pp. 23–24, Figure 8 · p. 23

4.2.2 Capability Tasks

  • On brick, zero-shot CoT yields ∆=+1.52 versus SAE steering ∆=−0.23; on stapler, persona+CoT yields ∆=+0.59 versus SAE steering ∆=+0.45.Strong prompting therefore outperforms SAE steering on brick and slightly exceeds it on stapler.Source: §4.2.2 · pp. 24–25, Figure 10 · p. 25
  • For brick creativity, probe steering raises the achieved mean from approximately 4.0 at target 3 to 8.2 at target 9, but remains below perfect control.The achieved-versus-target slope is below unity, consistent with creativity involving fluency, flexibility, originality, and elaboration rather than one linear direction.Source: §4.2.2 · p. 26, Figure 11 · p. 26

5 Discussion

The discussion presents SAEs and probes as complementary tools for interpreting and steering LLM agents, with method choice depending on research goals. It also identifies limits in prompting, probe control, and the scope of the experiments.

5 Discussion

  • Across preferences and capabilities, interpretability-driven methods produced more interpretable and controllable agent behavior than prompting alone.SAEs reveal human-readable features, while probes shift agents along pre-specified traits.Source: §5 · p. 27
  • Even when stronger prompts match steering on observed behavior, prompting is less effective at revealing or modulating underlying mechanisms.The discussion cautions that agents’ explanations can reflect narrative justifications rather than the processes that produced their actions.Source: §5 · p. 27
  • For multifeature creativity, SAEs offer better controllability; probes are cheaper but yield partial control with diminishing returns at extreme targets.The probe’s achieved-versus-target slope remains below unity, and probes require only hundreds of training examples rather than the massive datasets needed for SAEs.Source: §5 · pp. 27–28
  • SAEs suit inductive investigation of mechanisms, while probes offer efficient, fine-grained control for deductive tests of predefined measurable traits.The authors describe these as complementary stages in an inductive-to-deductive pipeline.Source: §5 · p. 28
  • The paper offers a comparative framework to help researchers choose among prompting, probes, and SAEs according to their theoretical aims.The framework distinguishes use cases including rapid prototyping, deductive hypothesis testing, and inductive discovery.Source: §5 · p. 28
  • Excessive SAE steering can produce hallucinations or activate off-target representations, making resulting behavioral shifts difficult to analyze.The authors recommend careful monitoring and calibration.Source: §5 · p. 29
  • The experiments focus on individuals, leaving interaction-driven group dynamics, organizational structure, social influence, and network effects for multi-agent settings.The authors identify these as future-work directions.Source: §5 · pp. 29–30
  • The experiments cover a small set of preference and capability traits, not the more abstract, context-dependent mechanisms embedded in complex structures.The authors say future work should examine mechanistic support when using SAE features for higher-order mechanisms.Source: §5 · p. 30

A Technical Appendix for Prompting and Sparse Autoencoder

The appendix documents prompting and SAE steering procedures across tasks, along with controls, reproducibility materials, and creativity-evaluation methods. It also describes a graded prompting comparison with SAE- and probe-based steering, including task-specific exemplar availability and multi-judge scoring considerations.

A.1.1 Stack Overview

  • EDSL defines games as typed interactions with strict I/O schemas and validators, while Goodfire connects Llama-3.3-70B-Instruct to on-demand SAE activations and decoding-time feature nudges.The bridge resets the Variant when steering is turned off to prevent bleed-through.Source: §A.1.1 · pp. 37–38

A.1.2 Agents, Personas, and Cohorts; * Prompting

  • The risk-choice cohorts used neutral baseline personas, a slight-risk-preference prompt, or SAE steering on neutral personas, with shared decoding and validation settings.Activation-collection runs disabled caching; the prompting setup left the controller unset and otherwise matched baseline.Source: §A.1.2 · pp. 38–39, * Prompting · p. 39

A.2.1 Task Setup; A.2.2 Agent Cohorts; A.2.3 Steering Configuration

  • In the Ultimatum Game, responders accepted an offer to receive it or rejected it so both players received zero; cohorts used neutral, altruistic-prompt, or altruism-feature-steered agents.The SAE feature was labeled “Altruistic and selfless behavior or intentions.”Source: §A.2.1 · p. 42, §A.2.2 · p. 42, §A.2.3 · p. 43

A.3.2 Prompting Configuration for Creativity; A.3.3 SAE Steering Configuration for Creativity

  • Creativity prompting used three trait dimensions aligned with three SAE features, which were boosted simultaneously during decoding at steering strength 0.3.The strength was empirically selected to produce meaningful behavioral changes without output degradation or hallucination.Source: §A.3.2 · pp. 45–46, §A.3.3 · pp. 46, 48

A.3.4 Creativity Evaluation; A.3.5 Multi-Judge Evaluation Details

  • Creativity was evaluated using a 1–10 LLM rubric and a blind, length-controlled panel of five LLM judges applying four Torrance dimensions.The dimensions were fluency, flexibility, originality, and elaboration.Source: §A.3.4 · p. 48, §A.3.5 · p. 50

A.3.5 Multi-Judge Evaluation Details

  • Across N = 680 responses, five-judge mean pairwise Spearman correlation was 0.769; agreement was 0.68 on prompting-ladder brick and 0.42 on prompting-ladder stapler.The passage attributes lower stapler agreement to the greater interpretive openness of what counts as a stapler improvement.Source: §A.3.5 · pp. 50–51
  • Response length explained Gemini-2.5-Pro R2 = 0.43, DeepSeek-V4-Pro R2 = 0.36, and GPT-5 R2 = 0.29, but condition rankings persisted when Gemini and DeepSeek were dropped.Kimi-K2.6 and Claude-Sonnet-4.6 each had R2 ≈0.13; length control reduced but did not eliminate the dependence.Source: §A.3.5 · p. 51

A.3.6 High Temperature Scenario; A.6 Invariants and Controls

  • For capability tasks, a high-temperature scenario used 0.9 or 1.0 while holding other parameters constant; decoding and scaffolding were otherwise identical across conditions.The scenario tested whether creativity changes reflected increased output variance or mechanistic changes in idea generation.Source: §A.3.6 · p. 51, §A.6 · p. 53

A.6 Invariants and Controls

  • The invariants and controls check reports that small weight perturbations do not arbitrarily flip decisions in obviously unstable regimes.Source: §A.6 · p. 53

A.7 Reproducibility and Artifacts

  • Reproducibility artifacts include timestamped results directories, condition- and reward-specific CSVs, steering controller JSON files, and top-k feature activations per response.Source: §A.7 · p. 54
  • For inspection, the artifacts expose prompt, generated-token, answer, and comment columns for the safe-risky choice task.Source: §A.7 · p. 54

A.8 Strong Prompting Baselines

  • The graded prompting ladder compares SAE- and probe-based steering with prompt conditions on the same Llama-3.3-70B-Instruct backbone across four tasks.Source: §A.8 · p. 54
  • Prompting runs use self-hosted bf16 vLLM on 2×H100 GPUs, with decoding held constant and task-specific generation limits.Conditions use temperature 0.7, top-p 1.0, and 40 agents; maximum generation lengths are 512 tokens for preference games and 1,536 for creativity tasks.Source: §A.8 · p. 54
  • The ladder spans neutral baseline and persona prompts, zero-shot CoT (Kojima et al., 2022), few-shot demonstrations (Brown et al., 2020; Wei et al., 2022), and dose-varied few-shot CoT.The six CoT demonstrations vary the target-disposition fraction from ρ = 0 to 1.0, with opposing dispositions in the remaining examples.Source: §A.8 · pp. 54–55
  • Brick-task exemplars come from human-rated Ocsai responses, while stapler few-shot prompting is omitted for lack of a comparable public dataset, leaving persona plus plan-then-list CoT as its strongest prompt.Brick exemplars are selected by human originality ratings rather than downstream LLM-judge scores; the appendix notes this asymmetry as one reason SAE/probe steering retains a clearer edge on stapler than brick.Source: §A.8 · p. 55

A.2 Ultimatum Game Implementation

The ultimatum game uses Baseline, Prompting, and Steering scenarios to measure altruism and fairness preferences.

A.2 Ultimatum Game Implementation

  • The ultimatum game compares Baseline, Prompting, and Steering scenarios to measure altruism and fairness preferences.Source: §A.2 · p. 41

A.2.4 Experimental Sweep

The experimental sweep varies offer amounts and compares three conditions at each amount, then plots their outcomes across the sweep.

A.2.4 Experimental Sweep

  • Offers range from 10 to 90 tokens, with three experiments run for each offer amount.Source: §A.2.4 · p. 44
  • Baseline uses a neutral persona without a controller; Prompting uses an altruistic persona without a controller; Steering boosts the altruism feature with a neutral persona.Source: §A.2.4 · p. 44
  • Each condition writes one CSV, and the three curves are plotted across offer amounts.Source: §A.2.4 · p. 44

A.3 Capability Task Implementation and Evaluation

Capability tasks compare Baseline, Prompting, and Steering with an added high-temperature scenario to test whether randomness enhances creative outputs.

A.3 Capability Task Implementation and Evaluation

  • For divergent creativity and product innovation, a high-temperature scenario tests whether randomness enhances creative outputs alongside Baseline, Prompting, and Steering.Source: §A.3 · p. 44

A.3.1 Task Implementation

The capability tasks elicit open-ended responses for divergent creativity and product innovation, while feature analysis records activated concepts for each prompt–response pair.

A.3.1 Task Implementation

  • Capability prompts ask agents to generate detailed uses for a brick or specific improvements to a stapler.The stapler task excludes new uses and focuses on changes to the object itself.Source: §A.3.1 · p. 45
  • Unlike preference tasks, capability tasks use open-ended Question List responses, with up to 10 items per answer.Source: §A.3.1 · p. 45

A.4 Feature Activation Analysis

  • Feature analysis reports each prompt, model response, and the top-k activated features for baseline, prompting, and steering runs.The baseline call sets use_base_variant=True to avoid stale controller state.Source: §A.4 · pp. 51–52

A.5 Experimental Sweep and Outputs

The probe pipeline steers preference and creativity behaviors by intervening on hidden states, with probe scores tracking targeted preference shifts and creativity control remaining partial across objects.

B.1 Stack Overview

  • Probes use a frozen base model and a linear probe trained on intermediate activations; the intervention layer is selected by held-out performance.The main Llama experiments use layer 48, and the pipeline is also replicated on Qwen at layer 17.Source: §B.1 · p. 56

B.2 Agents, Cohorts, and Trial Structure

  • Each agent is an independent decoding instance; baseline trials use λ = 0, while steering trials repeat prompts and decoding settings with a specified λ and direction.Source: §B.2 · p. 57

B.3 Tasks, Labels, and Targets

  • Probe labels use observed choices, with risky and accept choices as positive classes; this keeps early risk-seeking switches in the higher-risk class.Trials whose choices cannot be parsed are discarded before probe training.Source: §B.3 · p. 58

B.4.2 Steering

  • At inference, steering adds a scaled probe direction to hidden states at one layer, applying it over the final 20% of tokens with scale rising from 0.5 to 1.0.Source: §B.4.2 · p. 59

B.5 Lambda Calibration and Dose-Response

  • Target behaviors are reached by searching λ: a coarse grid identifies a region, followed by bracketed binary search until the achieved metric meets a fixed tolerance.The lottery target is the risky reward at which P(Risky) = 0.5; the ultimatum target is the offer at which P(Accept) = 0.5.Source: §B.3 · p. 57, §B.5 · p. 59

B.6 Internal Probe Dynamics

  • Probe scores tracked the intended steering direction in both preference tasks while preserving separation between risky versus safe and accept versus reject choices.For lottery, scores decreased as target switching points rose from 40 to 240 tokens; for ultimatum, scores increased with the target switching point.Source: §B.6 · p. 60

B.7 Cross-Object Generalization

  • Across four objects, achieved creativity scores rose monotonically with targets from approximately 3.7 to 8.3, but fell increasingly below targets at higher levels.The pattern indicates partial but meaningful control.Source: §B.7 · pp. 60–61
  • Cross-object probe accuracy remained reasonable but dropped approximately 5–6 percentage points versus in-distribution evaluation, indicating imperfect transfer of object-specific patterns.Source: §B.7 · p. 61

B.10 Robustness Check with Qwen-2-7B-Instruct

The Qwen replication reproduces probe-based steering patterns and transfers the pipeline to a smaller model, while appendix results show prompting control varies across tasks and creativity control generalizes across objects.

B.10 Robustness Check with Qwen-2-7B-Instruct

  • Qwen-2-7B-Instruct achieved approximately 85% in-distribution creativity classification accuracy, versus approximately 92% for Llama, and both probes substantially exceeded chance.The Qwen probe used layer 17, selected by a cross-validated score of 0.99.Source: §B.10 · p. 62
  • On Qwen, probe steering shifted lottery and ultimatum switching points monotonically, enabled out-of-sample creativity steering, and showed a creativity ceiling effect across objects.Source: §B.10 · p. 62

C Appendix Figures and Tables

  • For lottery prompting, even when every demonstration chose the risky action, the switch point stayed within 97–101 tokens, indistinguishable from baseline.The sweep varied the risky-action fraction across ρ = 0, 0.25, 0.5, 0.75, and 1.0.Source: Figure 12 · p. 63
  • Few-shot chain-of-thought prompting shifted the ultimatum acceptance threshold dose-dependently, from approximately 40 tokens at ρ = 0 to approximately 10 at ρ = 1.0.Source: Figure 14 · p. 64, Figure 15 · p. 65
  • Five judges showed mean off-diagonal Spearman correlation of 0.769 on the creativity dataset, with pairwise correlations ranging from 0.675 to 0.849.The caption reports the lowest pair as Claude–DeepSeek and the highest as Gemini–GPT-5.Source: Figure 17 · p. 66
  • A brick-trained divergent-creativity probe achieved a comparable controllability range on stapler, paperclip, and bowl prompts, despite those objects being out of sample.Source: Figure 20 · p. 67

Related papers

Loading 2609.16436v1…