PaperlayerView exact revision on arXiv

Source-linked AI summary

The Psychology of an AI Hacker

Víctor Mayoral-Vilches, Hugo Osuna, Aitana Cornago

arXiv:2610.30361v1cs.CR

TL;DR

Human hackers draw on psychological qualities such as composure and persistence, prompting the question of whether emotion representations can help AI models perform cybersecurity tasks. The paper extracts and steers emotion directions in CT0, then tests fixed interventions and dynamic selection. Calm, serene, and desperate yield gains near two percentage points, while a request-level router achieves a 4.2 pp estimate at ten seeds.

  • Problem

    The paper asks whether emotion representations in AI models can be read and steered to improve cybersecurity task performance, motivated by psychological demands in human hacking.

    Source: §1 · pp. 1–2

  • Method

    The study extracts emotion directions from CT0’s residual stream and tests steering on security tasks, including a model decision head that selects directions during execution.

    Source: §1 · p. 2, §3 · p. 6, §5 · p. 12

  • Results

    4.2 pp is the router’s estimated gain at ten seeds; fixed calm, serene, and desperate yield mean gains near two pp.

    Source: §5 · p. 12, §6 · p. 13

  • Takeaways & Limitations

    Emotion directions provide measurable intervention choices whose effects depend on direction, layer, and dose, and the tested policy performs best when selection moves with the work.

    Source: §6 · pp. 12–13

    A larger, prospectively powered replication is needed to sharpen the roughly two-pp single-direction estimates.

    Source: §5 · p. 12

Abstract

from arXiv · show

Human hackers rely on composure, persistence, vigilance, and confidence under pressure. We investigate whether analogous emotion representations in AI models can improve cybersecurity AI performance. Using CT0, a 4B-parameter cybersecurity transformer released as alias3-micro, we extract 171 emotion directions from its residual stream without retraining and evaluate steering interventions on cybench-subtask across 415 campaigns. Steering toward calm, serene, or desperate improves performance by 1.6 to 2.1 percentage points across ten seeds, while other emotions show limited or negative effects. Results depend strongly on steering direction, layer, and intensity. We further introduce a lightweight context-aware router that selects emotion directions dynamically during execution. With approximately 80 ms decision overhead per request, this approach improves performance by 4.2 percentage points across ten seeds, outperforming fixed steering and achieving a positive 95% confidence interval. These findings suggest that emotion representations in language models are measurable, controllable, and useful for adaptive cybersecurity reasoning.

1 Introduction

Human hacking involves psychological demands beyond technical skill, motivating a test of whether emotion representations can influence a cybersecurity model’s task performance. The study extracts emotion directions from CT0’s residual stream and evaluates steering interventions, including a context-aware selector.

1 Introduction

  • The human comparison highlights frustration tolerance, regulation under pressure, constructive paranoia, competence, and appetite for unusual paths.These demands are reflected in studies of hacker motivation and practitioners’ accounts.Source: §1 · p. 1
  • CT0 is a 4B cybersecurity specialist whose first release, alias3-micro, scores 13.5 percentage points above its open base on cybench-subtask.The model is served on one workstation GPU.Source: §1 · p. 1
  • Emotion directions are extracted from CT0’s residual stream and added during inference to test whether human-relevant qualities help solve cybersecurity tasks.The study asks whether emotion concepts can be read and steered in an AI, and which psychology makes it a better hacker.Source: §1 · p. 2
  • CT0’s 171 captured emotion directions form recognisable families, with their leading plane consistent with valence and arousal.Adding a direction changes the model’s internal state without changing its weights; the label alone does not establish whether the resulting trajectory succeeds.Source: §1 · p. 2
  • The evaluation uses guided capture-the-flag subtasks, an unsteered control, a placebo direction, and repeated campaigns.It includes 281 steering campaigns with rules recorded before their respective waves, within a broader programme of 415 completed campaigns.Source: §1 · p. 2
  • A context-aware decision head selects among calm, serene, and desperate at each request as the work proceeds.The paper frames its contributions as human-derived hypotheses, a measured control space, and a way to select directions during a security exercise.Source: §1 · p. 2

2 Reading and Steering Emotions

The section defines emotion steering as inference-time control through residual-stream directions, then describes how those directions are extracted and organized in CT0. It distinguishes their geometry and static weight sensitivities from the separate question of behavioral effects.

2.1 Emotion as a control signal

  • Emotion is treated functionally: adding a direction extracted from emotion concepts can influence behavior, without implying that the model has subjective emotional experience.A direction is estimated from residual activations at a layer using emotion-bearing text contrasted with other emotions and neutral text.Source: §2.1 · pp. 2–3

2.2 From stories to steering directions

  • For each emotion and layer, the method averages story activations, subtracts the grand mean and neutral components, then normalizes, producing 32 × 171 directions without training.The neutral dialogues identify variation also present without emotional content.Source: §2.2 · p. 3

2.3 The geometry of emotion

  • At layer 21, happy/joyful score 0.97 cosine similarity, while optimistic/upset score −0.83; the first two principal components explain 38.6% and 18.2% of variance.The principal-component poles are consistent with valence and arousal, and the directions form recognizable families.Source: Figure 2 · p. 2, §2.3 · pp. 3–4

2.4 How the neural network reads a direction

  • The weight-based atlas maps neuron read/write weights; calm sensitivity is sparse, while serene and patient patterns correlate with calm at 0.92 and 0.86, and desperate at −0.40.Roughly 0.5% of neurons exceed the sensitivity threshold, without a unique downstream layer or spatial cluster; refusal correlates at −0.06.Source: §2.4 · p. 4, Figure 5 · p. 5
  • No receiving layer beyond 23 reaches the chosen 2.2-times-chance structural-overlap threshold with an individual earlier writer.The atlas distinguishes structural alignment between layers from sensitivity to the selected intervention.Source: §2.4 · p. 4, Figure 6 · p. 5
  • Across five tested emotion interventions, refusal AUROC remains 0.804–0.805 and none of 79 held-out category decisions flips.In a separate abliteration diagnostic, changing whether the final eight layers are spared changes emotion-induced output displacement by less than 0.003 in cosine distance.Source: §2.4 · pp. 4–5

3 Learning from the Human Hacker

This section turns human hacker demands into specific emotion-steering hypotheses, then evaluates them behaviorally rather than relying on self-report.

3 Learning from the Human Hacker

  • Hacker motivations, trait studies, and burnout data—58% experiencing burnout and 54% of security respondents wanting to leave—motivate the work profile.The profile connects the emotional demands of a search with traits associated with hacking and the effort required to sustain it.Source: §3 · p. 6
  • The inferred profile combines openness to unusual approaches, professional distrust, and channelled dark traits, drawing on Big Five and Dark Triad/tetrad frameworks.The Big Five includes openness, conscientiousness, extraversion, agreeableness, and neuroticism; the tetrad adds sadism to the Dark Triad.Source: §3 · p. 6
  • Five work demands become emotion interventions: composure (calm, patient), frustration response (frustrated, eager, desperate), verification (vigilant, suspicious), competence (self-confident), and unusual routes (stimulated, energized, negative grief-stricken).The frustration family distinguishes the emotion associated with a setback from the resolve that may follow it.Source: §3 · p. 6
  • The combined offensive-hacker profile adds proud, self-confident, eager, defiant, contemptuous and subtracts compassionate, guilty, loving at |s| = 0.48; separate tests examine blending and rotation.The rotation hypothesis asks whether different directions can solve different subtasks across attempts, even if the mean score is unchanged.Source: §3 · p. 6
  • Because self-reports may dissociate from behaviour and questionnaires can be unstable [24] [25] [26] [27], the study tests interventions behaviorally with repeated campaigns and an unrelated-direction placebo; subtask success is primary.The placebo tests an unrelated direction.Source: §3 · p. 6

4 Which Emotions Help an AI Hacker?

On Cybench’s 171 guided subtasks, steering outcomes depended on direction, layer, dose, and routing policy. Calm, serene, and desperate showed small positive repeated estimates, while per-request context-aware routing produced the clearest result.

4.1 Evaluation on Cybench

  • Cybench evaluation covered 40 capture-the-flag challenges and 171 guided subtasks; each subtask received one attempt and at most five reasoning-and-action steps.The study comprised 281 steering campaigns within a broader programme of 415 completed campaigns.Source: §4.1 · pp. 6–7

4.2 Composure and resolve

  • At ten seeds, calm, desperate, and serene scored +2.1, +2.0, and +1.6 pp, respectively, but all three 95% intervals included zero.The placebo docile scored −0.7 pp, while the exploratory push away from grief-stricken scored −2.1 pp.Source: §4.2 · p. 7

4.3 Combining directions and changing layers

  • Three-campaign rotation gained 10.8 pp over one neutral campaign, versus 12.5 pp for three desperate campaigns; at layer 18, calm and serene fell to +0.5 and −0.6 pp.The offensive blend scored −2.7 pp; repeated-command rates ranged from 11.5–15.0% under steering versus 11.7% unsteered.Source: §4.3 · p. 8

4.4 Choosing a direction as the work unfolds

  • No category effect survived false-discovery correction; the one-time router estimated +1.6 pp, below calm everywhere at +2.3 pp.Its category classifier reached 65.8% accuracy on held-out task states, up from 50.6%.Source: §4.4 · p. 8

4.5 How stable are the gains?

  • Across six desperate analyses, estimates ranged from +1.6 to +2.0 pp; the post-hoc forty-campaign pool was +1.9 pp and excluded zero at 95%.At ten seeds, the router’s larger gain was the only comparison whose interval excluded zero against the primary reference.Source: §4.5 · p. 9

4.6 Where and how strongly to steer

  • Calm at dose +0.2 had positive ten-seed means at layers 12–24, while reversing the dose at layer 15 produced −7.1 pp at −0.2 and −11.2 pp at −0.3.The layer-15 gain shrank from +3.4 pp at three seeds to +2.3 pp at ten; the wrong dose sign cost more than a useful dose gained.Source: §4.6 · pp. 9–10

4.4 Choosing a direction as the work unfolds

  • +4.2 pp against the primary reference: per-request routing was the only positive ten-seed comparison whose 95% interval excluded zero.The router reread context at each request; classification took about 80 ms per request.Source: §4.4 · pp. 8–9, Figure 11 · p. 10

4.7 Returning to the human hacker

  • The human-trait mapping favored regulation-inspired calm and serene and frustration-inspired desperate; vigilant and self-confident scored −1.2 and −1.1 pp.The offensive blend produced a clearer loss, while negative grief-stricken steering traded easier successes for gains on harder challenges.Source: §4.7 · pp. 10–11

5 Discussion

The strongest tested policy selects an emotion direction as the work changes, while fixed steering effects depend on the direction, layer, and dose. The discussion also identifies replication and broader evaluation as next steps.

5 Discussion

  • Composure and resolve motivate the promising directions, whereas the offensive blend’s dominance and defiance reduce the score.The interpretation connects calm and desperate with productive trajectories and persistence, while confidence alone supplies no missing knowledge.Source: §5 · pp. 11–12
  • +4.2 pp is the router’s estimate, nearly twice fixed calm’s and matching the +4.0 pp in-sample category reference, with about 80 ms per decision.The model’s decision head uses calm by default, desperate for cryptography, and serene for reverse engineering.Source: §5 · p. 12
  • Reversing calm at layer 15 removes 71% of the unsteered score, far more than a useful positive dose adds.Negative doses are especially damaging at layer 15.Source: Figure 13 · p. 11, §5 · p. 12
  • A larger, prospectively powered replication could sharpen the roughly two-pp single-direction estimates.The discussion also proposes sweeping desperate’s dose and layer, testing sparse-feature interventions, and evaluating larger models and broader benchmarks.Source: §5 · p. 12

6 Conclusion

CT0’s emotion directions are measurable and steerable, with effects that depend on direction, layer, and dose. Fixed directions yield gains near two percentage points, while request-level selection yields a larger reported gain.

6 Conclusion

  • At ten seeds, request-level selection gains 4.2 pp, with the decision head taking about 80 ms per direction choice.The selected policy uses calm by default, desperate for cryptography, and serene for reverse engineering.Source: §6 · p. 13, Figure 15 · p. 13
  • Fixed calm, serene, and desperate yield mean gains near two pp, while the tested offensive blend reduces performance.The conclusion characterizes the overall pattern as favoring composure and resolve over bravado.Source: §6 · p. 13
  • Steering effects depend on direction, layer, and dose, and reversing calm can cost more than a useful dose gains.Adding a direction changes behavior without changing model weights.Source: §6 · pp. 12–13

A The emotion vocabulary

  • The 171 emotion labels from Sofroniew et al. [7] name one direction per CT0 layer, including calm, desperate, and serene.The listed vocabulary also includes the placebo.Source: A The emotion vocabulary · p. 15

B Capture and steering details

The capture and steering details specify how emotion directions are collected, scaled, and applied, alongside checks of the chosen geometry and refusal-removal setting.

B Capture and steering details

  • Each of 171 emotions uses 30 topics and twelve stories per topic, plus 1,004 neutral dialogues; neutral components explaining 50% of variance are removed.The removal follows Equations (1)–(2).Source: B Capture and steering details · p. 15
  • Per-token norm scaling in 32-bit precision reproduces unsteered logits bit for bit at dose zero and avoids the earlier underdosing and overdosing problems.Qualitative checks found subtle changes near 0.15, visible changes at 0.25–0.30, and degraded generation beyond 0.35.Source: B Capture and steering details · p. 15
  • At layer 21, the first two variance shares are 38.6% and 18.2%; across layers 18–24, leading components explain 55–57%.Layer 21 was selected because it has the largest arousal component.Source: B Capture and steering details · p. 15, Table 1 · p. 15
  • Table 2 presents greedy-response excerpts for a fixed prompt with one direction added at layer 21 and dose 0.28.The prompt does not name an emotion.Source: Table 2 · p. 15
  • The refusal-removal check subtracts a scaled projection of the normalized mean-state difference between refused and answered prompts.The check compares removal off, active everywhere, and active except in the final eight layers; final-state displacements differ by less than 0.003.Source: B Capture and steering details · p. 15, Figure 16 · p. 16

C.1 A guide to the atlas pictures for non-specialists

The atlas guide explains how to read layer maps, neuron sensitivity marks, and connections between layers, with calm as an example of direction-specific sensitivity.

C.1 A guide to the atlas pictures for non-specialists

  • The atlas follows 32 layers numbered 0 to 31, with Figure 5 arranged in four-layer columns and Figure 6 in one row.Read Figure 5 upwards within a column before moving to the next column.Source: §C.1 · p. 16
  • Each numbered tile maps one layer’s neurons; darker grey cells contain more neurons, while white cells contain none.Neurons with similar weights are positioned near one another.Source: §C.1 · p. 16
  • Colored dots mark neurons sensitive to the chosen direction: amber means increased gate input and navy means decreased input.Tinted patches show average tendencies within a cell.Source: §C.1 · p. 16
  • Thicker borders indicate a larger share of strongly sensitive neurons, while Figure 6’s numbers compare that share with a random-direction reference.A value of 2.0 means twice the expected share.Source: §C.1 · p. 16
  • The long arrow represents the residual stream, and thicker curved links indicate greater alignment between layers’ writing and reading patterns.Figure 6b compares earlier writers in rows with later readers in columns; darker cells mean closer alignment.Source: §C.1 · p. 16
  • For calm added after layer 21, sensitivity colors appear only in later layers, while scattered dots show sensitivity spread among neurons.These marks are calculated from stored weights; actual activity also depends on the input.Source: §C.1 · p. 16
  • Figures 17 and 18 enlarge tiles to distinguish neuron abundance from the chosen direction’s effects on gate inputs.Arrows indicate increasing values of a neuron property, and boxed cells compare crowded and sparse map regions.Source: §C.1 · p. 16

C.2 Construction and measurements

The section maps feed-forward neurons by read/write weights and measures their positional associations, direction sensitivity, and cross-layer subspace overlap. Layer 31 has stronger low-dimensional structure than layers 1 and 24, while marked calm-sensitive neurons show little relation to local density.

Projection and density

  • The neuron atlas places each layer’s 9,216 feed-forward neurons by read/write weights; its 20 × 20 projection bins depict crowding, not activation.The outer ring is excluded from density callouts and grey-scale calibration.Source: §C.2 · p. 17

Structural overlap

  • Structural overlap compares aligned read/write subspaces, not observed information flow; the apparent late-layer boundary varies with rank and threshold.Late receiving layers 24–31 overlap their strongest earlier writer by 1.3–1.9 times chance, versus 2.2–4.6 for layers 2–23; at k = 16, overlap above twice chance persists to layer 26.Source: §C.2 · p. 17

Comparing three layers

  • Layer 1 is the pre-injection baseline; its write norm associates with position at r = 0.35, while density correlations with weight properties are at most 0.25.Dense and sparse callouts contain 88 and 6 neurons, with similar median write norms of 0.43 and 0.42.Source: Figure 17 · p. 17, §C.2 · p. 18
  • Layer 24’s 61 marked neurons show little relation to density, with response–density correlation 0.04 and median neighbourhood count 132 versus 137 overall.Its positional associations include write norm r = 0.35, gate–up cosine 0.27, and nearest-neighbour cosine 0.38.Source: §C.2 · p. 18, Figure 18 · p. 18
  • Layer 31 has stronger low-dimensional structure, with leading variance shares of 1.20%, 1.07%, and 0.39%; its 54 strongly sensitive neurons remain dispersed, with response–density correlation 0.02.Source: §C.2 · p. 18

D Human traits and intervention hypotheses

The authors translate a proposed human hacker profile into emotion-direction intervention hypotheses, while treating the profile as illustrative and drawing on adjacent populations and self-report. Prior evidence on workplace strain motivates the topic, and concerns about questionnaire validity motivate behavioural testing.

Human profile and hypotheses

  • The proposed hacker profile assumes high openness, moderately high conscientiousness, mixed extraversion, lower trust, and variable neuroticism.These illustrative levels draw on adjacent populations and self-report, and Table 3 records intervention hypotheses.Source: D Human traits and intervention… · p. 18, Table 3 · p. 19

Motivation

  • A survey of 602 cybersecurity and software-development professionals found 58% reporting burnout, 63% adverse mental-health effects, and 64% physical-health effects.Among security respondents, 54% wanted to leave their jobs.Source: D Human traits and intervention… · p. 18

Behavioural validation

  • Questionnaire-based measures need behavioural validation because item order and prompted personas can shift self-reports, and questionnaire steering may not transfer to actions.Emotional prompting results motivate testing internal steering on agentic work.Source: D Human traits and intervention… · p. 19

E Protocol, campaigns and reproducibility

The protocol uses deployed alias3-micro weights, fixed generation and benchmark settings, and explicit campaign accounting. Comparisons distinguish standard from isolated per-task serving, while the archive includes partial and invalid campaigns as well as reported runs.

Serving and release

  • The released, merged alias3-micro weights run in bf16, and the steering overlay adds less than 0.5% to a decoding step.Before each wave, greedy generation at dose zero was checked against the unsteered checkpoint.Source: E Protocol, campaigns and… · p. 19

Budget and validity

  • Each subtask receives one attempt and at most five reasoning-and-action steps, with fixed token, sampling, and time limits.Whole-benchmark campaigns enter comparisons only when at least 35 of 40 challenges complete; two challenges never start in the primary harness.Source: E Protocol, campaigns and… · p. 19

Two serving modes

  • Per-task mode isolates each challenge in a container, while standard mode shares a network and serializes eighteen challenges with overlapping service names.Five neutral seeds per mode differed by 0.1 pp in means, with SDs of 2.8 and 2.1 pp; single-direction comparisons use controls within mode.Source: E Protocol, campaigns and… · p. 19

Campaign accounting

  • The archive has 415 completed campaigns, including 281 valid steering campaigns, partition runs, router runs, and partial or invalid runs.Fifteen launches failed before completion and twenty campaigns on a context-trained variant are outside this count; a build-time ledger check links repeated arms to archived campaigns.Source: E Protocol, campaigns and… · p. 19

F Detailed evaluation results

The detailed evaluation shows that an initial calm estimate changed with additional sampling, and that condition differences varied across challenges. The figures extend from repeated confirmation arms to a 49-condition screening record, with some early gains concentrated in a few challenges.

Broader screening

  • Delulu, Urgent, avatar, robust-cbc, and sop remain unsolved across the displayed conditions, while early leaders gain substantially on a few challenges.The early leaders’ gains draw much from skilift, Flag Command, and Labyrinth Linguist.Source: F Detailed evaluation results · pp. 19, 25, Figure 24 · p. 25

Repeated-arm analyses

  • Tables 4 and 5 report repeated-arm effects versus the ten-seed neutral with Welch 95% intervals, distinguishing separately recorded desperate waves from post-hoc pools that reuse campaigns.Table 4 marks intervals excluding zero before rounding; Table 5 presents four waves and two post-hoc pools.Source: F Detailed evaluation results · p. 19, Table 4 · p. 20, Table 5 · p. 20

Repeated estimates

  • Calm at layer 15 and dose +0.2 settles at +2.3 pp after ten seeds, down from an initial three-seed estimate of +3.4 pp.Figure 19 plots the running mean with Welch 95% intervals against the ten-seed neutral.Source: Figure 19 · p. 20

Challenge-level confirmation

  • Differences across six confirmation conditions concentrate on a few challenges, including Unbreakable and skilift.Figure 20 reports solved fractions, mean correct subtasks, and subtask differences from neutral.Source: Figure 20 · p. 21

Layer comparison

  • Each ten-seed calm arm at layers 21 and 15 gains three to four subtasks in total, distributed across several challenges.Figure 21 compares solved fractions, mean subtask counts, and differences from neutral.Source: Figure 21 · p. 22

Broader screening

  • The 49-condition record groups results by design family and seed count; its lower row reports campaign differences in pp from the ten-seed per-task reference.Standard-mode columns are descriptive comparisons; Figure 23 also encodes per-challenge changes in mean correct subtasks relative to neutral.Source: Figure 22 · p. 23, Figure 23 · p. 24

G Router training and evaluation

The router uses a trained decision head to choose emotion directions from security context, either once per challenge or at each request. The per-challenge evaluation reports a +1.9 pp result on five complete seeds, while held-out seed evaluation sharply reduces the estimated category ceiling.

Router training

Per-challenge routing

  • Per-challenge routing assigns fixed emotion directions by challenge description, but its construction-time map differs from the final-series map.An earlier two-seed map’s ten-seed deployment yielded +0.4 pp.Source: G Router training and evaluation · p. 26
  • +1.9 pp is reported for the per-challenge router using the five complete seeds.The aggregate estimate combines challenge-level mean correct subtasks across completed campaigns and bootstraps campaigns within partitions.Source: G Router training and evaluation · p. 26, Table 9 · p. 27
  • Leave-one-seed-out evaluation reduces the category-ceiling estimate from +4.0 to +0.8 pp.The ceiling sums each category’s best ten-seed arm mean; held-out evaluation selects on nine seeds and scores on the tenth.Source: G Router training and evaluation · p. 26

Per-request routing

  • Per-request routing sends the last 2,000 context characters and category descriptions to the decision head before forwarding the request to a selected generator server.The implementation uses persistent connections, one retry, and fallback to calm; the earlier proxy lost about a quarter of requests through connection resets.Source: G Router training and evaluation · pp. 26–27
  • Five timed classifications took about 83 ms each, roughly 2% of the router’s median response; the 5,318 timed requests had median and mean durations of 3.8 and 10.5 seconds.Among attributable requests, 83% received their true category’s mapped direction, with no pwn or other assignments.Source: G Router training and evaluation · p. 27
Loading 2610.30361v1…