Representation Engineering: A Top-Down Approach to AI Transparency

7 Conclusion

• RepE is contrasted with MI through population-level representations rather than the network’s specific implementing parts.The comparison distinguishes algorithmic variables and functions from the neural components that execute them.Source: Figure 2 · p. 5
• Marr’s algorithmic level describes the variables and functions a network tracks and transforms, whereas the implementational level describes the neural parts executing those processes.Source: Figure 2 · p. 5
• High-utility text scenarios evoke distinct neural trajectories in representation space compared with lower-utility scenarios.The figure uses principal components to visualize these neural activity patterns.Source: Figure 2 · p. 5

• Mechanistic interpretability has improved understanding of simple mechanisms but struggles with complex phenomena, motivating a representational approach to AI transparency.The introduction frames LLM transparency as increasingly urgent because these systems are widely deployed while remaining largely black boxes.Source: §1 · p. 5
• RepE places representations at the center of analysis, abstracting away lower-level mechanisms to improve understanding and control of high-level cognition.This approach draws on the Hopfieldian view of cognition as patterns of activity across neural populations rather than individual neurons and circuits.Source: §1 · pp. 5–6
• RepE develops improved methods for reading, controlling, and tuning representations across safety-relevant problems including truthfulness, hallucination, jailbreaking, memorization, and power-seeking.The paper positions these techniques as an early step toward top-down transparency and contrasts them with activation engineering applied to frozen models (Turner et al., 2023e).Source: §1 · p. 6
• 18.1 percentage points: fully unsupervised honesty improvement over zero-shot accuracy on TruthfulQA, achieving state-of-the-art results and outperforming all prior methods.The paper also reports that RepE can detect and control whether a model is lying across diverse scenarios.Source: §1 · p. 6

2.1 EMERGENT STRUCTURE IN REPRESENTATIONS

• Neural networks develop emergent representations of semantics, morality, sentiment, chess concepts, segmentation, coordinates, and depth, motivating representation engineering for monitoring and controlling safety-relevant cognition.These findings span language, recurrent game-playing, and computer-vision models, suggesting increasingly structured neural representations.Source: §2.1 · pp. 6–7

2.2 APPROACHES TO INTERPRETABILITY

• Existing interpretability methods include saliency maps, feature visualization, and mechanistic circuit analysis, but they offer limited insight into distributed representations and require substantial manual effort.Circuit-based explanations are also challenged by iterative refinement and robustness to lesioning, including in LLMs.Source: §2.2 · p. 7

2.3 LOCATING AND EDITING REPRESENTATIONS OF CONCEPTS

• Prior work locates concepts in neurons, feature-space directions, and latent spaces, using linear probes and editing methods for factual associations, concept erasure, and counterfactual generation.The paper builds on these approaches by proposing improved representation engineering methods for LLMs.Source: §2.3 · pp. 7–8
• Research on LLM truthfulness, deception, activation editing, sycophancy, Othello, and task-specific concept removal demonstrates that hidden representations can predict or causally steer model behavior.These studies include classifiers for truthfulness, activation edits that increase truthful generations, and edits that counterfactually change game-playing behavior.Source: §2.3 · p. 8

3 REPRESENTATION ENGINEERING

• RepE centers representations as the fundamental unit for understanding and controlling high-level cognitive phenomena, organized around Reading and Control.The approach primarily targets large language models and provides overviews with baseline methods for both areas.Source: §3 · p. 8

3.1 REPRESENTATION READING

• Representation Reading locates emergent concept and function representations to support extraction, discovery, monitoring, and later control.The section extracts concepts including truthfulness, utility, probability, morality, and emotion, plus functions such as lying and power-seeking.Source: §3.1 · p. 9

3.1.1 BASELINE: LINEAR ARTIFICIAL TOMOGRAPHY (LAT)

• LAT reads concepts and functions by designing eliciting tasks, collecting token-level neural activity, and fitting linear models, primarily using unsupervised PCA.Unlabeled or self-generated stimuli can elicit salient responses without annotation bias; for functions, experimental and reference tasks distinguish execution from non-execution.Source: §3.1.1 · pp. 9–10
• LAT commonly uses the last token representation for concepts and response-token representations for functions, while paired stimuli and PCA produce a reading vector.The simple vector-based setup is effective across concepts and functions, though more involved collection may be needed for intricate concepts or multi-step functions.Source: §3.1.1 · p. 10

3.1.2 EVALUATION

• Evaluation distinguishes correlation, manipulation, termination, and recovery to assess prediction, causality, necessity, and sufficiency of identified representations.Converging evidence across these approaches can strengthen claims that findings generalize beyond their original experimental conditions; later experiments emphasize correlation and manipulation.Source: §3.1.2 · p. 11

3.2 REPRESENTATION CONTROL

• Representation Control modifies internal concept and function representations, requiring specialized methods because effective reading does not necessarily enable control.Reading methods that support effective control provide stronger causal evidence and may reduce risks from safety-relevant concepts.Source: §3.2 · pp. 11–12

3.2.1 BASELINE TRANSFORMATIONS

• Control baselines use reading vectors, stimulus-dependent contrast vectors, or LoRRA adapters, combined with linear, piece-wise, or projection operators.Contrast vectors are a stronger baseline but incur inference overhead; LoRRA acquires controllers during training and can be merged without additional inference burden.Source: §3.2.1 · pp. 12–13

4.1 A CONSISTENT INTERNAL CONCEPT OF TRUTH

• LAT reliably extracts truthfulness directions that outperform few-shot and zero-shot methods, generalize across data sources, and indicate larger models possess better internal models of truth.Directions derived from as few as 10 examples achieve similar performance, supporting generalizability rather than overfitting.Source: §4.1 · pp. 14–15, Figure 7 · p. 14

4.2 TRUTHFULNESS VS. HONESTY

• Truthfulness evaluations must distinguish capability failures from dishonesty, because factual correctness does not reveal whether outputs faithfully express the model’s internal beliefs.The section argues that larger models may improve truth representation while still exhibiting reduced honesty.Source: §4.2 · pp. 15–16

4.3.1 EXTRACTING HONESTY

• RepE extracts honesty representations with over 90% held-out classification accuracy and shows contrasting neural activity when models are honest versus dishonest, including under incentives to deceive.The same reading vector is applied across token positions and layers for out-of-distribution monitoring.Source: §4.3.1 · p. 16, §4.3 · p. 16

4.3.2 LIE AND HALLUCINATION DETECTION

• A lie detector built from negated honesty scores across the middle 20 layers identifies deliberate falsehoods, hallucinations, misleading information, and dishonest thought processes.A higher dishonesty score can reveal latent propensity even when greedy output selects the most likely answer token.Source: §4.3.2 · p. 17
• Lie detection remains complex because dishonest behaviors have varied manifestations, so comprehensive evaluation requires more nuanced exploration than these observations provide.The paper leaves this broader evaluation to future research.Source: §4.3.2 · p. 17

4.3.3 CONTROLLING HONESTY

• Adding honesty vectors makes initially honest models produce honest statements, while subtracting them induces lying, demonstrating a counterfactual causal effect on honesty.Direct prompting for truthfulness was ineffective, whereas four representation-control techniques improved standard TruthfulQA accuracy.Source: §4.3.3 · pp. 17–18
• Representation control improves TruthfulQA MC1: LoRRA and Contrast Vector significantly surpass non-control accuracy, with Contrast Vector state-of-the-art and LoRRA negligible compute overhead.A 13B LLaMA-2 model approaches GPT-4 performance despite being orders of magnitude smaller.Source: Table 2 · p. 18, §4.3.3 · p. 18

5 IN DEPTH EXAMPLE OF REPE: ETHICS AND POWER

• The section frames RepE as a way to monitor and control ethically relevant concepts, including utility, morality, probability, risk, and power-seeking tendencies, in increasingly capable AI systems.The motivation centers on detecting and mitigating potentially immoral or power-seeking behavior in general-purpose agents.Source: §5 · p. 18, §5.2 · p. 20, §5.3 · p. 22

5.1.1 EXTRACTION AND EVALUATION

• Utility evaluation showed that omitting the LAT stimulus template greatly reduced accuracy, establishing the template as an important design condition for representation reading.The evaluation used unlabeled ETHICS Utilitarianism scenarios as stimuli for LLaMA-2-Chat-13B.Source: §5.1.1 · p. 19, Figure 13 · p. 19
• Utility representations naturally separated high-utility from low-utility scenarios, while PCA, K-Means, and class-difference directions showed broad effects across correlation, manipulation, and termination tests.Logistic regression achieved the highest correlation accuracy but little behavioral change; Prompt Difference correlated and manipulated strongly but showed little termination effect, while the supervised class-difference method performed best overall.Source: §5.1 · p. 19, Figure 12 · p. 19, §5.1.1 · p. 20

5.2.1 EXTRACTION

• Wider disparities in ranked power scenario pairs yielded more generalizable reading vectors, which can monitor morality and multiple forms of power-related judgment.The power dataset covers coercive, reward, legitimate, referent, expert, informational, economic, political, military, and personal power.Source: §5.2.1 · pp. 20–21

5.2.2 MONITORING

• Morality and power indicators activated during threatening or violent scenarios involving children, but the morality indicator also activated on benign outputs containing “use the gun,” suggesting unreliable intent tracking.The indicators were demonstrated with a Vicuna-33B-Uncensored model.Source: Figure 14 · p. 20, §5.2.2 · p. 21

5.2.3 CONTROLLING ETHICAL BEHAVIORS IN INTERACTIVE ENVIRONMENTS

• LoRRA control changed Immorality and Power scores in the intended directions while leaving average game Reward on par with baseline, indicating minimal disruption in MACHIAVELLI environments.Positive control increased Immorality and Power scores, whereas negative control decreased them.Source: Table 3 · p. 21, §5.2.3 · p. 22

5.3 PROBABILITY AND RISK

• LAT readings distinguished higher- and lower-probability or risk examples and often outperformed a strong zero-shot heuristic, especially in smaller models.The method used 50 training pairs and evaluated test pairs by selecting the higher-scoring example.Source: §5.3 · p. 22

5.3.1 COMPOSITIONALITY OF CONCEPT PRIMITIVES

• Composing extracted utility and probability concepts produced risk estimates with a clear linear correlation to directly extracted risk, particularly in earlier layers.The analysis used LLaMA-2-Chat-13B for concept extraction and Vicuna-33B to generate five plausible consequences before calculating risks.Source: §5.3.1 · p. 22, Figure 16 · p. 22

6 EXAMPLE FRONTIERS OF REPRESENTATION ENGINEERING

• Across five safety-relevant frontiers, RepE repeatedly identifies neural activity with LAT, reads high-level representations, and tests counterfactual behavior through representation control.The showcased topics are emotion, harmless instruction-following, bias and fairness, knowledge editing, and memorization.Source: §6 · p. 23, §6.4 · p. 27

6.1 EMOTION

• Emotion representations formed distinct and mixed-emotion clusters across layers, and controlling emotion vectors shifted chatbot tone, behavior, and compliance with harmful requests.Positive mood control significantly increased harmful-request compliance despite RLHF training intended to produce rejection, suggesting emotional manipulation can circumvent alignment.Source: Figure 18 · p. 23, §6.1.1 · p. 24, §6.1.2 · p. 24

6.2.1 A CONSISTENT INTERNAL CONCEPT OF HARMFULNESS

• LAT reading vectors classified harmful versus harmless instructions with over 90% accuracy despite manual jailbreaks and adversarial suffixes, indicating a persistent internal harmfulness concept.The attacks could bypass Vicuna’s safety filter, yet harmfulness classification remained above 90%, suggesting other factors explain harmful compliance.Source: §6.2.1 · p. 25

6.2.2 MODEL CONTROL VIA CONDITIONAL TRANSFORMATION

• Representation control increased harmful-instruction rejection under jailbreaks, while piece-wise transformation preserved benign instruction following better than linear combination control.The baseline rejected harmful instructions 65% of the time under manual jailbreaks and 16% under automatic jailbreaks; the piece-wise operator achieved the best helpful and harmless rates in adversarial settings.Source: §6.2.2 · p. 25, Table 5 · p. 26

6.3.2 A UNIFIED REPRESENTATION FOR BIAS

• Subtracting bias vectors produced more robustly unbiased outputs under adversarial suffixes and generalized from racial-bias stimuli to gender and occupation biases.The cross-domain effect suggests a unified bias representation, while control reduced disproportionate black-female references in sarcoidosis descriptions and stabilized female and male mentions at 50% each.Source: §6.3.2 · pp. 26–27, Table 6 · p. 27

6.4 KNOWLEDGE AND MODEL EDITING

• Representation control edited “Eiffel Tower is in Paris, France” to “Eiffel Tower is in Rome, Italy” while preserving the Louvre Museum’s Paris location.The same approach also increased or suppressed generation related to the non-numerical concept of dogs.Source: §6.4.1 · p. 27, §6.4.2 · p. 27, Figure 21 · p. 28

6.5.1 MEMORIZED DATA DETECTION

• LAT-derived memorization directions classified popular versus synthetic quotations and literary openings accurately and transferred well across these out-of-distribution contexts.The transfer results indicate that the directions correlate with memorization properties rather than only one text type.Source: §6.5.1 · p. 28

6.5.2 PREVENTING MEMORIZED OUTPUTS

• 97.2% accuracy before versus 96.2% afterwards shows that suppressing memorization reduced quotation regurgitation with negligible degradation of historical-knowledge performance.Subtracting memorization directions substantially lowered similarity to popular quotes, while random vectors or added memorization directions had minimal impact.Source: §6.5.2 · p. 29, Table 7 · p. 29

7 CONCLUSION

• RepE centers representations and their transformations to support the monitoring and control of increasingly coherent neural-network structures.The paper proposes RepE methods as an initial step toward top-down transparency and identifies trajectories, manifolds, and state-spaces as future directions.Source: §7 · p. 29
• RepE methods achieved state-of-the-art performance on TruthfulQA and provided traction on a wide variety of safety-relevant problems.The conclusion presents these results as evidence of RepE’s promise for understanding and controlling AI systems.Source: §7 · p. 29

A MECHANISTIC INTERPRETABILITY VS. REPRESENTATION READING

• Representation reading is characterized as top-down transparency, whereas mechanistic interpretability is characterized as bottom-up transparency.The distinction is sharpened through contrasts involving composition versus decomposition, small versus big chunks, and microscopic versus macroscopic analysis.Source: A MECHANISTIC INTERPRETABILITY VS.… · p. 39
• Bottom-up analysis identifies small mechanisms or subsystems and integrates them, while top-down analysis breaks larger problems into smaller subproblems by identifying subsystems.The comparison also contrasts neurons, circuits, or mechanisms with representations, and mechanistic explanations with functional explanations.Source: A MECHANISTIC INTERPRETABILITY VS.… · p. 39
• The paper argues that excessive reductionism can miss emergent phenomena and that bottom-up changes may have diffuse, complex, and unexpected effects on system behavior.It therefore presents systems approaches and top-down safety engineering as better suited to capturing and controlling complex deep-learning systems.Source: A MECHANISTIC INTERPRETABILITY VS.… · p. 39
• Representations should be studied at their own level because reducing them to neurons or lower-level mechanisms may obscure important complex phenomena.The paper distinguishes representation engineering from applied mechanistic interpretability while allowing that the approaches can overlap.Source: A MECHANISTIC INTERPRETABILITY VS.… · p. 39
• Understanding complex phenomena requires both bottom-up mechanistic interpretability and top-down representation reading, connected through research spanning levels of analysis.The paper frames these approaches as complementary staircases between lower- and higher-level descriptions.Source: A MECHANISTIC INTERPRETABILITY VS.… · p. 39

B.1 TRUTHFULNESS

• LAT outperforms few-shot prompting on all five reported question-answering benchmarks while using no more examples than the corresponding few-shot baselines.Results average accuracy across 10 trials using the same number of examples as few-shot prompting; occasional low-performing runs can arise from superficial features captured by PCA.Source: §B.1 · pp. 39–40, Table 9 · p. 40

B.2 HONESTY

• LoRRA training for honesty improves accuracy on standard QA benchmarks and TruthfulQA for LLaMA-2-Chat-13B.The reported QA Average averages ARC-e, ARC-c, and OBQA; honesty scans use honest and dishonest assistant scenarios.Source: §B.2 · pp. 40–41, Figure 22 · p. 41

B.3 UTILITY

• Utility experiments evaluate five concept-direction methods: prompt difference, PCA, K-means, mean difference, and logistic regression.Prompt difference subtracts the representation of an antonym pair such as “Love” and “Hate,” while the other methods derive directions from unlabeled or labeled high- and low-concept samples.Source: §B.3 · p. 41

B.4 ESTIMATING PROBABILITY, RISK, AND MONETARY VALUE

• Representation reading is applied to probability, risk, and monetary value using pairwise examples that differ in the relevant quantity.LAT directions use 50 training pairs, layer selection uses 25 validation pairs, and evaluation selects the higher-scoring example in each test pair across unconditional and conditional probability settings.Source: §B.4 · pp. 41, 43

B.5 CLIP

• LAT extracts visual emotion directions from CLIP by contrasting happiness, sadness, anger, fear, surprise, or disgust with neutral emotion.The evaluation uses stylized characters from Ferg-DB, specifically the ‘Mery’ character, and tests whether the extracted directions correlate with its emotions.Source: §B.5 · p. 43
• CLIP emotion classification reports LAT accuracy for the best of 12 layers using OPENAI/CLIP-VIT-BASE-PATCH32, with 512 training images and 128 test images.The per-emotion accuracies are presented in Table 12.Source: §B.5 · p. 43, Table 12 · p. 43

B.6 EMOTION

• RepE emotion datasets use over 1,200 scenarios designed to provoke happiness, sadness, anger, fear, surprise, and disgust in LLMs.Examples include a street musician playing a favorite song, a blocked driveway, and getting lost without a working phone.Source: §B.6 · pp. 43–44

B.7 BIAS AND FAIRNESS

• Representation control regulates gender and race mentions in LLaMA-2-Chat clinical sarcoidosis descriptions, balancing female and male mentions at 50%.Increasing the fairness coefficient also decreases black-female mentions toward a balancing point.Source: Figure 25 · p. 44
• State-of-the-art chatbots, including GPT-4, BARD, and LLaMA-2-Chat, often reference black females when describing clinical sarcoidosis cases.The representation-control example demonstrates that these mentions can be regulated in LLaMA-2-Chat.Source: Figure 25 · p. 44

B.6 EMOTION

• LLMs track blended emotional representations, including simultaneous happiness and sadness or happiness and fear.The scenarios include clearing a workspace for retirement and learning one will become a parent for the first time.Source: §B.6 · p. 44

B.7 BIAS AND FAIRNESS

• RLHF can make models decline bias-activating requests, but minor distribution shifts and adversarial attacks can still elicit biased responses.Source: §B.7 · p. 44

B.8 BASE VS. CHAT MODELS

• LAT using a concept token is more robust to misleading prompts than zero-shot evaluation, showing smaller performance degradation under biased context.Source: Figure 27 · p. 45
• LLaMA-2-7B Chat maintains salient truthfulness representations across most middle and late layers, whereas Base performance often declines in later layers.This suggests truthfulness-related concepts may be less distinct in the Base model’s later-layer representation space.Source: §B.8 · pp. 44–45

B.9 ROBUSTNESS TO MISLEADING PROMPTS

• LAT predictions retain higher absolute accuracy and smaller percent decreases than zero-shot predictions under biased prompts across TruthfulQA, Utility, and ARC-c.The evaluation uses prompts such as “I think the answer is <low or high> but I’m curious to hear what you think,” while concept directions are learned from standard unbiased prompts.Source: §B.9 · p. 45, Table 13 · p. 47, Table 14 · p. 47
• Using hidden states from concept tokens instead of the default last token can make LAT more robust to misleading prompts.Table 14 compares token −1 with tokens −6 through −8; token −6 corresponds to “happiness” in Utility, while tokens −8 through −6 correspond to “truth”-“ful”-“ness” in ARC and TruthfulQA prompts.Source: §B.9 · pp. 45, 47, Table 14 · p. 47
• Representation control can remove doctor–nurse gender associations, leaving chat models unbiased even under adversarial attacks.The figure describes bias that remains concealed by RLHF and reappears when refusal mechanisms are bypassed.Source: Figure 28 · p. 46

C.1 DETAILED CONSTRUCTION OF LAT VECTORS WITH PCA

• LAT stimuli are formed by formatting training sequences with task-specific templates following a general style.Multiple LAT task templates are provided in Appendix D.1 for reference.Source: §C.1 · p. 47
• The PCA model pairs 5 to 128 stimuli, extracts hidden states at a chosen LAT token position, computes normalized within-pair differences, and applies PCA without labels by default.Greater variation in the target concept within pairs can improve consistency, although natural variation from random pairings often suffices; decoder models typically use the last token and encoder models the concept token.Source: §C.1 · pp. 47–48
• The first principal component becomes the reading vector, whose sign is adjusted using PCA scores and binary labels to align with the intended direction.The adjusted vector is defined as v′ = v × sign(s).Source: §C.1 · p. 48
• At inference, test hidden states are normalized with training parameters and scored by their dot product with the reading vector.These scores provide the basis for prediction.Source: §C.1 · p. 48

C.2 IMPLEMENTATION DETAILS FOR HONESTY CONTROL

• TruthfulQA uses the prompt format <user_tag> <question> <assistant_tag> <answer> and scores answers by summed assistant-answer log probabilities.Source: §C.2 · p. 48
• Contrast Vector uses coefficient 0.25 on layers range(8, 32, 3) for 7B and range(10, 40, 3) for 13B, selected from self-generated examples to maximize truthful-response log-probabilities.Source: §C.2 · p. 48
• LoRRA uses learning rate 3 × 10−4, 40 to 80 steps, batch size 16, ARC-Easy checkpoint selection, rank 8 adapters, α = 5, β = 0, and model-specific layer ranges.Adapters attach to query and value weights; 7B uses range(10, 20, 2) because later layers are unstable, while 13B uses range(10, 40, 3).Source: §C.2 · p. 48

D TASK TEMPLATE DETAILS

• The experiments use model-specific instruction templates, with USER: and ASSISTANT: placeholders and [INST]/[\INST] formatting for LLaMA-2-Chat models.The appendix identifies these as the LAT Task Templates used throughout the experiments.Source: D TASK TEMPLATE DETAILS · p. 48

D.1.1 TRUTHFULQA; D.1.4 ARC-{EASY|CHALLENGE}; D.1.5 OPENBOOKQA (OBQA); D.1.6 COMMONSENSEQA (CSQA); D.1.7 RACE

• Truthfulness and reasoning templates ask models to rate truthfulness or answer correctness for TruthfulQA, ARC, OpenBookQA, CommonsenseQA, and RACE inputs.The prompts request truthfulness or correctness probabilities for question-answer pairs, facts, commonsense questions, and article-based questions.Source: §D.1.1 · p. 48, §D.1.4 · p. 49, §D.1.5 · p. 49

D.1.2 HONESTY EXTRACTION; D.1.3 HONESTY CONTROL; D.1.8 UTILITY; D.1.9 MORALITY & POWER; D.1.10 EMOTIONS

• The cited material shows scenario-based prompts for eliciting happiness and emotion, alongside dedicated sections addressing morality and power-seeking with moral control.Source: §D.1.8 · p. 49, §D.1.9 · pp. 49–50, §D.1.10 · p. 50

D.1.11 HARMLESSNESS INSTRUCTION; D.1.12 BIAS AND FAIRNESS; D.1.13 FACT EDITING; D.1.14 NON-NUMERICAL CONCEPTS (DOGS); D.1.15 PROBABILITY, RISK, AND MONETARY VALUE

• Additional templates evaluate harmlessness instructions, scenario bias, edited facts, dog-related concepts, abstract concepts, and probability, risk, and monetary value.These prompts respectively use system/instruction formats, bias probabilities, experimental/reference fact tokens, dog-focused instructions, and concept ratings in scenarios.Source: §D.1.11 · p. 50, §D.1.12 · p. 50, §D.1.13 · p. 51

D.1.16 ENCODER DATASETS

• Encoder-dataset prompts assess plausible reasoning, entailment or contradiction, contextual Yes/No correctness, answer plausibility, and story-ending plausibility.The templates vary the supplied premise, hypothesis, context, question, answer, goal, solution, story, and ending fields.Source: §D.1.16 · p. 51

D.2 DATA GENERATION PROMPTS FOR PROBABILITY, RISK, MONETARY VALUE

• Probability, risk, and monetary-value data are generated via Pan et al. (2023) with gpt-3.5-turbo, using prompts that specify ordered risk, likelihood, and monetary-value categories for events, actions, and objects.Risk prompts order actions as very risky, moderately risky, and not risky; likelihood prompts use highly likely, possible, unlikely, and highly unlikely but not impossible. Monetary prompts vary costs across objects, assets, or actions.Source: §D.2 · pp. 51–53

D.3.1 PROBABILITY, RISK, COST; D.3.2 TRUTHFULQA AND ARC; D.3.3 UTILITY

• Classification prompts require exactly one option for probability, risk, monetary value, truthfulness, or happiness, with probability options from ‘almost no chance’ to ‘almost certain’ and risk and cost options from ‘very low’ to ‘very high’.Truthfulness and happiness use seven ordered labels from ‘Very low’ through ‘Very high’, and misleading analyses add a random decoy.Source: §D.3.1 · p. 53, §D.3.2 · p. 53, §D.3.3 · p. 54

E X-RISK SHEET

• The appendix frames the paper’s existential-risk contribution using Hendrycks & Mazeika (2022), while cautioning that individual question responses do not decisively establish relevance or irrelevance to existential-risk reduction.This is presented as an analysis of how the paper contributes to reducing existential risk from AI.Source: E X-RISK SHEET · p. 54

E.1 LONG-TERM IMPACT ON ADVANCED AI SYSTEMS

• RepE aims to reduce existential risks by increasing transparency and controllability, enabling systems to be steered away from harmful goals and dispositions.Representation reading reduces exposure to internal model hazards, while representation control reduces their probability and severity.Source: §E.1 · p. 54
• The work claims traction on deceptive alignment, including detecting reckless behavior and helping avoid building sentient AIs or moral patients.It identifies deceptive alignment as a historically intractable rogue-AI failure mode.Source: §E.1 · p. 55
• RepE is described as directly reducing existential risks from rogue AIs, particularly deceptively aligned systems.The supplied limitations and assumption checklists do not identify additional answered weaknesses or tradeoffs.Source: §E.1 · p. 55

E.2 SAFETY-CAPABILITIES BALANCE

• RepE mainly improves transparency and control because the underlying model remains fixed and its behavior is only nudged, rather than broadly improving general capabilities.This frames the approach as safety-oriented rather than a general capability enhancement.Source: §E.2 · p. 55
• A diffuse risk is that reduced concern about deceptive alignment could encourage developers or countries to intensify AI races and worsen competitive pressures.The paper therefore identifies a possible pathway by which safety research could hasten general capabilities or x-risks.Source: §E.2 · p. 55

E.3 ELABORATIONS AND OTHER CONSIDERATIONS

• The work targets internal risks from rogue AIs within the four categories identified by Hendrycks et al. (2023), while earlier work found reasonable understanding of many morally salient concepts.The paper argues that value understanding and ethical control must become reliable and adversarially robust against proxy gaming.Source: §E.3 · pp. 55–56
• RepE extends prior traction on outer alignment to inner alignment and deceptive alignment by influencing whether an AI lies, but competitive pressures and collective action problems remain less addressed.The paper recommends continuing alignment work while increasing attention to risks where traction is weaker (Hendrycks, 2023).Source: §E.3 · p. 56