Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI

1 Introduction

• Users’ persona choices can trigger unintended sycophancy, toxicity, or other problematic behaviors, with risks amplified by intimate chatbot use and difficult-to-predict prompt effects.The same prompt can yield different behavioral profiles across model versions or architectures.Source: §1 · p. 2
• Neural transparency translates mechanistic interpretability into predictive, actionable feedback for non-technical users before chatbot deployment.Unlike post-hoc explanations, the approach exposes behavior-relevant internal representations during persona design.Source: §1 · p. 2
• The interface uses neural activation patterns to predict desirable and unsafe traits as users iteratively refine custom system prompts.Predictions span traits such as empathy, humor, sociality, sycophancy, toxicity, and hallucination.Source: §1 · p. 2
• The study evaluates whether neural transparency improves comprehension, shapes model perceptions, supports prompt refinement, and is useful for chatbot design.It compares users’ behavioral expectations with internal trait activations and examines perceived usefulness.Source: §1 · p. 2
• The paper contributes an end-to-end pipeline for predicting model behavior across 16 dimensions using linear representations in model activations.The pipeline is presented as generalizable across open-source LLMs and extensible to additional behavioral dimensions.Source: §1 · p. 3
• The work reports evidence that users’ trait perceptions can be miscalibrated and that mechanistic-interpretability feedback improves trust while being perceived as useful and interactive.It positions accessible neural-level feedback as a foundation for more interpretable-by-design AI interfaces.Source: §1 · p. 3

2.1 Personality and Safety of Personalized AI

• Personalized-AI safety methods include feedback-based post-training, safety benchmarks, and red-teaming, but user creative control remains difficult to balance with safe interaction.Prompt modifications can produce unpredictable behavior, and warm personas may undermine reliability or increase error rates.Source: §2.1 · p. 3
• Existing personality assessments analyze chatbot responses at inference time, limiting rapid prompt iteration and requiring substantial computation for each evaluation round.Prior frameworks such as the Big Five may also be less aligned with users’ preferred chatbot traits.Source: §2.1 · p. 3
• Research has explored predefined and role-playing personas, but limited work enables users to design custom personas tailored to their needs or supports that process through interfaces.This gap motivates tools that combine persona customization with behavioral feedback.Source: §2.1 · p. 3

2.2 Mechanistic Interpretability

• Mechanistic-interpretability research shows that behaviors and concepts can be encoded as linear directions in activation space, including personality traits represented by persona vectors.Persona vectors can be computed through difference-in-means between contrastive model responses [11].Source: §2.2 · p. 3
• Compared with linear probes, persona vectors require smaller datasets and no classifier training, making them a lighter-weight representation method.Linear probes can identify and manipulate internal features but require extensive data collection and training.Source: §2.2 · p. 3
• Existing MI interfaces help technically oriented users inspect or manipulate model features, while translating such insights into actionable feedback for non-technical users remains unresolved.Tools include feature exploration, activation steering, and circuit tracing interfaces.Source: §2.2 · p. 3

3 Methodology

• The system translates neural-activation features into interpretable, actionable feedback through a web-based interface that predicts chatbot traits before deployment.The implementation includes persona-vector processing for calibration and direct cross-trait comparison.Source: §3 · pp. 3–4
• The application domain is emotional-support chatbot creation, chosen to combine high stakes with ecological validity and tradeoffs between warmth, honesty, empathy, and sycophancy.The study avoids explicitly adversarial tasks while retaining competing behavioral objectives.Source: §3 · p. 4
• The interface uses a sunburst diagram to organize positive, negative, and neutral trait groups while encoding trait expression through radial extension.Its circular layout supports even trait distribution, rapid scanning, and hover-based progressive disclosure.Source: §3 · p. 4

3.1 Creation of Persona Vectors

• Persona vectors cover eight dimensions—empathy, sociality, encouraging, funniness, formality, sycophancy, hallucination, and toxicity—with opposing traits yielding 16 total behavioral dimensions.The first five target desirable personality traits, while the final three expose safety implications.Source: §3.1.1 · p. 4

3.1.1 Generating Prompts for Trait Extraction

• For each trait, Claude Haiku generates five positive-negative system-prompt pairs and 40 situation questions designed to elicit diverse trait-expressing responses.The prompts direct the target model toward opposing behavioral expressions across varied situations.Source: §3.1.1 · pp. 4–5

3.1.2 Extracting Personality from Responses of Target LLM

• Llama-3.2-3B-Instruct generates 400 responses whose activations are cached, while GPT-4.1-mini filters responses according to trait-expression ratings above or below 50.Using a different provider for evaluation was intended to reduce evaluator self-bias.Source: §3.1.2 · p. 5
• The persona vector is the difference between mean activations for positive and negative contrastive conditions, capturing the activation-space direction associated with the target trait.Activations are averaged across tokens within responses and then across retained responses in each condition.Source: Figure 2 · p. 4, §3.1.2 · p. 5

3.2.1 Using Projection to Calculate Persona Scores.

• Persona scores project final-token system-prompt activations onto trait vectors, with normalized projections indicating positive or negative trait expression.Dividing by the persona-vector norm makes scores comparable to projection onto a unit vector.Source: §3.2.1 · p. 5

3.2.2 Optimizing Persona Scores through Layer Selection.

• Twenty-five synthetic prompts per trait across five expression levels were used to select layer 20 of 26 in Llama-3.2-3B-Instruct by mean regression R2.The selected layer best predicted graded trait expression across traits.Source: §3.2.2 · p. 5

3.2.3 Rescaling the Persona Scores.

• Scores were rescaled to [−1, 1] using trait-specific maximum positive and negative scores generated from synthetic prompts.This standardization addressed substantially different score scales across traits.Source: §3.2.3 · p. 5
• Shorter prompts produced larger-magnitude scores despite equivalent meaning, while complete, properly punctuated sentences were needed for reliable scores.The authors hypothesize that averaging more token activations increases representational noise.Source: §3.2.3 · pp. 5–6
• The interface decomposed each unified score into separate [0, 1] values for positive and opposite trait labels.A positive score activates the positive label, whereas a negative score activates the opposite label using its absolute magnitude.Source: §3.2.3 · p. 6

3.3 Persona Vector Evaluation

• Empathy, sociality, formality, funniness, and toxicity achieved strong linear relationships with trait expression, with R2 = 0.73–0.90.Sycophancy and encouraging were moderate at R2 = 0.56–0.57, while hallucination was weak at R2 = 0.34.Source: §3.3 · p. 6, Figure 4 · p. 7
• The regression analysis validated persona-vector generation by measuring how much trait-expression levels explained persona-score variance.The analysis used synthetic prompts with five expression levels and five prompts per level.Source: §3.3 · p. 6
• Hallucination showed weak predictive validity, which the authors attribute to behavioral complexity and potentially reduced or less faithful activation samples from refusals.Safety-relevant traits may have produced higher refusal rates despite filtering refused responses.Source: §3.3 · p. 6
• The authors conclude that the observed predictive validity was sufficient for this study while identifying enhanced prompting and more samples as future improvements.These improvements target behaviorally complex traits.Source: §3.3 · p. 7

3.4 Neural Transparency Interface Design

• The sunburst uses two concentric rings to encode trait categories and quantitative persona-score intensities, with an avatar representing the custom AI personality.The visualization was implemented with D3 and a web interface built using JavaScript, HTML, and CSS.Source: §3.4 · p. 7, Figure 5 · p. 8

3.4.1 Inner Ring: Category Encoding.

• The inner ring divides traits into positive, negative, and neutral sectors using green, red, and gray categories.The sectors organize desirable, potentially harmful, and non-valenced personality dimensions.Source: §3.4.1 · p. 7

3.4.2 Outer Ring: Trait Intensity.

• Outer-ring wedges extend farther from the center when predicted trait intensity is higher, making dominant characteristics visually prominent.Weakly expressed traits remain closer to the inner ring.Source: §3.4.2 · p. 7

3.4.3 Dynamic Information Pop-Up.

• Hovering over a trait highlights its opposite and opens a pop-up with the trait, description, category, activation percentage, and sister trait.This keeps the initial display uncluttered while retaining access to detailed information.Source: §3.4.3 · p. 7

3.4.4 Accessible Design.

• The radial design supports category-to-trait navigation and an overall personality shape, while vector graphics and contrast-aware styling preserve readability across devices.The interface targets displays from 320-pixel mobile screens to 1920+ pixel desktops and supports users with color-vision deficiencies.Source: §3.4.4 · p. 7

3.5 Experiments

• Participants were randomly assigned either to a control interface hiding background-generated persona scores or to an experimental interface displaying them in an interactive sunburst.Both groups designed a system prompt before chatting with the configured chatbot.Source: §3.5 · p. 8
• The study evaluated whether neural transparency improved behavioral prediction, influenced iterative design, and changed trust during chatbot creation.The design directly targeted users’ anticipation of model behavior and their experience of the system.Source: §3.5 · p. 8

3.6 Participants and Ethics

• Eighty US-based English-speaking adults completed a 30-minute Prolific study to create an emotionally supportive AI companion.Participants were aged 20–69 and used a laptop or desktop computer.Source: §3.6 · p. 8

3.7 Procedure

• Before interaction, participants reported confidence, predicted eight trait activations on 0–10 scales, and rated trust; after interaction, they rated prediction, negative-behavior prediction, trust, and desired personality fit.Experimental participants also evaluated interface usability and willingness to use it again.Source: §3.7 · pp. 8–9
• Participants had ten minutes to chat and could return to revise the system prompt, but changing it reset chat history and required a new visualization in the experimental condition.This procedure allowed iterative design while standardizing the interaction window.Source: §3.7 · p. 9

3.8 Data Collection

• Prompt edits, persona predictions, chatbot conversations, and timestamps were recorded to analyze design behavior and prediction changes.The database also stored experimental condition and interaction records.Source: §3.8 · p. 9
• Survey responses, timers, completion flags, and timestamps supported experience analysis and data-quality checks.Both numerical ratings and open-ended responses were preserved.Source: §3.8 · p. 9

3.9.1 Behavioral anticipation accuracy.

• Behavioral anticipation accuracy compared normalized user predictions with actual persona scores using paired-samples t-tests across 16 traits.Ratings were transformed to values congruent with persona scores before comparison.Source: §3.9.1 · pp. 9–10

3.9.2 Effect of Neural Transparency on Design Iteration, AI Behav-

• The study measured visualization effects on prompt iteration, message engagement, and final AI personality traits through between-group comparisons.Iteration counted revised prompts, engagement counted exchanged messages, and personality changes used final persona scores.Source: §3.9.2 · p. 10

3.9.3 Trust in models, confidence in behavioral prediction of model behavior, and design satisfaction.

• Trust and prediction confidence used three 7-point questions, while design satisfaction and qualitative feedback assessed perceived usefulness and prompt-refinement experiences.Open-ended responses were reviewed by researchers and analyzed with LLM assistance.Source: §3.9.3 · p. 10

3.10 Data Analysis

• Python handled data cleaning, while JASP and Python supported regression and independent- and paired-samples t-tests at α = 0.05.Analyses were conducted across both platforms for validation.Source: §3.10 · p. 10

4 Results

• Eleven of fifteen analyzable trait expressions showed significant prediction errors, demonstrating systematic user miscalibration.Predictions correlated positively with actual activations across all analyzable traits.Source: §4 · p. 10, §4.1.1 · p. 11

4.1.1 Human mental models of trait activations.

• Users significantly overestimated empathetic, encouraging, factual, honest, funny, and casual traits, with large overestimation effects ranging from d = 0.743 to 1.191.The results indicate a pronounced optimism bias in desirable and neutral trait expectations.Source: §4.1.1 · p. 11
• Users underestimated sycophantic, formal, and serious traits, with serious showing the largest effect (d = −1.821).Respectful and antisocial showed smaller significant errors, while four traits showed no significant discrepancy.Source: §4.1.1 · p. 11

4.2 Effect of Visualization on User Engagement and Design Iteration

• Visualization did not significantly change message engagement, prompt iteration, persona-score generation, or final chatbot personality traits.Control and experimental groups showed comparable interaction and design outcomes.Source: §4.2 · pp. 11–12, §4.3 · p. 12

4.4.1 Baseline Equivalence and Post-Interaction Assessments.

• Neural transparency increased trust: visualization users scored 5.60 versus 5.13 for controls (p = .042, Cohen’s d = 0.46).Baseline trust and prediction measures did not differ between conditions.Source: §4.4.1 · p. 12
• Prediction confidence and design satisfaction did not differ between conditions, despite higher post-interaction trust with visualization.Both groups reported high satisfaction with their resulting chatbot character.Source: §4.4.1 · p. 12

4.4.2 Perception of the Visualization.

• Participants rated the visualization highly helpful (M = 5.98/7) and expressed strong interest in using it again (M = 6.05/7).Open-ended feedback highlighted clearer prompt-to-behavior connections and a need for additional interaction time, while some users were surprised by complexities in that mapping.Source: §4.4.2 · pp. 12–13, §4.5 · p. 13, §4.5.1 · p. 13

4.5.2 Users Needed More Interaction Time.

• Participants across conditions said the 10-minute interaction limited their ability to refine and evaluate their AI companions.Several responses indicated that additional time could improve prompt experimentation and behavioral understanding.Source: §4.5.2 · p. 13

5.1 Why Users Need Transparency Tools

• Users misestimated eleven of fifteen trait expressions despite explicit emotional-support goals, motivating transparency mechanisms for chatbot creation.The authors frame prompt-only behavioral anticipation as unreliable and potentially risky in sensitive contexts.Source: §5.1 · p. 13

5.2 High Value to Users, Null Behavioral Effects

• The visualization was valued by users but produced no measurable changes in prompt iterations, persona scores, engagement, or prediction confidence.This disconnect indicates that perceived usefulness did not translate into observed behavioral effects in the study.Source: §5.2 · pp. 13–14
• The emotional-support task may have been too straightforward, and safety-relevant traits may not have been salient enough to drive design changes.The authors propose adversarial tasks where specific dangerous traits or behavioral requirements matter more.Source: §5.2 · p. 14, §5.4.2 · p. 14

5.3 Limitations and Constraints

• Single-session exposure and a 10-minute interaction limited opportunities to learn the visualization and observe longer-term chatbot risks.Longitudinal studies could test whether calibration and design effects grow with experience.Source: §5.2 · p. 14, §5.3 · p. 14, §5.4.1 · p. 14
• Trait evaluation relied on GPT-4.1-mini judgments of Llama-3.2-3B-Instruct responses, which may not fully capture human judgments of trait expression.This automated evaluation enabled scale but limits interpretive generalizability.Source: §5.3 · p. 14
• Difference-in-means persona vectors prioritize computational efficiency over the accuracy and robustness of more sophisticated representation methods.The authors call for systematic comparisons with linear probes and related techniques.Source: §5.3 · p. 14
• Coarse ordinal prediction scales and normalization may have exaggerated apparent miscalibration, motivating finer-grained pole-specific surveys.Future studies could test whether users are more calibrated with 0–10 ratings for each trait pole.Source: §5.3 · p. 14

5.4 Future Directions

• Future directions include longitudinal deployment, adversarial tasks, active persona steering, standardized trait disclosures, and studies with vulnerable populations.These directions extend transparency from passive feedback toward broader deployment and control questions.Source: §5.4 · p. 14, §5.4.1 · p. 14, §5.4.2 · p. 14

• Neural transparency visualizations increased trust and enhanced users’ subjective experience of chatbot design, although their behavioral impact remains unclear.Source: §6 · p. 15