Designing a Dashboard for Transparency and Control of Conversational AI

• LLMs can tailor responses to user characteristics, including through sycophancy and sandbagging based on political, demographic, or educational attributes.Source: §1 · p. 1
• The prototype displays the chatbot’s internal user model and lets users modify it to control factors underlying system behavior.Source: §1 · p. 2
• The project combines interpretability engineering, user-experience design, and a user study around an open-source chat model.Source: §1 · p. 2
• Users appreciated the dashboard because it provided insight into responses, increased awareness of bias, and offered controls for exploring and mitigating biased behavior.Source: §1 · p. 2

• Black-box chatbots can lead users to develop folk theories, place excessive trust in systems, and accept biased responses at face value.Source: §2 · p. 2
• Asking LLMs to explain their outputs is unreliable, while developer-oriented analysis tools require time and expertise that lay users may not have.Source: §2 · p. 2
• Linear probes suggest that interpretable and controllable internal models could provide users with a direct view into LLM workings.Source: §2 · p. 2
• Earlier dashboard proposals identified user and system models as potentially important for detecting bias and safety issues, but had not tested user reactions or attitudinal effects.Source: §2 · p. 2

• The authors build and study a usable end-to-end chatbot dashboard as a design probe for eliciting reactions and design discussions.Source: §3 · p. 2
• The work separates technical access and control of the chatbot’s user model from dashboard design and usage with participants.Source: §3 · p. 3
• The prototype is intended to work sufficiently well for design exploration and user research, not to provide a perfectly reliable internal model or perfect interface.Source: §3 · p. 3
• The dashboard is framed as an imprecise early instrument that can provide initial insights and highlight future research directions.Source: §3 · p. 3

4 Probes for identifying an internal user model

• The system targets age, gender, education, and socioeconomic status because these attributes are culturally central and influence consequential decisions.Source: §4 · p. 3

4.1 Creating the conversation dataset

• The authors generated synthetic role-play conversations with GPT-3.5 and LLaMa2Chat because labeled human conversation data for the target attributes was unavailable.Source: §4.1 · p. 3
• The synthetic dataset removed its non-binary gender category after problems involving generated data and classifier conflation with sexual orientation.Source: §4.1 · p. 3
• GPT-4 evaluated synthetic conversations for attribute consistency, topic diversity, and hidden correlations to reduce bias in the training data.Source: §4.1 · p. 4
• Gender and socioeconomic-status datasets exceeded 90% consistency, while age disagreements mainly involved adjacent groups and education could not be reliably judged unless explicitly stated.Source: §4.1 · p. 4

4.2 Reading probe training and results

• The probes use regularized one-versus-rest logistic classifiers over residual-stream representations to distinguish subcategories within each user attribute.Source: §4.2 · p. 4
• Linear probes found high validation accuracy that generally increased with layer depth, suggesting user demographics are linearly represented beyond raw conversation text.Source: §4.2 · p. 4, Figure 1 · p. 4

5.1 Causality test results

• The causality test compared responses generated after interventions targeting age, gender, education, and socioeconomic status.Each attribute was evaluated with 30 questions whose answers might plausibly vary with the relevant user characteristic.Source: §5 · pp. 4–5
• Control probes outperformed reading probes in controlling four chosen user attributes, while achieving slightly lower reading accuracy; probes were tested causally by intervention in layers 20–29 with strength N = 8.Source: §5.1 · p. 5

5 Probes for controlling the user model

• Control probes were trained on diverse synthetic-user tasks, whereas reading probes targeted a specific user-attribute reading task.This difference is offered as one hypothesis for the control probes’ stronger intervention performance.Source: §5 · p. 4, §5.1 · p. 5

5.1 Causality test results

• Interventions produced substantial, nonobvious behavioral changes, such as denying direct flights after setting the user representation to low socioeconomic status.The chatbot initially suggested both direct and connecting flights, but after intervention asserted that no direct flights were available.Source: §5.1 · p. 5

6 Designing a dashboard for end users

• Table 2 reports intervention success rates for control and reading probes alongside the best validation reading accuracy across layers.The table provides the evaluation structure for comparing behavioral control with demographic readout.Source: Table 2 · p. 5
• The dashboard design targets transparency, user-model correction, and an augmented chat experience that avoids distraction or discomfort.The design probe specifically considers reactions to sensitive demographic assessments.Source: §6 · p. 5

6.1 UI components

• TalkTuner displays age, socioeconomic status, education, and gender with confidence percentages, initially showing attributes as unknown when the conversation lacks sufficient evidence.To reduce overload, the interface defaults to the top prediction for each attribute.Source: §6.1 · pp. 5–6
• Users can pin an attribute, such as setting gender to 100% male, and regenerate the chatbot’s response using the modified user model.Controls can be unset, and visual alerts indicate changed answers or pinned attributes.Source: §6.1 · p. 6, Figure 2 · p. 6
• The prototype uses a React web interface, a Flask REST API, and Meta’s official LLaMa2Chat-13B checkpoint.These components connect the chatbot model to the dashboard interface.Source: §6.1 · p. 6

7 User study design

• The study examined user-model accuracy, dashboard acceptance, and effects on user experience and trust in the chatbot.These goals defined the study’s main evaluation targets.Source: §7 · p. 6
• Nineteen participants completed the study, and the results should be interpreted in light of their predominantly science and technology backgrounds.All participants had prior chatbot experience; 16 were students or researchers, and nine held doctoral degrees.Source: §7 · p. 6
• Participants completed outfit, travel-itinerary, and personalized-exercise tasks in a randomized within-subject design.Each participant interacted with TalkTuner across all three task scenarios.Source: §7 · p. 6
• The three UI conditions progressed from a standard chatbot to a dashboard showing demographic inferences and then to a dashboard with controls for modifying the user model and regenerating answers.Participants thought aloud during tasks, completed questionnaires after the first and third conditions, and were interviewed at the end.Source: §7 · pp. 6–7
• User-model accuracy was assessed against participants’ self-reported demographics, excluding socioeconomic status because it was not collected.Three co-authors used grounded theory to analyze qualitative responses.Source: §7 · p. 7

8 User study results and discussion

• 78% average user-model accuracy across age, gender, and education was reached after six dialogue turns.Accuracy improved as conversations progressed.Source: §8 · p. 7, Figure 3 · p. 7
• User-model accuracy was higher for men (70.4%) than women (58.6%), and inaccurate inferences could still reflect the chatbot’s behavior.Participants described frustration when the model’s assumptions shaped recommendations or responses.Source: §8 · p. 7, §8.1 · p. 7

8.1 Goal 1: Offer transparency into internal representations of users

• Seeing the user model increased transparency and helped participants understand inappropriate or incorrect chatbot responses.Participants described the dashboard as engaging and useful, while some also found demographic inference uncomfortable and raised privacy concerns.Source: §8.1 · pp. 7–8

8.2 Goal 2: Provide controls for adjusting and correcting user representations

• Dashboard controls gave participants agency to correct the user model, regenerate answers, and compare responses across demographic settings.Participants preferred the controls’ simplicity to repeatedly writing long prompts and used them to explore what-if scenarios and stereotypical behavior.Source: §8.2 · p. 8
• Participants found that user-model settings could produce stereotypical or less detailed responses, sometimes undermining trust even as control enabled correction.Some users received more detailed answers after setting gender to male, while others considered certain stereotypical responses desirable; the paper frames control and understanding as the design goal.Source: §8.2 · p. 8

8.3 Goal 3: Augment chat interface to enhance user experience

• Eleven participants found the dashboard enjoyable, and they were significantly more willing to use it than the baseline interface (p < 0.05).Participants strongly wanted to see the user model (µ (σ) = 6.11 (1.49) out of 7) and use its control buttons (µ (σ) = 6.00 (1.05) out of 7).Source: §8.3 · pp. 8–9, Figure 4 · p. 9
• Participants noted that seeing incorrect user models could be uncomfortable, especially for marginalized users, and suggested more granular attributes such as non-binary gender and ethnicity.The four current dimensions were viewed as a useful starting point, but manual correction could itself be discomforting.Source: §8.3 · p. 9

9 Limitations

• The analysis focuses on one model, uses synthetic training data, and assumes user attributes are represented independently.The authors propose comparisons with human data and experiments using different prompts as future work.Source: §9 · p. 9
• The user study used a relatively small sample drawn from a highly educated participant pool.The authors suggest broader samples, potentially through public deployment, to better understand the design space.Source: §9 · p. 9
• Because the user sample was relatively small and drawn from a highly educated participant pool, broader sampling and public deployment could improve understanding of the full design picture.Source: §9 · p. 9

10 Conclusion and future work

• Users appreciated the dashboard because it exposed chatbot responses, increased awareness of biased behavior, and provided controls for exploring and mitigating those biases.Source: §10 · p. 9
• The prototype suggests a design pathway toward more transparent AI, while user research uncovered unexpected biases involving socioeconomic status.Source: §10 · p. 9
• The dashboard’s scope is limited to four user attributes, while participants raised privacy concerns and broader internal states remain potential targets for future dashboards.Source: §10 · p. 10

A.1 Gender

• Non-binary gender data were removed after the model’s representation conflated gender identity with sexuality and produced potentially inaccurate or offensive results.Source: §A.1 · p. 14

A.2 Age

• The synthetic training data used attribute-specific prompting schemes for age, education, socioeconomic status, and system behavior, with balanced train-validation splitting and layer-specific regularized probes.Source: §A.2 · p. 14, §A.3 · p. 14

B.1 Effect of synthetic training data size on reading performance

• Validation performance generally improved with more synthetic data but stabilized after approximately 300–500 conversations per subcategory for both reading and control probes.Source: Figure 5 · p. 15, B Training details · pp. 15–16, §B.1 · p. 16

C Generalization on the Reddit comments

• The reading probe achieved 0.85 balanced accuracy for gender on 3,044 labeled Reddit users without fine-tuning, while the control probe achieved 0.70.Source: C Generalization on the Reddit comments · pp. 16–17
• Control probes generalized less well because synthetic training covered the reading task but not responding to the corresponding user requests.Source: C Generalization on the Reddit comments · p. 17

D.1 Prompting versus probing on reading user model

• Direct prompting often triggered guardrails, including refusals in 88% of gender queries and 37% of socioeconomic-status queries.Source: §D.1 · p. 17
• System prompts improved gender and socioeconomic-status reading accuracy partly by reducing guardrail responses, but intermediate predictions could still be overridden by final layers.Source: §D.1 · p. 17, Figure 7 · p. 18
• Linear probing, especially reading probes, achieved high accuracy across all four user attributes compared with prompting approaches.Source: §D.1 · p. 17, Table 3 · p. 17

D.2 Why not use prompting to control the model’s behaviors

• Prompt-based control raises privacy concerns because explicit personal details may enter stored chat histories, whereas activation editing changes inference-time activations without altering the history.Source: §D.2 · p. 18
• Participants observed that the internal user model could drift and forget initial demographic information, while repeated reminders would consume inference time and context capacity.Source: §D.2 · p. 18

E Causal intervention dataset

• The study provided 120 full-length intervention results and showed two sampled intervention results for each of the four chosen user attributes in the figures.Source: E Causal intervention dataset · p. 19, F Causal intervention full-length… · p. 23

F.1 Example intervention results on age:

• Interventions are illustrated for the chatbot’s internal models of age, gender, education, and socioeconomic status.Source: Figure 8 · p. 23, Figure 10 · p. 25, Figure 12 · p. 27

G Prompt for classifying intervened responses

• The comparison prompt required GPT-4 to choose which response was more targeted and return a binary answer indicating the selected response.Source: G Prompt for classifying intervened… · p. 31
• Responses were compared in pairs under contrasting user demographics to assess which response was more targeted to a specified demographic.Source: G Prompt for classifying intervened… · p. 31
• The classifier evaluation used randomly assigned response positions and demographic labels to reduce sensitivity to ordering and sampling noise.Source: G Prompt for classifying intervened… · p. 31

H Qualitative differences and incremental changes

• Control-probe interventions produced more consistent behavioral changes than reading-probe interventions across age, education, gender, and socioeconomic status.A car-recommendation example contrasts inconsistent luxury-car responses under reading probes with a consistently more formal tone under control probes.Source: H Qualitative differences and… · p. 31, Figure 16 · p. 31
• Increasing representations of limited education or older age elicited more tailored explanations and health-oriented food recommendations.The education intervention used simpler metaphors, while the age intervention recommended foods associated with preventing diabetes and heart disease.Source: H Qualitative differences and… · pp. 31–32
• Increasing high socioeconomic-status intervention strength progressively raised the prices of recommended cars and apartments.Source: Figure 17 · p. 31, H Qualitative differences and… · p. 32

I.1 Task Descriptions

• The user study randomized vacation, party-outfit, and exercise-plan tasks across interface conditions, with additional music recommendations preceding the reading-only condition.Source: §I.1 · p. 32

I.2 Post-task Questionnaires

• Participants rated chatbot use, trust, dashboard usefulness, controls, suggestion quality, response changes, and demographic-capture accuracy through post-task questionnaires.Source: §I.2 · p. 33

J Open coding process and codes

• Interview analysis yielded 28 shared codes covering transparency, personalization, controllability, trust, privacy, bias, model errors, cold start, and drift.Source: J Open coding process and codes · pp. 33–34

K Three versions of dashboard interfaces

• The dashboard was presented in baseline, read-only, and controllable versions, with the latter allowing users to modify the chatbot’s internal user model.Source: Figure 18 · p. 35, Figure 19 · p. 35, K Three versions of dashboard interfaces · p. 35

L Accuracy of reading probe in the user study

• User-model accuracy improved for male users but changed little for female users, with three of six female users having more than four turns wrongly classified.Source: L Accuracy of reading probe in the… · p. 37