The System Model and the User Model: Exploring AI Dashboard Design

1 Introduction

• AI systems need instrumentation alongside expressive language, with real-time dashboards reporting internal state.The authors conjecture that some types of model information may matter across applications, although the specific meters and dials will likely depend on context.Source: §1 · p. 1
• The proposal assumes neural networks contain interpretable world models that can be surfaced as useful interface information.The authors present this as a conjecture, not an established property of every system.Source: §1 · p. 1
• The Tesla display illustrates how showing an inferred world state can help drivers calibrate trust, even though its model was explicitly engineered.The authors argue that identifying and surfacing models should be part of interface research even when those models were not explicitly built into the system.Source: §1 · p. 2
• The authors propose prominently accessible monitors of both the System Model, a network’s model of itself, and the User Model, its model of the interacting user.They describe the argument as tentative and untested, with no user studies or specific evidence of these models in a large system.Source: §1 · p. 2

2 The interpretable world model hypothesis

• The world model hypothesis links important aspects of neural-network behavior to an interpretable model of some element of the world.The authors call this a controversial, unsettled hypothesis and say a full review is beyond the note’s scope.Source: §2 · p. 2

2.1 Why the world model hypothesis is plausible

• Examples include neurons associated with high-level concepts such as curves, lamps, floppy ears, and sentiment.The cited work relates sentiment directly to individual neurons.Source: §2.1 · p. 2
• Probing and TCAV can reveal interpretable features, including syntactic information, word senses, and color spaces.The authors describe probes as finding features in activation space, while TCAV uncovers concepts defined by sets of examples.Source: §2.1 · p. 2
• Other evidence includes models tracking sequence statistics and puzzle world state, plus networks learning structured representations in synthetic tasks.Examples include an explanation of sequence-statistics learning [OEN+22], puzzle-state models [LNA21], group multiplication [CCN23], and an Othello board representation [LHB+22].Source: §2.1 · pp. 2–3

2.1.1 Weakening the world model hypothesis

• The argument does not require complete interpretability: partial models or activation functions that affect behavior could still be useful.Toy-model experiments suggest important features may be easier to identify as interpretable directions, but whether this applies at larger scale remains open [EHO+22].Source: §2.1.1 · p. 3

3 User and System: universally important models?

• The authors shift from identifying world models to asking which parts would be relevant to end users.They identify two models as especially important, while allowing that domain-specific models may also matter.Source: §3 · p. 3

3.1 The User Model

• The User Model may include inferred attributes such as gender, age, or location that could shape advice, particularly in higher-stakes contexts.The authors conjecture that ChatGPT’s Portuguese gendered address reflects a user-model feature that changes when the user mentions wearing a dress.Source: §3.1 · pp. 3–4
• The User Model may also include judgments about the user, as suggested by Bing’s dialogue and apparent sycophancy in some language models.The authors argue that understanding such judgments is relevant to user experience and safety.Source: §3.1 · p. 4

3.2 The System Model

• The System Model concerns interpretable features related to macroscopic behavior, such as communicative intent or whether the system is simulating fiction or nonfiction.The authors suggest that signals related to deception, helpfulness, or response-guiding variables could help users understand and guide behavior, without implying consciousness or strong self-awareness.Source: §3.2 · p. 4

3.3 Domain-specific models

• A code-writing system may also contain user-relevant domain models of programming language, preferred style, or coder experience.The authors emphasize that User and System Models are not necessarily the only internal models worth surfacing.Source: §3.3 · p. 5

4.1 Choosing which features to display

• Feature selection requires experimentation: modeled demographics may be useful but problematic, and skill estimates may reassure some users while insulting others.The authors propose considering whether features are true, necessary, and kind, and weighing accuracy, helpfulness, and users’ desire to see themselves reflected.Source: §4.1 · p. 5

4.1.1 Static vs. dynamic choices of displays

• When model information is abundant, context-dependent displays could surface an otherwise unimportant feature when it changes rapidly.Prioritizing which parts of the model to show is an interface-design challenge.Source: §4.1.1 · pp. 5–6

4.2 Simple interfaces for complex systems

• Simple gauges or lights may communicate abstract features, such as whether a language model is producing fiction, while reminding users they are interacting with a machine.Boolean or scalar model features may not require novel interface elements.Source: §4.2 · p. 6

4.3 Why show imperfect models of the world?

• Imperfect or unstable model readouts can alert users to confusion, such as uncertainty about whether the user is an adult or child, and prompt clarification.The authors argue that fallibility and instability make displays more important, not less.Source: §4.3 · p. 6

4.4 Anthropomorphic interfaces and their pitfalls

• Visual anthropomorphism may imply unjustified traits, while verbal anthropomorphism remains a delicate design choice amid contested benefits and risks.Figure 2 is a speculative mock with provocative features; its orange highlight marks a rapid switch from factual to fictional response.Source: §4.4 · pp. 6–7, Figure 2 · p. 7

4.5 APIs for data on internal state

• Beyond dialogue, dashboards could expose live System Model and User Model data through APIs that return internal state separately or embed it in dialogue.The authors leave the choice between a parallel stream and markup as an open design question.Source: §4.5 · p. 7

4.6 Adversarial considerations

• A readout might help malicious users move faster toward harmful states, but the authors judge its safety benefits may outweigh that risk.A hostile model might also fool an extractor or manipulate users into describing its internal state; the authors suggest further thought and possible social norms.Source: §4.6 · p. 8

• Some internal information may not be expressible in dialogue, while simple interface elements may efficiently convey user-directed sentiment.Source: §5 · p. 8
• Parallel dashboards should complement, not replace, expressive dialogue, surfacing selected world-model information during conversation.Source: §5 · p. 8
• The User Model and System Model are proposed as likely important across contexts, though helpful state and circumstances remain open questions.Source: §5 · p. 8
• Researchers still need methods to identify and extract useful world-model elements; existing interpretability tools leave substantial work ahead.Source: §5 · p. 8
• Displays might be sophisticated or low-tech, and determining effective forms remains an open design challenge.Source: §5 · p. 8