Automating the Application of HCI Principles: Skills for On-Demand UI Construction, the Human-AI Space to Think, and the Future of HCI

1 Introduction

• LLMs, task-decomposing agents, and dialogue surfaces make on-demand UI construction feasible, while a conceptual frame for the paradigm remains missing.The established workflow anticipates tasks in advance, leaving users workarounds, rework, or abandoned features when their needs diverge from that anticipation.Source: §1 · p. 1
• The paper proposes a Space to Think in which dialogue supports shared context and task decomposition, with the UI constructed as a workspace rather than a separate artifact.Its contributions also include a framework and research agenda focused on evaluation, predictability, and accessibility of dynamically generated content.Source: §1 · p. 1
• Machine-readable skill.md files let agents apply HCI design knowledge as runtime procedural guidance, making design principles constraints on generation rather than post-hoc audits.The skills are declarative, inspectable, version-controlled, and editable, and can encode heuristics, affordance prescriptions, WCAG criteria, cognitive-load constraints, and mixed-initiative principles.Source: §1 · pp. 1–2

2 Generative UI and LLM-Driven User Interface Construction

• Generative UI uses LLMs to construct task-specific interfaces instead of linear text, with reported cognitive gains on structured, information-dense tasks.The area is characterized by computational co-creation, expanded design-space exploration, representational fluidity, contextual adaptation, and synthesis-over-selection.Source: §2 · p. 2
• Systems including v0, Artifacts, and Canvas support dialogue-driven construction, while Design2Code, UICoder, and retrieval-augmented generation demonstrate improving UI-generation fidelity.Jelly generates interfaces around task-driven data models that users can reshape through natural language and direct manipulation.Source: §2 · p. 2
• The paper connects contemporary conversational systems to mixed-initiative interaction, dialogue-driven planning, discourse theory, and earlier model-based UI generation.Mixed-initiative principles address balancing autonomous assistance and user control under uncertainty.Source: §2 · p. 2

3 On-Demand UI as Cognitive Extension

• A Space to Think is a shared, persistent, structured workspace where a human and AI externalize, manipulate, and refine task representations.Its structure includes sub-goals, hypotheses, evidence, decisions, and tools under construction, and it is operational for the user rather than an ordinary chat log.Source: §3 · p. 3
• Either the human or AI may propose, contest, or revise sub-goals, with uncertainty guiding when the AI should ask rather than act.This is the section's mixed-initiative interaction model.Source: §3 · p. 3
• Each dialogue-identified sub-goal produces a corresponding UI affordance, so the interface mirrors the user's thought and extends the shared context.Using the interface provides feedback to the AI for future concept iterations; dialogue and UI state share a task-driven data model.Source: §3 · p. 3

4 Automating HCI Design Principles

• UI construction can be automated while applying critical design principles procedurally to each artifact, targeting coverage that hand design rarely matches in practice.The section presents HCI principles as constraints on the generator rather than only as evaluations of finished interfaces.Source: §4 · p. 3
• Nielsen's ten heuristics can become UI-generation constraints, with downstream testing verifying whether generated interfaces satisfy them.The paper cites UICoder's compiler and vision-language feedback in a self-training loop as a related approach.Source: §4 · p. 3
• Component-library rules can enforce that controls' visual forms signal their actions, such as buttons appearing pressable and editable fields signaling editability.The proposed design can be validated before reaching the user.Source: §4 · p. 3
• A minimum-cognitive-footprint policy exposes a capability when dialogue first articulates the need, making progressive disclosure part of UI construction.Direct manipulation adds visible objects, rapid reversible incremental actions, and object manipulation in place of command syntax.Source: §4 · p. 3
• The paper proposes treating WCAG success criteria as hard constraints, with automated checks for contrast, focus order, ARIA roles, alt text, keyboard reachability, and screen reading.It aligns this approach with ability-based personalization.Source: §4 · p. 3
• The paper argues that interfaces growing with users' articulated needs support learnability, while dialogue back-links provide provenance and recognition over recall.Each affordance carries a link to the dialogue that justified it.Source: §4 · p. 3

• A skill.md file specifies when a capability applies, its procedural steps and constraints, and how success is verified.The agent loads it on demand when its triggering description matches the task; researchers can author or audit skills without retraining a model.Source: §5 · p. 3
• The proposed library encodes Nielsen heuristics, Norman affordances, WCAG constraints, user-specific accommodations, progressive disclosure, mixed initiative, and direct manipulation.Examples include automated checks for contrast and keyboard reachability, and confirmation, reversibility, audit trails, and confidence signals for actions affecting the world.Source: §5 · pp. 3–4
• HCI researchers can add skills as research advances, while practitioners and organizations can audit, fork, extend, or add domain-specific conventions.The paper frames design patterns, accessibility lint rules, and layout heuristics as runnable deliverables.Source: §5 · p. 4
• Skills are uniform, inspectable, editable, and verifiable: they check every generated component, explain controls, accept updates, and can be compiled into tests.The same skill can be reviewed by regulators, accessibility advocates, or peer reviewers.Source: §5 · p. 4

• Skills can be authored, reviewed, version-controlled, and deployed in hours, while updating model weights takes months.Organizations and individuals can layer skills without forking the underlying model.Source: §6 · p. 4
• The paper frames good HCI as a property of the generative process, with the field's craft becoming runnable, inspectable, and shareable.This conclusion follows the proposal to turn HCI design knowledge into procedural constraints on generation.Source: §6 · p. 4
• Generated interfaces remain difficult to verify, and current user studies and heuristic walkthroughs are costly, slow, and inconclusive on learnability, error resistance, and long-term usability.The paper calls for formal validation criteria, machine-interpretable checks, and predictive models for cognitive load and learnability.Source: §6 · p. 4
• The authors anticipate that HCI skills will become an open ecosystem, with papers shipping runnable artifacts and the unit of knowledge transfer shifting from heuristic checklists to executable skills.They forecast that pre-designed UIs may become rare for heterogeneous tasks and that accessibility may become a default.Source: §6 · p. 4

• The diagram is described as a cycle of six panels.Source: §A.1 · p. 5
• The figure presents six components: Space to Think, shared task model, generation pipeline, automatic HCI-principles enforcement, generated UI, and bidirectional coupling.The text links the shared task model to panels 1 and 3, the generation pipeline and enforcement to panel 5, and the generated UI to panel 6.Source: §A.1 · p. 5
• The overview identifies AI-enabled software engineering tools and a result as additional boxes in the diagram.Source: §A.1 · p. 5

• The dialogue is used to articulate intent, goals, and preferences.Source: §A.2 · p. 5
• In the example, a user requests ETF tracking, active-pick performance, benchmark comparisons, and an alert when SPY drops 3% in a day.Source: §A.2 · p. 5
• The AI's dialogue decomposition organizes the task into goals and sub-goals, constraints and preferences, domain concepts, actions and information needs, and success criteria.Source: §A.2 · p. 5
• The diagram links Panel 1 to Panel 2, the Shared Task Model.Source: §A.2 · p. 5

• The listed tools cover LLM reasoning and planning, task and data modeling, UI generation, tooling and frameworks, and testing and verification.Source: §A.3 · p. 5
• The tools lead into the shared task model.Source: §A.3 · p. 5

A.4 Panel 2. Shared Task Model

• The shared task model is a single source of truth: a structured representation of task and dialogue state covering entities, views, actions, and provenance linked to dialogue turns.Source: §A.4 · p. 5

A.5 Panel 3. Generation Pipeline

• The generation pipeline interprets dialogue, plans the UI and information architecture, selects components and interactions, generates code and styles, and instantiates the interface.Source: §A.5 · p. 5

A.6 Panel 4. Automatic HCI Principles Enforcement

• Design knowledge is applied procedurally through heuristics and WCAG practices, while automated checks and an AI critic verify compliance.Examples include confirming destructive actions, providing recovery and help, and supporting keyboard navigation and focus order.Source: §A.6 · p. 5

A.7 Panel 5. Generated User Interface

• The example generated dashboard combines an ETF watchlist, portfolio-versus-benchmark chart, configurable SPY price-drop alert, and a link explaining the alert's conversational provenance.The illustrated chart ends at +10% for the Portfolio and +5% for the Benchmark.Source: §A.7 · p. 6

A.8 Panel 6. Bi-Directional Coupling

• Users can refine the interface in conversation or interact with it directly, and those UI interactions update the shared task model for subsequent dialogue.Source: §A.8 · p. 6

A.9 Result

• The stated result is a living, malleable interface that grows with the user's thinking, preserves provenance and spatial memory, and aims for good HCI by construction.Source: §A.9 · p. 6