PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Interfaces for Language Models

Jiaqi Chen, Yanzhe Zhang, Yutong Zhang, Yijia Shao, Diyi Yang

arXiv:2508.19227v3cs.CLcs.AIcs.HC

TL;DR

Conversational interfaces remain static and linear, while Generative Interfaces create adaptive, goal-driven interactions by generating task-specific UIs. The paper combines structured representations and iterative refinement with systematic evaluation, finding advantages over conversational approaches, particularly for structured and information-dense tasks.

  • Problem

    Conversational UIs remain static and linear, and the paper asks how LLMs can support adaptive, goal-driven interactions that serve human needs.

    Source: §1 · p. 1

  • Method

    The paper introduces Generative Interfaces that dynamically generate task-specific UIs using structured representations and iterative refinement, alongside a systematic evaluation framework.

    Source: §1 · pp. 1–2

  • Results

    Generative interfaces show advantages over conversational approaches across diverse query types and interaction patterns, particularly in structured, information-dense tasks.

    Source: §1 · pp. 2–3, §6 · p. 9

  • Takeaways & Limitations

    Users prefer generative interfaces for their visual organization, interactivity, and reduced cognitive load, while conversational formats remain competitive in some cases.

    Source: §1 · p. 2, §6 · p. 9

    The system supports only HTML/JavaScript frontends without backend logic, restricting the complexity of generated interfaces.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) are increasingly seen as assistants, copilots, and consultants, capable of supporting a wide range of tasks through natural conversation. However, most systems remain constrained by a linear request-response format that often makes interactions inefficient in multi-turn, information-dense, and exploratory tasks. To address these limitations, we propose Generative Interfaces for Language Models, a paradigm in which LLMs respond to user queries by proactively generating user interfaces (UIs) that enable more adaptive and interactive engagement. Our framework leverages structured interface-specific representations and iterative refinements to translate user queries into task-specific UIs. For systematic evaluation, we introduce a multidimensional assessment framework that compares generative interfaces with traditional chat-based ones across diverse tasks, interaction patterns, and query types, capturing functional, interactive, and emotional aspects of user experience. Results show that generative interfaces consistently outperform conversational ones, with up to a 72% improvement in human preference. These findings clarify when and why users favor generative interfaces, paving the way for future advancements in human-AI interaction. Data and code are available at https://github.com/SALT-NLP/GenUI.

1 Introduction

Conversational UIs remain static and linear, while Generative Interfaces dynamically create task-specific interfaces for adaptive, goal-driven interactions. The paper introduces infrastructure and evaluation methods for these interfaces and reports advantages over conversational approaches across diverse query types and interaction patterns.

1 Introduction

  • Conversational UIs remain static and linear, limiting support for the diverse ways users learn, explore, and interact.Source: §1 · p. 1
  • Generative Interfaces dynamically create interface structures tailored to users’ goals and interaction requirements rather than returning static text.Examples include an interactive neural-network animation or piano-practice tool with real-time feedback.Source: §1 · p. 1
  • The framework combines structured interface-specific representations and iterative refinement with a systematic comparison of generative and conversational interfaces.The evaluation framework assesses functionality, interactivity, and emotional perception.Source: §1 · p. 2
  • Generative interfaces show advantages across diverse query types and interaction patterns, especially in structured, information-dense domains.The reported evaluation also identifies visual organization, interactivity, and reduced cognitive load as reasons users prefer them.Source: §1 · pp. 2–3

2 Generative Interface for LMs

The system turns user queries into structured representations that guide interface generation, then uses query-specific evaluation and iterative refinement to improve candidate UIs. Its design spans interaction-flow and component-level behavior modeling, a multistep generation pipeline, and adaptive reward-driven refinement.

2.1 Structured Interface-specific Representation

  • The structured representation models interaction flows as user trajectories and finite state machines as component behaviors and UI logic.The flow is a directed graph of views or subgoals and transitions; each component FSM specifies state changes in response to user events.Source: §2.1 · p. 3

2.2 Generation pipeline

  • The generation pipeline converts a query into a requirement specification, a structured representation, and executable HTML/CSS/JS code.It supplies reusable UI components and retrieved examples or data sources as context for code synthesis.Source: §2.2 · p. 3

2.3 Iterative UI Refinement

  • The iterative refinement procedure uses an LLM to build a reward function tailored to each user query, generating fine-grained evaluation metrics with associated weights and verification rules to score candidates.Source: §2.3 · p. 3
  • Refinement regenerates interfaces from the highest-scoring candidate and its evaluation until the score reaches 90 or five iterations are complete.The feedback loop targets structural, semantic, or visual-design issues.Source: §2.3 · p. 4, Figure 2 · p. 4

3 Evaluation Framework

The evaluation framework combines a diverse query suite, multidimensional measures of user perception, and pairwise human judgments of interface outputs. It uses both broad prompt coverage and trained annotators to assess interfaces across functional, interactive, and emotional dimensions.

3 Evaluation Framework

  • The evaluation framework combines a diverse user-query suite, multidimensional metrics, and an integrated human study.Source: §3 · p. 4

3.1 User Queries

  • UIX contains 100 queries spanning ten domains, concise and detailed prompts, and conversational and task-oriented query types.The prompts were generated using Claude 3.7 and designed to cover varied specificity and complexity.Source: §3.1 · p. 4

3.2 Evaluation Metrics

  • The metrics cover functional, interactive, and emotional perception through seven dimensions, including query consistency, task efficiency, usability, learnability, information clarity, aesthetic appeal, and satisfaction.For each query, annotators compare two interfaces across the seven dimensions and overall preference.Source: §3.2 · pp. 4–5

3.3 Human Evaluation

  • Each evaluation instance was judged by three annotators using pairwise choices, with majority voting and moderate inter-annotator agreement (Fleiss’ Kappa 0.525).The study involved 428 annotators who were native English speakers in the United States and regular AI-chatbot users.Source: §3.3 · p. 5

4 Experimental Results

GenUI outperforms conversational interfaces overall, with advantages varying by task and domain. Ablations favor structured representations, iterative refinement, and adaptive rewards, while user comments and real-user queries provide additional evidence about preferences.

4.1 Main Results and Findings

  • GenUI achieves an 84.0% win rate over ConvUI (Claude 3.7), with emotional appeal and interactive functionality identified as primary drivers of its advantage.The comparison spans functional, interactive, and emotional dimensions.Source: §4.1 · p. 5, Table 1 · p. 6
  • GenUI preference is strongest in Data Analysis & Visualization (93.8%) and Business Strategy & Operations (87.5%), while Advanced AI/ML Applications receives 50.0% preference.Users also favor GenUI for interactive tasks (80.0%) and detailed queries (80.0%), compared with 73.0% for concise queries.Source: §4.1 · p. 6

4.2 Ablation Study

  • The full framework outperforms direct-instruction UI generation by a 58.0% higher win rate; structured representations raise overall win rate from 13% to 17% versus natural-language representations.Iterative refinement adds +14.0% overall win rate versus one-shot generation, while removing adaptive rewards drops overall win rate by 17.0%.Source: §4.2 · pp. 6–7

4.3 Human Preference Analysis

  • Among users mentioning Cognitive Load & Intuition, 78.5% preferred GenUI, while comments linked its modular layouts and visual hierarchy to more organized presentation.ConvUI was preferred in some easier scenarios where added tools could impose unnecessary cognitive load.Source: §4.3 · p. 8

4.4 Real-User Query Evaluation

  • Across 380 self-authored queries from 76 participants, GenUI received 50.8% wins, 8.2% ties, and 41.1% losses against ConvUI.30.3% of participants strongly preferred GenUI, compared with 18.4% who strongly preferred ConvUI.Source: §4.4 · p. 8

5 Related work

Prior adaptive interfaces often relied on predefined states, while newer LLM-based systems can generate interface elements from prompts. This work distinguishes itself by generating complete interfaces for diverse queries and using interaction flows and finite state machines to model their behavior.

Context-Aware and Adaptive Interface

  • Earlier adaptive systems used predefined states and faced scalability, predictability, and user-control challenges; LLMs enabled prompt-driven interface generation.Prior LLM-based work often combined text with predesigned UIs or addressed narrower scenarios, such as clarification, chart editing, or modifying fixed components.Source: §5 · pp. 8–9

Prior LLM Interfaces

  • Unlike systems using predesigned UIs or modifying fixed components, this framework generates complete interfaces customized to diverse user queries.The related work also notes that the framework requires no UI specifications from users.Source: §5 · p. 9

Automatic UI Generation

  • Prior UI-generation methods span vision-based reverse engineering, neural synthesis, and LLM generation from natural-language descriptions, screenshots, and sketches.Some approaches use iterative LLM feedback or intermediate representations to control code generation.Source: §5 · p. 9

Intermediate Representations

  • The framework models reactions to user actions and interface evolution with interaction flows and finite state machines.Related approaches include graph-based representations, UI grammars, and data-schema-driven specifications.Source: §5 · p. 9

6 Conclusion

The paper introduces Generative Interfaces, in which LLMs generate adaptive, interactive interfaces to support complex user goals. It reports advantages over conversational approaches, especially for structured and information-dense tasks, while noting conversational formats remain competitive in some contexts.

Conclusion

  • Generative Interfaces are a paradigm in which LLMs proactively generate adaptive, interactive interfaces to support complex user goals.The paper's future directions include multimodal input, domain-specific templates, and collaborative multi-user environments.Source: §6 · p. 9
  • The evaluation finds clear advantages over traditional conversational approaches, particularly for structured and information-dense tasks.The findings also identify contexts where conversational formats remain competitive.Source: §6 · p. 9

Limitations

The system's supported frontend scope, refinement latency, and decision to generate interfaces for every query constrain its use.

Limitations

  • HTML/JavaScript frontends without backend logic restrict the complexity of generated interfaces.More complex tasks may require representations beyond interaction flows and finite state machines.Source: Limitations · p. 9
  • Iterative refinement can add latency of up to several minutes, which may be undesirable in real-time settings.The paper notes that model-efficiency and infrastructure advances could help mitigate this latency.Source: Limitations · p. 9
  • The system generates interfaces for all queries, including those where interaction is unnecessary.The authors suggest a classifier could determine when interaction is needed and selectively invoke the system.Source: Limitations · p. 9

Ethical Considerations

The paper identifies ethical risks from generated interfaces and says transparency and oversight are needed to keep them usable and trustworthy. Its prompt suite spans practical domains and varies prompts by detail and interaction type.

Ethical Considerations

  • Generative interfaces may create accessibility barriers, shape decisions through biased framing, and increase trust in outputs that may contain misinformation.The paper calls for improved transparency and oversight to keep interfaces usable and trustworthy.Source: Ethical Considerations · pp. 9–10

A Prompt Suite

  • The prompt suite covers ten practical domains and organizes queries into four quadrants by detail level and whether they are general or interactive.Examples range from concise general learning queries to detailed interactive website-building requests.Source: A Prompt Suite · p. 12

B LLM Evaluation

The section describes LLM-based evaluation of interfaces and contrasts natural-language descriptions with a structured representation. The structured approach separates task-level interaction flows from component-level behavior.

B LLM Evaluation

  • LLM scores are compared with human pairwise annotations, yielding 69.0% agreement, though length bias may favor GenUI.Claude 3.7 receives UI code and screenshots and assigns 0–100 scores across evaluation dimensions.Source: B LLM Evaluation · p. 13

C Representation: Natural language vs. Structured

  • Natural-language representation describes UI requirements without constraining interface states or their transitions.It expresses goals, salient features, technical requirements, and user preferences through descriptive fields.Source: C Representation: Natural language vs.… · p. 13

D Adaptive Reward Function

The adaptive reward function tailors evaluation metrics to each query, while supplementary examples illustrate generated interfaces and their refinement. The section also describes questionnaire materials and procedures for filtering human annotations.

D Adaptive Reward Function

  • Query-specific reward criteria can capture intent that generic static rewards miss, as shown by adaptive and static quantum-physics visualizations.The adaptive criterion checks whether interactive models demonstrate phenomena such as wave-particle duality; the static version produces incoherent particle clusters without proper interference visualization.Source: D Adaptive Reward Function · pp. 15–16

E Supplementary Examples

  • Figure 7 reports that GenUI’s layout improves perceived clarity, trustworthiness, and professionalism.The section also presents a Business Strategy & Operations comparison and a continuous-integration dashboard refined for usability and clarity.Source: E Supplementary Examples · p. 16

F Human Evaluation Questionnaire Interface

G Human Annotation Filtering

  • Trap questions, consistency checks, and manual review remove inattentive or inconsistent annotations and low-agreement responses judged careless.The authors report that this filtering retained attentive, internally consistent annotations.Source: G Human Annotation Filtering · pp. 16–17

H Real-User Query Evaluation

The real-user evaluation collects participants’ chatbot queries and asks them to compare generated and conversational interfaces tailored to those queries. A supplementary example reports higher perceived clarity, trustworthiness, and professionalism for GenUI’s layout.

H Real-User Query Evaluation

  • Participants submitted five chatbot queries, then compared paired interfaces generated for their own queries and reported an overall preference.Queries were used without post-processing, and Gemini-2.5-flash generated both interface types.Source: H Real-User Query Evaluation · p. 17
  • Figure 6 illustrates successive continuous-integration interface iterations that reduce clutter, add onboarding guidance, and clarify performance and process feedback.Source: Figure 6 · p. 18
  • GenUI’s layout is reported to improve perceived clarity, trustworthiness, and professionalism.Source: Figure 7 · p. 18

Related papers

Loading 2508.19227v3…