Enabling Creative Exploration for Vibe Design Agents

Yifan Zhang; Nghi D. Q. Bui; Georgios Evangelopoulos; Arnaud Benard

Source-linked AI summary

Enabling Creative Exploration for Vibe Design Agents

Yifan Zhang, Nghi D. Q. Bui, Georgios Evangelopoulos, Arnaud Benard

arXiv:2609.15078v1Sep 14, 2026cs.AI

TL;DR

Vibe design agents need to explore coherent alternatives without sacrificing precise code generation. The paper inserts structured design specifications and external selection between the request and fixed-setting generation, finding broader theme variation but mixed judged and online outcomes. The results support structured specifications as a control point while leaving human preference and universal temperature choices unresolved.

Problem
Vibe design agents typically produce valid interfaces, but useful exploration requires helping users inspect meaningfully different design directions without entangling aesthetic variation with syntax-sensitive code generation.
Method
A proposal stage generates structured theme or asset specifications with typicality scores, an external selector samples one, and fixed-setting downstream generation realizes it with the original request.
Results
Theme sampling broadens observed selection coverage and screenshot variation, while LLM-judge preferences vary across interventions, prompt complexity, and viewport; online code-export increase remains statistically uncertain.
Takeaways & Limitations
Structured design specifications provide a practical control point for exploring alternative UI concepts while keeping downstream decoding settings fixed.
Takeaways & Limitations
The experiments do not establish human design preference or a universally preferred temperature, and online aggregate causes remain unidentified.

Abstract

from arXiv · show

Vibe design agents turn natural-language briefs into rendered interfaces and frontend code. Yet a useful design agent should do more than produce one valid page: it should help users explore coherent alternatives. Increasing token-level temperature is a blunt solution because it varies aesthetic decisions and syntax-sensitive code at the same time. We instead separate exploration from implementation through an inference architecture that makes design direction an explicit intermediate decision. Inspired by Verbalized Sampling, a pre-pass proposes structured design specifications with typicality scores, an external selector samples one, and the downstream generator realizes the selected specification together with the original request under fixed settings. We apply this approach to UI themes and visual-asset prompts. Across 168 prompts, with 1,255 paired comparisons per temperature for each intervention, theme sampling broadens observed selection coverage and screenshot variation, while LLM-judge preferences vary across interventions, prompt complexity, and viewport. In an online experiment with more than 300,000 tasks, the observed code-export increase remains statistically uncertain, while fewer negative feedback events coexist with more correction interactions and modest operational costs. Together, these findings identify structured design specifications as a practical control point for exploring alternative UI concepts while keeping downstream generation settings fixed.

1. Introduction

Vibe design agents must support coherent design exploration alongside precise interface and code generation. The paper separates these goals by making design direction an explicit intermediate choice while keeping downstream generation settings fixed.

Motivation and approach: Token-level temperature entangles aesthetic variation with syntax-sensitive implementation changes, while “be creative” provides no inspectable distribution for balancing alternatives.The challenge is to direct variation toward coherent design concepts rather than perturbing the entire output.
Motivation and approach: Structured design specifications let the agent vary design direction independently of fixed-setting downstream generation.A proposal stage produces alternatives, an external selector chooses one, and the generator realizes the selected specification with the original request.
Evaluation scope: The evaluation separates selection coverage, visual and structural diagnostics, LLM-judge preferences, and online behavior because these outcomes capture different properties of generated interfaces.The paper examines exploration breadth, judged quality, and behavior during use rather than treating quality as a single measure.
Findings: Across 168 prompts with 1,255 paired comparisons per temperature and intervention, theme sampling broadens selection coverage and screenshot variation, while judge preferences vary by intervention, prompt complexity, and viewport.The reported findings distinguish broader exploration from model-judge outcomes rather than assuming that variation improves preference uniformly.
Findings: In the online experiment, fewer negative feedback events coexist with more correction interactions and modest operational costs, while the observed code-export increase remains statistically uncertain.The paper therefore reports behavioral changes without treating the export estimate as established.
Scope and limitation: The findings do not establish human design preference or a universally preferred temperature, so blinded professional evaluation remains necessary.These limitations qualify interpretation of both offline judge results and temperature settings.

2. An Architecture for Explicit Design Exploration

The architecture separates proposing, selecting, and generating design directions. A selected structured specification conditions downstream interface or asset generation while the original request and downstream decoding settings remain fixed.

Architecture: The architecture separates design-specification proposal, external selection, and interface generation conditioned on the selected specification.The selected specification serves as the interface between exploration and implementation.
Structured design specifications: Each candidate includes a structured specification, rationale, and typicality score, with theme candidates bundling palette, fonts, display mode, and corner roundness.Asset candidates instead specify subject, composition, and visual style for image generation.
Selection policy: The selector normalizes valid nonnegative scores into weights and applies temperature scaling before sampling a candidate.Invalid or unusable scores trigger a re-prompt and may fall back to the baseline path.
Selection policy: At τ=1 selection follows normalized elicited weights, lower values favor higher-weight directions, and higher values increase the chance of lower-weight directions.Temperature changes odds within the proposed set and cannot introduce directions that were not proposed.
Conditioned generation: The downstream generator receives the original request plus the selected theme or image specification, while the existing generation pipeline and decoding settings remain fixed.Theme and visual-asset interventions are enabled separately to examine their respective effects.

3. Experimental Setup

The experiments evaluate exploration breadth, structural change, judged preference, and behavior during use across separate theme and asset interventions. They use three elicited directions, two prompt suites, multiple temperatures, and matched baseline comparisons.

Evaluation design: The evaluation treats variation, preference, and product use as separate outcomes, using selection coverage, screenshot similarity, DOM similarity, an LLM judge, and an online experiment.This design avoids collapsing exploration breadth and quality into one verdict.
Interventions: The proposal stage uses three directions—Safe, Premium, and Experimental—whose labels guide elicitation but are not measured quality or risk levels.Candidates are generated and evaluated with Gemini models, while Nano Banana 2 generates in-page images.
Selection settings: The study tests τ∈{0.5, 1, 1.5, 2, 5}, with reported defaults of 1.5 for themes and 2.0 for assets.These defaults are implementation settings rather than universal recommendations.
Interventions: Theme sampling varies theme generation, asset sampling varies image prompts with theme sampling disabled, and the joint condition appears only in the qualitative case study.This setup separates the two intervention sources in the main experiments.
Prompt suites: The standard suite contains 83 short open-ended prompts, while the complex suite contains 85 detailed prompts with layout, style, composition, and functionality constraints.Prompts are evaluated at mobile and desktop viewports with repeated baseline-versus-intervention comparisons.
Prompt suites: Each intervention contributes 1,255 paired comparisons per temperature across the two suites, rather than 1,255 unique outputs across the full temperature sweep.Pairs compare matched baseline and intervention outputs for the same prompt and viewport.
Measures: Screenshot embedding similarity measures visual variation, compiled DOM similarity measures structural change, and neither metric alone establishes aesthetic quality or functional validity.The LLM judge reports descriptive model-judge outcomes rather than human preferences.
Online experiment: The online experiment reports task completion, latency, error signals, exports, feedback, and corrections over 309,870 created tasks and 505,940 generated screens.Correction is evaluated on a subset of conversations, with denominators and confidence intervals specified separately.

4. Preliminary Results

Theme sampling broadens observed design coverage and screenshot variation, but judged quality varies by intervention, prompt suite, viewport, and temperature. Online deployment shows fewer negative-feedback events alongside more corrections, modest operational costs, and uncertain export gains.

4.1. Theme Selection, Rendered Variation, and Judged Quality: Candidate theme selection increases observed coverage from 1.00 to 2.10–2.94 options, with every prompt showing more than one option at 𝜏≥1.5.The result uses one selected theme across five baseline runs for each of 83 prompts.
4.1. Theme Selection, Rendered Variation, and Judged Quality: Screenshot similarity falls from 0.6560 at 𝜏=0.5 to 0.5438 at 𝜏=2.0, while 𝜏=2.0 reaches a 1.10 win/loss ratio before preference reverses at 𝜏=5.0.The reversal despite nearly identical similarity indicates that recorded variation and judged quality do not change monotonically together.
4.2. Visual-Asset Variation and Judged Quality: Asset sampling achieves its highest observed win/loss ratio at 𝜏=1.0, 1.32, while whole-screen similarity changes only from 0.6765 to 0.6616.Higher asset temperature does not consistently improve either judge preference or screenshot separation.
4.3. Variation Across Prompt Suites and Viewports: The best observed settings differ across prompt suites: 𝜏=2.0 for themes, 𝜏=1.0 for standard assets, and 𝜏=1.5 for complex assets, where the aggregate ratio is 1.19.These descriptive comparisons suggest that a setting selected for one prompt suite may not transfer to another, and viewport comparisons require prompt-suite context.
4.3. Variation Across Prompt Suites and Viewports: Compiled HTML similarity ranges from 0.7935–0.8339 for intervention outputs versus 0.8779 for baseline theme outputs, indicating shared structure alongside change without establishing preserved functionality.The structural diagnostic complements screenshot variation but does not establish equivalent DOMs or functional validity.
4.4. Behavior in the Online Experiment: Online results show fewer negative-feedback events, more correction interactions, modest latency and completion costs, and an uncertain code-export increase.The observed export change is +8.23%, but its 95% interval [−13.03%, +29.48%] includes zero.

5. Qualitative Case Study: Joint Theme and Asset Sampling

The joint theme-and-asset condition illustrates how the pipeline can vary multiple design attributes within one complex prompt, while the case study does not establish a general effect.

The case study illustrates joint sampling but cannot isolate either intervention’s contribution or establish preference or executable validity.It uses a single prompt rather than a controlled ablation.
Across five inspected runs, the joint condition varies palettes, fonts, corner radii, meals, camera angles, and lighting.

6. Limitations and Implications

The study finds that exploration breadth and evaluation outcomes depend on intervention, prompt, and viewport, while several design and measurement limitations constrain interpretation. It recommends controlled replication and stronger adherence, execution, accessibility, and professional-quality checks.

Empirical findings: Reported option coverage increases and screenshot similarity decreases in the standard theme study, whereas asset interventions can shift judge preference with much smaller whole-screen changes.These results indicate that evaluation metrics may respond differently to broad palette or layout changes versus localized asset changes.
Interpretation limits: The integrated pipeline does not isolate typicality weighting from multiple-candidate proposal, and temperature comparisons therefore describe tested configurations rather than selection-only effects.Unweighted elicitation, uniform selection, source-faithful VS, and temperature alternatives are needed for separation.
Interpretation limits: Candidate selection is restricted to proposed options, whose weights are not calibrated probabilities, while option identity across repeated runs remains unclear.Direct execution and adherence checks are also needed to determine whether outputs satisfy both the selected specification and the original request.
Evaluation limits: Strongest settings differ across interventions and prompt suites, so reported defaults are not universal recommendations, and judge preferences do not establish professional design quality.LLM judges are vulnerable to position, verbosity, and alignment biases; offline aggregates also lack prompt-clustered uncertainty.
Online evidence: In the online experiment, fewer negative ratings coexist with more correction interactions, modest latency and task-success declines, and an export interval that includes zero.The aggregate correction result does not identify its cause, while sparse voluntary feedback and an incomplete analysis snapshot limit interpretation.
Future validation: The proprietary data and executable analysis are unreleased, and the experiments use one model configuration and UI pipeline, limiting replication across models, domains, languages, and accessibility-constrained tasks.A confirmatory study should freeze prompts, candidates, models, seeds, renderer, and exclusions; retain invalid and unrated outputs; and report clustered uncertainty.
Implications: Exploration usefulness depends on the brief and work stage, while accessibility, privacy, brand requirements, and task constraints should govern proposal and downstream validation.The study motivates adaptive policies and user steering but does not evaluate an adaptive policy.

7. Related Work

Related work frames the approach against training-time alignment, multi-response prompting, screenshot-to-code systems, creativity measurement, and LLM-based evaluation. The paper positions inference-time structured exploration alongside these traditions while retaining separate visual, judge, structural, and behavioral assessments.

Inference-time exploration: RLHF and DPO address instruction following and preference alignment during training, whereas Verbalized Sampling intervenes at inference time by requesting representative candidates.The paper treats prompted typicality values as normalized operational weights rather than calibrated probabilities.
Inference-time exploration: Multi-response prompting can increase diversity relative to independent sampling, while VS adds typicality annotations and probability thresholds.Prompted numbers may diverge from next-token probabilities, motivating calibration before probabilistic interpretation.
UI generation: Screenshot-to-code research maps images to UI programs or hierarchies, with newer benchmarks covering fidelity, frameworks, editing, repair, and interactions.Generation systems additionally study filtering, segmentation, hierarchical construction, refinement, efficiency, and component reuse.
Creativity measurement: Creativity research cautions against single-dimension measurement, while the Human Creativity Benchmark separates convergence on adherence and execution from divergence on aesthetic direction across ideation, mockup, and refinement.This distinction motivates evaluating different aspects of creative interface generation separately.
Evaluation: LLM judges provide scalable evaluation but exhibit position, verbosity, self-family, and task-dependent biases, so heterogeneous panels do not replace human assessment.The paper reports visual variation, model-judge preference, structural diagnostics, and online behavior separately; professional ratings and direct functional checks remain extensions.

8. Conclusion

The paper identifies structured design specifications as a practical control point for exploring UI alternatives while keeping downstream decoding fixed. Its experiments show broader theme variation, localized asset effects on judge preference, and online behavioral changes whose interpretation remains limited.

Conclusion: Structured design specifications make design choices explicit by proposing alternatives, selecting a direction, and conditioning downstream generation while retaining fixed decoding settings.The approach separates exploration from implementation in UI agents.
Conclusion: Theme experiments show broader rendered variation, while asset experiments show that judge preference can change with much smaller shifts in whole-screen similarity.These findings distinguish exploration breadth from the visual sensitivity of evaluation interventions.
Conclusion: The online experiment reports fewer negative feedback events alongside more correction interactions, but the provided conclusion passage does not establish the cause of that pattern.The result is therefore behavioral evidence rather than a demonstrated mechanism.

A. Extended Qualitative Case Study

The extended case study compares repeated meal-planning-dashboard generations, showing broader design-direction variation under sampling while remaining illustrative rather than quantitative evidence.

Design tokens: Table 3 records observed design-token variation across five runs, with percentages applying only to this single prompt.
Image prompts: Table 4 qualitatively compares generated image prompts for the same example and marks the selected direction with a dagger.
Joint sampling: The full set suggests joint sampling changes multiple visible design decisions together, rather than merely recoloring a fixed template.The observations cover the single meal-planning-dashboard prompt and five runs per condition.
Baseline: Baseline runs continue to favor similar green palettes, rounded cards, sans-serif typography, and familiar overhead food imagery.
Observed variation: Sampled runs add terracotta, forest, and sage directions, varied type pairings, and different culinary subjects and camera treatments.