PaperlayerView exact revision on arXiv

Source-linked AI summary

From Natural Language Requirements to Graphical User Interfaces: Automated Prototyping and Verification with Pretrained Language Models

Kristian Kolthoff

arXiv:2608.24749v1cs.SE

TL;DR

Die Arbeit untersucht, wie der manuelle Aufwand bei der Umwandlung natürlicher Sprachanforderungen in GUI-Prototypen und bei ihrer Verifikation reduziert werden kann. Sie entwickelt dafür Retrieval-, Reranking-, LLM- und multimodale agentenbasierte Verfahren sowie passende Datensätze. Die Ansätze zeigen Wirksamkeit für GUI-Prototyping und die Verifikation statischer und dynamischer GUI-Systeme.

  • Problem

    Die Erstellung von GUI-Prototypen aus natürlichen Sprachanforderungen sowie deren Verifikation sind zeitaufwendig und erfordern häufig Expertenwissen.

  • Method

    Die Arbeit kombiniert NL-basiertes GUI-Retrieval und Reranking, LLM-basierte GUI-Generierung sowie LLM- und MLLM-agentenbasierte Verifikation.

  • Results

    Die vorgeschlagenen Verfahren zeigen hohe Wirksamkeit bei GUI-Retrieval, Prototyping und der Verifikation komplexer Anforderungen in statischen und dynamischen GUIs.

  • Takeaways & Limitations

    Die Ergebnisse zeigen, dass pretrained LLMs und MLLMs für die Automatisierung von GUI-Prototyping und Requirements Verification eingesetzt werden können.

  • Takeaways & Limitations

    Die Generalisierbarkeit ist möglicherweise eingeschränkt, weil die Evaluationsdaten auf dem Rico-GUI-Datensatz basieren und seltene Domänen unterrepräsentiert sein können.

Abstract

from arXiv · show

Requirements elicitation is essential for developing interactive software systems, as it helps ensure that the resulting product meets stakeholder needs. Since elicitation typically relies on natural language (NL), misunderstandings can arise from its inherent ambiguity. Formal specifications can reduce ambiguity but require technical expertise. GUI prototyping therefore provides a valuable alternative by turning requirements into tangible visual artifacts that support communication, elicitation, and validation. However, creating high-fidelity prototypes remains time-consuming and costly. Similarly, requirements verification, which ensures that implementations conform to specified requirements, is still largely manual, while existing automated approaches are often limited to static, rule-based techniques. This work addresses two challenges: (C1) reducing the effort required to transform NL requirements into GUI prototypes, and (C2) reducing the effort required for requirements verification in GUI applications and prototypes. For C1, we introduce novel NL-based GUI retrieval and reranking methods, new benchmarks, and techniques for efficiently adapting LLMs to GUI generation, including proprietary GUI representations. Their effectiveness is demonstrated on a large benchmark with human annotations. For C2, we propose LLM-based methods for verifying semantically complex NL requirements on static GUI prototypes and introduce a multimodal LLM-based agent for verifying complex functional and non-functional requirements in dynamic GUI applications through automatically generated and evaluated interaction trajectories. Overall, the proposed methods substantially reduce manual effort in GUI prototyping and requirements verification.

Zusammenfassung

Die Arbeit adressiert die Reduktion des Aufwands für die Umwandlung natürlicher Sprachanforderungen in GUI-Prototypen. Dazu kombiniert sie GUI-Suche und -Neubewertung, LLM-basierte GUI-Generierung sowie neue Datensätze und Benchmarks.

  • Zusammenfassung: Die Arbeit zielt darauf, natürliche Sprachanforderungen schneller in anpassbare GUI-Prototypen zu übersetzen.Sie behandelt dafür GUI-Suche, Reranking und LLM-basierte Generierung proprietärer GUI-Repräsentationen.
  • Zusammenfassung: Die Arbeit basiert auf Beiträgen zu automatisiertem GUI-Prototyping und Requirements Engineering, die teilweise als Preprints vorlagen.Ein genannter Preprint behandelt Zero-Shot-Prompting für LLM-basierte GUI-Generierung.
  • Zusammenfassung: Die Beiträge umfassen mehrere veröffentlichte Arbeiten zu GUI-Generierung, GUI-Retrieval, User Stories und LLM-basierten Assistenten.Die aufgeführten Arbeiten erschienen unter anderem bei ASE, RE, CHI und weiteren Software-Engineering-Konferenzen.

Introduction

This work addresses the high effort and ambiguity involved in transforming natural-language requirements into GUI prototypes and verifying requirements in GUI systems. It develops pretrained-language-model-based methods, datasets, and benchmarks for automated GUI retrieval, generation, elicitation, and verification.

  • Introduction: Natural-language requirements are heterogeneous and unstructured, while GUIs are complex, multidimensional objects, making automated transformation difficult.The requirements may combine functional and non-functional aspects without fixed syntax or semantics, whereas GUIs include structural and other dimensions.
  • Introduction: Creating GUI prototypes from natural-language requirements is labor-intensive, costly, and dependent on expert knowledge and experience.
  • Approaches: The proposed C1 approach treats mapping from natural-language requirements to GUI prototypes as a retrieval problem and develops multiple LLM-based generation and elicitation techniques.Contributions include GUI retrieval and reranking, dialogue-based self-elicitation, prompt decomposition, and retrieval-augmented GUI generation.
  • Requirements verification: LLM-based verification methods address semantically complex requirements in static prototypes and dynamic web-based GUI applications through interaction steps and updated GUI states.The dynamic approach supports automatic verification of complex functional and non-functional requirements with generated interaction trajectories.
  • Datasets: The work introduces datasets and benchmarks for GUI retrieval, reranking, generation, and requirements verification across static and dynamic GUI settings.The resources include a 450-requirement retrieval benchmark, Rico annotations and embeddings, GUI descriptions and prototypes, interlinked user stories and GUI components, and verification data for dynamic applications.
  • Approaches: A data-driven GUI prototyping editor, RaWi, integrates the proposed GUI retrieval techniques.

Theoretical Background

Requirements Engineering systematically elicits, specifies, validates, verifies, and manages software requirements, with prototyping supporting communication and validation. The background contrasts linear and iterative RE processes and characterizes GUI prototypes by fidelity.

  • Requirements Engineering: Natural Language Requirements can be documented using constraints, system names, modal verbs, process verbs, objects, and object-specific details.
  • Requirements Engineering: Requirements Engineering covers elicitation, documentation, specification, negotiation, validation, verification, and management of software requirements.
  • Requirements Engineering: Requirements elicitation gathers and refines requirements from stakeholders and related sources, while validation detects ambiguity, vagueness, incompleteness, and contradictions.
  • Requirements Engineering: Linear RE models progress from elicitation through analysis, negotiation, documentation, and validation, whereas iterative models connect elicitation, specification, and validation non-linearly.
  • GUI Prototyping: Prototypes are requirements models that support validation by combining specifications with domain knowledge and user feedback.
  • GUI Prototyping: GUI prototype fidelity describes similarity to the final GUI and ranges from low-fidelity abstractions to high-fidelity representations close to the final system.

Natural-Language-Based GUI Retrieval

This section presents retrieval-based methods for mapping natural-language requirements to GUI prototypes, including RaWi, BERT-LTR ranking, and a crowdsourced evaluation benchmark. The approaches improve GUI retrieval and prototyping productivity, while remaining constrained by repository and dataset limitations.

  • Natural-Language-Based GUI Retrieval: RaWi maps natural-language requirements to GUI prototypes by retrieving screens from a large repository and deriving partly editable screens for reuse.The approach is designed to facilitate rapid, high-fidelity prototyping during requirements elicitation.
  • Natural-Language-Based GUI Retrieval: The work introduces a crowdsourced gold standard, an in-depth comparison of retrieval and ranking methods, and a multi-metric productivity evaluation methodology.The benchmark contains three-level relevance annotations for NLR-GUI pairs, while the evaluation includes a user study in a rapid prototyping environment.
  • Natural-Language-Based GUI Retrieval: BM25 is an effective baseline, with the first relevant GUI appearing at rank 1.92 on average and at least one relevant GUI appearing in the top five for 69% of queries.The nBoW model improves several small-k ranking metrics over plain TF-IDF, while BM25 remains stronger across most examined mean values.
  • Natural-Language-Based GUI Retrieval: Text-segment-wise PRF achieves the highest scores among most considered mean metric values and outperforms BM25 across all mean metric values.Weighted PRF-KLD variants improve some binary metrics but underperform on NDCG@k, which accounts for the full relevance scale.
  • Natural-Language-Based GUI Retrieval: BERT-LTR models consistently outperform BM25 across mean metric values, with statistically significant improvements for multiple AP, H@k, P@k, and NDCG@k metrics.BERT-LTR models also consistently outperform SBERT across mean metric values, although significance is reported for HITS@5.
  • Natural-Language-Based GUI Retrieval: In a controlled study, RaWi substantially outperforms traditional prototyping in basic GUI-component counts at each time step, although the difference narrows over time.Participants often started with more suitable screens in RaWi, but the approach could also produce prototypes containing more unrequested components.

GUI-ReRank: Enhancing GUI Retrieval with Multi-Modal LLM-based Reranking

GUI-ReRank combines multi-dimensional retrieval with MLLM-based reranking to represent GUI prototypes more richly and improve ranking from natural-language requirements. Image-based reranking improves effectiveness, while text-based variants offer a substantially better cost trade-off.

  • Approach: GUI-ReRank combines constrained multi-dimensional retrieval, MLLM-based reranking, and customizable GUI annotations for natural-language queries.The framework extracts positive and negative constraints across dimensions such as domain, functionality, design, and GUI components, then uses textual and visual GUI information.
  • Effectiveness: GPT-4.1 (Image) improves AP over pairwise BERT-LTR (2) by 67.66%, while relevant GUIs appear first in 87% of cases.The 87% HITS@1 result represents an 89.13% improvement over the best baseline reported in the passage.
  • Effectiveness: MLLM-based reranking substantially improves ranking performance over BERT-LTR baselines across nearly all evaluated metrics.The reported improvements are statistically significant across all metrics except HITS@10 and HITS@15.
  • Effectiveness–cost trade-offs: Image-based models generally outperform text-based models, but text-based reranking provides a more favorable effectiveness–cost trade-off.GPT-4.1 (Image) improves mean AP over GPT-4.1 (Text) by 3.32% while increasing API costs by 543.9%; all text-based GPT-4.1 and Gemini-2.5 models are Pareto-efficient.
  • Limitations: The approach remains limited by run-to-run variance, high MLLM resource costs, and the expertise required to tune multi-dimensional retrieval weights.The authors report few ranking deviations across manually checked runs, while text-based reranking reduces costs substantially.
  • Approach: The approach addresses the semantic representation gap by incorporating GUI screenshot information alongside textual representations derived from GUI repositories.This extends prior approaches based mainly on text fragments extracted from GUI hierarchy data.

Self-Elicitation of Requirements with Automated GUI Prototyping

SERGUI supports self-elicitation by helping stakeholders rapidly build GUI specifications from natural-language requirements, recommended features, and reusable GUIs. Its evaluations show effective retrieval, recommendation, reranking, and high perceived usability, while its data-driven design limits support for very special and custom GUIs.

  • Approach: SERGUI enables stakeholders to independently build GUI prototypes with little prior prototyping experience through automated natural-language assistance.The approach combines GUI retrieval, feature recommendations, and interactive prototype construction.
  • Approach: SERGUI combines NLR-based GUI retrieval, feature-based reranking, contextualized LLM feature recommendations, and Rico-based prototype construction.Aspect-GUIs visualize individual relevant features and can be saved alongside selected GUIs in the prototype.
  • Evaluation: Precision=.9818 / Recall=.7012 was achieved on a separate test set for the GUI-filtering model, with precision prioritized to avoid removing adequate GUIs.The filtering process removed 23,817 GUIs and retained 48,402.
  • Evaluation: SERGUI’s feature recommendation achieved mean AP of 74.1 and MRR of 81.6, while aspect-GUI retrieval achieved MRR of 39.0 and HITS@15 of 69.1.The recommendation results indicate relevant features are often ranked near the top, while aspect-GUI retrieval finds relevant matches across the ranking.
  • Evaluation: 68.05% of participant cases involved selecting a better-matching GUI after feature-feedback reranking, with an average improvement of +61.89 ranks.The ensemble retrieval model also outperformed the individual SBERT and S2W models in MRR.
  • Evaluation: SERGUI achieved a SUS of 84.16 versus an average SUS of 68, and participants reported needing little to no prior knowledge.Participants also found feature recommendations helpful for incorporating relevant features they had not initially considered.
  • Limitations: SERGUI is constrained by its data-driven repository and cannot support the self-elicitation process for very special and custom GUIs.The interaction model is also inflexible, and feature feedback is currently restricted to binary inclusion or exclusion.

Retrieval-Augmentation and Self-Critique Prompting

The paper develops retrieval-augmented and self-critique prompting methods for generating high-fidelity GUI prototypes from natural-language requirements without training or finetuning. Across evaluations, SC and RAGG generally outperform simpler prompting approaches, while content generation further improves prototype realism and perceived quality.

  • Retrieval-Augmentation and Self-Critique Prompting: RAGG retrieves GUI examples, reranks them with multimodal LLMs, and conditions generation on their feature and layout knowledge.The pipeline filters for relevance before supplying GUI screenshots to the generation model.
  • Retrieval-Augmentation and Self-Critique Prompting: SC systematically applies self-critique prompting to end-to-end natural-language GUI generation, refining prototypes through iterative feedback.The approach adapts self-critique methods that use model-generated feedback to improve an earlier output.
  • Retrieval-Augmentation and Self-Critique Prompting: LLM-based content generation augments base HTML/CSS with realistic content and DALL-E-3 images to improve prototype visual quality, completeness, and satisfaction.The pipeline generates and integrates images rather than leaving generated interfaces with generic or insufficient example content.
  • Retrieval-Augmentation and Self-Critique Prompting: RAGG improved Feature Completion, Feature Extensiveness, Visual Appeal, Overall Satisfaction, and Complete App, while differences were not significant for Feature Implementation, Information Organization, and Minimal Errors.SC showed a similar pattern of broad gains, with significant improvements on Feature Completion, Feature Extensiveness, Feature Implementation, Visual Appeal, Overall Satisfaction, and Complete App.
  • Retrieval-Augmentation and Self-Critique Prompting: RAGG input-token use grows with retrieved examples, whereas SC input and output use grow across iterations; most SC gains occur after the first iteration and costs remain moderate.Both approaches use more tokens than baseline and prompt-decomposition methods, but the reported absolute costs are negligible in practice.
  • Retrieval-Augmentation and Self-Critique Prompting: SC significantly outperformed baselines and prompt decomposition across most metrics, while RAGG with k = 7 significantly improved over at least one baseline on most metrics.No significant differences separated RAGG (k = 7) and SC (k = 4) across all metrics except (H) Complete App.

7.1 Motivation

The work introduces LLM-based assistance for GUI prototyping, combining retrieval, reranking, detection, matching, and component generation to reduce manual effort. Evaluations report strong component quality, token-efficiency gains, and improved prototype outcomes.

  • Approach: A two-stage RAG approach integrates proprietary GUI component libraries into LLM-based generation of editable GUI representations.The approach is implemented for Material Design within Figma.
  • Approach: The Figma plugin assesses user-story completion, identifies matching GUI components, and recommends or generates components for requirements.Generated components are previewed so users retain control over whether to incorporate recommendations.
  • Evaluation: Generated GUI components received high crowdworker ratings for functional-requirement fulfillment, user-story fulfillment, and component correctness.These criteria received three median ratings of eight and one median rating of seven on nine-point Likert scales.
  • Evaluation: 60.82% lower mean total token consumption was achieved with two-stage RAG than with one-stage RAG.The reported mean total tokens were 9,359.44 for one-stage RAG and 3,666.54 for two-stage RAG.
  • Evaluation: The treatment configuration produced higher user-story completion for most evaluated stories across the three applications.For the third application, all ten stories favored treatment, with seven differences statistically significant.

GUIDE: LLM-Driven GUI Generation Decomposition

GUIDE converts high-level natural-language requirements into editable GUI prototypes by decomposing them into GUI features and generating components through retrieval-augmented methods in Figma. In a user study, GUIDE significantly outperformed unaided creation across all nine GUI quality metrics.

  • GUIDE architecture: GUIDE decomposes high-level natural-language requirements into low-level GUI features before generating editable prototypes with Material Design components.Its generation decomposition selects component combinations and produces fully specified implementations through a RAG approach.
  • GUIDE architecture: GUIDE combines a Figma plugin with an LLM-based generation decomposition that selects and implements components from a predefined library.Users enter short, high-level requirements directly into the plugin interface.
  • Evaluation: 3.2 GUI prototypes per participant were commenced with GUIDE, compared with 2.33 without assistance during the 45-minute study.The treatment group started 16 GUIs across six participants, while the control group started 14 across five participants.
  • Evaluation: Median requirement-meeting ratings were 8 with GUIDE and 5 for the control group.The corresponding means were 7.393 and 4.746.
  • Evaluation: Median ratings for using correct GUI components were 7 with GUIDE and 5 for the control group.The corresponding means were 7.107 and 4.769.
  • Evaluation: GUIDE significantly outperformed the control group across all nine considered GUI quality metrics.The study also reports substantially fewer errors and improvements in design-related metrics.
  • Limitations: The current implementation is limited to Figma, Material Design, individual GUI prototypes, and interactions within single prototypes.Future extensions target additional tools, libraries, whole applications, interactions, and cross-GUI dependencies.

Interlinking GUIs and User Stories: A Semi-Automatic LLM-Based Approach

This work studies whether LLM prompting can automate verification of user stories in simplified static GUI representations. It evaluates implementation classification and component matching, finding high classification performance and moderate-to-good matching performance, while identifying representation-related errors.

  • Approach: The feasibility study evaluates zero-shot, few-shot, and chain-of-thought prompting for detecting implemented user stories and matching them to GUI components.The approach transforms structural Rico GUI hierarchies into textual representations for LLM processing.
  • Dataset: The interlinked dataset contains 231 user stories and GUIs after independent labeling and dispute resolution.The authors excluded 96 user stories, and the initial inter-annotator agreement was Cohen’s Kappa κ = .548.
  • RQ1: Implementation detection: 85.2 accuracy was achieved by zero-shot prompting for detecting user-story implementation in static GUIs.The passage also reports 84.8 for FS5 and 82.9 for CoT_t=1.0.
  • RQ1: Implementation detection: Zero-shot prompting achieved precision 83.0, recall 88.6, and F1-measure 85.7 for implementation detection.No significant pairwise differences were observed between prompting approaches.
  • RQ2: Component matching: Matching errors commonly involved semantically related but incorrect components and missing or ambiguous component descriptions.Examples include selecting room-count controls for a guest-count requirement and failing to identify image-based download functionality.
  • RQ2: Component matching: Zero-shot prompting achieved Macro precision 78.4, recall 81.9, and F1-measure 75.5 for matching user stories to GUI components.Significant differences were observed between ZS-CoT and FS-CoT.
  • Limitations: The study is limited to verifying user stories in simplified, static Rico GUI representations.The authors characterize the work as a feasibility study and identify the Rico representation as a limitation.

GUISpector: An MLLM Agent Framework for Automated Verification of Natural Language Requirements in GUI Applications

This chapter presents GUISpector, an automated MLLM-based approach for verifying natural-language requirements in dynamic GUI applications. It extends an earlier static-GUI feasibility study toward fully fledged applications.

  • GUISpector applies an MLLM-based computer-use agent to verify natural-language requirements in GUI applications.The approach is presented as a novel automated verification method for dynamic applications.
  • The framework is implemented as a tool prototype for GUI verification and test data.The author reports implementing the entire framework and tool prototype.
  • The chapter’s work was accepted for publication at ICSE-Companion and is reported as in press.The section is adapted from the cited GUISpector publication.
  • Source code, evaluation datasets, and a demonstration video are provided as supplementary materials.The materials are linked through the GUISpector repository and demo video.

C. Agentic Implementation-Verification Loop

The agentic loop connects GUI verification with LLM-based software implementation: GUISpector reasons over GUI states, predicts and executes actions, and can trigger further implementation changes. Evaluation reports high verification effectiveness but nontrivial runtime and monetary costs.

  • MLLM GUI Agent Verification Loop: GUISpector iteratively reasons over GUI states, predicts next actions, executes them, and obtains new states for verification.The loop forms the core MLLM computer-use-agent verification process.
  • Agentic Implementation-Verification Loop: The MCP interface lets LLM programming agents invoke GUISpector within an autonomous implementation-verification loop.When a requirement is not met, the programming agent can implement further changes before repeating verification.
  • Evaluation Dataset: 150 requirements produced very high inter-annotator agreement, with Krippendorff’s α = 0.929 and eight LLM-human disagreements.The gold standard covers five applications and was based on agreed human evaluations.
  • Verification Effectiveness: 94.0 and 87.0 are the average F1 scores for met and unmet acceptance criteria, respectively, across five applications.Both classes achieved high average scores in binary acceptance-criteria classification.
  • Verification Effectiveness: 90.5, 87.8, and 63.1 are the average F1 scores for met, unmet, and partially met requirements, respectively.The partially met class is lower and associated with greater semantic ambiguity and boundary confusions.
  • Cost and Runtime: Average verification cost ranges from $0.524 to $0.936, while steps range from 22.3 to 34.2 per requirement.Execution effort varies across applications because prerequisite flows differ; parallelization can mitigate runtime.
  • Conclusion: The approach achieves high verification accuracy at both acceptance-criteria and requirements levels, but proprietary MLLM usage entails substantial runtime and monetary costs.The conclusion emphasizes effectiveness alongside the cost boundary reported in the evaluation and discussion.

Discussion

The discussion frames pretrained language models as effective tools for automating GUI prototyping and verification, while emphasizing their costs, hallucination risks, and the need for efficient adaptation. The thesis spans retrieval, reranking, generation, and verification for natural-language requirements.

  • Discussion and Future Work: LLM-based methods are effective across the investigated tasks but entail high costs, hallucination risks, and a need for closer human collaboration.Future work highlights smaller task-adapted models as a possible efficiency direction while acknowledging reduced general capabilities.
  • Motivation: GUI prototyping supports requirements elicitation and validation, but traditional prototyping and verification remain time-consuming, costly, and limited for complex natural language.The thesis targets these effort-intensive activities through automated approaches.
  • C1-1 NLR-based GUI Retrieval Gap: BERT-based retrieval reduces the gap between natural-language requirements and textual GUI-prototype representations.The BERT-LTR model is trained on crowdsourced GUI relevance data.
  • C1-2 Semantic Representation Gap: MLLM-based reranking addresses semantic gaps caused by simplified representations that omit multidimensional GUI characteristics and exclusion criteria.The approach considers aspects such as components, relationships, text, icons, layouts, and design.
  • C1-3 GUI Prototyping Automation for Self-Elicitation: SERGUI integrates GUI retrieval with guided dialogue and LLM-based feature recommendations to support self-elicitation through GUI prototyping.The approach combines stakeholder dialogue with feature-level searches.
  • C1-4 Efficient LLM Adaptation: Zero-shot prompting methods, including retrieval-augmented generation and self-consistency, improve LLM-based GUI generation over zero-shot baselines.The evaluation uses over 20,400 GUI annotations from 101 UI/UX experts.
  • C1-5 Editable Prototypes: A two-stage retrieval-augmented approach adapts LLMs to generate proprietary GUI representations directly integrable into Figma.It first selects components from abstracted library views, then retrieves full specifications for generation.
  • C2 GUI Verification: The thesis extends requirements verification from static Rico GUIs to highly dynamic, fully fledged GUI applications using MLLM-based computer-use agents.The dynamic approach predicts meaningful interactions from trajectories containing screenshots, reasoning, and actions.

B.1 Experimental Details

The experimental details describe statistical comparisons and mixed-effects models for GUI retrieval and prototyping productivity. The supplied passages identify the evaluated metrics, reference conditions, and model structure, but not their numerical outcomes.

  • GUI Retrieval Evaluation: The GUI retrieval experiments compare models using AP, MRR, HITS@k, P@k, and NDCG@k with Holm-corrected Wilcoxon tests.The tests evaluate whether one model is greater than another, with significance at α = 0.05 after correction.
  • Productivity Comparisons: RaWi and Mockplus are compared on GUI-component productivity using paired one-sided Wilcoxon signed-rank tests with Holm correction.The reported metrics include component counts, diversity, negative-adjusted productivity, means, medians, differences, and Cliff’s Delta.
  • Mixed-Effects Models: The mixed-effects models include approach, time, task, GUI, condition, an approach-by-time interaction, and participant random intercepts.RaWi, task 1, GUI 1, and condition 1 serve as reference levels, with time centered at minute 3.

C.1 Experimental Details

The experiments report Holm-corrected statistical comparisons across retrieval and GUI-generation models using several evaluation metrics.

  • Tables C.1–C.3 report Holm-corrected statistical tests for AP, MRR, HITS@k, P@k, and additional metrics.
  • The comparisons test whether model 1 performs better than model 2 across the reported metrics.
  • The experiments distinguish image, text, Mini, Nano, and GPT-4.1 model variants.

D.1 Experimental Details

The appendix evaluates overall differences among zero-shot prompting approaches on both the full and cleaned GUI datasets.

  • The Friedman tests assess whether at least one zero-shot model differs significantly from the others for each evaluation metric.
  • The analyses cover the full dataset of 50 GUIs and a cleaned dataset of 15 GUIs.

D. APPENDIX FOR CHAPTER 6

The appendix reports corrected pairwise tests comparing prompting methods, retrieval-example counts, loop counts, and content-generation settings across GUI-quality dimensions.

  • Prompting comparisons: Post-hoc Wilcoxon tests compare zero-shot prompting methods across feature completion, implementation, organization, visual appeal, and related quality dimensions.
  • Retrieval examples: RAGG is evaluated with different numbers of retrieved examples, including k = 1, 3, 5, and 7.
  • Loop counts: SC is compared across loop counts k = 0, 1, 2, 3, and 4, with k = 0 representing the underlying zero-shot baseline.
  • Content generation: Additional tests compare prompting techniques with and without content generation, using RAGG and SC with k = 3.

E.1 Experimental Details

The experiments evaluate paired correctness and per-user-story F1 differences among zero-shot, few-shot, and chain-of-thought prompting approaches.

  • McNemar tests assess paired correctness for user-story detection and compare whether one model differs from another.
  • Pairwise one-sided Wilcoxon tests evaluate whether Model 1 has greater per-user-story F1 than Model 2.
  • The reported F1 difference is defined as ∆F1 = F1₁ − F1₂, with Holm correction applied to significance tests.
Loading 2608.24749v1…