PaperlayerView exact revision on arXiv

Source-linked AI summary

PhishGen: Understanding LLM Capabilities in Automated Phishing Website Generation

Fujiao Ji, Yerim Kim, Yena Cho, Seonhye Park, Hyoungshick Kim, Doowon Kim

arXiv:2610.30226v1cs.CR

TL;DR

Prior work did not establish whether generated phishing pages work end-to-end, how convincing people find them, or how they compare with deployed kits. PHISHGEN evaluates pages from 14 LLMs across visual and functional dimensions and benchmarks them against real-world kits. Pages often render, but perceived fidelity and credential-harvesting success remain limited, with substantial differences across models.

  • Problem

    Prior studies assessed functionality by inspection, visual fidelity through automated metrics or limited internal ratings, focused on commercial models, and did not benchmark against deployed phishing kits.

    Source: §I · p. 1

  • Method

    PHISHGEN varies input modalities and rewriting strategies across 14 LLMs, evaluates visual fidelity and end-to-end functionality, and benchmarks results against real-world phishing kits.

    Source: §I · p. 1

  • Results

    Perceived similarity averages 2.02/5, while only 38.7% of rendered pages complete the full credential-harvesting pipeline.

    Source: §I · p. 2

  • Takeaways & Limitations

    LLMs trade lower average completeness than kits for scalable, low-barrier generation, while generator fingerprints suggest defenses should look beyond visual similarity.

    Source: §I · p. 2, §IX · p. 14

    The study covers a limited set of representative brands, prioritizing analytical depth and feasibility over exhaustive coverage.

    Source: §VIII · p. 13

Abstract

from arXiv · show

Phishing relies on visual deception, traditionally requiring web-development skills or pre-built kits. LLMs erode this barrier by generating functional websites from natural-language prompts, yet prior work does not answer how far LLM-driven generation actually reaches: functionality is assessed by inspection rather than execution; visual fidelity is scored by automated metrics or a few internal coders rather than the humans such pages aim to deceive; evaluation is confined to commercial models; and no study benchmarks generated pages against phishing kits deployed in the wild. We present PhishGen, a framework that evaluates the appearance fidelity and end-to-end functionality of LLM-generated phishing pages. PhishGen constructs prompts across three input modalities in seven combinations with twelve safeguard-bypassing rewriting strategies, generates pages using 14 LLMs spanning commercial, open-source, and uncensored categories, and assesses each via automated similarity metrics, reference-based detectors, a crowdsourced user study, and execution-based functionality testing, benchmarking all results against 64 real-world phishing kits. Although 96.2% of safeguard-bypassing outputs render, their perceived fidelity is modest (2.02/5), and only 38.7% complete the full credential-harvesting pipeline, with the Unsaved error as the largest failure mode. Contrary to the belief that uncensored models pose the greatest threat, GPT 4.1 reaches 86.3 to 94.1% functional success while uncensored models rarely exceed 45.4%. Phishing kits remain more complete per sample, yet three strong LLMs surpass the kit-level rate (70.9 to 89.7%), showing that LLMs trade average completeness for scalable, low-barrier generation of working phishing pages. We further identify generator-level fingerprints that current similarity-based defenses miss, and distill recommendations for hybrid detectors.

I. INTRODUCTION

LLMs lower the skill barrier to phishing, but prior work left execution-level functionality, human-perceived fidelity, broader model coverage, and comparison with deployed kits unresolved. PHISHGEN evaluates those dimensions and finds that pages often render but remain limited in perceived fidelity and end-to-end credential harvesting.

I. INTRODUCTION

  • Prior evaluations relied on inspection for functionality, automated or limited internal visual ratings, narrow commercial-model samples, and no comparison with deployed phishing kits.These gaps leave open how far LLM-driven generation reaches and how its practical threat compares with real-world attacks.Source: §I · p. 1
  • PHISHGEN evaluates appearance fidelity and end-to-end functionality across three model categories, combining automated analysis, a crowdsourced user study, and benchmarking against real-world kits.The framework spans 14 LLMs, multiple input modalities, and 64 phishing kits.Source: §I · pp. 1–2
  • Although 96.2% of safeguard-bypassing outputs render, perceived similarity averages 2.02/5, and LLMs reproduce brand text more often than logos (84.4% vs. 18.2%).Commercial models show the strongest visual fidelity, while uncensored models maximize generation success at the cost of visual quality.Source: §I · p. 2
  • Generated pages reproduce much of the DOM, but styling is asymmetric: buttons are reproduced faithfully while form typography is routinely omitted.This identifies a structural-stylistic fingerprint of generated pages.Source: §I · p. 2
  • Only 38.7% of rendered pages complete the full credential-harvesting pipeline; GPT 4.1 reaches 86.3%-94.1% functional success, while uncensored models rarely exceed 45.4%.Failures are dominated by backend persistence rather than frontend rendering.Source: §I · p. 2
  • Phishing kits are more complete per sample than LLM pages (57.8% vs. 38.7%), but three strong LLMs exceed the kit-level functional rate at 70.9%-89.7%.Textual brand mimicry is comparable (90.6% vs. 84.4%), while kits reproduce logos more faithfully (42.2% vs. 18.2%).Source: §I · p. 2
  • The study identifies generator-level fingerprints, including DOM-CSS asymmetry and stronger textual than visual brand mimicry, that existing similarity-based defenses miss.It also distills recommendations for hybrid detectors.Source: §I · p. 2

B. Problem Statement

The paper frames unresolved questions about end-to-end functionality, model coverage, site characteristics, and real-world grounding. Its controlled scope attributes generation outcomes to prompting and the model, then varies prompt modalities and rewriting strategies to study those capabilities.

B. Problem Statement

  • Prior work leaves four gaps: end-to-end execution testing, controlled coverage across model types, characterization of generated sites, and benchmarking against real-world kits.Earlier studies checked whether components appeared rather than whether the credential-harvesting pipeline executed.Source: B. Problem Statement · p. 2

C. Our Threat Model and Scope

  • The threat model limits adversarial effort to prompting and regeneration, excludes hand-editing, and omits downstream campaign stages such as hosting and domain registration.This isolates generation capability from manual development and external infrastructure.Source: C. Our Threat Model and Scope · pp. 2–3

III. EXPERIMENTAL SETUP AND ANALYSIS PLAN

  • PHISHGEN organizes the study into prompt design, controlled generation, and analysis of model performance, generated-site characteristics, and differences from phishing kits.The overview describes three experimental phases.Source: §III · p. 3

A. Prompt Design for Real-world Scenarios

  • Prompts vary text, screenshot, and HTML inputs across seven combinations, with 12 rewriting strategies and six rewriting models to broaden prompting behavior and reduce single-model bias.Uncensored models also receive a parallel prompt set containing the keyword “phishing”; rewriting applies only to manually crafted textual descriptions.Source: A. Prompt Design for Real-world… · p. 3
  • The prompt dataset combines 72 rewriting variations with two baseline prompts, yielding 518 prompts for image-supported LLMs and 296 for text-only LLMs.The total depends on the number of prompt combinations applied.Source: A. Prompt Design for Real-world… · p. 3

B. Selection of Target Brands and LLMs

The study selects four widely targeted brands and representative models from commercial, open-source, and uncensored categories. It generates and screens pages, curates a 64-kit comparison set, and evaluates visual and end-to-end functional characteristics.

B. Selection of Target Brands and LLMs

  • The study selects Amazon, PayPal, Facebook, and Wells Fargo as targets and samples representative commercial, open-source, and uncensored LLMs.Wells Fargo provides a more complex design challenge than the other targets.Source: B. Selection of Target Brands and LLMs · pp. 3–4

C. Phishing Website Generation

  • Models generate HTML and PHP, after which pages are rendered and screened; researchers refine prompts for 9,383 samples before subsequent analyses.Screening uses FastDup, CLIP, and manual inspection; only prompts, not generated code, are refined.Source: C. Phishing Website Generation · p. 4

D. Phishing Kits

  • The kit benchmark is curated from 4,409 shared kits through brand-keyword filtering, screenshot clustering, and manual inspection, yielding 64 kits targeting the four selected brands.The source kits were collected between July 2021 and March 2025.Source: D. Phishing Kits · p. 4

E. Analysis Plan for Characteristics

  • Appearance analysis combines rendered-page metrics and user ratings with HTML-based structure, similarity, and CSS measures; functionality requires successful credential capture, submission, and local persistence.For kits, the analysis instead verifies the credential-exfiltration destination.Source: E. Analysis Plan for Characteristics · pp. 4–5

IV. RESULTS OF SCREENSHOT-BASED EVALUATION

Rendered screenshots show that perceived fidelity and detector coverage vary substantially across models and brands. The strongest visual results come from a few commercial models, while logo omission, target complexity, and model-specific imitation patterns shape both user judgments and detection.

A. Multidimensional Visual Traits across LLMs and Brands

  • 96.8% of prompts bypassed safeguards on average, and 96.2% of bypassed responses were valid websites, but willingness and capability varied across model-brand pairs.Uncensored models responded at 98.3%-100.0% yet had the lowest and most variable validity; commercial models refused most often.Source: A. Multidimensional Visual Traits… · p. 5
  • 73.0% of screenshots fell into models’ dominant visual clusters, with commercial models showing the strongest convergence and open-source models ranging from tightly clustered to diverse.Commercial-model clustering was 76.0%-97.8%; Qwen 3 clustered tightly, while Phi was substantially more diverse.Source: A. Multidimensional Visual Traits… · p. 5
  • 2.02/5 was the mean User-perceived Website Similarity (UWS); commercial models scored higher than open-source and uncensored models, led by GPT 4.1 at 3.05 and Gemini at 2.94.GPT 4.1 peaked at 3.50 on Amazon; uncensored models averaged below 1.90.Source: A. Multidimensional Visual Traits… · p. 5
  • Wells Fargo received the lowest mean UWS across models, indicating that its visually rich login page exposed rendering limitations more sharply.Its score was significantly lower than Amazon and PayPal, but not Facebook.Source: A. Multidimensional Visual Traits… · p. 5
  • Across 570 rated instances, style and layout were the most cited reasons for similarity judgments, while uncensored models drew more broken-image or text complaints than commercial models.Style was cited 464 times and layout 459 times; broken-image or text complaints numbered 75 for uncensored models and 22 for commercial models.Source: A. Multidimensional Visual Traits… · p. 7
  • 84.4% of valid pages matched brand text, compared with 18.2% matching logos; GPT 4.1 most consistently reproduced both textual and visual cues.Wells Fargo logo matching was 0% across evaluated models, while GPT 4.1 achieved 77.8%-91.4% on Amazon, Facebook, and PayPal.Source: A. Multidimensional Visual Traits… · p. 7
  • GPT 4.1 reproduced textual and visual brand cues, GPT 5 largely suppressed explicit branding, and most other models relied primarily on text cues.Wells Fargo was consistently impersonated without logo imagery.Source: A. Multidimensional Visual Traits… · p. 7
  • 15.5% of valid pages were detected by Phishpedia, with logo-omitting pages—especially for Wells Fargo—often evading its logo-based coverage.Detection ranged from roughly 87.5% for Facebook to single digits for PayPal; Wells Fargo was typically below 1%.Source: A. Multidimensional Visual Traits… · p. 7

B. Multidimensional Visual Traits Across Prompt Combinations

Prompt configuration changes generation success, perceived realism, output diversity, and brand mimicry. Compared with generated pages, real-world kits reproduce logos more faithfully, while generation costs and HTML structure add further dimensions to the comparison.

B. Multidimensional Visual Traits Across Prompt Combinations

  • 98.5%-99.2% generation success came from textual inputs, while screenshot-based prompts achieved higher UWS (2.35-2.54 versus 1.35-2.07 for text prompts) but lower output diversity.Text and HTML prompts had 45.5%-58.8% top-1 clustering, compared with 60.7%-65.0% for screenshot-based prompts.Source: B. Multidimensional Visual Traits… · pp. 7–8, Table II · p. 8
  • 80.5%-88.9% of pages matched brand text across prompt types, while logo matching was 5.8%-30.6%; screenshot prompts roughly doubled logo reproduction.Logo matching rose from 5.8%-16.3% without screenshots to 29.7%-30.6% with screenshots.Source: B. Multidimensional Visual Traits… · pp. 7–8

C. LLM-Generated vs. Real-World Phishing Kits

  • 42.2% of real-world kits matched brand logos, versus 18.2% of LLM-generated pages; kits also had higher reference-based detection, while brand-text matching was comparable.Reference-based detection was 53.1% for kits and 15.5% for generated pages; brand-text matching was 90.6% and 84.4%, respectively.Source: C. LLM-Generated vs. Real-World… · p. 8, Table III · p. 8
  • Kit-to-LLM differences varied by brand: Facebook kits were less visually homogeneous, while PayPal kits matched text less often than corresponding LLM pages but had the strongest kit logo matching.PayPal kit logo matching was 83.3%; Facebook kit homogeneity was 7.7%, versus up to 38.5% for Amazon kits and 76.0%-94.6% for commercial LLMs.Source: C. LLM-Generated vs. Real-World… · p. 8

V. HTML-BASED EVALUATION

  • The HTML-based evaluation examines generation cost, overall structure, structural similarity to legitimate sites, and CSS styling of login forms and buttons.Table IV covers HTML structure for generated phishing websites and real-world kits.Source: §V · p. 8, Table IV · p. 9

A. Cost Analysis of Generation

  • 22 s and $0.002 made Gemini comparatively fast and inexpensive, while GPT 5 took 93 s and cost $0.71 per sample.WhiteRabbitNeo required 358 seconds per sample, whereas Dolphin took 6 seconds; the paper identifies high cost and latency as limits on scale.Source: A. Cost Analysis of Generation · pp. 8–9

B. HTML Structure

LLM-generated pages converge on compact HTML and overlapping CSS patterns, while real phishing kits are structurally heavier and LLM styling differs from legitimate pages. HTML similarity remains high, complicating DOM-based detection.

B. HTML Structure

  • Real phishing kits average 28,551 tokens, DOM depth 16, and 150 tags, compared with LLM outputs’ 199–811 tokens, depth 5–8, and 17–49 tags.Across LLMs, DOM depth stays relatively stable, suggesting convergence on a limited set of structural templates.Source: B. HTML Structure · p. 9

C. HTML Similarity

  • HTML cosine similarity to legitimate pages ranges from 0.92–0.99 for both LLMs and real kits, making DOM-feature detection difficult.For PayPal, most models have median similarity above 0.98; DeepSeekVL2 is an outlier, and adding “phishing” to prompts barely changes similarity.Source: C. HTML Similarity · p. 9, Figure 4 · p. 9

D. CSS Styling

  • LLM-generated forms favor layout and decorative CSS, while legitimate forms prioritize typography; phishing kits retain legacy and vendor-prefixed styles useful as forensic signals.Across LLMs, CSS property selection is highly overlapping, unlike the older styles inherited by kits.Source: D. CSS Styling · pp. 9–10, Table V · p. 10
  • LLM-generated buttons emphasize surface styling but often omit layout properties; open-source models converge on generic templates, while commercial models show greater stylistic diversity.Figure 5 shows open-source pairwise cosine similarity of at least 0.87, mostly above 0.92; GPT 5’s similarity to open-source models is 0.78–0.83.Source: D. CSS Styling · p. 10, Figure 5 · p. 10, Table V · p. 10

VI. FUNCTIONALITY OF GENERATED WEBSITES

LLM-generated sites often render but fail to complete credential harvesting, with backend persistence a major bottleneck. Kits are more often functional overall, though three LLMs exceed the kit-level success rate.

VI. FUNCTIONALITY OF GENERATED WEBSITES

  • Only 38.7% of valid LLM-generated websites complete the credential-harvesting pipeline, with 7,061 “Unsaved” cases and other PHP-side failures pointing to backend persistence as the main bottleneck.The failures contrast plausible frontend rendering with incomplete end-to-end execution.Source: §VI · pp. 10–11, B. Failure Analysis · p. 11, Table VI · p. 12

A. Characterizing LLMs & Brand Differences

  • GPT 4.1 reaches a per-brand maximum success rate of 94.1% (450/478), while uncensored models’ regular-prompt maximum is 35.9% (89/248); phishing prompts improve uncensored-model functionality.Functionality also varies by brand within models, with Facebook showing the largest variation and model-specific brand preferences.Source: A. Characterizing LLMs & Brand… · p. 11, Figure 6 · p. 11

B. Failure Analysis

  • “Unsaved” failures account for 7,061 cases: the backend returns HTTP 200, but credentials are not written to the local file system.Other failure modes include frontend submit timeouts and missing forms, plus server-side redirection without capture and execution errors.Source: B. Failure Analysis · p. 11

C. LLM-Generated vs. Real-World Phishing Kits

  • Three LLMs exceed the 57.8% functional rate of real-world kits: GPT 4.1 at 89.7%, Llama 4 at 79.1%, and Gemini at 70.9%.Overall, 38.7% of generated sites are functional, versus 57.8% of kits; 13 kits were non-verifiable because their exfiltration destinations were unconfigured placeholders.Source: C. LLM-Generated vs. Real-World… · pp. 11–12
  • LLM failures most often involve non-persistence, whereas non-functional kits primarily show backend syntax or incomplete-code faults and missing frontend forms or exfiltration logic.These failure categories can overlap within a kit.Source: C. LLM-Generated vs. Real-World… · p. 12

VII. RELATED WORK

PhishGen distinguishes LLM-generated phishing pages from traditional kits in both their visual and structural signatures, while finding that end-to-end functionality remains uneven. Its evaluation motivates combining human-centered and generator-level signals, with conclusions bounded by model and brand coverage.

VII. RELATED WORK

  • Prior phishing-page evaluations often checked requested components rather than executing credential harvesting, and did not jointly assess user-perceived fidelity or benchmark against real-world kits.Earlier visual assessment also relied on three in-house coders, leaving systematic user-perceived and operational comparisons unexplored.Source: §VII · p. 12

VIII. DISCUSSION AND IMPLICATIONS

  • Syntactically correct HTML/CSS does not ensure convincing pages: fine-grained human-salient cues such as accurate logos and typography can be missing.This motivates human-centered evaluation rather than relying only on structural correctness.Source: §VIII · p. 13
  • Uncensored models’ high willingness does not translate into comparable functional success; removing refusal constraints alone is insufficient to improve end-to-end functionality substantially.The paper concludes that practical risk depends also on the capability to construct reliable systems.Source: §VIII · p. 13
  • Kits and LLMs leave opposite structural signatures, so the paper recommends combining resilient brand-text cues, CSS-level layout-versus-typography signals, and existing kit-oriented and visual-similarity signals.The proposed CSS discriminator complements brand-text and visual-similarity signals rather than replacing them.Source: §VIII · p. 13
  • Prompt-design bias cannot be eliminated, models continue to evolve, and the study’s open-source censored and uncensored models are relatively small.The conclusions about uncensored models are scoped to the currently available small-model ecosystem; the study also covers a limited set of representative brands.Source: §VIII · p. 13

IX. CONCLUSION

  • Across 28,121 rendered samples from 14 LLMs, 96.2% were visually plausible, but only 38.7% were functionally viable, with coding deficiencies—not refusal behavior alone—identified as the primary barrier to end-to-end credential harvesting.LLM-generated phishing websites also showed distinguishable patterns, including homogenized HTML templates and CSS anomalies.Source: §IX · pp. 13–14

USE OF GENERATIVE AI

The authors use LLMs as experimental tools to generate phishing websites and investigate their capabilities and risks. Their ethics discussion describes controlled research practices, limits on artifact access, and anticipated impacts on affected stakeholders.

USE OF GENERATIVE AI

  • LLMs are used to generate phishing websites and examine variation in generation capabilities; authors conduct the study’s design, analysis, and conclusions themselves.Generated websites are used only in a controlled research environment and are neither deployed nor published online.Source: USE OF GENERATIVE AI · p. 14

ETHICS CONSIDERATIONS

  • The study frames its aim as characterizing LLM-assisted phishing generation and informing defenses, while acknowledging possible misuse of research artifacts.Full prompts and generated artifacts are temporarily available for artifact evaluation, then removed; simplified prompts and querying code are retained for reproducibility.Source: USE OF GENERATIVE AI · p. 14, ETHICS CONSIDERATIONS · p. 14
  • The authors identify model providers, end users, impersonated organizations, researchers, and attackers among stakeholders, framing anticipated benefits chiefly around improved defenses.They also note that attackers may gain some knowledge, while arguing that the models are openly accessible and the contribution primarily strengthens defensive research.Source: ETHICS CONSIDERATIONS · p. 14
  • The user study was IRB-approved, used informed consent, collected no personally identifiable information, and exposed participants only to static screenshots in a controlled setting.Participants could withdraw their data at any point during or after the study.Source: ETHICS CONSIDERATIONS · p. 14

APPENDIX A ANALYSIS PLAN

The appendix describes prompt combinations and rewriting strategies, illustrates a Facebook Text+Function prompt, and lists the models and versions used. It also clarifies how visual and functional prompt inputs differ.

A. Prompt Information

  • The site-generation prompt set includes seven combinations; visual inputs describe appearance and layout, whereas functional inputs describe interface purpose and interaction.The targeted login-page examples are for Amazon, Facebook, PayPal, and Wells Fargo.Source: A. Prompt Information · p. 17, Table VII · p. 17

B. An Example of Text+Function Prompt

  • The Facebook Text+Function example requests a branded login page with account links and a footer, and specifies PHP processing and saving after submission.The original prompt also contained CSS, JavaScript, HTML-layout, and PHP restrictions, which are omitted from the example for security reasons.Source: B. An Example of Text+Function Prompt · p. 17

C. LLM Information

  • The study uses commercial, open-source, and uncensored models for website generation, and model versions vary with experiment timing and release availability.The listed website-generation models include GPT 5, GPT 4.1, Gemini, Llama 4, six open-source models, and four uncensored models.Source: C. LLM Information · p. 17

D. User Study

The user study adapts its similarity rubric and uses screened pairwise judgments to assess generated pages. The accompanying HTML analysis plan measures both structural properties and similarity to legitimate reference pages.

D. User Study

  • A pilot led the authors to replace technical Website Appearance Scale wording with concise, user-friendly similarity descriptors and rationale categories derived from participant explanations.Two additional participants verified the revised instrument, with no new rationale codes observed.Source: D. User Study · pp. 17–18
  • Figure 7 shows login pages for four targeted brands, and Figure 8 illustrates the survey; the study also collected demographic and brand-familiarity information.The four brands were Amazon, Facebook, PayPal, and Wells Fargo.Source: Figure 7 · p. 18, Figure 8 · p. 18, D. User Study · p. 18
  • The survey used 16 pairwise trials with similarity, confidence, and rationale questions; the final sample comprised 61 participants recruited in two waves.The primary group comparisons filtered low-confidence responses and used a balanced subset of 36 participants; 25 others contributed to supplementary analyses.Source: D. User Study · pp. 18–19

E. HTML Analysis Plan

  • The HTML analysis measures token length, DOM depth, tag counts, and unique tags, and uses MarkupLM to compare generated pages with legitimate references.Tag counts include CSS and JavaScript elements, while token length and DOM depth exclude them; CSS is analyzed separately, and MarkupLM excludes JavaScript and CSS.Source: E. HTML Analysis Plan · p. 19

F. CSS Styling Analysis Plan

The analysis identifies buttons and forms in generated pages using manual inspection and HTML heuristics, then reconstructs button styling by resolving CSS as a browser would.

F. CSS Styling Analysis Plan

  • 200 manually inspected samples inform heuristics for detecting buttons and form elements across the dataset.The analysis focuses on displayed buttons and form-related elements because they facilitate credential submission and other sensitive-data entry.Source: F. CSS Styling Analysis Plan · p. 19
  • 46.1% of samples use pseudo-forms, so the analysis treats qualifying generic containers as valid forms alongside standard <form> elements.Form candidates also need relevant class or ID keywords or enclosed interactive inputs, and unusually large or minimal structures are filtered out.Source: F. CSS Styling Analysis Plan · pp. 19–20
  • Button styles are recovered by combining internal, external, and inline CSS, then resolving properties according to CSS specificity.Inline styles receive the highest priority group, and invisible buttons are excluded.Source: F. CSS Styling Analysis Plan · p. 20
  • Buttons are identified through <button> tags, button-like input types, ARIA roles, or class and ID names containing “btn” or “button.”Elements styled with display: none are excluded from analysis.Source: F. CSS Styling Analysis Plan · p. 20

G. Three Steps for Checking Functionality

The paper describes browser-based testing of credential-harvesting functionality and supplementary analyses of screenshots, HTML tags, and user ratings. User ratings show a commercial-model advantage, while defect patterns distinguish some model categories but not others.

G. Three Steps for Checking Functionality

  • Playwright tests pages by injecting context-appropriate values into input fields and checking standard form submissions and JavaScript-handled asynchronous requests.A page counts as functional only if it completes credential harvesting and correctly stores user-provided information.Source: G. Three Steps for Checking… · p. 20

A. Additional Screenshot-Based Results

B. Additional Analysis of User Evaluation

  • Commercial models scored 2.64 mean UWS versus 1.84 for open-source and 1.66 for uncensored models; commercial models significantly exceeded both non-commercial groups.Open-source and uncensored models did not differ significantly; ordinal ratings across 968 responses had Krippendorff’s α = 0.54.Source: B. Additional Analysis of User… · p. 21
  • Commercial models were selected below aggregate rates for logo defects, while WhiteRabbitNeo and Dolphin were selected above them; Layout and Style defects were common across models.For Missing or Broken Logo, Gemini and GPT were selected at 20.7% and 35.3%, versus 90.7% for WhiteRabbitNeo and 86.0% for Dolphin.Source: B. Additional Analysis of User… · p. 21

A. Top-10 Tag Distribution Across LLMs

B. HTML Similarity

Generated HTML is highly similar to legitimate reference pages, and CSS patterns vary with model lineage while remaining stable when phishing intent is explicitly stated. The appendix also reports brand-dependent functionality variation and a shared persistence-related failure pattern.

B. HTML Similarity

  • Generated HTML similarity exceeds 0.92 across all four brands, with PayPal around 0.98 and DeepSeekVL2 at 0.93–0.96 across most brands.Adding “phishing” to prompts yields nearly identical HTML similarity distributions to unmodified prompts.Source: B. HTML Similarity · p. 22, Figure 12 · p. 22

C. HTML Style Heatmap

  • CSS styles are highly similar within architectural lineages, and Daredevil’s style similarity reaches 1.0 with and without explicit phishing keywords.The reported heatmaps cover form and button CSS across 18 evaluated models.Source: C. HTML Style Heatmap · pp. 22–23, Figure 11 · p. 22

D. CSS Properties for Legitimate Websites

  • All top-20 button CSS properties occur on at least three legitimate websites, while LLM-generated pages use a broader variety of CSS properties.The legitimate-site comparison covers four samples, six buttons, and eight forms; common properties include line-height in forms and background-color in buttons.Source: D. CSS Properties for Legitimate… · p. 23

APPENDIX D FUNCTIONALITY OF GENERATED WEBSITES

  • Facebook shows the largest within-model variation in functional-page counts; Amazon, PayPal, and Wells Fargo vary little within each model.Phi produces at most 9 functional pages for the other three brands but 49 for Facebook; the reported trends differ by model.Source: Appendix D · p. 23, Figure 13 · p. 23
  • Unsaved is the most frequent Facebook failure type at 46.5%, followed by Submit Timeout at 14.9%, and is the dominant model-level failure for nearly all models.The authors connect cross-model performance differences largely to handling this shared persistence bottleneck.Source: Appendix D · p. 23
Loading 2610.30226v1…