Source-linked AI summary
Secret MCP: Evidence-Bounded and Context-Isolated Design Specification Generation from Web Screenshots
Yeongjin Jo
TL;DR
Screenshot-to-code systems omit structure, interaction logic, responsive rules, and provenance, while multi-reference prompts risk transferring evidence or inferences between works. Secret MCP generates one auditable design specification per reference through separated, sequential processing and evaluates isolation and contract structure. The evaluation confirms orchestration and syntactic properties, but not semantic or visual reconstruction accuracy.
Problem
Screenshots omit document structure, interaction logic, responsive rules, and provenance, while multi-reference prompts risk transferring evidence or inferences between works.
Method
Secret MCP separates retrieval, evidence preparation, model invocation, storage, and inspection while processing references sequentially into one contract and artifact set per work.
Results
The evaluation confirms one request and one artifact per successful reference, prospective exclusion, evidence metadata recording, cross-reference detection, and required contract headings in tested fixtures.
Takeaways & Limitations
The reference-scoped sampler interface preserves isolation invariants across model APIs and transports, providing a practical baseline for auditable, model-agnostic design specification generation.
Takeaways & Limitations
The measures establish observable orchestration and lexical properties, not semantic correctness, because identifiers and required headings can be present even when content is wrong.
Abstract
from arXiv · showhide
Screenshot-to-code systems optimize for rendered implementations, but screenshots omit document structure, interaction logic, responsive rules, and provenance needed to distinguish observation from guesswork. Multi-reference prompts also risk contaminating one reference with evidence or inferences from another. We present Secret MCP, an open-source local system that produces one auditable design specification per public web reference. It separates retrieval, evidence preparation, model invocation, storage, and inspection. Long captures are resized and tiled with overlap; evidence records preserve prepared- and source-space coordinates and a measured color palette. A 19-section contract covers page inventories, navigation geometry, responsive matrices, components, accessibility, acceptance criteria, and explicit labels for measured, observed, inferred, and unknown claims. References are processed sequentially through a sampler interface. The evaluated MCP adapter sends one sampling/createMessage request per reference with includeContext set to none; a fresh-process adapter provides a stronger boundary. We evaluate commit c130c9c at two levels. A live retrieval and fixture-model integration run selected two references after excluding a third, prepared nine evidence images, issued two sampling requests, and produced two documents with zero cross-reference identifier occurrences. A static audit of three externally generated design indexes found all 19 required sections in every document, one unique reference identifier per document, eight page specifications, 1,943 pixel-valued measurements, and 319 color literals. These tests establish orchestration invariants and syntactic contract compliance, not semantic or visual reconstruction accuracy. The sampler abstraction preserves these boundaries across direct model APIs and transports despite MCP sampling's 2026 deprecation.
1 Introduction
Secret MCP addresses multi-reference screenshot analysis as a provenance and isolation problem, generating inspectable design specifications rather than directly proposing a new vision or language model. Its evaluation covers evidence preparation, a 19-section contract, and narrow reproducibility properties while explicitly withholding claims about visual fidelity.
- Multi-reference prompts can transfer colors, routes, components, or inferred interactions between works, while making provenance and partial-failure recovery difficult.
- Secret MCP processes each reference separately and stores its evidence, request contract, and document in a provenance-preserving DESIGN_INDEX artifact.
- The system contributes reference-scoped generation, long-screenshot evidence preparation, a 19-section specification contract, and reproducibility evaluation.
- The evaluation verifies system invariants and output structure, not visual fidelity, while human evaluation and rendered-page similarity experiments remain future work.
2 Related Work
Earlier screenshot-to-code systems generate executable code or benchmark model capabilities, whereas Secret MCP targets an explicit intermediate specification with reference-scoped orchestration. This distinction preserves uncertainty when screenshots incompletely support implementation decisions, without comparing model quality against prior benchmarks.
- Earlier systems generated platform-specific tokens, simplified HTML, or HTML from synthetic and instruction-tuning data.
- Recent benchmarks evaluate visual similarity, element completeness, relative layout, perception, programming, understanding, and end-to-end generation across real-world or large image–code datasets.
- Secret MCP instead produces an intermediate specification that labels values as measured, visible, inferred, or unknown when evidence is incomplete or conflicting.
- MCP sampling provides the adapter capability for delegating one analysis to the host model, while direct API adapters can preserve the same request and storage invariants.
3 Problem Formulation
The formulation constructs per-reference evidence and contracts, invokes a sampler for each reference, and stores each resulting document before preparing the next reference. Its invariants target isolation, local failure handling, and inspectability, with stronger execution boundaries requiring fresh processes and workspaces.
- For each reference r_i, evidence E_i is prepared from source screenshots, a contract C_i is constructed, and sampler G returns markdown D_i.Evidence items include image bytes, a coordinate transform, and palette measurements.
- The system aims to preserve invariants governing reference-scoped inputs, context boundaries, sequential storage, and inspectable per-item artifacts.
- D_i is stored before preparation of r_i+1, making partial failures local and observable.
- The MCP adapter requests includeContext: none, but fresh processes and workspaces are needed when execution-level non-reuse is required.
4 System Design
Secret MCP keeps retrieval, evidence preparation, contract construction, model invocation, and storage reference-scoped, while producing auditable design indexes from normalized screenshot evidence. Its contract distinguishes measured, observed, inferred, and unknown claims, and its evaluation preserves local failures, context boundaries, and inspectable artifacts.
- Retrieval and exclusion: Filtering occurs before evidence loading and model requests, while the server internally retains selected references and constructs their processing sequence.Persistent exclusions affect future searches without deleting prior evidence or documents.
- Long-image evidence preparation: Long desktop screenshots are normalized, downscaled to width Wp ≤1200 pixels, JPEG-encoded, tiled at heights up to 1,600 pixels, and recorded in prepared and source coordinates.Tiles overlap vertically by 80 pixels; mobile images remain separate evidence rather than being merged with desktop tiles.
- Measured palette extraction: Palette extraction resizes each tile to 180 × 180, converts pixels to raw RGB, quantizes channels into 17-unit multiples, and stores eight frequent bins with color formats and coverage.Coverage for bin b is n_b/N rounded to four decimals, and sampled colors are labeled Measured rather than recovered CSS tokens.
- Specification contract: The DESIGN_INDEX contract requires 19 sections covering page canvases, ordered content, components, state rules, responsive transitions, assets, accessibility requirements, and acceptance checks.A long scroll remains one page unless distinct route screens are visibly present.
- Generation, isolation, and inspection: The generator persists each artifact before advancing, marks empty, non-text, timeout, or token-limit responses as item-local failures, and stores contracts, documents, evidence, and logs separately.The MCP adapter requests includeContext: none, while a fresh process and workspace provide stronger execution-level non-reuse when required.
5 Evaluation
The evaluation tests Secret MCP’s repository health, reference-isolated orchestration, evidence persistence, and syntactic compliance with its 19-section contract. Results support structural and orchestration invariants, but not visual fidelity or semantic reconstruction accuracy.
- RQ1: Build health: All RQ1 checks passed: installation, vulnerability audit, TypeScript compilation, and ESLint validation succeeded.The audit reported zero vulnerabilities, with no compilation or lint errors or warnings.
- RQ2: Isolation and artifacts: The live RQ2 run selected two references after excluding one, issued two sampler requests, and produced two output files.The run prepared nine images and preserved source and crop coordinates, palettes, contracts, models, documents, and completion timestamps.
- RQ2: Isolation and artifacts: No selected request contained the other selected reference identifier, and the excluded identifier was never sampled.This directly tests cross-reference isolation and prospective exclusion in the fixture-backed integration run.
- RQ3: Contract compliance: The RQ3 audit found all 19 required section names and exactly one unique GDWEB identifier per document.Across three documents, it also counted eight page specifications, 1,943 pixel-valued measurements, and 319 six-digit color literals.
- Interpretation and limits: These results establish buildability, reference-scoped orchestration, evidence metadata persistence, and lexical contract compliance, not screenshot understanding or pixel-perfect reconstruction.The audit explicitly warns that identifier isolation can miss semantic style transfer and lexical measurements can be numerically incorrect.
6 Discussion
Secret MCP argues that an intermediate, reference-scoped specification makes screenshot analysis auditable and implementation-agnostic. Its isolation layers reduce accidental mixing, but adapter security, protocol evolution, and rights constraints remain practical boundaries.
- Why an intermediate specification?: An intermediate specification separates screenshot evidence, implementation needs, and model guesses while allowing downstream framework choice.The same page specification can guide React, static HTML, game-engine web export, or manual development.
- Why an intermediate specification?: Stable page and section identifiers, canonical coordinates, responsive matrices, and acceptance criteria make specifications useful for multi-page and long-scroll evidence.Later implementations can be evaluated against explicit checks rather than a vague visual-similarity instruction.
- Isolation layers: Reference-scoped requests, sequential generation, separated paths, and single-document viewing reduce accidental cross-reference mixing and expose failures.The design maintains separate evidence, requests, and document bodies for each work.
- Isolation strength: includeContext: none is not a security boundary because clients may alter parameters and persistent model processes may retain hidden state.Execution-level non-reuse requires a trusted fresh-process adapter and workspace, or a direct API request built solely from one reference and its evidence.
- Protocol evolution: Sampling’s 2026 deprecation affects the current MCP adapter, while the dependency-injected sampler preserves migration paths that retain the system’s invariants.Direct provider integration changes the trust model by adding credential, retention, retry, and consent requirements.
- Rights and use: Public screenshots and assets may still require permission for reproduction or redistribution, so the system is intended for analysis, planning, or authorized reconstruction.Users are instructed to adapt copyrighted copy and brand assets unless authorized.
7 Threats to Validity and Limitations
The evaluation establishes lexical and orchestration properties rather than semantic, visual, or broadly generalizable reconstruction accuracy. Its main limitations concern narrow contamination proxies, fixture-based testing, limited scope, missing rendered benchmarks, and operational dependencies.
- Construct validity: Identifier counts are only a narrow contamination proxy, because visual patterns can transfer without identifier copying.Required headings and numeric literals can also appear when content is incorrect.
- Construct validity: Lexical section and literal counts establish observable structure, not semantic correctness.A stronger validator would check evidence links, coordinate consistency, page coverage, and executable acceptance criteria.
- Internal validity: The deterministic fixture sampler tests request construction and artifact persistence without measuring model quality, cost, or latency.Live retrieval results also depend on the GDWEB state observed during the run.
- External validity: The live test used two selected works and one excluded work, while the bundled audit covered three documents from one domain scenario.Results may not generalize across galleries, languages, page lengths, route counts, or mobile coverage.
- Missing downstream evaluation: No rendered reconstruction benchmark compares implementations produced from the specifications with reference pages using visual, element, or layout metrics.A blinded developer study is proposed as a stronger downstream evaluation.
- Operational limitations: Live retrieval depends on GDWEB markup and image endpoints, and MCP sampling is deprecated as of specification version 2026-07-28.Proposed priorities include a direct provider adapter, conformance tests, evidence hashes, and further evaluation of explicit Unknown labels.
8 Reproducibility and Availability
The paper provides the source, documentation, viewer, bundled indexes, audit script, and a reproducible verification procedure. The live smoke test uses deterministic fixture text, so it does not require external model credentials.
- Availability: The public repository contains source code, English and Korean documentation, verification procedures, a viewer, bundled design indexes, and an audit script.The evaluated revision is commit c130c9c80876ddf9594df9517174a163f48d523f.
- Reproduction: The core reproduction commands install dependencies, audit, build, lint, run the GDWEB isolation smoke test, and evaluate artifacts.The commands are listed as npm ci, npm audit, npm run build, npm run lint, npm run smoke:gdweb-isolation, and node paper/evaluate_artifacts.mjs.
- Reproduction: The live smoke test depends on current GDWEB availability but uses deterministic fixture text and requires no external model credential.The arXiv package includes main.tex, references.bib, and generated main.bbl.
9 Conclusion
Secret MCP reframes multi-reference screenshot analysis as an evidence and isolation problem, producing reference-scoped specifications with explicit provenance and uncertainty. Its evaluation confirms orchestration and structural properties while leaving semantic and visual accuracy open.
- Conclusion: Secret MCP preserves recoverable coordinates, separates measurement from inference, requires a page-complete contract, and stores one request and artifact set per work.Its sampler interface also separates durable invariants from the deprecated MCP sampling transport.
- Conclusion: The evaluation confirms orchestration and structural properties but does not establish semantic or visual accuracy.The conclusion therefore presents the system as a practical baseline for auditable, model-agnostic design specification generation.