PaperlayerView exact revision on arXiv

Source-linked AI summary

ProductWebBench: Evaluating Website Continuity Beyond Task Completion

Jiajun Wu, Jian Yang, Yaxin Du, Shukai Liu, Linzheng Chai, Tuney Zheng, Wei Zhang, Haitao Yang, Yan Bai, Yihang Lou, Xianglong Liu, Bryan Dai

arXiv:2610.31118v1cs.SEcs.AI

TL;DR

Frontend agents may implement a requested feature while disrupting previously working website behavior, a continuity obligation not captured by target completion alone. ProductWebBench evaluates that obligation through staged tasks and cumulative browser checks. Across its runs, requested-work completion often coexists with continuity or interaction failures.

  • Problem

    Frontend changes can satisfy a local requirement while breaking previously working states, workflows, or visual consistency, so evaluation must consider the website beyond the patch.

    Source: §1 · p. 1

  • Method

    ProductWebBench evaluates 320 Change and 80 Build tasks with progressively revealed requirements and checks that keep previously cleared behaviors active throughout the run.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    28.2% of Change runs passing every requested-work check still failed continuity checks, and 10.8% regressed browser states they had previously passed.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Requested-work completion and preservation of established website behavior are separate challenges in frontend development.

    Source: §1 · p. 2

    The finding that from-scratch construction scores below matched in-place changes is established on 21 length-matched tasks, mostly static or vanilla-node sites, and is not claimed for arbitrary stacks.

    Source: H LIMITATIONS AND SCOPE · p. 25

Abstract

from arXiv · show

Frontend coding agents are usually graded on one question: did the requested target get built. Real engineering changes carry a second obligation, because the rest of the site has to keep working. We call that second obligation website continuity, and introduce ProductWebBench, a benchmark that scores both on living frontend repositories: 320 Change tasks over existing code and 80 long-horizon Build tasks. Every task freezes a repository snapshot, staged requirements, browser evidence and deterministic checks before evaluation, and enters only once a reference passes every check and plausible-wrong ones fail. A simulated user reveals the requirements one stage at a time, and every check cleared earlier stays armed for the rest of the run. Across 8,737 runs from 13 models the two come apart: of the Change runs passing every requested-work check, 28.2% still fail a continuity check -- a blank route, horizontal overflow, console errors -- and 10.8% push a browser state they had already passed back to failing. Because the same checks also run on the untouched repository, each regression carries exact provenance, not a post-hoc label: on the 586 runs where both readings exist, 18.3% are caught breaking a verified state. Interaction is a further blind spot, failing in 29.5% of runs whose static DOM already passes. ProductWebBench therefore scores a patch against the living system it lands in, not the target alone.

1 INTRODUCTION

Frontend changes can satisfy a local request while breaking previously working states, workflows, or visual consistency, so success must account for the website the patch enters. ProductWebBench evaluates that continuity alongside requested work, and its results show the two can diverge.

1 INTRODUCTION

  • Frontend changes can propagate beyond the requested feature to shared state, checkout flows, responsive layouts, and components across routes.Existing benchmarks cover parts of the problem but do not evaluate continuous improvement of an existing website while preserving established behavior.Source: §1 · pp. 1–2
  • ProductWebBench evaluates long-horizon frontend agents on 320 Change tasks and 80 Build tasks using progressively revealed requirements and a cumulative workspace.Its design tests whether agents complete requested changes and preserve behaviors established during development.Source: §1 · p. 2
  • 28.2% of 674 Change runs passing every requested-work check still failed continuity checks, and 10.8% regressed browser states they had previously passed.Across 8,737 valid trajectories from 13 models, interaction checks also failed in 29.5% of runs whose target DOM requirements passed.Source: §1 · p. 2
  • Across 586 covered completing runs, two regression channels with exact provenance jointly caught 18.3% damaging a verified state; 13.3% still failed a continuity check at the end.The channels use states previously passed during a run and gates passed by the untouched repository.Source: §1 · p. 2

2 EVIDENCE-BACKED TASK CONSTRUCTION

ProductWebBench packages each task as a frozen repository snapshot, ordered change request, and executable evidence of required behavior. Tasks use captured browser evidence to define stages and are admitted only when the reference passes and plausible wrong implementations fail.

2.1 TASK PACKAGE AND REGIMES

  • Figure 2 depicts repository states supplying evidence for ordered stages, with admission requiring a passing reference and failing baseline and smart-wrong controls.Admission also requires a distinct artifact.Source: Figure 2 · p. 3
  • A task package Q contains the repository snapshot, allowed paths, stage plan, browser evidence, deterministic checks, reference patch, and negative controls.The reference patch and negative controls are used for admission and never reach the solver.Source: §2.1 · pp. 2–3
  • The 400-task suite contains 320 Change tasks on runnable repositories and 80 Build tasks with 5–14 ordered milestones, spanning multiple frontend stacks.Change tasks target a continuity facet, and repositories must be installed, built, served, and exercised before authoring.Source: §2.1 · p. 3

2.2 BROWSER EVIDENCE AND STAGE CONSTRUCTION

  • Browser states record route, viewport, and action prefix; requirements and preservation gates are grounded in those states and ordered by their dependencies.Captured evidence includes screenshots, visual anchors, DOM structure, and accessible names; prior-state gates are replayed at every checkpoint.Source: §2.2 · p. 3

2.3 FROZEN CONTRACT AND ADMISSION CONTROLS

  • Every package field is frozen, and admission requires the reference patch to pass every applicable check while plausible wrong implementations fail.The controls include an inert but convincing control, a layout that overflows at one viewport, and a local edit that drops an existing route.Source: §2.3 · pp. 3–4

3 CUMULATIVE MULTI-TURN EVALUATION

Evaluation presents staged requirements in a cumulative workspace and keeps cleared milestones active as obligations at later checkpoints. Verification distinguishes completion from continuity, while the Website Continuity Score retains partial progress and penalizes regressions.

3 CUMULATIVE MULTI-TURN EVALUATION

  • Cleared milestone assertions remain standing obligations and are retested after every later edit, so evaluation reflects the trajectory rather than only its endpoint.Rework reuses the workspace, including successful edits and unresolved state.Source: §3 · p. 4, §3.2 · p. 5

3.1 TASK PRESENTATION AND SIMULATED USER

  • In staged evaluation, a capability-conditioned simulated user reveals pre-authored work units, while the harness applies edits, serves the project, checks browser states, and returns grounded failures.Figure 3 shows alternative PM, QA, or designer personas; the harness checks each milestone and replays prior gates.Source: §3.1 · p. 4, §3.2 · p. 4, Figure 3 · p. 4

3.2 WORKSPACE UPDATES AND MILESTONE REPLAY

  • A milestone clears only when the workspace builds, serves, reaches the specified route, viewport, and action state, and passes its assertions; later checkpoints replay earlier gates.A patch that fixes the current stage but invalidates an earlier state is recorded as a regression.Source: §3.2 · p. 5

3.3 VERIFICATION AND CONTINUITY SCORE

  • Whole-task pass requires every stage and rework to clear with no regression breaks, whereas final gate pass checks all 19 gates only on the final workspace.The Website Continuity Score combines stage progress, deterministic gate pass rate, final gate pass, and regression breaks; it retains partial progress while penalizing regressions. Completion gates measure requested work, while continuity checks cover regression text, visual anchors, layout, state artifacts, and runtime health.Source: §3.3 · p. 5

4 EXPERIMENTS

The experiments use a frozen, staged evaluation protocol whose task references and wrong-solution controls are validated before model runs. Results show sparse whole-task passes, distinct model rankings across measures, and substantial score variation across scaffolds and backbones.

4.1 SETUP

  • Table 3 frames the Change results as completion versus additional checks across 6,979 cells.The evaluation uses the same frozen 19-gate verifier at every checkpoint of each run.Source: Table 3 · p. 6, §4.1 · p. 6
  • All 479 validated reference implementations pass and all 479 smart-wrong controls fail, establishing gate discrimination before model evaluation.This is an admission property of the authoring pool, not an estimate of solver accuracy on unseen output.Source: §4.1 · p. 6

4.2 MAIN RESULTS

  • Only 198 of 8,737 runs pass every stage without a regression, and whole-task rates span 1.25–3.50%; Gate% spans 49.41–61.71 and gives a different top-model ordering.Claude-Opus-4.8 leads whole-task success, while GPT-5.5 leads Gate% and Stage%.Source: §4.2 · p. 6
  • On the matched 4 × 2 grid, changing scaffolds moves the component mean by 8.06 points on Sonnet-4.6 and 10.52 points on the other reported backbone.The Figure 4 caption reports a 5.08-point change when swapping the backbone under a fixed scaffold on GPT-5.5.Source: §4.2 · p. 6, Figure 4 · p. 7

5 ANALYSIS

The analysis separates requested-work completion from continuity failures and tracks regressions through both replayed states and untouched-repository baselines. It also finds that Build and Change differ, while prompt changes can shift Gate% even when other interventions do not.

Completion and continuity

  • 28.2% of Change runs passing every completion-family gate still fail a continuity check, rising to 64.1% under the visual-inclusive reading.Among interactive cells whose static DOM requirement passes, interaction assertions still fail 29.5% of the time.Source: §5 · p. 7

Regression channels

  • 10.8% of completing runs regress a state they previously passed; on runs with both channels observable, the two provenance channels catch 18.3% damaging a verified state.The untouched-repository channel catches failures of gates that passed before the run; the protected-state channel replays states established during the run.Source: §5 · p. 7, Table 4 · p. 7

Long-horizon construction

  • Build WCS is 0.240 versus 0.348 for Change on the nine-model dual-modal panel, and matched greenfield twins score 11.82 points lower than source Change tasks.The twin comparison covers 105 paired cells, with all five model-legs negative; the pooled Build–Change gap alone cannot separate regime from trajectory length.Source: §5 · pp. 7–8, Figure 6 · p. 8

What moves the continuity signal

  • Truncating solver history to five turns drops Gate% by 9.40 points, while adding a preservation clause lifts it by 7.81 points.Changing simulated-user wording, edit format, or screenshot availability moves nothing distinguishable from zero.Source: §5 · p. 8, Figure 7 · p. 9

6 RELATED WORK

Prior benchmarks cover interface generation, repository-level code changes, and browser agents, but do not combine these concerns as cumulative verification under a frozen contract. ProductWebBench grades requested completion, additional verification failures, and provenance-exact regressions separately.

Frontend and repository benchmarks

  • Prior work spans frontend generation and repair, executable-test repository benchmarks, and browser-agent environments, with lifecycle coverage, feedback, and preservation as shared concerns.The cited benchmark families include WebCompass, SWE-bench, and SWE-bench Multimodal.Source: §6 · p. 9

Browser environments, simulated users, and what PRODUCTWEBBENCH adds

  • ProductWebBench binds requirements to route, viewport, and action-history evidence, stages work with a simulated user, and replays cleared states at later checkpoints.A trajectory yields separate quantities for requested completion, additional verification failure, and provenance-exact regression.Source: §6 · p. 9

7 CONCLUSION

ProductWebBench evaluates frontend changes against the continuing site rather than the target alone. Its results show that completion checks alone do not establish continuity, while the reported scope also supports recording the scaffold alongside agent capability scores.

7 CONCLUSION

  • Across 13 models, 28.2% of Change runs passing every completion gate still fail a conservative check, and 18.3% of runs with both readings show provenance-exact damage to a verified state.The latter figure is from 586 runs; 13.3% of those still fail a conservative check at the end.Source: §7 · p. 9

ETHICS STATEMENT

The benchmark evaluates code-generating agents on public frontend repositories without human subjects, personal data, user studies, deployment, or production credentials; its scores describe systems under one fixed protocol, not general vendor capabilities.

ETHICS STATEMENT

  • The study uses public frontend repositories and involves no human subjects, personal data, or user studies.Runs are sandboxed, with no public-service deployment or production credentials; released materials retain upstream provenance and license.Source: ETHICS STATEMENT · p. 10
  • Reported scores describe systems under one fixed protocol and should not be read as general capability claims about vendors.The paper notes that model coverage is unbalanced where provider availability differs.Source: ETHICS STATEMENT · p. 10

USE OF LARGE LANGUAGE MODELS

The section records the benchmark’s task and gate conventions, staged evaluation interface, and checks on verifier validity. Its audits quantify agreement and error patterns while identifying limits on interpreting conditional regression estimates.

A TASK AND FACET INVENTORY

  • The locked Change split has 320 task slots, but only 304 carry capability labels; overall and per-capability columns therefore use different denominators.The labeled slots are 106 Edit, 76 Repair, and 122 Consistency.Source: A TASK AND FACET INVENTORY · p. 13

B GATE FAMILIES

  • Gate families are derived from frozen verifier names; visual regression is excluded from the conservative family because its reference-render comparison does not establish target-versus-prior provenance.A configured state-URL signal never fails, but Gate% remains divided by all 19 signals; using the 18 that can fail lowers model scores by two to three points without changing Table 1 ordering.Source: B GATE FAMILIES · p. 13

C PRESENTATION INTERFACES AND EVALUATION ARMS

  • The two presentation arms are reported separately; scaffold and staged scores are not comparable as model-quality estimates because their task-success measures differ.The staged arm requires the whole-task notion, including maintaining previously passing states; the scaffold arm grades the final gate pass.Source: C PRESENTATION INTERFACES AND… · pp. 13–14
  • One capability-conditioned simulated user reveals preauthored requirements and grounded feedback while keeping acceptance fixed across models.Multi-role interaction is identified as a natural extension on the same frozen states and gates.Source: C PRESENTATION INTERFACES AND… · p. 14

D VERIFIER VALIDITY

  • On 359 cases where both blind raters committed, they agreed 94.2% of the time (κ = 0.820); against consensus, the verifier reached κ = 0.766.The consensus comparison used 338 cases; the false-pass rate was 0.044 [0.025, 0.075], and the false-fail rate was 0.190.Source: D VERIFIER VALIDITY · p. 14
  • The audit rejudged sampled solver-output verdicts using captured evidence without access to gate implementations, reference patches, or verifier verdicts.The audit’s agreement varies by capability and gate family, with κ = 0.657 on Repair and κ = 0.711 on continuity.Source: D VERIFIER VALIDITY · p. 14
  • Conditional post-completion figures are not conservative bounds: false fails on completion gates can change the completing sample’s composition in an unknown direction.False passes can also admit runs that do not belong in that subset; the 4.4% false-pass rate affects its denominator.Source: D VERIFIER VALIDITY · p. 14
  • False fails in the audit sample all fell on completion-family gates, while none of seven consensus FAIL verdicts on conservative continuity gates was false.The continuity sample is too small to bound its false-fail rate tightly (Wilson-95 [0%, 35.4%]).Source: D VERIFIER VALIDITY · p. 14

E STATE PROVENANCE AND THE REGRESSION ESTIMATE

The section separates post-completion failures from pass-to-fail transitions in states established during a run, then uses untouched-repository baselines to identify provenance-exact regressions. It also qualifies these estimates by baseline coverage, task conditioning, and unresolved experimental evidence.

E STATE PROVENANCE AND THE REGRESSION ESTIMATE

  • 28.2% is a post-completion verification gap: 190/674 runs passing requested-work checks still fail a conservative non-completion check, and only 35 also record protected-state regression.The gap does not identify whether a failure is a regression in an initial state, a newly established state, or a global hygiene check that was already failing.Source: E STATE PROVENANCE AND THE REGRESSION… · pp. 15–16
  • A protected-state regression is recorded when an assertion that passed at an earlier cleared stage fails at a later checkpoint; the harness rechecks the accumulated protected set.The sequence is frozen in advance, and admission ensures the reference passes every protected assertion at each later checkpoint.Source: E STATE PROVENANCE AND THE REGRESSION… · p. 16
  • The two measures partly overlap: 155 completing cells fail conservatively without a recorded replay regression, while 38 record a mid-run regression but finish with conservative checks clean.Neither the final-state gap nor replay alone captures the other channel’s cases.Source: E STATE PROVENANCE AND THE REGRESSION… · p. 16, Table 8 · p. 16
  • 18.3% of 586 completing cells with an untouched-repository baseline show provenance-exact regressions across the two channels, compared with 9.7% from protected-state replay alone.Across all 674 completing runs, the protected-state rate is 73/674 = 10.8%.Source: E STATE PROVENANCE AND THE REGRESSION… · p. 16, Table 9 · p. 17
  • Untouched-repository baselines cover 234 of 320 Change slots; among covered completing cells, 50 fail a gate the original repository passed and 76 fail a gate already failing before the run.The other 29 cells in this comparison have no baseline because their repositories do not render as handed.Source: E STATE PROVENANCE AND THE REGRESSION… · p. 16
  • Baseline exclusions constrain the provenance estimate: 86 slots lack a baseline, and excluded slots skew toward stacks with a build step, though exclusion is not specific to bundlers.The protected-state rate on the covered completing subset is 9.7% versus 10.8% across all completing cells; the paper detects no shift of that size.Source: E STATE PROVENANCE AND THE REGRESSION… · p. 16
  • The 10.8% protected-state rate is descriptive across models because task sets differ and models clearing more stages have more checkpoints at which flips can be observed.Per-model comparisons are therefore limited to commonly completed tasks.Source: E STATE PROVENANCE AND THE REGRESSION… · pp. 16–17
  • Replay is a trajectory measure: 38 of 73 runs recording a regression finish with every conservative check clean, making the trajectory rate roughly twice the end-state rate.The untouched-repository channel is a final-state measure, so the paper reports both observation points rather than merging them.Source: E STATE PROVENANCE AND THE REGRESSION… · p. 17

F ADDITIONAL RESULT DETAIL

The additional analyses show that target completion can coexist with continuity failures, and clarify how regression evidence, long-horizon behavior, model comparisons, and evaluation design qualify the results.

F ADDITIONAL RESULT DETAIL

  • The locked evaluation suite uses 400 of 479 validated tasks, while only 234 Change tasks provide a capturable initial baseline; repository-level clustering is not reported.The released manifest permits that grouping to be recomputed, but it is not part of the reported analysis.Source: F ADDITIONAL RESULT DETAIL · pp. 17–18
  • The Build–Change WCS gap tracks stage completion, not gate quality; 98.3% of failing cells land at least one edit, while only 1.7% land none.The failure counts bound, but do not eliminate, patch-tooling explanations.Source: F ADDITIONAL RESULT DETAIL · p. 18, Table 12 · p. 18
  • On the matched scaffold grid, scaffold spread is 8.06 points on Sonnet-4.6 and 10.52 on GPT-5.5, versus 5.08 for a backbone swap; the paired backbone swap favors GPT-5.5 on every scaffold.Scaffold rank order is largely preserved across backbones (Spearman ρ = 0.80), so the factors are close to additive.Source: F ADDITIONAL RESULT DETAIL · p. 20
  • Regression-continuity is the easiest facet for all 13 models, construction is hardest for 12, and the same category ordering holds on the smallest task quartile.The category comparison also survives edit-size control.Source: F ADDITIONAL RESULT DETAIL · pp. 20–21, Figure 13 · p. 24
  • On Build tasks with 7–14 stages, turns and stages cleared correlate negatively in every bucket, with a median of −0.91; failed units can consume their full 12-turn budget.The negative relation is limited to long tasks; five- and six-stage buckets have positive correlations, and the turn cap means this is not a causal estimate of what extra turns buy.Source: Figure 9 · p. 19, F ADDITIONAL RESULT DETAIL · p. 21, G DISCUSSION · p. 23
  • Screenshot assignment changes WCS by −0.0007, indistinguishable from zero, and no robustness-ablation arm yields a net strict-pass gain.The screenshot arm improves the experience facet but not aggregate performance; prompt-scale intervention effects are reported at the gate level, not the protected-state endpoint.Source: F ADDITIONAL RESULT DETAIL · p. 21, Table 13 · p. 21

G DISCUSSION

  • 28.2% of Change runs passing every requested-work gate still fail a conservative continuity check, and 29.5% of target-DOM-passing interactive runs fail interaction checks.Both failures are invisible to an endpoint DOM assertion.Source: G DISCUSSION · p. 22
  • 18.3% of runs with both readings break a state verified in the untouched repository, while 10.8% of completing runs break a state they had previously cleared.The untouched-repository checks and replayed protected states provide two distinct provenance channels; some replay-channel damage ends with all conservative checks clean.Source: G DISCUSSION · p. 22

H LIMITATIONS AND SCOPE

The scope notes qualify the category, long-horizon, twin-task, and intervention findings, and identify limits in coverage and measurement.

H LIMITATIONS AND SCOPE

  • The category result survives edit-size control, and the category ordering is unchanged on the smallest quarter of tasks.The figure compares category coefficients against the Edit baseline while controlling for task size and model.Source: Figure 14 · p. 25
  • The greenfield-twin result is established on 21 length-matched Change tasks, mostly static or vanilla-node sites, not arbitrary stacks.The authoring chain yields blank baselines only for those stacks.Source: H LIMITATIONS AND SCOPE · p. 25
  • Turn count is endogenous: failed units consume the stage cap while cleared units stop early, so persistence to the cap is not evidence of what extra turns cause.The result is reported as an observation about trajectory endings, not a causal estimate.Source: H LIMITATIONS AND SCOPE · p. 25
  • Coverage is uneven and cells share tasks; task-clustered intervals address repeated cells, but repository-level clustering is not reported.The released manifest includes source paths for recomputing repository groups.Source: H LIMITATIONS AND SCOPE · p. 25
  • At temperature zero, across fifteen pairs of one slot and one model replayed three times, mean lh-score standard deviation was 0.093, while binary verdicts stayed unchanged in all fifteen pairs.Replays re-executed the whole solver loop, so this measures end-to-end run variability.Source: Figure 15 · p. 26
  • The scaffold comparison is limited to two backbones and four shared scaffolds, with time and token budgets not equalized.Any budget difference therefore remains part of the measured scaffold effect.Source: Figure 16 · p. 26
  • Visual anchors check authored properties, not unrestricted aesthetic quality; prompt-scale interventions target gate metrics, not the rarer protected-state endpoint.Testing effects on that endpoint requires a separately powered arm.Source: H LIMITATIONS AND SCOPE · p. 26

I REPRODUCIBILITY AND RELEASE SAFEGUARDS

The paper documents cross-environment calibration and release safeguards, then illustrates how staged contracts test live behavior and preserve earlier requirements. The selector case shows that a visually active control is insufficient unless interaction changes the page as specified.

I REPRODUCIBILITY AND RELEASE SAFEGUARDS

  • Five served workspaces support, but do not establish, cross-environment equivalence for the 18 gates used in the class-(a) statistic.All served recaptures matched the cluster on nonvisual gates; the authors say a same-environment initial-versus-final replay would settle the question.Source: I REPRODUCIBILITY AND RELEASE SAFEGUARDS · p. 27
  • Across 13 slots and 247 gate verdicts, late baseline recaptures agreed with frozen baselines on 85%, with disagreements concentrated in environment-sensitive checks.The late captures moved the union count from 106 to 107; the paper treats each slot’s baseline as environment-conditional.Source: I REPRODUCIBILITY AND RELEASE SAFEGUARDS · p. 27

J EVIDENCE-BASED CASE STUDIES

J.1 CASE (1): A STAGE SELECTOR THAT HAS TO ACTUALLY REWIRE THE PAGE

  • A selector stage requires a real click, a changed state, and the exact “Showing 2 of 5” readout; a merely highlighted chip fails the contract.Clicking Launch also changes the selected-project panel. The case clears s2–s4 and all 19 final gates, but not s1 or either rework, yielding a long-horizon score of 0.875.Source: §J.1 · p. 28

J.2 CASE (2): NEON ALBUM CONTINUITY ACROSS A CONSISTENCY TASK

These cases show how PRODUCTWEBBENCH checks both requested changes and what must remain working as edits accumulate. Examples range from preserved album content and scroll interactions to full staged construction, contract discrimination, and regression provenance.

J.2 CASE (2): NEON ALBUM CONTINUITY ACROSS A CONSISTENCY TASK

  • In the Neon Dreams task, the new rail tile is added while the artwork, ordered ten-track list, byline, and release metadata remain consistent with the reference.The contract also rechecks the rail interaction, distinguishing a valid tile from a placeholder.Source: §J.2 · pp. 28–29, Figure 18 · p. 30

J.3 CASE (3): A CINEMATIC 3D SCROLL SURVIVES REVALIDATION

  • After a scene-composition edit, the demo-2 scene renders while the ten chapter cues, scroll-linked cylinder, and runtime metadata remain verified.The replay checks chapter navigation and the scene-director binding; the accompanying figure shows the second scene variant and bound scroll interactions.Source: §J.3 · pp. 29–31, Figure 19 · p. 31

J.4 CASE (4): AN ANALYTICS DASHBOARD PASSES EVERY CHECKPOINT

  • All nine dashboard stage and rework units clear within budget, and the final replay passes all 19 gates with the reference KPI values and chart series.The verified state includes the Support/Traffic toggle, notifications rail, and mobile containment at 390 pixels.Source: §J.4 · pp. 31–32, Figure 20 · p. 32

J.5 CASE (5): THE CONTRACT REJECTS AN AUTHORED NEAR-MISS

  • All 479 references pass and all 479 authored controls fail, so the contract rejects plausible near-misses before model evaluation.This lets subsequent failures be read as model behavior rather than impossible requirements or gates that accept plausible regressions.Source: §J.5 · p. 33, Figure 21 · p. 33

J.6 CASE (6): A CLEAN ENDPOINT ON TOP OF A DAMAGED TRAJECTORY

  • Eleven previously cleared assertions fail during the dental-site run and are repaired before its final replay, leaving a clean endpoint that conceals the intervening flips.The case belongs to the 38 of 674 completing runs that regress a protected state yet finish with every conservative check clean.Source: §J.6 · pp. 33–34, Figure 22 · p. 34

J.7 CASE (7): A PROPERTY THE UNTOUCHED REPOSITORY ALREADY HAD

  • The explorer repair passes its requested checks, but the untouched-repository channel catches that its previously clean browser console no longer passes.Protected-state replay cannot detect this because no stage asserted the console; the figure likewise notes that the console failure is not visible in the image.Source: §J.7 · p. 34, Figure 23 · p. 35
Loading 2610.31118v1…