PaperlayerView exact revision on arXiv

Source-linked AI summary

A11yLTLNav: Automatic Detection of Accessibility Navigation Failures

Chenming Ge, Kewen Peng, Chengyang Shi, Ben Greenman, Yue Jiang

arXiv:2609.17959v1cs.HCcs.SE

TL;DR

Static snapshots and many automated checkers miss accessibility failures that emerge as BLV users interact with websites. A11yLTLNav combines a failure taxonomy, executable temporal properties, and random keyboard exploration to check interaction behavior. Across 31 website environments, it reported 309 failures, confirmed 274, and achieved 88.7% precision.

  • Problem

    Static accessibility checkers primarily assess single-state properties and miss runtime navigation failures that emerge only during interaction.

    Source: §1 · p. 2

  • Method

    A11yLTLNav formalizes browser-observable failures as executable LTL properties over action-state traces and checks them during random keyboard exploration.

    Source: §1 · p. 3, §2.4 · p. 5

  • Results

    88.7% precision across 31 website environments; A11yLTLNav confirmed 274 of 309 reported failures and identified more confirmed failures than comparison checkers.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The approach turns documented accessibility failures into reusable checks of interface behavior across interaction sequences.

    Source: §7 · p. 18, §8 · p. 20

    A11yLTLNav checks browser-observable interaction states, but does not verify actual screen-reader announcements or cover failures requiring other assistive-technology or cross-view evidence.

    Source: §7.2 · p. 19

Abstract

from arXiv · show

For blind and low-vision (BLV) screen-reader users, a website that appears accessible in a static snapshot can become difficult or impossible to navigate once interaction begins. Yet, most automated accessibility checkers miss failures involving focus, interface state, and accessible feedback across interactions. We present A11yLTLNav, a property-based approach for automatically detecting accessibility navigation failures. Through a structured review of prior research, we organize accessibility navigation failures into a failure taxonomy and formalize a browser-observable subset as executable Linear Temporal Logic properties over action-state traces. A11yLTLNav combines random keyboard exploration with runtime property monitoring to detect these failures during interactions. We evaluate A11yLTLNav on 31 generated websites based on real-world websites and tasks. It reported 309 accessibility failures, of which 274 were confirmed, achieving 88.7% precision and identifying more confirmed failures than the comparison checkers. Our results show that A11yLTLNav transforms accessibility knowledge into reusable checks of interface behavior over time.

1 Introduction

A11yLTLNav targets navigation failures that emerge during interaction, which static snapshots can miss. It combines a failure taxonomy and executable temporal properties with keyboard exploration; evaluation found 274 confirmed failures among 309 reports across 31 website environments.

1 Introduction

  • A static snapshot can appear accessible even when later interface changes make navigation difficult, block access to new content, or provide insufficient feedback.Such barriers can prevent users from continuing to navigate or completing a task, even when individual elements have appropriate labels or roles.Source: §1 · p. 2
  • The unified taxonomy organizes documented failures by the evidence needed to detect them and distinguishes browser-observable behavior from failures needing additional context.Its categories include element-, interaction-, and task-level evidence.Source: §1 · pp. 2–3
  • The properties make reports traceable to a triggering action, observed outcome, and violated expectation, and can be reused without specifying a complete task for each website.The same property can be checked when its trigger arises across different interfaces and exploration paths.Source: §1 · p. 3
  • 88.7% precision across 31 website environments; A11yLTLNav confirmed 274 of 309 reported failures and identified more confirmed failures than comparison checkers.Precision ranged from 85.9% to 88.7% across three independent exploration runs; runtime was 18.52× lower than the adapted agentic checker, with no language-model inference during testing.Source: §1 · p. 3
  • A11yLTLNav formalizes browser-observable navigation failures as executable LTL properties and checks them over action-state traces during random keyboard exploration.The checker separates exploration, which selects keyboard actions, from property monitoring, which evaluates resulting behavior.Source: §1 · pp. 2–3

2 Background & Motivation

Dynamic website interactions can create BLV navigation barriers that static element checks do not capture, while the impact of an issue may depend on task and interaction history. The paper defines navigation failures around their effects on users, reviews detection approaches, and motivates reusable temporal properties for analyzing interaction sequences.

2.1 Accessibility Barriers in Dynamic Web Navigation

  • Dynamic, visually structured interfaces make accessibility failures more dependent on how users interact with a webpage over time, and the same issue may be recoverable in one task but blocking in another.User studies document failures and behavioral patterns as they unfold in real interactions, but those observations remain fragmented across studies.Source: §2.1 · pp. 3–4

2.2 Accessibility Navigation Failures

  • Accessibility navigation failures include website behaviors that hinder BLV users’ access to content, operation of controls, orientation, or receipt of information needed to proceed.The definition includes both conformance violations and usability problems when they directly affect navigation.Source: §2.2 · p. 4

2.3 Automated Detection and Repair

  • Static analyzers detect many element-level issues but cannot detect runtime failures emerging only during interaction; scripted approaches also depend on predefined sequences and checks.A11yLTLNav instead pairs random keyboard exploration with explicit temporal properties, so judgments are grounded in a reusable specification rather than model inference alone.Source: §2.3 · pp. 4–5

2.4 Formal Model for Failure Analysis

  • The paper translates browser-observable failures into LTL properties evaluated over ordered traces of keyboard actions and accessibility-relevant interface states.Each property specifies a triggering condition and expected outcome, such as focus entering a dialog after it opens, making expected and prohibited behaviors explicit and reusable.Source: §2.4 · p. 5

3 Failure Taxonomy

The paper synthesizes prior research into a taxonomy of BLV web accessibility failures and uses the evidence needed to observe them to connect evaluation levels with formal, automated reasoning.

3.1 Scoping Review Method

  • The taxonomy organizes prior research on accessibility navigation failures into 13 categories according to interaction stage.It was iteratively coded, consolidated, and refined from the final 56 studies in a structured review.Source: §3.1 · p. 6
  • A PRISMA-guided, multi-source review screened 936 candidate papers and identified 476 meeting the inclusion criteria for empirically observed web failures.The final taxonomy drew on failures extracted from 56 studies.Source: §3.1 · p. 6

3.2 Levels of Accessibility Failure

  • The paper distinguishes conformance-level element checks, interaction-level navigation checks, and task-level evaluation of real-world goal completion.L1 tools identify static node-level issues; L2 examines interaction steps; L3 examines whether interaction sequences support user goals.Source: §3.2 · p. 7
  • Organizing failures by required evidence identifies which can be evaluated from browser-observable behavior and formalized as temporal properties.The taxonomy includes both accessibility violations and usability problems when they impede BLV users’ navigation or operation.Source: §3 · p. 5, §3.2 · p. 7

4 A11yLTLNav: A Failure-driven Checker for Navigation Accessibility Failures

A11yLTLNav turns a literature-based failure taxonomy into executable temporal checks over keyboard interactions and resulting accessibility states. Its exploration and property checking are separated so the same properties can be reused across interfaces and paths.

4 A11yLTLNav: A Failure-driven Checker for Navigation Accessibility Failures

  • A11yLTLNav checks accessibility navigation failures by exploring keyboard interactions and monitoring browser-observable interface behavior.The workflow records violations while exploration continues from the resulting state.Source: §4 · p. 7, Figure 1 · p. 9

4.1 Temporal Property-Based Testing for Accessibility Navigation Failures

  • LTL expresses interaction expectations over action-state traces, linking a triggering interaction to an expected subsequent accessibility state.The properties are generic to interaction level and do not require knowledge of a website-specific task goal.Source: §4.1 · p. 9

4.2 Property-Based Checker for Accessibility Navigation Failures

  • Bombadil supplies state-aware random exploration and temporal checking, while the checker extends it with accessibility-specific components.Its default browser observations and interaction space are not designed for keyboard navigation, focus management, or accessibility state.Source: §4.2 · p. 10

4.2.1 Action Selector.

  • The action selector generates valid keyboard actions based on the currently focused element and widget context.Exploration and checking are decoupled, and exploration continues after a defect to find additional violation witnesses.Source: §4.2.1 · p. 10

4.2.2 Accessibility State Extractor.

  • The accessibility state layer captures semantics, DOM and ARIA state, focus, and interaction feedback from each browser state.Feedback observations include live-region updates, status messages, and alerts.Source: §4.2.2 · pp. 10–11

4.2.3 Runtime Property Monitor.

  • The runtime monitor starts a property-specific transaction when a trusted keyboard action meets its trigger, then evaluates the settled post-action state.It reports a violation only when the property applies and sufficient evidence is available.Source: §4.2.3 · p. 11

4.3.1 From Failure Taxonomy to Temporal Properties.

  • Each taxonomy failure is formalized through its trigger, expected outcome, and required observations, yielding reusable properties across websites and paths.For example, opening a modal should move focus inside it, while closing one can require focus to return to its invoker.Source: §4.3.1 · pp. 11–12

4.3.2 From Temporal Properties to Executable Scripts.

  • Each executable script binds a triggering interaction to relevant state, records pre-action observations, and checks the expected behavior after the interface settles.A script can check a common accessibility expectation without prescribing one implementation mechanism.Source: §4.3.2 · p. 12

5 Evaluation

The evaluation compares A11yLTLNav with static rule-based checkers and a task-driven agentic checker. It uses generated local website environments and manually reviewed reports to assess precision, runtime, and cost.

5 Evaluation

  • The evaluation compares A11yLTLNav with axe, WAVE, and a TaskAudit-based agentic checker across precision, runtime, cost, and detected failures.The research questions also examine precision stability across exploration runs.Source: §5 · p. 13

5.1 Baselines

  • Axe and WAVE inspect pages for static accessibility-rule violations, while the agentic baseline attempts website tasks using screen-reader transcripts and analyzes interaction traces.The agentic checker follows TaskAudit.Source: §5.1 · p. 13

5.2 Website Generation

  • The study evaluates 31 locally hosted generated website environments selected from Mind2Web subdomains.The environments reconstruct real websites; the authors used generated environments because live sites can block tools and change over time.Source: §5.2 · p. 13, §5 · p. 13

5.3 Metrics

  • Authors manually reviewed deduplicated reports against interaction traces and taxonomy definitions, annotating correct detections and false positives.Evaluation measures precision, execution time, and cost.Source: §5.3 · p. 13

6 Results

A11yLTLNav reported 309 failures with 88.7% precision, exceeding the agentic checker’s precision; its reports also covered different failure domains and its exploration required less runtime.

6.1.1 Agentic Checker.

  • 57.2% precision: the agentic checker had 158 confirmed detections and 118 false positives among 276 reports.Its reports depend on correctly executing and interpreting task traces.Source: §6.1.1 · p. 14

6.1.2 A11yLTLNav.

  • 88.7% precision: A11yLTLNav had 274 confirmed detections among 309 reports, 31.4% higher than the agentic checker.Precision remained between 85.9% and 88.7% across three independent runs.Source: §6.1.2 · p. 14

6.2 RQ2: Runtime and Computational Cost

  • 18.52× as much runtime: the agentic checker required approximately that much more runtime than A11yLTLNav across the 31 websites.The agentic checker also used approximately 1.01 billion tokens, while A11yLTLNav uses no LLM during exploration or property evaluation.Source: §6.2 · pp. 14–15

6.3 RQ3: Comparative Analysis of Detected Failures

  • A11yLTLNav reports concentrated in feedback and state (112), keyboard and focus (103), and dynamic content (51), while agentic reports were dominated by composite widgets (160).These distributions reflect different checking boundaries: reusable browser-observable relationships are more readily expressed as temporal properties than varied widget conventions.Source: §6.3 · pp. 15–16

6.3.1 A11yLTLNav.

  • The illustrated A11yLTLNav detections include missing accessible confirmation after a form update, focus loss after a quantity change, and focus remaining behind a newly opened dialog.In each case, the interface visibly responds but fails the expected accessibility outcome.Source: §6.3.1 · p. 17

6.3.2 Agentic Checker.

  • Agentic-checker false positives arose from misreading interface state, mistaking task progress for focus failure, or judging before screen-reader exploration was complete.The examples include a changed size selection, a collapsed combobox after Tab, and price information encountered later in a listing.Source: §6.3.2 · p. 18

7 Discussion

A11yLTLNav checks interaction-driven navigation expectations with temporal properties, complementing static rule-based checkers and agentic systems. Its browser-observable scope leaves out some failures requiring screen-reader output, task context, or broader state coverage.

7 Discussion

  • 88.7% precision across websites supports the effectiveness of temporal property checking for navigation errors involving focus, interface state, and accessible feedback.A11yLTLNav combines random exploration with properties derived from empirical failures.Source: §7 · p. 18

7.1 Complementing Existing Accessibility Checkers

  • Rule-based checkers assess single-state violations, temporal properties check interaction transitions, and agentic checkers can reason about richer task and screen-reader context.The approaches provide complementary forms of evidence that may broaden coverage when combined.Source: §7.1 · p. 19

7.2 Limitations and Future Work

  • The current state model checks browser-observable focus, keyboard, widget-state, and feedback signals, but not actual screen-reader announcements or some semantic and persistent-state evidence.Future extensions could add assistive-technology observations, richer semantic relationships, and persistent interaction state.Source: §7.2 · p. 19
  • Reusable, task-independent properties cannot check outcomes that depend on a user’s goal, such as task-relevant descriptions, appropriate control order, or path efficiency.The proposed direction is to instantiate task-specific predicates in parameterized property templates, potentially using an agent to identify targets and expected outcomes.Source: §7.2 · p. 19
  • A violation is evaluated only if stochastic exploration reaches its triggering interaction, so a fixed-budget run that reports none does not establish absence.Property-aware exploration is proposed to prioritize untriggered properties, including sequences such as opening, navigating within, and dismissing a dialog.Source: §7.2 · p. 20
  • The work detects failures but does not evaluate whether detected violations can be repaired safely.A proposed future workflow would re-check revisions against explicit behavioral expectations and ensure authoring and feedback interfaces remain accessible.Source: §7.2 · p. 20

8 Conclusion

The paper concludes that empirically grounded temporal properties can express and detect browser-observable navigation failures across interactions, complementing existing accessibility evaluation. It also supplies formal LTL semantics and documents the literature-search record counts used in the review.

8 Conclusion

  • Across 31 website environments, A11yLTLNav achieved 88.7% precision and remained stable across three independent runs, complementing rule-based and agentic checkers.The approach organizes empirical failures into a taxonomy, formalizes a browser-observable subset as temporal properties, and combines them with state-aware keyboard exploration.Source: §8 · p. 20

A LTL Semantics

  • An LTL formula describes a set of traces, modeled here as infinite sequences of atom sets; satisfaction is evaluated over the trace and its suffixes.The supplied semantics include the “until” condition: one formula holds until another eventually holds.Source: A LTL Semantics · pp. 25–26

B PRISMA records

  • The literature search yielded 113 unique records for screening after deduplication; Q1 contributed 112 abstract-field and 3 title-field records, while Q2’s 16 records were subsumed by Q1.The record-count passage describes the searches and deduplication before screening.Source: Table 8 · p. 26

C A11y-CUA Trace Annotation

The taxonomy draws on a structured literature review and analysis of BLV users’ interaction traces, grouping recurring interface-caused navigation errors as a basis for temporal properties.

C A11y-CUA Trace Annotation

  • Published studies often lack the step-by-step execution traces needed for formalization, and existing taxonomies may miss temporal errors in dynamic websites.The authors analyzed A11y-CUA traces to complement the literature synthesis.Source: C A11y-CUA Trace Annotation · pp. 26–27
  • The trace analysis treated navigation as sequences of actions, recording task context, preceding interactions, and effects on user progress when navigation was interrupted, unnecessarily extended, or led to unexpected states.Cases were not reduced to a single binary task-completion outcome.Source: C A11y-CUA Trace Annotation · p. 27
  • Similar errors across tasks were grouped into general failure types, while user mistakes and operating-system behavior were excluded from the interface-behavior focus.Examples include excessive sequential navigation and focus resets; the resulting types underpin the temporal properties.Source: C A11y-CUA Trace Annotation · p. 27

D Detailed Failure Taxonomy

The taxonomy organizes BLV web accessibility failures by the evidence needed to observe them, with cumulative levels distinguishing element, interaction, and task-related failures. Its examples span keyboard and widget operation, feedback and state, dynamic content, and task fit.

D Detailed Failure Taxonomy

  • The taxonomy’s evidence levels are cumulative: an L3 task-level failure can persist even when L1 and L2 checks pass, and the same domain can recur across levels.For example, a missing text alternative is L1, an unannounced state change is L2, and a present but goal-irrelevant description is L3.Source: Table 9 · p. 27
  • Keyboard and focus failures include focusable hidden content, ineffective role-specific activation, and controls that capture shortcuts needed by screen readers.Examples include collapsed-carousel items announced with empty strings and a calendar cell that responds only to Space.Source: D Detailed Failure Taxonomy · p. 28
  • Composite-widget failures include missing active options or cells, arrow keys that do not move the active item, and selections that fail to commit.Examples cover expanded comboboxes and managed grids such as date pickers.Source: D Detailed Failure Taxonomy · p. 28
  • Feedback and state failures include missing activation confirmation, disconnected responses, contradictory ARIA and native states, and absent progress or expanded-state signals.Examples include an unlinked results-panel update and an upload with no indication of whether it is running or stalled.Source: D Detailed Failure Taxonomy · p. 28
  • Dynamic-content failures include dialogs without useful names or dismissal controls, focus that stays behind an overlay, missing Escape dismissal or focus return, and unreachable media controls.A consent banner announced only as “dialog” gives no indication what it is or how to close it.Source: D Detailed Failure Taxonomy · p. 28
  • Context and task-fit failures include labels that omit nearby context, names that change across reading modes, and descriptions that omit goal-relevant information.A price without its product row or column illustrates missing context; a password field may lose its name announcement while typing.Source: D Detailed Failure Taxonomy · p. 28
  • Task-level mismatches include reading order or grouping that conflicts with task logic, such as a chart summary too brief for comparing series or a link named “read more” among many others.A page’s structure may be technically present yet fail to convey relationships needed for the task.Source: D Detailed Failure Taxonomy · p. 28
  • Navigation can also be inefficient or hard to resume: controls may be far from related content, require many Tab presses to reach, or return users to the top instead of their prior position.Other examples include excessive heading marks and re-reading a thread because completeness is not confirmed.Source: D Detailed Failure Taxonomy · pp. 28–29

E Failure-to-Temporal-Property Mapping

The section maps formalized accessibility failure categories to executable temporal properties, with compact identifiers linking each check to the failures it targets. Some categories map to multiple properties, while categories outside the executable set are documented separately.

E Failure-to-Temporal-Property Mapping

  • Table 10 maps formalized accessibility failures to simplified temporal properties, with each row representing one executable property.A blank Failure cell continues the failure category immediately above.Source: Table 10 · p. 29, E Failure-to-Temporal-Property Mapping · p. 29
  • Other mapped checks address absent activation feedback, contradictory ARIA and native states, and missing expanded state after a reveal.The corresponding examples include L2-11, L2-12, and L2-15.Source: E Failure-to-Temporal-Property Mapping · p. 30
  • The mapping also covers dialogs lacking an accessible name or named dismissal control, and rejected actions lacking recovery guidance.A failure may map to multiple properties when distinct observable interaction mechanisms need separate checks.Source: E Failure-to-Temporal-Property Mapping · pp. 29–30
  • Keyboard-navigation checks identify three failures: focus does not move into a newly opened dialog; Escape does not dismiss dialog or return focus; and a role-specific activation key has no effect.Source: E Failure-to-Temporal-Property Mapping · p. 30

E.1 Meaning and scope of the executable properties

Table 11 explains what the executable properties check and where their claims are bounded; Table 12 describes the notation used in the simplified properties. The checks span focus, keyboard widgets, feedback, state, dialogs, forms, and navigation, with scopes tied to browser-observable conditions.

E.1 Meaning and scope of the executable properties

  • L1-06 and L1-07 cover duplicate local links to the same destination and sequential keyboard reachability of eligible standalone semantic controls.The duplicate-link check excludes unreachable or dead URLs; the reachability check excludes composite-widget descendants and controls without declared interactive semantics.Source: §E.1 · p. 31
  • L2-01 to L2-03 check focus after local invalidation, orientation after major view transitions, and progress when Tab is blocked despite another usable target.The blocked-Tab check is a single-step no-progress test, not complete cycle detection; L2-01 requires concrete exposed focus after a stable local-view invalidation.Source: §E.1 · p. 31
  • L2-04 to L2-10 check hidden-element tab stops, valid active options or cells, inferable arrow-key movement, exact option commitment, and declared role/key effects.The arrow-key checks apply when the adjacent option or grid cell can be inferred.Source: §E.1 · p. 31
  • L2-11 requires an interface-changing activation to produce an AT-surfaceable feedback signal, such as a focus, name, state, live-notification, or same-document-navigation change.The property does not establish that a screen reader actually speaks the signal.Source: §E.1 · p. 31
  • L2-12 to L2-14 require redundant ARIA checked, selected, and value states to agree with their corresponding native control states.The checked-state comparison includes native checked or indeterminate state; the value comparison concerns a native range control.Source: §E.1 · p. 31
  • L2-15 and L2-16 check that expanded state matches exposed content after expansion or collapse, including a local disclosure reveal.The disclosure property relates the revealing control’s expanded state to the controlled region.Source: §E.1 · p. 31
  • L2-17 to L2-21 check dialog naming, a named enabled focusable action, focus entry, Escape dismissal, and focus return to an eligible invoker.The named action is only an identifiable candidate; the check does not prove that it dismisses the dialog.Source: §E.1 · pp. 31–32
  • L3-01 and L3-02 check for actionable guidance after native validation rejection or a newly invalid field, but exclude arbitrary task-level prerequisites.The guidance predicate covers linked accessible guidance or focused native validation guidance.Source: §E.1 · p. 32

E.3 Unformalized Failure Taxonomies

The executable property set operationalizes 22 of the taxonomy’s 36 failure categories, but the remaining categories require predicates not currently observable and attributable from browser state alone. The mapped categories are coverage by at least one check, not exhaustive coverage of every instance.

E.3 Unformalized Failure Taxonomies

  • 22 of the taxonomy’s 36 failure categories have at least one executable temporal property; the remaining 14 are not excluded because they are inexpressible in LTL.They require predicates that are unavailable from browser state alone or cannot be attributed to a concrete interaction.Source: §E.3 · p. 33, Table 13 · p. 33
  • Some failures require task or relevance information, such as judging whether descriptions, reading order, grouping, or Tab order fit the user’s goal.The browser exposes document order and grouping, but not the task-dependent relationships that determine useful ordering or grouping.Source: §E.3 · pp. 33–34
  • Other unformalized checks need thresholds or task models for excessive heading density, efficient routes, and knowing when a review or search is complete.The checker lacks a task-independent threshold for excessive heading density and cannot infer when enough task-relevant evidence establishes completeness.Source: §E.3 · pp. 33–34
  • Resumption and learned-position failures require persistent cross-view identity or comparison, while screen-reader shortcut capture requires OS- and assistive-technology-level attribution.A single browser execution state cannot establish comparable prior positions or identify which component consumed a shortcut.Source: §E.3 · pp. 33–34
  • The 22 mapped categories indicate coverage by at least one property, not exhaustive detection; implemented checks intentionally target narrower, high-confidence observable subtypes.This boundary aligns checks with what the monitor can establish from an execution trace.Source: §E.3 · pp. 33–34

F Prompts for Website Generation

  • Website-generation runs use original Mind2Web tasks associated with each selected website as a closed workflow specification, while keeping the remaining prompt instructions fixed.The generation prompt is adapted from uxCUA’s procedure.Source: F Prompts for Website Generation · pp. 34–35

G Prompts Used in Task-based Agentic Accessibility Checker

Appendix G documents prompts for a task-based agentic accessibility checker, covering how agents explore and reflect on tasks and how an analyzer evaluates outcomes, distinguishes accessibility errors from legitimate screen-reader boundaries or no-change behaviors, and reports evidence in JSON.

G Prompts Used in Task-based Agentic Accessibility Checker

  • The checker reuses taskAudit’s architecture but omits its task generator and adapts its prompts from TalkBack-based mobile navigation to web browsing with Orca on Linux.The websites are generated from tasks rather than discovered through button detection.Source: G Prompts Used in Task-based Agentic… · p. 36

G.1 Decision

  • The decision prompt treats prior decisions, observations, and reflections as one continuous task and requires exactly one immediate action, reserving completion or impossibility for established outcomes.It directs the agent to gather information through navigation or reading when needed.Source: §G.1 · pp. 36–37
  • The decision prompt distinguishes Orca’s virtual-navigation position from browser keyboard focus and treats the transcript’s last item as the latest utterance, not necessarily either position.Structural navigation moves Orca’s virtual position without activating or editing a control.Source: §G.1 · pp. 36–37

G.2 Reflection

  • The reflection prompt supplies the preceding operation’s thought and action alongside the resulting screen-reader transcript and focus state for the website explorer’s reflection turn.Source: §G.2 · pp. 38–39

G.3 Accessibility Analyzer

  • The analyzer makes separate overall-task and immediate-action judgments, using task evidence for completion and before/after evidence to assess the action’s expected observable effect.A failure is a screening candidate when the expected effect is absent, contradicted, or unestablished; required widget context must be established by before evidence.Source: §G.3 · p. 39
  • For a failed action, the analyzer compares the intended action with the transcript-reported outcome and classifies the cause as an accessibility error or a non-accessibility error.Non-accessibility explanations include agent mistakes, harness or infrastructure issues, inappropriate action context, and website state changes; certainty that the interface caused the failure is not required.Source: §G.3 · pp. 40–41
  • The analyzer must not treat a screened failure as an accessibility error when it reflects a legitimate screen-reader boundary or no-change behavior.It is asked to explain the accessibility problem when an error occurred, or the likely non-accessibility cause otherwise.Source: §G.3 · p. 41
  • The analyzer returns JSON fields for error status, problematic element, element index, and a concise evidence-based root-cause explanation.It uses null for the element index if the transcript provides no usable index.Source: §G.3 · p. 41
  • The prompt supplies task specifications, a Stage 1 failure reason, and screen-reader evidence as inputs to the analyzer.The provided excerpt includes placeholders for the task specification and Stage 1 failure reason, followed by a screen-reader evidence block.Source: §G.3 · p. 41
Loading 2609.17959v1…