PaperlayerView exact revision on arXiv

Source-linked AI summary

Invisible in Space, Visible in Time: Motion Vision CAPTCHA against GUI Agents

Zeyu Zhang, Dingyi Rong, Zijian Chen, Zicheng Zhang, Xiongkuo Min, Guangtao Zhai

arXiv:2609.27461v1cs.CV

TL;DR

Many visual CAPTCHAs expose answer-relevant static cues that current MLLM-based GUI agents can increasingly access. MVCAP makes target semantics depend on temporal segregation of motion-defined foregrounds, and its benchmark shows a large human–agent gap. Controlled comparisons identify dynamic background camouflage as the main difficulty.

  • Problem

    Many visual CAPTCHAs encode answer-relevant information in static appearance, local structure, or interface state, cues increasingly accessible to MLLMs and GUI agents.

    Source: §1 · p. 1

  • Method

    MVCAP uses motion-defined foreground semantics across coherent, structural, and biological motion, evaluated with a browser benchmark and matched foreground-only control.

    Source: §1 · p. 2

  • Results

    99.6% human accuracy versus 16.8% for the best GUI agent on MVCAP-Bench, with the latter near six-way chance.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MVCAP-Bench makes the human–agent gap in recovering motion-defined semantics under dynamic camouflage measurable.

    Source: §1 · p. 2

    The attacker setting is limited to live-browser GUI interaction and requires iterative interaction rather than offline static classification.

    Source: §3 · p. 4

Abstract

from arXiv · show

Most existing visual CAPTCHAs remain spatially solvable: the required information is exposed by static appearance, local structure, and interface state. This assumption is weakened by advances in multimodal large language models (MLLMs) and Graphical User Interface (GUI) agents, which exhibit strong visual perception, reasoning, and browser interaction capabilities. We propose Motion Vision CAPTCHA (MVCAP), a hierarchical motion-based CAPTCHA framework in which target semantics are instantiated as motion-defined foreground structures and become recoverable only through temporal segregation from a dynamically evolving background. Built on this shared principle, MVCAP is instantiated in three perceptually progressive levels: coherent motion, structural motion, and biological motion. To evaluate this framework, we introduce MVCAP-Bench, a browser-based benchmark with 600 live CAPTCHA instances, together with a matched foreground-only control benchmark, MVCAP-Bench-FG. We evaluate humans, Browser Use agents, native computer use agents, and a supplementary offline VQA setting derived from the same instances. Results reveal a substantial human--agent gap: on the full MVCAP-Bench, human accuracy reaches 99.6%, whereas the best GUI agent achieves only 16.8%, close to the six-way chance level. The foreground-only control further shows that the key difficulty comes from dynamic background camouflage rather than answer format or browser interaction alone. These findings identify a measurable human--agent perception gap and position MVCAP-Bench as a benchmark for studying motion-defined perception in current agents.

1 Introduction

Existing visual CAPTCHAs expose answer-relevant information through static cues that current MLLM-based GUI agents can increasingly access. MVCAP instead makes semantics motion-defined, and its benchmark reveals a large human–agent gap.

1 Introduction

  • Static appearance, local structure, and interface state remain common CAPTCHA cues, but MLLMs and GUI agents increasingly access those cues through perception and interaction.Source: §1 · p. 1
  • MVCAP makes target semantics recoverable through temporal segregation of motion-defined foreground structures, with levels spanning coherent, structural, and biological motion.Its design keeps target semantics inaccessible in isolated frames and relies on temporal integration.Source: §1 · p. 2
  • 99.6% human accuracy versus 16.8% for the best GUI agent on MVCAP-Bench, near six-way chance; controls locate the main difficulty in dynamic camouflage.The controlled comparisons distinguish camouflage from answer format or browser interaction alone.Source: §1 · p. 2
  • MVCAP-Bench contains 600 instances across three levels, and MVCAP-Bench-FG is a matched control that removes background camouflage.The evaluation covers humans, browser-use and native computer-use agents, and supplementary offline VQA.Source: §1 · p. 2

2 Related Work

Prior visual CAPTCHAs largely expose answer-relevant evidence spatially, while progress in MLLMs and GUI agents makes such cues increasingly accessible. MVCAP instead targets perception that depends on temporal motion.

2.1 Visual CAPTCHAs

  • Most visual CAPTCHAs remain spatially solvable, and animation alone does not require temporal integration when extracted frames expose the answer.MVCAP differs by embedding a motion-defined target in dynamic camouflage, requiring temporal foreground–background segregation.Source: §2.1 · pp. 2–3

2.2 MLLMs and GUI Agents

  • MLLM perception and GUI interaction support increasingly capable agents, while existing CAPTCHA success is especially evident on spatial-perception and shallow-interaction tasks.Source: §2.2 · p. 3

2.3 Motion-Defined Perception

  • Coherent motion, form-from-motion, and biological motion convey meaningful structure, yet current models remain unreliable when semantics depend primarily on temporal patterns.MVCAP measures this motion-perception gap without claiming permanent machine incapability.Source: §2.3 · p. 3

3 Threat Model

The threat model considers an MLLM-based GUI agent trying to bypass browser verification without human intervention. Its scope is live, iterative browser interaction rather than offline static classification.

3 Threat Model

  • The attacker’s goal is to solve MVCAP autonomously and bypass browser-based human verification, potentially enabling unauthorized automated access to protected services.Source: §3 · p. 3
  • The interaction is modeled as a partially observable Markov decision process with hidden session state, observations, GUI actions, and environment transitions.Observations may include screenshots, visible prompts and options, and structured interface information.Source: §3 · p. 4
  • The agent may access the live webpage and use Browser Use tools and multistep reasoning, but must solve through iterative GUI interaction rather than offline static classification.Source: §3 · p. 4

4.1 Overview

MVCAP shifts CAPTCHA solving from static recognition to semantic perception grounded in motion over time. It organizes challenges into three motion levels.

4.1 Overview

  • MVCAP is designed so target semantics remain effectively inaccessible in isolated frames and become perceptible through temporal integration of motion cues.Source: §4.1 · p. 4
  • Its three levels progress from motion-grouped text perception to motion-defined 3D object structure and then action perception from point-light biological motion.Source: §4.1 · p. 4
  • Across the levels, target semantics are instantiated as temporally evolving foreground structures against backgrounds that suppress reliable recovery from static spatial cues alone.Source: §4.1 · p. 4

4.2 Shared Motion Vision Principle

MVCAP instantiates target semantics as temporally evolving foreground structures, then presents them against a separately evolving background. Across levels, targets are designed to emerge through temporal integration of differential motion rather than reliable recovery from isolated frames.

4.2 Shared Motion Vision Principle

  • Target semantics define a level-specific foreground structure that evolves over time, from text-shaped regions to 3D objects or articulated human motion.The foreground structure is rendered frame by frame as part of the challenge sequence.Source: §4.2 · p. 4
  • The foreground and background have distinct temporal dynamics, so competing motion conceals the target rather than merely adding noise.The levels use opposite drift, 3D rotation with differential drift, or biological motion against drifting distractors.Source: §4.2 · p. 4
  • Across dense and sparse renderings, target semantics become perceptible through temporal integration of differential foreground–background motion, not isolated frames.Dense dot fields characterize L1 and L2; L3 uses a point-light foreground and sparse distractors.Source: §4.2 · p. 4

4.3 Level Instantiations

MVCAP instantiates motion-defined semantics at three levels: coherent text motion, evolving 3D object structure, and sparse biological motion. Each targets a different kind of recognition over time.

4.3.1 L1: Coherent Motion.

  • L1 embeds procedurally generated four-character text masks in a similar dot background, where coherent motion groups the weak or ambiguous text over time.The target remains fixed while foreground and background follow distinct motion patterns.Source: §4.3.1 · p. 4

4.3.2 L2: Structural Motion.

  • L2 renders ShapeNetCore-derived 3D object categories under controlled rotation, making object structure perceptible through temporal integration rather than a single frame.The evolving object is embedded in a visually similar dot background.Source: §4.3.2 · p. 4

4.3 Level Instantiations

  • MVCAP-Bench combines browser-based evaluation, procedurally generated instances, and motion-defined challenges that are not solvable from static spatial cues alone.The comparison is with representative CAPTCHA benchmarks and systems.Source: Table 1 · p. 5

4.3.3 L3: Biological Motion.

  • L3 uses BABEL-aligned CMU motion-capture actions as sparse point-light sequences against sparse distractors, targeting action recognition through biological motion.The corresponding answer options are rendered with the challenge videos.Source: §4.3.3 · p. 5

4.4 Perceptual Progression Across Levels

The three MVCAP levels engage distinct kinds of human motion perception, forming a progression in perceptual demands rather than a simple increase in task difficulty.

4.5 Perceptual Progression Across Levels

  • L1 tests coherent-motion perception by asking humans to group motion-defined text from background noise.Source: §4.4 · p. 5
  • L2 tests structural-motion perception by requiring recovery of object structure from temporally evolving 3D motion.Source: §4.4 · p. 5
  • L3 tests biological-motion perception through action recognition from sparse articulated motion cues.Source: §4.4 · p. 5

4.5 Benchmark Construction

MVCAP-Bench evaluates motion-defined CAPTCHA solving as an interactive browser task, with a matched foreground-only variant to isolate the role of dynamic background camouflage.

4.5 Benchmark Construction

  • 600 instances across three levels form looping-video browser tasks with a prompt and six answer options, requiring agents to select and submit an answer.Each challenge video is 3 seconds long and rendered at 30 frames per second; there are 200 instances per level.Source: §4.5 · p. 5
  • MVCAP-Bench-FG removes background content while retaining targets, prompts, options, and interface, providing a controlled reference for the role of dynamic background camouflage.Source: §4.5 · p. 5
  • The benchmark jointly combines browser-based evaluation, procedurally generated instances, and motion-defined challenges that static spatial cues alone cannot solve.Source: §4.5 · p. 5

5 Experiment

The experiments compare browser-based GUI agents with humans across MVCAP-Bench and a matched foreground-only control, then examine costs, a paired case, and offline VQA. Agent performance stays near chance on the full benchmark, while exposing the foreground substantially improves results.

5.1 Experimental Setup

  • The evaluation uses ten Browser Use backbones, three native computer-use agents, human and random-guess baselines, and single-attempt Pass@1 plus monetary cost.Agents solve live browser pages containing a looping video, prompt, and six options; 30 human participants each see a balanced 60-instance subset per benchmark.Source: §5.1 · pp. 5–6

5.2 Experiment Results

  • Humans reach 99.6% accuracy, while the best Browser Use agent, GPT-5.4, reaches 16.8% on MVCAP-Bench, near the 16.7% six-way chance level.The gap is consistent across levels; no Browser Use agent exceeds 20.5% on any full-benchmark level.Source: §5.2 · p. 6, Table 2 · p. 6
  • Removing dynamic background camouflage raises average Pass@1 from 16.8% to 61.7% for GPT-5.4, from 13.8% to 68.0% for Gemini-3.1-Pro, and from 15.8% to 63.0% for Gemini-3-Flash.The foreground-only control preserves targets, prompts, answer options, and browser interface, isolating foreground visibility from answer format and interaction changes.Source: §5.2 · p. 6
  • Higher cost does not predict higher Pass@1 on the full benchmark, and camouflage removal produces larger, more consistent gains than switching between Browser Use and native computer use.For example, GPT-5.4 scores 16.8% with Browser Use and 20.8% natively on MVCAP-Bench; removing camouflage improves every aligned backbone by more than 40 percentage points under either embodiment.Source: §5.2 · p. 7

5.3 Case Study

  • In the paired L2 case, Claude-Sonnet-4.6 misses the planter in the cluttered full scene but identifies it when the same target is shown without the distracting background.The full-scene trace fails to recover stable object-level structure, whereas the foreground-only trace uses structural cues to select the correct answer.Source: Figure 4 · p. 7, §5.3 · p. 7

5.4 Offline VQA Experiment

  • Offline accuracy on the full benchmark rises only from 13.8%–16.8% in the browser to 17.0%–20.9%, with Gemini-3-Flash best at 20.9%.Removing browser interaction yields larger gains on the foreground-only control; for example, Qwen3.5-Plus rises from 53.8% to 65.3%.Source: §5.4 · p. 8, Table 3 · p. 8, Figure 5 · p. 8

6 Conclusion

The paper presents MVCAP as a motion-based CAPTCHA framework and introduces MVCAP-Bench with a matched foreground-only control. Across browser, native computer-use, and offline VQA evaluations, humans solve the full benchmark almost perfectly while current agents remain near random guess.

6 Conclusion

  • MVCAP shifts CAPTCHA solving from static spatial recognition to motion-defined semantic perception across coherent, structural, and biological motion.The paper presents this framework for human–machine differentiation.Source: §6 · p. 8
  • MVCAP-Bench and its matched foreground-only control support evaluation of motion-defined CAPTCHA challenges.Experiments span live browser, native computer use, and offline VQA settings.Source: §6 · p. 8
  • Humans solve the benchmark almost perfectly, whereas current agents remain near random guess on the full benchmark.The paper concludes that motion-defined perception is a promising direction for CAPTCHA design and a measurable human–agent gap remains.Source: §6 · p. 8

Related papers

Loading 2609.27461v1…