Source-linked AI summary
Breaking Darknet CAPTCHAs with general purpose LLM
Benjamin Fehrensen, Jens Hubler
TL;DR
The paper asks how effectively general-purpose MLLMs can solve JavaScript-free darknet CAPTCHAs, especially when tasks require precise geometry. It evaluates several solving strategies and proposes an MCP-based hybrid in which MLLMs orchestrate deterministic image processing. Across the evaluated CAPTCHA types, hybrid solving combines semantic interpretation with more reliable geometric computation.
Problem
JavaScript-free darknet CAPTCHAs differ from mainstream challenges, while MLLMs often struggle with precise localization and geometric transformations despite recognizing relevant visual structures.
Method
The study evaluates MLLMs and image-processing methods and uses MCP to let an MLLM orchestrate deterministic geometric computations.
Results
The hybrid approach enables reliable and efficient automated solving across all evaluated CAPTCHA types, while MLLMs perform strongly on semantic object recognition.
Takeaways & Limitations
Combining MLLM reasoning with classical image processing is effective for structurally constrained, geometry-heavy CAPTCHAs.
Takeaways & Limitations
For well-defined geometric tasks, MLLM verification can increase latency and token consumption or reduce accuracy, so direct algorithms remain preferable.
Abstract
from arXiv · showhide
Our work evaluates the effectiveness of automated methods for solving CAPTCHA challenges commonly encountered in darknet environments. These CAPTCHAs are typically designed to operate without JavaScript, resulting in distinct characteristics compared to mainstream CAPTCHA systems. Our study considers three representative challenge types: open-circle localization, rotation-based alignment, and object-selection CAPTCHAs. The experiments reveal a systematic limitation of contemporary MLLMs: while they are generally capable of identifying relevant visual structures, they frequently struggle with precise spatial localization and geometric transformations. These deficiencies can be mitigated either through task reformulation or by augmenting the models with specialized image processing tools. These deficiencies can be mitigated by task reformulation or by equipping the model with specialized image-processing tools. We therefore propose a hybrid framework in which an MLLM serves as a high-level reasoning and orchestration layer while delegating geometric computations to deterministic algorithms via the Model Context Protocol (MCP). The resulting system achieves success rates above 90% across all evaluated CAPTCHA types and demonstrates that combining the complementary strengths of MLLMs and classical computer vision yields a more accurate and efficient solver than either approach alone.
1 Introduction
The paper studies JavaScript-free CAPTCHAs common on darknet platforms and evaluates classical methods, MLLMs, and hybrid systems across representative challenge types. It finds that MLLMs understand visual content but struggle with geometric precision, motivating tool-augmented solving.
- JavaScript-free CAPTCHAs form a distinct class because privacy-sensitive users often disable JavaScript, leading platforms to avoid JavaScript-dependent challenges.
- The study evaluates open-circle, rotation-based, and object-selection CAPTCHAs using classical image processing and contemporary MLLMs.
- MLLMs are effective at semantic image understanding but systematically weak at precise localization and mental rotation.
- These weaknesses can be mitigated through task reformulation or specialized image-processing tools.
- The proposed MCP-based hybrid framework uses an MLLM for orchestration while delegating geometric computations to dedicated image-processing algorithms.
2 Related Works
Related work shows that general-purpose multimodal models can solve many CAPTCHA tasks, but the paper identifies a persistent gap in precise geometric reasoning. Its hybrid design delegates geometry to deterministic tools while retaining MLLM reasoning and semantic strengths.
- Recent VLM systems use planning, tool use, and multi-step reasoning to solve previously unseen CAPTCHA types.
- On darknet-style JavaScript-free CAPTCHAs, MLLMs often recognize relevant structures but fail at precise localization and mental rotation.
- The framework delegates open-circle detection and rotation estimation to deterministic algorithms exposed through MCP.
- Task reformulation, such as presenting pre-rotated candidates, further reduces the need for direct angle regression.
- Classical-plus-MLLM hybrids target structurally constrained, geometry-heavy CAPTCHAs, whereas agentic VLMs target open-ended interactive challenges.
3 Methods and Experiments
The experiments compare classical image processing, prompt-engineered MLLMs, task reformulation, and MCP-based hybrid solving on real-world CAPTCHA samples. Evaluation considers accuracy, latency, token consumption, reproducibility, model selection, and three CAPTCHA types.
- Methods: The study evaluates classical image processing, prompt engineering, task reformulation, and hybrid MLLM-assisted solving.
- Methods: Rotation tasks can be reformulated as selecting among discrete pre-rotated candidate images instead of predicting an angle directly.
- Methods: MCP integration lets MLLMs invoke specialized image-processing operations during inference.
- Limitations: Because evaluated models had to expose inference-parameter controls, experiments used GPT-4o rather than newer 5.x-series models.
- Models: The evaluation uses four MLLMs from three providers, including reasoning and non-reasoning Grok-4.20 variants.
- Evaluation: Accuracy, latency, and token consumption measure solution success, end-to-end response time, and MLLM processing cost.
- Dataset: Experiments use real-world samples covering open-circle, rotation, and insect-selection CAPTCHAs.
4 Results
The experiments show that MLLMs often recognize CAPTCHA structures but struggle with precise localization and geometric transformation. Classical image processing and MCP-based hybrid solving improve accuracy, efficiency, and reliability across open-circle and rotation tasks.
- Open-circle CAPTCHA: MLLMs frequently identified open circles correctly but failed to report coordinates precisely enough for successful interaction.Grok-4.20 recognized the target ring semantically but produced insufficiently precise coordinates.
- Open-circle CAPTCHA: The hybrid open-circle configurations matched the image-processing pipeline’s perfect accuracy and localization precision.Both reasoning and non-reasoning Grok-4.20 variants successfully invoked the MCP tool and returned its coordinates without modification.
- Open-circle CAPTCHA: GPT-4o improved from 20% to 100% accuracy, while both Grok-4.20 variants improved from 10% to 100% with the hybrid architecture.The improvement indicates that coordinate computation and reporting, rather than target identification, limited standalone performance.
- Open-circle CAPTCHA: Delegating geometric localization reduced Gemini 3.1 Pro response times from 43.9 s to 8.6 s and from 88.1 s to 18.2 s despite higher token consumption.The results show that additional tool-related tokens did not necessarily increase latency.
- Rotation CAPTCHA: Direct rotation-angle estimation was near chance for most models, with approximately 10% accuracy versus 12.5% expected from eight-way random guessing.Reasoning traces showed that models recognized relevant structures but struggled to mentally rotate and align them.
- Rotation CAPTCHA: The image-processing rotation pipeline achieved perfect accuracy in 0.006 s, while the best standalone MLLM reached 70% accuracy with several orders of magnitude more computation time.For geometrically well-defined tasks, specialized algorithms remained superior in accuracy, runtime, and resource consumption.
- Rotation CAPTCHA: Hybrid GPT-4o and Grok configurations retained perfect rotation accuracy, whereas Gemini 3.1 Pro reduced hybrid accuracy to 80% by sometimes overriding correct algorithmic solutions.The findings suggest that MLLM verification can degrade a reliable deterministic solution when alignment artifacts are present.