PaperlayerView exact revision on arXiv

Source-linked AI summary

SpiderSapien: Client-Centric Web Crawler and Security Scanner

Eric Olsson, Benjamin Eriksson, Adam Doupé, Andrei Sabelfeld

arXiv:2609.02532v1cs.CR

TL;DR

Modern web applications’ dynamic, interactive client sides limit black-box scanners’ ability to reach deep states and test vulnerabilities. SpiderSapien uses browser feedback to detect and order client-side actions and employs an LLM for forms. It improves average code coverage by 16% over the union of all other scanners and finds 32 XSS vulnerabilities across 7 applications.

  • Problem

    Modern web applications’ dynamic client-side interactions and stateful workflows make deep functionality difficult for black-box scanners to discover and test.

  • Method

    SpiderSapien redesigns black-box crawling around high-level user-facing browser feedback, runtime interactable-element detection, UI interaction ordering, and LLM-based form solving.

  • Results

    16% higher average code coverage than the union of all other scanners and 32 XSS vulnerabilities found across 7 applications were reported.

  • Takeaways & Limitations

    Immersive interaction enables SpiderSapien to explore deeper application states and improve both code coverage and vulnerability detection across the evaluated applications.

Abstract

from arXiv · show

Black-box web application crawling and scanning play an important role for security testing of web applications. Yet state-of-the-art scanners fall short of addressing key characteristics of a modern web application: its extreme dynamism and interactivity on the client side. This paper identifies immersive interaction as a key ingredient for scanners to deeply explore modern web applications. We propose SpiderSapien, a client-centric crawler and security scanner. SpiderSapien incorporates a unique combination of high-level, user-facing feedback channels from the web application to achieve immersive interaction in a black-box crawling loop. These feedback channels include both novel methods to detect interactable elements and sensibly order UI interactions, and orthogonally using an LLM to solve forms. In doing so, we demonstrate how to reliably discover and test deep states of modern web applications. Furthermore, our modular approach and useful abstraction layer can serve as a building block for future scanners. The evaluation of our approach shows substantial improvements in both code coverage and vulnerability detection over previous work. Our approach increased average code coverage across applications by at least 46% over any other scanner, or 16% when compared to the union of all other scanners. We find XSS vulnerabilities in 7 web applications, while any other scanner finds XSS in up to 2 applications.

I. INTRODUCTION

Modern web applications combine dynamic statefulness with rich client-side interactions that black-box scanners often fail to explore deeply. SpiderSapien addresses this gap through immersive, user-facing interaction and improves coverage and XSS detection.

  • Motivation: Modern web applications expose complex, multi-step client-side workflows whose deep behavior can remain untested by existing black-box scanners.Prior scanners often operate on URLs, requests, HTML tags, or event listeners rather than rendered interfaces and valid user actions.
  • Approach: SpiderSapien redesigns black-box crawling around high-level browser feedback and valid client-side actions.Its loop discovers interactable elements, orders UI interactions, and uses an LLM for complex forms.
  • Approach: The approach combines framework-agnostic interactable-element discovery, interaction ordering, and LLM-based form solving in a recurrent crawler.This recurrent design targets applications where later actions appear only after earlier interactions.
  • Evaluation: 46% higher average code coverage than any other scanner, and 16% higher than the union of all other scanners, was achieved across applications.The evaluation compared SpiderSapien with six black-box scanners on nine open-source web applications.
  • Evaluation: SpiderSapien found 32 XSS vulnerabilities across 7 of 9 applications, whereas other scanners found 29 across 3 applications.Any individual competing scanner found XSS in at most 2 applications.

II. CHALLENGES

Modern applications challenge black-box scanners through dynamic client-side elements, state-dependent interaction order, and diverse form validation. SpiderSapien frames immersive interaction as using the rendered interface and user-facing signals to reach deeper states.

  • Response: Immersive interaction raises the abstraction level to user-facing signals and valid client-side actions for navigating stateful workflows and discovering deep states.The proposed approach combines interactable-element detection, interaction ordering, and LLM-assisted form solving.
  • Challenges: Modern applications require scanners to navigate multi-step client-side interactions because available options depend on previous actions.Without handling these workflows or composing structured inputs, deep behavior remains hidden and untested.
  • Challenges: Custom tags, semantically meaningless elements, and dynamically assigned event handlers can obscure interactable elements from scanners.Frameworks may attach handlers to parent elements or the document body, separating event evidence from the relevant element.
  • Challenges: Interaction order must reflect client-side state because visible overlays can block underlying elements and hidden clicks may not produce intended effects.A modal form can expose new controls while making previously visible controls inactive.
  • Challenges: Complex forms require valid inputs that scanners may struggle to infer from optional HTML annotations, descriptive text, or server-side validation.Validation can be implemented in custom JavaScript or invisible server-side logic.

B. Client-side Crawling

SpiderSapien performs client-side crawling by detecting runtime interactability, prioritizing currently active actions, and repeatedly exploring the resulting application-state graph. This avoids relying solely on semantic tags or event-listener analysis.

  • Detecting Interactable Elements: SpiderSapien extends semantic HTML elements with runtime signals to identify elements that users can interact with.The method uses properties such as pointer cursor, text behavior, and nonnegative tabindex values.
  • Detecting Interactable Elements: Runtime visibility checks remove elements hidden beneath others, restricting the interactable set to currently visible and active elements.The scanner uses element coordinates and the top element at those coordinates to test whether an element is actionable.
  • Detecting Interactable Elements: The scanner uses the browser-rendered page directly rather than modeling the DOM or relying on specific tags and JavaScript event listeners.This helps identify custom elements whose interactability is indicated by runtime presentation.
  • Ordering UI Interactions: A directed graph represents application states and possible actions, while a two-phase strategy first finds linked pages and then prioritizes client-side interactions and payload inputs.The main phase orders new actions, active current-page elements, and input elements.
  • Ordering UI Interactions: The crawler prioritizes active elements in a DFS-like manner but uses random diversions to avoid becoming stuck in one application area.Unlike approaches that separate exploration and exploitation, it continuously selects the best available option without requiring termination.

C. LLM-based Input Generation

SpiderSapien uses an LLM to generate acceptable form inputs when validation is diverse or hidden from scanner-visible HTML. The evaluation compares the complete scanner on open-source applications using single scanner–application runs.

  • LLM-based Input Generation: LLM-based form solving addresses validation constraints that may be expressed in custom JavaScript or server-side logic rather than simple patterns.Such constraints can include numeric ranges and other requirements not reliably inferable from limited HTML semantics.
  • LLM-based Input Generation: The form solver supplies few-shot examples and summarized form information to an LLM, which infers acceptable values from field descriptions.For example, it can infer that “number of hours” should receive a numeric value in an appropriate range.
  • Implementation: SpiderSapien sends form HTML and selected input metadata to the LLM, receives selector–value pairs, and submits the form with those values.The prompt includes input elements, required fields, and elements with regex patterns.
  • Implementation: Additional LLM checks identify potentially dangerous forms and select the intended submit button, while the implementation also handles drag-and-drop and file uploads.The dangerous-form check targets changes to usernames or passwords.
  • Evaluation: The scanner evaluation uses open-source web applications, primarily single runs of scanner–application pairs, and restricts the application set to PHP for uniform server-side coverage.The selected applications include DokuWiki, HotCRP, Kanboard, Leantime, Nextcloud, osCommerce, Piwigo, TinyFileManager, and WordPress.

2) Compared Black-box Scanners:

SpiderSapien is evaluated against six black-box scanners using server-side unique lines of code and manually verified XSS vulnerabilities. It generally achieves broader coverage and vulnerability discovery across the evaluated applications, while other scanners retain complementary findings.

  • Evaluation setup: The evaluation compares SpiderSapien with six black-box scanners on nine open-source web applications using code coverage and verified XSS findings.Coverage is defined as unique server-side lines of code executed, while vulnerability reports are manually verified and deduplicated.
  • Coverage: SpiderSapien outperforms the other scanners in 7/9 evaluated applications for server-side code coverage.The comparison uses Figures 3 and 4 for scanner-level coverage differences and reports exact values elsewhere.
  • Caveats: SpiderSapien’s advantage is not universal: ZAP performs better on TinyFileManager coverage, while Arachni reports two additional vulnerabilities there but with substantial false-positive uncertainty.ZAP triggers more error states in TinyFileManager, and only 3 of Arachni’s 10 reported vulnerabilities were manually confirmed.
  • Complementarity: Other scanners still discover thousands of lines of code SpiderSapien misses, but those lines contain fewer vulnerabilities in the reported comparison.The authors suggest distinguishing intended or high-quality coverage from coverage of missed code, without claiming that all missed code is irrelevant.

D. Repeated Evaluation Case Study

The repeated-run case study examines coverage stability, vulnerability discovery, and state-depth analysis for SpiderSapien and YuraScanner. Results are mostly close across runs, but application-specific state changes and incomplete trace coverage affect interpretation.

  • Repeated evaluation: The repeated evaluation compares two runs of SpiderSapien and YuraScanner, reporting minimum, maximum, and union coverage.The authors describe the case study as limited and note that one extra run is not statistically significant.
  • Coverage stability: Most repeated-run coverage results differ by about 10% from the maximum, indicating generally similar performance between runs.Piwigo is a notable exception because SpiderSapien destroys application state while attempting a full application update.
  • Coverage stability: Repeated runs reveal application-dependent saturation: TinyFileManager exercises largely the same code, whereas osCommerce exposes thousands of new lines between runs.Longer or repeated scans are therefore expected to have less effect on TinyFileManager than on osCommerce.
  • Vulnerability stability: In the repeated vulnerability evaluation, YuraScanner finds 11 vulnerabilities in osCommerce, while SpiderSapien finds 17 there compared with 19 in the main evaluation.The repeated SpiderSapien run overlaps with 12 of its main-evaluation findings.
  • Vulnerability stability: SpiderSapien finds 31 XSS vulnerabilities across 6 applications in the repeated run, missing one Kanboard vulnerability because it did not resubmit a correctly identified vulnerable form.Piwigo performance also changes after the state-destroying update, while Leantime improves from 2 to 9 XSS findings.
  • State-depth method: State depth is analyzed from event traces, where first executions of source files or annotated lines form the states reached by a scanning run.State graphs then infer relationships among these events, simplifying them into shortest paths to relevant application events.
  • Limitations: The state-depth definition can overestimate prerequisite states when traces omit alternative paths or scanners discover non-overlapping portions of the application.The authors mitigate this by adding runs and identify test suites or user traces as future sources of additional traces.

F. Client-side Coverage

SpiderSapien measures client-side byte coverage and finds interactable elements beyond semantic HTML, while ablations show both LLM form solving and element discovery improve exploration.

  • Client-side Coverage: SpiderSapien collects byte-level client-side coverage before and after each action using Chrome’s built-in profiler.Coverage is restricted to URL-backed application scripts, excluding anonymous functions and eval contents.
  • Client-side Coverage: SpiderSapien outperforms Black Ostrich across all applications, with coverage improvements ranging from 5% on Kanboard to 1020% on Piwigo and averaging 287%.SpiderSapien covers 65% of the loaded bytes on average, leaving room for improvement.
  • Client-side Coverage: In HotCRP, SpiderSapien reaches every state in the page graph, while EvoCrawl and YuraScanner reach only overlapping subsets.EvoCrawl additionally reaches purple states, YuraScanner blue states, and green states are exclusive to SpiderSapien.
  • Non-semantic Interactable Elements: SpiderSapien discovers thousands of non-semantic interactable elements that purely semantic selection would miss, including menu <li> tags and clickable <span> or <label> icons.The analysis counts unique interactable elements and non-semantic elements across applications.
  • Ablation Study Results: Ablation results show average coverage gains of 42% from LLM form solving and 32% from improved element discovery.The LLM helps especially on applications requiring complex forms early, while removing element discovery can reduce coverage when framework-specific controls are missed.
  • Ablation Study Results: Neither ablation finds XSS in as many applications as SpiderSapien, with the element-discovery ablation finding 28 XSS across 4 applications and the LLM ablation 27 across 3.The reported ablation counts include a concentration of vulnerabilities in osCommerce and a form in Leantime.

V. ANALYSIS / DISCUSSION

SpiderSapien’s components are intended as complementary building blocks for future crawlers, but the discussion identifies trade-offs involving LLM evaluation, state destruction, search strategy, and scan duration.

  • The Future of Black-box Crawling: SpiderSapien combines with machine-learning and LLM-based crawlers as modular building blocks for form solving and interactable-element detection.The authors suggest LLM form solving can replace prior heuristics and that element detection can improve future page abstractions.
  • The Future of Black-box Crawling: Fully LLM-based crawling can perform worse overall, so black-box LLMs require careful evaluation despite strong form-solving performance.The authors identify single-pass task solving or abstract page representation as possible explanations for YuraScanner’s weaker results.
  • Exploration Goals: Total server-side coverage can mislead because fuzzing unintended error paths differs from exploring intended user code paths.On TinyFileManager, ZAP’s coverage included error handling for incorrect CSRF tokens, whereas SpiderSapien submitted forms successfully with valid tokens.
  • Exploration Goals: SpiderSapien is not strictly superior to other exploration methods, and the authors suggest combining scanners because substantial functionality remains unexplored.The discussion frames breadth of testing as a reason to use complementary approaches rather than a single universally dominant strategy.
  • State Destruction: SpiderSapien can break an application by discovering state-destroying actions, as in its poor repeated-evaluation performance on Piwigo.The authors propose safer-action avoidance, breakage heuristics, or application resets as future directions.
  • osCommerce Coverage: On osCommerce, the broad shallow surface favors BFS-like strategies within limited evaluation time, while SpiderSapien’s deeper navigation is closer to DFS.Coverage also tends toward BFS when forms cannot be solved or required interactable elements cannot be detected.
  • False Negatives and Length of Scan: Different runs discover different vulnerability sets because evaluation is limited to 8 hours and scans can destroy application state.The authors note that academic scanners may continue increasing coverage after eight hours.
  • False Negatives and Length of Scan: DokuWiki’s XSS remained undetected because its configuration form combines server-side existence checks with payload constraints, although partial-form attacks may succeed.The authors identify partial form submission as a possible avenue for future scanners.

F. Exploitability of Vulnerabilities

The paper distinguishes vulnerability discovery from exploitability and situates its scanner among complementary approaches for state creation, input validation, and black-box crawling.

  • Exploitability: Scanners generally do not model exploitability against protections such as CSP, CSRF, SameSite cookies, or administrator-only access.Using valid administrator accounts and CSRF tokens means scanner conditions differ from those of a real attacker.
  • Exploitability: 26 of 32 XSS vulnerabilities are exploitable under the paper’s definition: enabling a user to gain additional privileges.Three reflected XSS cases are protected by CSRF, while one stored XSS does not require webmaster privileges.
  • Disclosure: SpiderSapien disclosed all vulnerabilities; Kanboard fixed its reported issue, TinyFileManager is preparing a fix, and responses remain pending from several applications.WordPress developers classify the relevant administrator and editor roles as trusted and will not fix the case.
  • Related Approaches: SpiderSapien’s general approach can interact with React elements, although most evaluated applications do not use React.The authors demonstrate adding and marking notes as completed in the TodoMVC React example.
  • Related Approaches: Unlike single-shot grey-box fuzzers, SpiderSapien’s black-box approach addresses application state and diverse client-side interactions.The related-work discussion contrasts coverage-guided injection fuzzing with methods that reason about state.
  • Related Approaches: EvoCrawl improves navigation through evolutionary search but limits interactable discovery to predefined tags, which SpiderSapien’s method could extend.This comparison identifies element discovery as a complementary improvement rather than a replacement for evolutionary navigation.
  • Related Approaches: SpiderSapien creates application state through the client-side interface, contrasting with approaches that synthesize databases or insert payloads directly into storage.This positions client-side state creation as complementary to database-state and direct-insertion techniques.
  • Related Approaches: YuraScanner uses an LLM for task-driven scanning and form solving, while SpiderSapien assumes a recurrent crawler; their results can therefore be complementary.The two approaches differ in how they define and execute workflows for reaching deeper states.

APPENDIX

The ablation study examines how SpiderSapien’s LLM form solving and element discovery contribute to coverage and XSS detection. Results show that removing either capability reduces performance on applications requiring complex forms or nonstandard interaction patterns.

  • Coverage: 42% and 32% average coverage improvements come from adding the LLM and element-discovery method, respectively.These comparisons are made against the corresponding ablated versions.
  • Coverage: Complex forms in Kanboard, HotCRP, and Leantime make LLM-based solving especially beneficial for reaching deeper functionality.Without advanced form solving, scanners cannot create projects and tasks or submit papers in these applications.
  • Element discovery: Nonstandard controls and state-dependent visibility cause element-discovery ablations to miss actions or interact with elements in the wrong state.Examples include Piwigo label-based buttons and submenu items clicked before their menus are opened.
  • Vulnerabilities: 28 XSS vulnerabilities across 4 applications are found by one ablation, while the other finds 27 across 3 applications.The ablations overlap with SpiderSapien on 13 and 15 vulnerabilities, respectively.
  • Vulnerabilities: Neither ablated version finds XSS in as many applications as SpiderSapien.The evaluation also notes that more time or repeated runs can help minimize false negatives.

VERSIONS SS-LLM

SpiderSapien uses a recurrent crawling loop and an LLM-based form-solving interface to generate structured inputs for complex web forms. The prompt combines realistic-value instructions with few-shot examples and automated browser input execution.

  • Crawler loop: SpiderSapien’s recurrent loop exercises functionality uncovered only after earlier actions, unlike task solvers that execute a static task list.This design supports exploration of deeper application functionality.
  • Form solving: The LLM returns form-solving results as structured (CSS locator, user input) pairs through a function-calling API.The pairs are parsed for subsequent browser automation.
  • Form solving: Input automation handles checkboxes, radio buttons, select elements, file uploads, and text fields.The LLM output is converted into interactions with these form controls.
  • Prompt: The prompt instructs the model to generate realistic values without assuming what values to plug into functions, while matching language and region.The instruction to proceed with plausible values is deliberate.
  • Prompt: Six few-shot form examples provide additional domain context and demonstrate the function calls expected from the LLM.The complete prompt template is shown in Figure 6.

E. LLM Parameters and Cost

The evaluation fixes Gemini 2.5 Flash at temperature 1.7 but does not systematically compare parameter choices. A limited comparison finds greater model-and-temperature variance on Kanboard, while coverage measurement focuses on server-side code.

  • LLM parameters: Gemini 2.5 Flash with temperature 1.7 is the evaluation configuration, and other parameter choices are not systematically evaluated.The authors identify careful input selection and parameter search as potential future improvements.
  • LLM parameters: Kanboard shows more variance across models and temperatures than TinyFileManager in the limited 8-hour comparison.The authors report potential improvement from systematic model and temperature search.
  • LLM parameters: 19291 different LoC are discovered by all model-and-temperature parametrizations in the comparison tables.Only a few particularly successful parametrizations discover especially broad coverage.
  • Coverage and cost: Table VII reports token usage and LLM cost for an 8-hour Gemini 2.5 Flash evaluation, including lower costs when repeated inputs are cached.The caption distinguishes uncached input/output billing from cached repeated-prompt billing.
  • Coverage and cost: Coverage evaluation reports server-side code coverage because client-side metrics are harder to collect in dynamically generated applications.The paper notes that dynamic scripts and links perturb client-side measurements.

H. Client-side Coverage Data

The paper supplements server-side coverage with client-side coverage and interactable-element data, while illustrating deep-state exploration through a manually annotated TinyFileManager state graph. Coverage improvement is also tabulated against other crawlers.

  • Client-side coverage: Table X compares byte-level client-side JavaScript coverage between SpiderSapien and Black Ostrich.The total column reports total JavaScript bytes from Chrome, with percentages shown in parentheses.
  • Element data: Table XI reports interactable elements, non-semantic elements, and non-semantic tags found by SpiderSapien.These categories characterize the elements discovered during crawling.
  • Coverage comparison: Table XII presents SpiderSapien’s coverage improvement over individual crawlers and their union, both per application and averaged.The improvement is defined as (|A| − |B|)/(|B|).
  • State graph: Figure 7 constructs a TinyFileManager state graph from manually annotated state-changing lines of code.Gray states are found only by ZAP, while other states use coloring corresponding to Figure 5.
  • Coverage comparison: Tables XIII and XIV compare server-side executed lines of code between SpiderSapien and other crawlers, including latest state-of-the-art crawlers.The tables separate SpiderSapien-only, shared, and other-crawler-only coverage.
Loading 2609.02532v1…