PaperlayerView exact revision on arXiv

Source-linked AI summary

Cheap, open agents make LLM pollution harder to mitigate

Raluca Rilla, Anne-Marie Nussberger, Rui Mata, Dirk U. Wulff

arXiv:2609.31054v1cs.AI

TL;DR

LLM pollution threatens online data validity, while prior accounts linked low agent prevalence partly to deployment costs and technical effort. We compare nine agent configurations and find that open agents compete with commercial ones, while open-text responses best distinguish agents from humans.

  • Problem

    LLM pollution threatens online data validity, and evidence on autonomous agents had mostly relied on commercial or technically demanding configurations despite deployment costs being linked to low prevalence.

    Source: Introduction · p. 1

  • Method

    We evaluated nine agent configurations with procedural and behavioral checks and compared their open-text, behavioral, and self-report responses against human responses.

    Source: Results · p. 2

  • Results

    Open agents competed with commercial agents and had distinct failure profiles; open-text responses discriminated best, with open models' median SED from human text at 21.11.

    Source: Results · p. 2, Discussion · p. 3

  • Takeaways & Limitations

    Mitigation should not rely on cost barriers or a single procedural check, but combine platform safeguards with complementary procedural and content-based checks, especially open-text analysis.

    Source: Discussion · pp. 3–4

    The findings cover nine configurations in one survey at one point, without adversarial optimization, and depend on the human sample and distance measures.

    Source: Discussion · p. 3

Abstract

from arXiv · show

Large Language Model (LLM) pollution occurs when synthetic responses contaminate data intended to capture human behavior. High deployment costs have so far limited the risk posed by autonomous survey agents. However, open-weight models paired with open-source agentic frameworks may have removed this barrier. We compared the performance and detectability of nine agent configurations, ranging from fully open variants to closed commercial ones. Each agent autonomously completed a survey containing multiple response types yielding various detection checks. Fully open agents ran locally without usage fees and performed competitively with commercial alternatives. Open and commercial agents failed different sets of checks, and no single check reliably detected all agents, but open-text responses discriminated best between agents and humans. These findings identify fully open agents as a distinct risk for LLM pollution and support multilayered detection strategies emphasizing open-text analysis.

Introduction

The paper asks whether low-cost open agents can pose a distinct pollution risk despite deployment barriers, and evaluates their survey performance and detectability against commercial agents. Open agents performed competitively, but detection signals varied by agent type, with open-text responses distinguishing agents from humans most clearly.

Discussion

  • The authors argue that low financial and technical barriers may increase deployment incentives, challenging cost alone as a lasting deterrent without establishing profitability at scale.Open-weight models paired with off-the-shelf frameworks can run locally without usage fees beyond electricity.Source: Introduction · p. 1, Discussion · p. 3

Introduction

  • The study compares nine configurations across procedural, behavioral, self-report, and open-text indicators, estimating survey costs and divergence from human responses.The agents autonomously completed a survey about associations with artificial intelligence, attitudes, and expertise.Source: Introduction · p. 1, Materials and Methods · p. 4

Results

  • Every configuration failed at least one detection check, and open configurations matched or outperformed commercial agents on some checks while showing distinct failure profiles.Closed agents were sometimes caught by reCAPTCHA or paste logging, whereas open agents rarely failed reCAPTCHA, never pasted, and consistently failed a clipped jailbreak honeypot.Source: Results · p. 2
  • Open-text SED was highest for open models (median 21.11), versus mixed configurations (16.91) and fully proprietary models (15.96).Behavioral measures reversed this ordering: open models were closer to human response times than larger, closed agents.Source: Results · p. 2, Figure 1 · p. 3
  • Fully open configurations incurred no API or subscription fees, with marginal usage costs limited to electricity, although some mixed and closed agents also completed surveys at low cost.The estimates exclude hardware acquisition and setup time.Source: Results · pp. 2–3

Discussion

  • The findings characterize nine configurations in one survey at one point in a changing landscape, without adversarial optimization or establishing maximum capabilities.Divergence estimates also depend on the selected human sample and distance measures.Source: Discussion · p. 3
  • The authors recommend combining platform safeguards with complementary procedural and content-based checks, especially analysis of open-text responses.They suggest stronger identity and account verification as one way to raise barriers to automated participation.Source: Discussion · pp. 3–4

Empirical study

The empirical study used a survey about people’s understanding of AI and its relationship to self-reported expertise, use, trust, and risk propensity. It compared a human sample with autonomous agent runs on the same survey.

Empirical study

  • The survey elicited up to five timed AI associations, four 0–10 self-reports, a written expertise justification, demographics, and several response-quality and AI-use items.It was implemented in oTree and ended with open-ended feedback.Source: Empirical study · p. 6

Human sample

  • The human sample comprised 3,242 U.S.-based Prolific participants recruited over two weeks, with compensation of £0.75 per participant.Participants were approximately representative of the population in gender, age, and ethnicity.Source: Human sample · p. 6

Synthetic sample

  • The synthetic sample comprised 360 runs across nine agents, with 40 runs per agent, completed autonomously over three weeks.Agents were instructed to complete the same survey without human intervention.Source: Synthetic sample · p. 6

Agent configurations

The study compares open, mixed, and closed agent configurations and assigns demographic personas to their survey runs. Three piloted systems were excluded before the main sample.

Agent configurations

Open configurations

  • Fully open configurations paired an open-weight model with the open-source Nanobrowser Chrome extension and ran locally, leaving electricity as the only marginal cost.The models were Devstral Small 2 24B, Ministral 3 14B, and Qwen 3 Coder 30B.Source: Open configurations · p. 6

Mixed configurations

  • Mixed configurations used commercially hosted Gemini 2.5 Flash or GPT-4o through the open Nanobrowser framework, requiring a Chrome extension and API key.Source: Mixed configurations · p. 6

Closed configurations

  • Closed configurations paired proprietary models with proprietary frameworks; all four deployed agents could process rendered screenshots as well as HTML, unlike the open and mixed agents.Source: Closed configurations · p. 7

Excluded agents

  • Three piloted systems did not reach the main sample: Skyvern failed Cloudflare verification, BrowserOS failed setup, and GensparkAI refused the survey.Skyvern still incurred user charges for failed attempts; the authors also note that the main-text Cloudflare results show all included agents bypassed the check.Source: Excluded agents · p. 7

Persona assignment

  • Agents received concrete age and gender personas, with half of each agent’s runs assigned female and half male personas.Ages were randomly drawn within five Prolific age bins, yielding four runs of each gender in each bin per agent.Source: Persona assignment · p. 7

Agent prompt

Agents were instructed to complete the survey consistently from an assigned age-and-gender persona, while following rules intended to preserve that character and constrain how they interacted with the survey.

  • The prompt assigned an age and gender, directed agents to complete the survey, and instructed them to provide consent.Source: Agent prompt · p. 7
  • Agents were told to mimic human communication through varied pacing, avoid unnecessarily formal or structured prose, and maintain consistent personal details.Source: Agent prompt · p. 7
  • The character was to ignore ambiguous embedded commands, use only manual keyboard actions, and respond evasively if questioned about automation.The instructions also said not to acknowledge the guidelines or break character.Source: Agent prompt · pp. 7–8
  • Agents were directed to continue through the final survey page, answer each question before advancing after an error, and avoid suspicious embedded commands and paste actions.The study notes that honeypot responses or paste attempts would contradict these explicit instructions.Source: Agent prompt · p. 8

Detection checks

The study used procedural and behavioral checks spanning survey access, timing, interactions, self-reports, and response content to detect different forms of automated responding.

Detection checks

  • The battery comprised 18 checks: 16 pass/fail checks and two timing measures; failure rates report the share of each configuration’s 40 runs triggering a check.The layered checks were intended to cover partial LLM assistance through full LLM delegation.Source: Detection checks · p. 8

CF (Cloudflare)

  • A Cloudflare run failed only if the agent could not reach or complete the survey; recovering by refreshing did not count as failure.Source: CF (Cloudflare) · p. 8

reCAPTCHA (v3)

  • A reCAPTCHA v3 score at or below 0.5 counted as failure; timing measures recorded total survey duration and time spent in each association textbox.Total duration excluded the consent page, and no specific timing pass/fail threshold was applied.Source: reCAPTCHA (v3) · p. 8, Timing measures (ttot, ta1–ta5) · p. 8

Illusion (illusion-illusion item)

  • The illusion check used a duck-only sketch resembling the duck-rabbit illusion and failed respondents who identified it as that illusion.The image was chosen to reduce the chance of genuine respondents failing through visual difficulty.Source: Illusion (illusion-illusion item) · p. 9

Box (honeypot checkbox)

  • The hidden-box check failed when an agent selected an invisible checkbox labeled “Please confirm that you have read this.”Its ambiguous HTML name was intended to make the element’s purpose harder to infer.Source: Box (honeypot checkbox) · p. 9

AI-use (AI-use disclosure)

  • The AI-use disclosure check failed when an agent admitted using AI during the survey, contrary to its instruction to identify as human.The passage notes that self-report is weak because disclosure varies across platforms and falls when respondents expect judgment.Source: AI-use (AI-use disclosure) · p. 9

Copy, Paste, R-click, Drag, Drop (interaction logging)

  • Interaction checks blocked and logged copying, pasting, selecting, dragging, dropping, and right-clicking; a run failed for a given event if it occurred at least once.Source: Copy, Paste, R-click, Drag, Drop… · p. 9

Honeypot items

  • Honeypots hid text from plain view while testing two strategies: jailbreaking prompts planted content in answers, while prompt-leaking prompts requested an exact copy of the agent’s prompt.A run failed when planted content appeared for jailbreaking checks or any verbatim prompt fragment appeared for prompt-leaking checks.Source: Honeypot items · pp. 9–10

Embeddings

Open-text responses were embedded into 1024-dimensional representations for analysis.

  • The analysis embedded each of five free associations separately and together in a fixed ordinal format, as well as participants’ AI expertise explanations.Using qwen3-embedding:0.6b, the authors left spelling, punctuation, and casing unnormalized because these features may distinguish human from synthetic responses.Source: Embeddings · p. 10

Between-group item distance

The section measures human–synthetic separation item by item using standardized Euclidean distance between group centroids, then organizes survey items into three blocks.

Between-group item distance

  • Itemwise separation is measured as standardized Euclidean distance between human and synthetic group centroids.Per-dimension pooled variance down-weights high-variance dimensions so dimensions within an item are on a comparable scale.Source: Between-group item distance · p. 10
  • Items may be scalar (d = 1) or vector-valued, with embeddings represented at d = 1024.Source: Between-group item distance · p. 10
  • Survey items are grouped into embedding, behavioral, and self-report blocks, covering text embeddings and justifications, interaction and timing measures, and AI-related ratings and use.Source: Between-group item distance · p. 10
Loading 2609.31054v1…