PaperlayerView exact revision on arXiv

Source-linked AI summary

A Corpus of Real Scam- and Spam-Call Conversations from an Active Voice-Agent Honeypot

Ethan Traister, Dennis Tsang Ng, Siyu Zhang, Huaiyu Guo, Tommy Duong, Tyler Wu, Yuchen Zhou, Xingyu Shen, Jiaqi Wu, Simiao Ren

arXiv:2609.29528v1cs.CRcs.CLcs.LG

TL;DR

Complete, real scam conversations over voice are difficult to obtain at scale, while existing collection approaches often capture automated calls or do not scale to sustained dialogue. This paper builds an active voice-agent honeypot that seeds numbers into lead-generation channels, engages inbound callers, and records and labels their calls. The resulting corpus contains 10,015 inbound calls, including 6,601 with two or more turns, and synthetic-trained detectors collapse in precision on real traffic.

  • Problem

    Scalable sources of complete, real, consistently labeled scam conversations over voice are missing, while passive honeypots, manual scam-baiting, and text-focused systems each capture only part of the picture.

    Source: Introduction · p. 1

  • Method

    The system seeds dedicated numbers into lead-generation channels, uses a conversational persona agent to engage inbound callers, and records, transcribes, and automatically labels calls.

    Source: Introduction · pp. 1–2

  • Results

    10,015 inbound calls, including 6,601 with two or more turns, were collected over 53 days; synthetic-trained detectors collapse in precision on real traffic.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    The paper presents a publicly released corpus of real, multi-turn scam- and spam-call conversations collected autonomously by LLM voice agents.

    Source: Introduction · p. 2

    The corpus is English-language and U.S.-centric, with most calls answered by one persona on one number; its automatic labels are silver annotations and the human-annotated subset is modest and singly annotated.

    Source: Limitations · p. 7

Abstract

from arXiv · show

Real conversations between fraudsters and their targets are among the most informative artifacts for studying telephone scams, yet also the scarcest: passive honeypots overwhelmingly capture automated messages and hang-ups, large-scale studies characterize call metadata rather than dialogue, and manual scam-baiting does not scale. We present a dataset of real scam-call conversations collected by an active voice-agent honeypot. Dedicated numbers are seeded into the lead-generation channels fraud operations harvest; inbound callers are answered by a low-latency conversational agent that adopts a plausible target persona and sustains the interaction while every call is recorded, transcribed, and automatically labeled. Over an initial 53-day window we captured 10,015 inbound scam and spam calls (6,601 with two or more turns): roughly 895 hours of audio and 328,869 transcribed turns from 5,665 distinct originating numbers. Under a holistic classifier the substantive calls are predominantly predatory-but-legal lead generation ("spam", about three in five), while about one in seven is an outright "scam" (949 in this snapshot). Each call carries a turn-level transcript, three-channel audio, per-turn latency telemetry, and layers of automatic labels, including a holistic scam/spam/legitimate judgment corroborated by independent human review (75% agreement on the binary decision). We describe the collection system, the record structure, and technical validation of the corpus's realism and label quality, including that the agent is recognized as non-human in only about 5% of engaged calls. We also benchmark established scam-detection methods, where detectors trained on published synthetic dialogue collapse in precision on real traffic.

Introduction

The paper presents an actively collected corpus of real scam- and spam-call conversations to address the scarcity of complete, consistently labeled voice dialogue. It combines seeded inbound calls, a conversational voice agent, and automatic labeling, then reports the corpus and its validation.

Introduction

  • Real scam conversations remain difficult to obtain at scale, despite their value for examining how operators deliver pretexts, apply pressure, and solicit money or credentials.Telephone fraud carries large reported losses, and voice-enabled agents can also place fraudulent calls at negligible cost (Federal Trade Commission 2025; Federal Bureau of Investigation, Internet Crime Complaint Center 2025; Fang, Bowman, and Kang 2024; Badhe 2025).Source: Introduction · p. 1
  • Passive honeypots, manual scam-baiting, fixed-script bots, and text-focused LLM systems each leave a gap in scalable, complete real voice conversations (Gupta et al. 2015; Prasad et al. 2020; Sahin, Relieu, and Francillon 2017; Siadati, Jafarian, and Jafarikhah 2025; Hossain et al. 2025).The paper identifies complete, real, consistently labeled scam conversations over voice as the missing resource.Source: Introduction · p. 1
  • The system seeds dedicated numbers into lead-generation channels, answers callers with an adaptive persona agent, and records, transcribes, and labels every call.The collection grew from roughly 127 calls in its first week to more than 2,500 in a peak week, and the methodology is being extended to multiple personas and numbers.Source: Introduction · pp. 1–2
  • The authors describe the corpus as the first large, publicly released collection of real multi-turn scam- and spam-call conversations autonomously gathered by LLM voice agents.They distinguish it from conversationally inactive honeypots, text-based scam-baiting, and simulated or role-played dialogue (Prasad et al. 2020, 2023; Siadati, Jafarian, and Jafarikhah 2025; Hossain et al. 2025; Basta, Atkins, and Kaafar 2025; Charnsethikul and Mirkovic 2025).Source: Introduction · p. 2
  • 10,015 inbound calls, including 6,601 with two or more turns, form the 53-day corpus; it contains about 895 audio hours, 328,869 turns, and 5,665 originating numbers.The corpus includes transcripts, three-channel audio, latency telemetry, and multi-layer labels, including a human-corroborated holistic judgment.Source: Introduction · p. 2

Related Work

Prior work measures inbound telephony traffic, studies particular fraud types, or engages operators in other settings. The paper positions its corpus as conversational, publicly released voice-call data and notes a gap in evaluating synthetic-trained detectors on real conversations.

Related Work

  • Passive honeypots characterize inbound traffic and robocall content but do not converse with live fraud operators or release transcripts; this corpus is conversational and publicly released (Gupta et al. 2015; Prasad et al. 2020, 2023).Related work also studies phone numbers as identifiers and particular fraud verticals such as technical-support scams (Costin et al. 2013; Marzuoli et al. 2016; Balduzzi et al. 2016; Gupta et al. 2018, 2016; Miramirkhani, Starov, and Nikiforakis 2017).Source: Related Work · p. 2
  • LLM scam engagement has focused chiefly on email and text, while voice-baiting studies have used simulated or role-played scammers rather than real inbound calls (Siadati, Jafarian, and Jafarikhah 2025; Hossain et al. 2025; Basta, Atkins, and Kaafar 2025; Charnsethikul and Mirkovic 2025; Wood et al. 2023).Earlier approaches include a fixed-script bot and a phone virtual assistant that screens robocalls (Sahin, Relieu, and Francillon 2017; Pandit et al. 2023).Source: Related Work · p. 2
  • Detectors trained on synthetic dialogue or proprietary, unreleased call sets had not been evaluated on real in-the-wild conversations; the paper reports that synthetic-trained detectors collapse in precision on its real calls (Shen et al. 2024).Figure 1 describes daily inbound volume and cumulative calls over the collection window, while Table 1 summarizes corpus counts and excludes development and self-test calls.Source: Related Work · p. 2, Figure 1 · p. 2, Table 1 · p. 2

The Collection System

The collection pipeline continuously seeds numbers into lead-generation channels, engages callers with a low-latency persona agent, and captures calls with layered labels. Its stages are designed to generate inbound calls, sustain dialogue, and preserve the resulting records.

The Collection System

  • The pipeline has three stages—seeding, engagement, and capture and labeling—that run continuously on a single commodity server.Seeding generates inbound calls, engagement answers and sustains conversations, and capture and labeling record and annotate calls.Source: The Collection System · pp. 2–3, Figure 2 · p. 3

Seeding

  • A web-search and form-filling system submits a consistent target identity and dedicated honeypot number to lead-generation forms, which aggregators may resell within hours.The bulk of collection uses a fictitious elderly, recently widowed retiree persona; the system is being extended to personas varying in age, gender, region, and disposition.Source: Seeding · p. 3

Conversational Engagement

  • The production agent transcribes callers, generates replies with a language model, and renders speech in a persona voice; its prompt uses clarifying questions, caller premises, and affective cues to sustain engagement.Median agent reply latency is 1.17 seconds, within the range of ordinary telephone conversation; a second variant uses native speech-to-speech, and calls record which agent answered.Source: Conversational Engagement · p. 3

Capture

  • Each call is stored in three synchronized audio channels, while each turn records transcript, speaker, timestamp, and latency components; recording continues through late-arriving content.Source: Capture · p. 3

Automatic Labeling

  • Labeling first removes non-substantive calls, then applies lenient behavioral, strict solicitation-based, and holistic scam/spam/legitimate classifiers, with auxiliary opening and ending labels.The strict classifier waits through an inactivity settling interval, and the holistic rubric matches the one given to human reviewers; an external commercial classifier scores a subset.Source: Automatic Labeling · p. 3

Data Records

The dataset pairs relational call and turn records with synchronized audio files. Its substantive calls range from short pitches to prolonged engagements and cover varied pretexts and solicitation behaviors.

Data Records

  • The relational store separates one-row-per-call session records from one-row-per-turn message records, alongside 30,687 WAV files totaling 614 GB.The store is 98 MB and can be distributed independently for text-only use.Source: Data Records · p. 3
  • Most substantive calls last one to five minutes, while the distribution has a tail beyond twenty minutes and genuine engagements can exceed 1,200 turns over roughly 1.8 hours.The one-to-five-minute mass reflects lead-qualification pitches; the longer tail reflects prolonged engagements.Source: Data Records · pp. 3–4, Figure 3 · p. 4
  • Calls span Medicare, insurance, home-security, autowarranty, debt-relief, and government, bank, and technical-support pretexts, as well as requests for addresses, birth dates, and Social Security numbers.Figure 4 shows the most frequent lenient-classifier signals, with scam pretexts and tactics in blue and direct requests for personal or credential information in orange.Source: Data Records · p. 4, Figure 4 · p. 4

The Public Release

The public release is a de-identified, 1,000-call subset of the corpus, structured for analysis but deliberately selected for long engagement. Its labels preserve distinct operational judgments, and its caller-disjoint folds support evaluation that tests generalization to unseen callers.

The Public Release

  • The openly downloadable, de-identified subset contains 1,000 calls, while the complete corpus is available under a data-use agreement.The subset is released without audio; the complete corpus includes three-channel audio and unredacted transcripts.Source: The Public Release · p. 4

Record Schema

  • Each self-contained JSON record combines per-call metadata, every label layer, and the full turn sequence, with a flat CSV for call-level use.Categorical fields use closed vocabularies and can serve directly as targets; Table 2 describes the schema.Source: Record Schema · p. 4, Table 2 · p. 5

Composition and Selection Bias

  • The release pairs 500 longest-engagement scams with 500 longest-engagement predatory-spam calls, yielding 183,491 turns; legitimate calls and audio are excluded.The calls span 28 May to 19 July 2026, and 573 of the 1,000 calls use the original seeded persona.Source: Composition and Selection Bias · p. 4, Table 3 · p. 5
  • Selection is nonrandom: released calls have a 19.8-minute median duration versus 2.2 minutes corpus-wide, and none is shorter than 4.8 minutes.The subset is a high-engagement stratum, not a basis for estimating prevalence or base rates; within it, scam and spam medians are 14.7 and 23.0 minutes, respectively, so length is not a fraud proxy.Source: Composition and Selection Bias · p. 4

What the Label Layers Separate

  • Credential requests alone do not define fraud: 64% of released scams and 40% of spam calls reached an explicit credential request, while institutional impersonation occurred in 25% and 1%, respectively.The lenient layer flags 74% of scams and 84% of spam, making it a sensitive screen rather than a decision rule.Source: What the Label Layers Separate · p. 4

Suggested Evaluation Protocol

  • Of 1,000 calls, 420 came from numbers appearing elsewhere in the subset, so random splits risk placing the same operator and script in both partitions.The archive supplies caller-disjoint folds, and reported benchmarks use that protocol to avoid rewarding caller-specific memorization.Source: Suggested Evaluation Protocol · pp. 4–5

Technical Validation

The validation evidence supports natural, sustained engagement with seeded callers, while the labeling review and benchmark results show both useful label quality and a marked synthetic-to-real transfer gap. The traffic’s weekly pattern and layered call categories further characterize what the corpus captures.

Technical Validation

  • 1.17 s median reply latency, 50 mean turns per substantive call, and only 339 of 6,601 calls ending in apparent bot recognition support natural, sustained engagement.In an example, the caller recites seeded identity details before escalating to a Social Security number request.Source: The Calls Are Real, the Engagement… · p. 5

Seeding Drives the Traffic

  • Near-zero weekend volume followed by Monday recovery matches commercial lead-generation business hours and is presented as evidence of organized operations working seeded lists.The holistic classification also identifies 949 outright scams, including 552 with explicit credential requests; Figure 6 describes opening and ending taxonomies as a behavioral fingerprint.Source: Seeding Drives the Traffic · p. 6, Figure 5 · p. 6, Figure 6 · p. 6

The Labels Are Meaningful

  • The holistic classifier labeled 949 substantive calls as scams and 3,949 as predatory-but-legal spam, while the strict classifier isolated 710 calls with explicit sensitive-information requests.The holistic label also covers payment demands, impersonation, and bulk identity harvesting; human reviewers corroborated the holistic scam/not-scam decision on 75% of sampled calls.Source: The Labels Are Meaningful · p. 6, Human Review Corroborates the Labels · p. 6, Figure 5 · p. 6

Human Review Corroborates the Labels

  • 75% of binary scam/not-scam judgments agreed with human reviewers for the holistic label, versus 67% for the strict credential-ask label.Reviewers labeled 60 calls, and residual disagreement centered on the boundary between outright scams and predatory spam.Source: Human Review Corroborates the Labels · p. 6
  • ROC-AUC 0.75–0.78 and F1 0.02–0.40 were retained when detectors trained on synthetic data were applied unchanged to real calls, with precision collapsing.On the real corpus, TF-IDF baselines reached ROC-AUC 0.94–0.96 and F1 0.70–0.75; the zero-shot LLM reached ROC-AUC 0.94 and F1 0.79.Source: Established Detection Methods Are… · p. 6, Figure 7 · p. 7

Usage Notes

The corpus is intended to support fraud-detection, social-engineering, taxonomy, and real-time dialogue research. Its scope is constrained by a U.S.- and English-focused collection, limited persona diversity, silver labels, and active engagement; release practices also restrict access to audio.

Intended Uses

  • The corpus supports realistic-dialogue fraud detection, analysis of social-engineering tactics and escalation, scam-script taxonomies, defensive agents, and real-time spoken-dialogue research.Source: Intended Uses · p. 7

Limitations

  • The collection is English-language and U.S.-centric, largely uses one persona on one number, and reflects behavior toward an engaged plausible target rather than neutral inbound traffic.Labels are automatic silver annotations, the human-reviewed subset is modest and singly annotated, and transcripts and synthesized speech may contain artifacts.Source: Limitations · p. 7

Ethics and Responsible Use

  • Honeypot lines are researcher-controlled and inbound-only; the release excludes legitimate calls, de-identifies released text, and withholds audio because of voiceprint reidentification risk.Users must not re-identify parties or use the corpus to defraud, harass, or deceive.Source: Ethics and Responsible Use · p. 7

Code and Data Availability

The evaluation and analysis code for reported benchmarks and release statistics accompanies the data, while the collection pipeline is withheld on dual-use grounds.

Code and Data Availability

  • Evaluation and analysis code reproducing the reported benchmarks and release statistics is released with the data.Source: Code and Data Availability · p. 7
  • The collection pipeline is withheld on dual-use grounds and is available from the authors.Source: Code and Data Availability · p. 7
  • The public release is distributed under CC BY-NC with a datasheet, while the full corpus and audio are available under a data-use agreement.A persistent identifier will be assigned on publication.Source: Code and Data Availability · p. 7
Loading 2609.29528v1…