Source-linked AI summary
Anatomy of a Scam Call: What 10,000 real scam and spam calls reveal about how phone scammers operate
Ethan Traister, Ankit Raj, Jiaqi Gan, Xingyu Shen, Tyler Wu, Yuchen Zhou, Tommy Duong, Kidus Zewde, Siying Chen, Simiao Ren
TL;DR
Telephone fraud is costly but its conversational mechanics are rarely observed at scale. Using a complete corpus of real calls and randomized fictitious identities, the paper finds templated operations that vary effort by apparent target age without changing their information requests, while scam escalation is predictable from opening lines. The authors also identify annotation and experimental-scope limits that constrain interpretation.
Problem
Telephone fraud is pervasive and costly, but its pretexts, pressure, and requests are seldom captured end to end at a scale supporting measurement.
Method
The paper analyzes 10,211 complete scam and spam calls collected by an active AI voice-agent honeypot and uses uniformly randomized fictitious identities to study target effects.
Results
Telephone fraud is templated: scammers work older-sounding targets harder but do not change what they request, and escalation reaches 0.87 ROC-AUC by the eighth opening line.
Takeaways & Limitations
The findings portray fraud as an industry that rations conversational effort by apparent target while preserving a stable objective and recognizable early content.
Takeaways & Limitations
The ten-identity experiment cannot fully disentangle apparent age from sex or accent, and automated labels should be revalidated against larger hand-coded samples.
Abstract
from arXiv · showhide
Telephone fraud is pervasive and costly, but its inner workings are rarely observed at scale. We analyze a complete corpus of 10,211 inbound scam and spam calls -- 913 hours of audio and 330,956 transcribed turns from 5,780 distinct numbers -- collected over 54 days by an AI voice-agent honeypot that answered callers and kept them talking, and introduced in a companion data descriptor. We separate outright scams, which solicit sensitive information, from the larger stream of predatory but legal lead generation ("spam") that feeds them. Scam operations keep office hours (6.6x more calls per weekday than weekend day); thousands of disposable numbers run a small catalog of recycled scripts (thirty opening clusters, half the traffic in the top five); and callers solicit identity anchors -- a home address and a date of birth -- far more often than payment credentials, pressing through persistence and manufactured authority rather than overt threats. Our central experiment asks: does it matter who picks up? Every seeded lead carried one of ten fictitious identities drawn uniformly at random, so the identity a fraud operation reaches is fixed before the caller exists. Across 1,823 randomized calls, scammers spent about 15% more conversational turns per decade of the target's apparent age (rate ratio 1.15, 95% CI 1.08-1.23; randomization p = 0.005) -- yet what they asked for did not change (26.3% of calls reached a request for sensitive information; odds ratio 0.99 per decade, 95% CI 0.90-1.08). A second experiment casts early detection as a benchmark: from a scammer's opening lines alone, on a caller-disjoint split, escalation is predictable at 0.72 ROC-AUC from the first line and 0.87 by the eighth, and a plain bag-of-words classifier matches a fine-tuned on-device language model. Telephone fraud emerges as a templated industry that varies how hard it works a target, but not what it wants.
1 Introduction
The paper analyzes scam calls from inside complete conversations, organizing the study around empirical questions about operations, persuasion, detection, and target identity. Its randomized identity experiment finds that apparent age changes conversational effort, not the information scammers seek.
- Scope and approach: 10,211 real scam and spam conversations were recorded end to end over 54 days using an active AI voice-agent honeypot.The companion corpus contains the conversations analyzed here; this paper studies the corpus rather than introducing it.
- Research questions: The analysis tests whether scammers keep office hours, recycle scripts, reveal scams early, solicit particular information, apply pressure, and respond differently to target identities.These ordinary-person questions are mapped to measurable properties of the corpus.
- Randomized identity experiment: Ten fictitious identities were assigned uniformly at random to manufactured leads, making the reached identity a randomized treatment outside the caller’s influence.The design functions as a correspondence audit of live fraud operations.
- Randomized identity experiment: About 15% more conversational turns per decade of apparent age occurred, while the probability of requesting sensitive information remained statistically indistinguishable across identities.The experiment’s central result is that who the target appears to be changes how hard scammers work, not what they want.
- Operational and behavioral findings: 6.6× more weekday-to-weekend calling, thirty recycled opening scripts, early-detection benchmarks, and measured solicitation and pressure patterns characterize telephone fraud as industrialized and templated.The benchmark reports 0.72 ROC-AUC from the first line and 0.87 by the eighth, while bag-of-words matches fine-tuned on-device language models.
2 Literature review
Prior work observes abusive calling, tracks disposable-number infrastructure, or engages scammers, but leaves end-to-end conversational measurement and randomized target audits open. This paper combines active engagement with randomized identities and content-based detection to study those gaps in live telephone fraud.
- Passive telephony honeypots: Passive telephony honeypots reveal which numbers call and how often, but usually capture robocalls, dead air, and hang-ups rather than scam substance.They do not engage callers, limiting observation of conversational pretexts and pressure.
- Metadata and infrastructure: Number churn weakens blacklist tracking, while this paper’s content-based detection targets scam evidence that persists as telephone numbers change.The related-work comparison frames content as complementary to infrastructure-level defenses.
- Detection benchmarks: The benchmark extends prior work that evaluates fraud detectors on real, uncurated deceptive content rather than curated proxies.Here, that methodological question is carried over from images to the voice channel.
- Targeting and causal identification: Research on elder targeting is contested, and no prior audit had tested live telephone fraud with randomized target identities.Complaint statistics are commonly interpreted as evidence of disproportionate elder targeting, but the underlying evidence is disputed.
- Targeting and causal identification: The honeypot supplies the missing design by manufacturing leads, randomizing each identity, and answering calls to observe behavior as a function of an assigned target attribute.This makes the comparison causal rather than merely observational.
3 Methodology
The study combines a closed corpus of answered calls with descriptive statistics, caller-disjoint prediction, and randomized identity assignment. Its methods distinguish scam from spam, use automated transcript labels, and test how early escalation can be predicted and how identity affects calls.
- Study design: The analysis separates descriptive corpus statistics, a caller-disjoint predictive benchmark, and a randomized field experiment by what each design can support.Q1, Q2, and Q4 are descriptive; Q3 evaluates prefix-based prediction; Q5 uses randomized identity assignment.
- Corpus: 6,619 substantive calls with at least two conversational turns were analyzed from a corpus of 10,211 calls collected over 54 days.The corpus contains 913 hours of audio, 330,956 transcribed turns, and 5,780 distinct originating numbers.
- Corpus collection: The honeypot submitted seeded lead-generation forms and used a conversational voice agent to answer callers, keep them engaged, record calls, and transcribe individual turns.The agent used speech-to-text, a language model, and text-to-speech, with a median reply latency of 1.16 seconds.
- Labeling: Calls received automated three-pass labels, including scam, spam, legitimate, unsure, and a fixed vocabulary of caller requests.The holistic pass used the full transcript and identified requests such as Social Security numbers, dates of birth, addresses, cards, bank details, money transfers, and gift cards.
- Early detection: Escalation prediction used the first 1, 2, 3, 5, or 8 scammer utterances and compared classifiers from a majority baseline through bag-of-words, embeddings, and higher-capacity models.The target was whether the full call eventually reached a request for sensitive information; 1,115 of 6,374 calls escalated.
- Randomized identity experiment: The identity experiment randomized one of ten fictitious personas to each seeded lead while holding the agent’s behavioral protocol and technical systems constant.Identities differed in voice, name, stated age, biography, address, and date of birth; each used its own telephone line and telephony project.
- Robustness: The randomized analysis was specified after collection ended and reported robustness checks across alternative samples, transformations, exclusions, and fixed-effects specifications.The authors describe it as a secondary analysis rather than a preregistered trial and report every examined persona-level outcome.
- Scam versus spam: Scam and spam were operationally distinguished at the observable solicitation stage rather than treated as nested categories.The holistic label classified 3,949 calls as spam, 949 as scam, 380 as legitimate, and 1,096 as too brief to judge among 6,374 substantive calls.
4 Results and discussion
Across 10,211 calls, telephone fraud appears industrialized: it follows business schedules, recycles scripts across disposable numbers, and uses identity data through persistence and manufactured authority. Random assignment shows that older-presenting targets receive more conversational effort, but not different requests.
- Operational structure: 253 weekday calls versus 38 weekend calls produced a 6.6-fold difference, with volume concentrated during daytime business hours.The pattern spans the East Coast opening through the West Coast closing, resembling an organizational schedule rather than random or automated noise.
- Operational structure: 5,916 substantive openings from 3,652 numbers collapsed into thirty recurring clusters, with the five largest accounting for 50% of openings.The median cluster used 102 numbers and the largest used 685, indicating scripts were recycled across a churn of phone numbers.
- Operational structure: A shared lead-resale pipeline feeds both legal spam and scams, so the same industrialized opener can precede a sales transfer or a sensitive-information request.Among calls with verdicts, 3,949 were spam and 949 were scams; 1,115 conversations ended in a sensitive-information request.
- Defensive implications: 0.72 ROC-AUC from the first line and 0.87 by the eighth show that escalation is predictable early, while TF-IDF with logistic regression matches or beats fine-tuned small language models.The benchmark used a caller-disjoint held-out split, making a cheap content-based detector a practical starting point.
- Targets and tactics: Home addresses and dates of birth were requested more often than payment credentials, while persistence and manufactured authority dominated pressure tactics over overt threats.Among 2,654 substantive requesting calls, home addresses appeared in 1,722 and dates of birth in 1,518; direct money transfers occurred in 123 and gift-card requests in 10.
- Who picks up: 1.152 was the rate ratio for scammer turns per decade of apparent age, showing that older-sounding targets received substantially more conversational effort.This estimate used 1,823 substantive randomized calls and remained positive when holding opening strategy or pretext vertical fixed.
- Who picks up: 26.3% of substantive calls reached a sensitive-information request, with odds ratio 0.99 per decade, indicating that apparent age changed effort but not what scammers asked for.The age relationship was flat overall, and none of the nine individual request types varied significantly with age.
5 Conclusion
Across 10,211 scam and spam conversations, telephone fraud appears predictable and industrialized: callers work business hours, recycle scripts, reveal escalation early, and vary effort by target age without changing their requests.
- 5 Conclusion: 10,211 conversations show phone scammers working business hours, recycling scripts across disposable numbers, and revealing the scam early.The conclusion characterizes telephone fraud as a predictable industry whose regularities can support inexpensive defense.
- 5 Conclusion: Older-presenting targets receive measurably more scammer time, while callers ask every target for the same things at the same rate.The experiment separates variation in conversational effort from variation in the requested information.