PaperlayerView exact revision on arXiv

Source-linked AI summary

Users really do respond to smishing

Muhammad Lutfor Rahman, Daniel Timko, Hamid Wali, Ajaya Neupane

arXiv:2212.13312v1cs.CR

TL;DR

Smishing is a growing mobile threat, and limited evidence explains who responds and which message features influence susceptibility. The paper conducts consented experiments with realistic SMS messages and records CLICK, REPLY, and CALL behaviors. It finds substantial first-round and repeat responses, with statistically significant effects associated with user actions, entity scenarios, and demographics, while consent may have biased results.

  • Problem

    Smishing is a growing mobile threat, but little research has examined who falls for these attacks, why they respond, and how message attributes and demographics affect success.

  • Method

    The authors conduct a consented empirical study using realistic smishing messages, manipulated attributes, recorded CLICK, REPLY, and CALL actions, and post-test feedback from participants.

  • Results

    16.92% of participants potentially fell for the first attack and 12.82% fell for a second attempt; response actions, entity scenarios, and demographic features showed statistically significant effects.

  • Takeaways & Limitations

    Realistic smishing messages can be effective, supporting the development of detection mechanisms, user training, notification messages, and defensive systems.

  • Takeaways & Limitations

    Participants’ prior consent and awareness that they had joined a mobile-user study may have biased the results.

Abstract

from arXiv · show

Text phish messages, referred to as Smishing is a type of social engineering attack where fake text messages are created, and used to lure users into responding to those messages. These messages aim to obtain user credentials, install malware on the phones, or launch smishing attacks. They ask users to reply to their message, click on a URL that redirects them to a phishing website, or call the provided number. Thousands of mobile users are affected by smishing attacks daily. Drawing inspiration by the works of Tu et al. (USENIX Security, 2019) on Robocalls and Tischer et al. (IEEE Symposium on Security and Privacy, 2016) on USB drives, this paper investigates why smishing works. Accordingly, we designed smishing experiments and sent phishing SMSes to 265 users to measure the efficacy of smishing attacks. We sent eight fake text messages to participants and recorded their CLICK, REPLY, and CALL responses along with their feedback in a post-test survey. Our results reveal that 16.92% of our participants had potentially fallen for our smishing attack. To test repeat phishing, we subjected a set of randomly selected participants to a second round of smishing attacks with a different message than the one they received in the first round. As a result, we observed that 12.82% potentially fell for the attack again. Using logistic regression, we observed that a combination of user REPLY and CLICK actions increased the odds that a user would respond to our smishing message when compared to CLICK. Additionally, we found a similar statistically significant increase when comparing Facebook and Walmart entity scenario to our IRS baseline.

1 INTRODUCTION

Smishing is a growing mobile threat, but limited research has measured who falls for it and which message attributes and demographics influence responses. This study addresses that gap with a consented empirical study of realistic smishing attacks and reports evidence for response rates, repeat susceptibility, and statistically significant predictors.

  • Research gap: Smishing research remains limited despite a 700% increase during the first six months of 2021 and prior evidence from only 24 targeted users.The authors distinguish their study from Blancaflor et al. by using eight message types and 265 unique users.
  • Research gap: The study examines how message attributes and demographics affect smishing success across user responses, entities, scenarios, area codes, and motivational factors.The research questions ask about overall success, message attributes, and demographic effects.
  • Study contribution: The study uses consented, near-realistic attacks involving university students and members of the general public recruited through social media platforms.The contribution emphasizes a real smishing attack study conducted without participants’ prior knowledge of the smishing study.
  • Study contribution: 16.92% of 260 participants responded to first-round smishing messages, while 12.82% of 195 participants fell for a second attempt.Responses included calling, clicking, or replying; the second round used different messages.
  • Implications: The findings are intended to inform smishing detection, user training, notification design, and defensive mechanisms, although few participants explained why they fell for messages.The authors specifically identify reluctance to provide post-test feedback as a direction for future research.

2 STUDY DESIGN & DATA COLLECTION

The study manipulates message entities, scenarios, area codes, motivations, and response actions across twelve SMS experiments. Participants’ CLICK, REPLY, and CALL behaviors are recorded through a flowchart-based procedure and used to measure smishing success.

  • 2.1 Attributes: The study defines entity as the organization or subject that appears to contact the user, enabling comparisons such as Bank and Facebook’s Security Team.Entity variation is used to examine what convinces victims to respond.
  • 2.1 Attributes: Scenarios represent message content and are organized around reward or fear motivations to examine what persuades users to respond.Examples include suspended Social Security numbers, bank-transaction approvals, and rewards.
  • Response measures: The measured smishing actions are CLICK, REPLY, and CALL, each representing a distinct way a participant can respond to an attack.CLICK redirects users to a fake website; REPLY and CALL provide alternative response channels.
  • Experimental design: Twelve experiments use eight unique SMS messages combining entity, scenario, area code, user action, and motivation attributes.Experiments E1–E7 comprise the first round and E8–E12 the second round.
  • Experimental procedure: A flowchart controls message handling and debriefing, with survey responses collected after participants respond through the relevant medium.The procedure is illustrated in Figure 1 and adapts a debriefing approach from Tu et al..

3 STUDY PROCEDURES

The study used deceptive, ethically reviewed smishing experiments with recruited participants, while recording message responses and protecting participant data. It also tracked delivery outcomes across experiments using messaging-service records.

  • Ethical and safety considerations: The study used incomplete disclosure to obtain bias-free responses, then debriefed participants about the study’s true purpose and smishing risks.The researchers coordinated with their university’s IRB and IT department to address ethical and security concerns.
  • Delivery tracking: Table 4 summarizes final delivery status for unique messages, and potential smishing rates count only delivered messages.The table distinguishes absent user actions marked with an asterisk from actions not mentioned in an experiment, marked with a dash.
  • Delivery tracking: Table 5 reports the status of all unique messages that Twilio failed to deliver in each experiment.This provides a separate view of undelivered-message outcomes from the delivered-message accounting used for potential smishing rates.
  • Data collection: The experiments collected demographic information, text-message responses, and survey responses while storing participant data in a secure author-accessible folder.Recorded demographics included age, gender, education, profession, race, and weekly mobile usage.
  • Participants: Participants were recruited primarily from the US through online channels, consented before testing, and provided mobile numbers and email addresses for the study.The researchers received 1002 registrations, removed invalid or duplicate entries, and identified 622 available participant targets.
  • Experiment implementation: Researchers used Twilio’s API and a custom C#, HTML, and JavaScript control system to send messages during weekdays and regular office hours.The experiments ran from November 25, 2021, to December 23, 2021.

4 PERFORMANCE MEASURES

The study defines smishing success as any participant click, reply, or call, then measures overall rates and how message attributes, repeated exposure, actions, and demographics relate to responses.

  • Message Attributes: 17.87% (37/207) responded to reward-motivated messages, compared with 12.96% (32/247) for fear-motivated messages.
  • Message Attributes: Facebook and Walmart entity scenarios significantly affected potential smishing odds relative to the IRS baseline, with p = 0.004 and p = 0.041, respectively.The fear-based entity comparison was also significant at p = .016, whereas the reward-based entity comparison was not significant at p = .714.
  • User Actions: The combined CLICK+REPLY action significantly increased potential-fall odds versus CLICK alone, while individual CLICK, CALL, and REPLY comparisons were not significant.The combined-action comparison reported p = 0.025; individual tests found no significant differences between CLICK and REPLY, CLICK and CALL, or CALL and REPLY.
  • Repeated Exposure and Awareness: Repeated smishing did not significantly change success rates, and participants frequently replied STOP, including 31.82% of the time when no STOP option was present.The repeated-smishing comparison reported p = .064; 11 of 22 text replies were STOP responses.
  • Demographics: 11–20 hours of weekly phone use significantly increased potential-fall odds versus the 6–10-hour baseline, while 18–24-year-olds had significantly lower odds.The mobile-use comparison reported p = 0.005, and the age comparison reported p = 0.005.

5 SURVEY RESPONSES

Survey feedback shows that participants sometimes responded despite recognizing scams, with realism, urgency, local numbers, government themes, and curiosity shaping responses.

  • Participants sometimes responded even when they were aware that the message was a scam.One participant attributed falling for the message to its local number, government subject, and tax-lawsuit threat.
  • Realism, local numbers, threatening language, and government information made the IRS fear-motivated message persuasive.
  • Curiosity was the main reported motivation for potentially falling for click-based smishing messages.One participant previewed the destination before clicking, but spoofed websites can remain difficult to distinguish from authentic sites.
  • Previewing a link helped one participant inspect its destination before clicking, but spoofed websites can still resemble legitimate sites.

6 DISCUSSION

The discussion finds that crafted messages can produce substantial response rates, while repeat exposure lowers responses and study design constraints limit interpretation.

  • 34.62% of participants responded in E4, the highest rate, while repeat exposure produced 12.82% versus 16.92% initially.E1 and E12 produced the next-highest reported rates, 30.00% and 18.33%, respectively; proactive responses fell from 7 to 2.
  • The study found no statistically significant effect for area-code type or for fear versus reward motivation.
  • Potential message filtering by Twilio or telecommunications carriers may have affected which SMSes reached participants.The authors call for further study of current SMS blocking schemes and suggest automated or third-party detection systems.
  • Consent may have biased behavior because participants could remember giving their phone numbers before the experiment.The authors state that collecting numbers and obtaining consent was required for IRB compliance.
  • Delivered-message status could include false positives because carrier confirmation does not guarantee inbox receipt.
  • The sample is not census-representative, timing may affect response rates, and changing message content limits perfect comparisons.

7 RELATED WORK

Prior work spans empirical phishing susceptibility, spam-SMS detection, mobile-fraud analysis, and human factors, but systematic smishing evidence remains limited.

  • Blancaflor et al. conducted a phishing campaign across email, social media chat, and SMS, finding limited user interaction with one phishing link.
  • Spam-SMS research has emphasized classification and fraudulent-base-station detection rather than users’ responses to smishing.Reported directions include SMS classification, fake-base-station detection, spammer characteristics, and machine-learning comparisons.
  • Other studies examine human vulnerabilities, training, and ways users identify social-engineering attacks.
  • Mobile-fraud research has examined SMS-based credential theft, malware installation, victim understanding, and attacker motivations.
  • Earlier phishing studies link susceptibility to demographic factors, security knowledge, awareness, behavior, training, employment, and education.

8 CONCLUSION

The paper presents a systematic empirical study of smishing using experiments with 265 users and finds susceptibility across actions, entity scenarios, and demographic features.

  • The study conducted twelve consented smishing experiments with 265 unique users and assessed responses across user actions, entity scenarios, and demographics.
  • Participants potentially fell for smishing across user actions, entity scenarios, and several demographic features relative to baseline values.The conclusion also notes that curiosity and greater knowledge may not prevent victimization.
  • The authors identify greater user awareness and automatic SMS phishing detection as needed defenses against smishing.

APPENDIX

The appendix documents the survey sample, post-experiment debriefing, response-specific survey questions, and study closeout information.

  • The survey sample comprised 265 participants, with demographics reported in Table 12.
  • The debriefing explained that the study measured text-phishing efficacy, clarified that the messages were not an actual scam, and stated that personal details were not collected.
  • Participants were asked what most influenced their decision to reply, click the link, or call back.
  • The experiment ended with participation thanks, researcher contact information, and institutional-review-board contact information for participant-rights or risk concerns.
Loading 2212.13312v1…