PaperlayerView exact revision on arXiv

Source-linked AI summary

Assessing Socio-Cyber Vulnerability Using Survey and Social Media Data

Shutonu Mitra, Qi Zhang, Tomas Neguyen, Hossein Salemi, Fengxiu Zhang, Michin Hong, Chang-Tien Lu, Hemant Purohit, Jin-Hee Cho

arXiv:2608.26388v1cs.SI

TL;DR

Existing vulnerability metrics separately emphasize technical remediation or community fragility, leaving individual socio-cyber susceptibility insufficiently modeled. The paper proposes and evaluates SCVI, which fuses user susceptibility with attack severity across survey and social-media data. SCVI shows stable behavior under weight variation, distinct relationships with CVSS and SVI, and stronger victim-group separation than those indices.

  • Problem

    CVSS is primarily technical and SVI is community-oriented, leaving individual cyber exposure and susceptibility insufficiently modeled.

  • Method

    SCVI fuses an Individual Vulnerability Index covering awareness, behavior, psychology, and experience with an Attack Severity Index covering frequency, consequence, and sophistication.

  • Results

    SCVI was operationalized on iPoll data from 4,596 U.S. adults and 450 Reddit r/scams reports, with stable rankings under sensitivity analysis and 10,000-iteration Monte Carlo simulations.

  • Takeaways & Limitations

    SCVI supports identification of high-risk populations and prioritization of targeted interventions through distinct socio-technical coverage and stronger victim–non-victim discrimination.

  • Takeaways & Limitations

    Self-reporting, sampling and self-selection effects, uneven state sample sizes, and uniform weighting constrain reliability and generalizability.

Abstract

from arXiv · show

The rapid growth of social media participation has increased exposure to socially engineered cyber threats (e.g., phishing, romance fraud, and tech-support scams), yet prevailing assessment tools remain fragmented: the Common Vulnerability Scoring System (CVSS) is primarily technical and largely omits human susceptibility, while the Social Vulnerability Index (SVI) is community-oriented and lacks cyber-specific modeling. To address this gap, we propose the Social Cyber Vulnerability Index (SCVI), an interpretable, uncertainty-aware metric combining two components: (i) an Individual Vulnerability Index (IVI), capturing awareness, behavior, psychological factors, and prior victimization, and (ii) an Attack Severity Index (ASI), capturing attack frequency, consequences, and sophistication. We validate SCVI across heterogeneous modalities: a nationally scoped survey (iPoll; 4,596 U.S. adults) and social-media narratives (450 Reddit r/scams reports, 2016-2024), demonstrating computation from both structured questionnaires and CI-driven feature extraction from text. Sensitivity analysis and 10,000-iteration Monte Carlo simulations show stable rankings under plausible weight variability and reveal context-dependent drivers. SCVI captures distinct socio-technical signals (Spearman correlation with CVSS $ρ= 0.33$; with SVI $ρ\approx -0.01$) and surfaces demographic and regional disparities. SCVI also provides substantially stronger separation between victim and non-victim groups than CVSS and SVI, supporting identification of high-risk populations and prioritization of interventions against emerging AI-enabled scams.

I. INTRODUCTION

Social scams exploit human trust and behavior, while CVSS and SVI omit important individual socio-cyber interactions. SCVI addresses this gap by combining susceptibility and attack characteristics across heterogeneous evidence sources.

  • Social cyber threats exploit trust and attention, causing financial, psychological, and reputational harms that generative AI may amplify.
  • CVSS emphasizes technical remediation, whereas SVI captures community-level socioeconomic fragility without modeling individual cyber exposure or susceptibility.
  • SCVI combines awareness, behavioral and psychological traits, and prior experiences with attack frequency, sophistication, and consequences.
  • SCVI is designed for scalable, interpretable vulnerability estimates using complementary survey and social-media evidence to support targeted resource allocation.
  • The manuscript extends prior work with heterogeneous-data operationalization, robustness and uncertainty analyses, benchmarking, and demographic and geographic disparity analyses.

B. Key Predictors of Vulnerability

Prior research identifies demographic, cognitive, behavioral, psychological, geographic, and temporal correlates of scam risk but rarely produces a unified individual-level measure. SCVI organizes these factors into susceptibility and attack-severity dimensions.

  • Prior studies associate scam risk with internet activity, self-control, well-being, cognitive ability, personality, and honesty-humility across demographic groups.
  • Existing predictor studies are constrained by post-incident measurement, self-reported outcomes, and limited sample diversity, and generally lack a unified comparable vulnerability score.
  • Geographic and temporal analyses reveal spatial clustering and demographic disparities but face dataset bias, proxy limitations, secondary-data dependence, and underreporting.
  • Individual Vulnerability Index (IVI): SCVI structures susceptibility around awareness and knowledge, behavioral patterns, psychological factors, and past experience.
  • The seven ABPE–FCS predictors are operationalized with survey and social-media datasets to provide a context-aware, interpretable measure.
  • Attack Severity Index (ASI): ASI represents attack frequency, consequences, and sophistication, including exposure volume, harm magnitude, realism, and personalization.

B. Social Cyber Vulnerability Index (SCVI)

SCVI is a convex combination of an Individual Vulnerability Index and an Attack Severity Index. Its components encode user susceptibility and perceived attack severity on normalized, weighted scales.

  • SCVI jointly models individual susceptibility and attack-side severity for each individual and scam type.
  • SCVI combines IVI and ASI using weights α and β constrained to [0, 1] with α + β = 1.
  • Individual Vulnerability Index (IVI): IVI aggregates awareness and knowledge, behavioral patterns, psychological factors, and past experience using weighted subcomponents.
  • Individual Vulnerability Index (IVI): Past experience includes prior scam encounters and individual response or recovery strategies.
  • Attack Severity Index (ASI): ASI aggregates attack frequency, consequences, and perceived sophistication, including mimicry, personalization, and advanced social engineering.
  • Scaling and operationalization: Components use an integer scale in [0, 5], while IVI can come from surveys or social-media traces and ASI from reported attack characteristics.

IV. SCVI ESTIMATION USING IPOLL AND REDDIT DATA

SCVI estimation uses iPoll for structured susceptibility and attack measures, while Reddit reports add contextual evidence about scam experiences. Survey responses are ordinally encoded and averaged into IVI and ASI components.

  • The two modalities provide complementary evidence: iPoll supplies structured susceptibility measures, while Reddit contributes contextual user-generated scam experiences and characteristics.
  • iPoll examines scam awareness, experiences, and behaviors among 4,596 U.S. adults surveyed in January 2020.
  • IVI estimation: IVI aggregates seven iPoll-derived subcomponents covering awareness, behavior, psychological factors, and prior scam encounters.
  • Response encoding: Categorical survey responses are mapped to ordinal scores where larger values indicate higher vulnerability, then averaged across associated items.
  • Prior experience: Prior scam encounters are represented through reported outcomes such as financial loss and emotional distress.

2) Modeling the Attack Severity Index (ASI):

ASI summarizes attack severity through frequency, consequence, and sophistication, computed by aggregating encoded survey responses and averaging the three components.

  • ASI combines frequency, consequence, and sophistication, with larger component values indicating greater attack severity.Frequency captures scam exposure prevalence, consequence captures financial and well-being impacts, and sophistication captures perceived plausibility and realism.
  • ASI is computed as the average of its frequency, consequence, and sophistication components.
  • Detailed IVI and ASI feature-extraction and response-to-score mappings are provided in supplementary Tables I and II.

B. SCVI Using the Reddit Scam Reports

The Reddit analysis derives SCVI from 450 r/scams reports using normalized text, annotations, interaction traces, and linguistic features to estimate individual vulnerability and attack severity.

  • The Reddit dataset contains 450 scam reports selected from 5,000 posts, spanning 2016–2023 with additional records from October–November 2024.The sample was balanced across years to support comparative temporal analysis.
  • Text preprocessing removes URLs, mentions, hashtags, and nonalphanumeric characters while normalizing slang, case, contractions, and elongated words.Punctuation is preserved to support consistent annotation and feature extraction.
  • Reddit IVI uses language and interaction traces to represent awareness, behavior, psychological susceptibility, and prior experience.Awareness, behavioral, and psychological factors are inferred from annotations and LIWC-based linguistic indicators, while experience reflects victimization-related annotations.
  • ASI annotations cover scam type and binary success, with frequency estimated from category prevalence, consequence from loss and distress, and sophistication from within-type success rates.Higher success rates are treated as evidence of more effective deception and greater attack sophistication.
  • The experimental setup evaluates SCVI using heterogeneous data sources, weighting strategies, simulations, and comparisons with CVSS and SVI.

A. Sensitivity Analysis of Weighting Schemes on SCVI Using iPoll and Reddit Data

The evaluation varies SCVI weights, uses Monte Carlo simulations, and compares SCVI with CVSS- and SVI-based baselines across iPoll and Reddit data.

  • SCVI sensitivity analysis varies IVI and ASI weights to measure effects on mean SCVI, variability, and component contributions.The analysis identifies the vulnerability dimensions most influential in score variation.
  • 10,000 Monte Carlo iterations perturb IVI and ASI weights within plausible simplex-constrained ranges to assess stability and influential configurations.
  • CVSS-like baselines map iPoll fraud attributes to technical vulnerability factors, while SVI baselines use iPoll demographic indicators.CVSS includes Base, Temporal, and Environmental metrics; SVI represents community-level socioeconomic and infrastructure vulnerability.
  • SCVI is compared with CVSS and SVI across groups using statistical tests, correlations, and demographic outlier analyses.The comparisons assess alignment and divergence across age, race or ethnicity, and gender.
  • In iPoll, IVI values cluster between 2 and 3, whereas ASI is concentrated near 0 with a smaller fraction near 4–5.This indicates moderate vulnerability for most users alongside concentrated high-severity incidents.
  • iPoll ASI components show bimodal frequency, moderate consequence, and broadly distributed sophistication with mass near both ends.
  • The iPoll analysis reports moderate vulnerability coexisting with heterogeneous and sometimes severe scam exposure.

2) Analysis of Reddit Dataset:

Reddit reports show low-to-moderate individual vulnerability alongside higher attack severity, while sensitivity analyses identify robust but context-dependent component influences across datasets.

  • Analysis of Reddit Dataset: Reddit behavioral vulnerability is generally low, psychological vulnerability is moderate, and awareness is widely concentrated at the lower end.Experience is concentrated at discrete values, with many users reporting no prior encounters and a smaller group showing high experience scores.
  • Analysis of Reddit Dataset: Most Reddit IVI scores fall between 1 and 2, while ASI scores mostly range from 3 to 4.The contrast indicates substantial scam severity even among users with lower individual vulnerability.
  • Analysis of Reddit Dataset: Reddit ASI frequency is polarized near 0 and 5, consequence peaks around 3, and sophistication peaks near 2 and 4.
  • Analysis of Reddit Dataset: The Reddit findings describe a mismatch in which low IVI commonly coexists with frequent exposure to impactful and sophisticated scams.
  • Sensitivity Analysis: Sensitivity results show consistent awareness, experience, consequence, and frequency effects, while psychological, behavioral, and sophistication effects vary by dataset.
  • Sensitivity Analysis: The authors characterize SCVI as robust yet context-sensitive and identify additional datasets and settings as future work for generalizability.
  • Sensitivity Analysis: Monte Carlo results form two dominant SCVI clusters: one led by Experience and Sophistication, and another by Awareness, Psychological factors, and Frequency.The primary cluster reports mean contributions of 0.421 for Experience and 0.447 for Sophistication; the secondary cluster reports means of 0.372 for Awareness and 0.436 for Frequency.

2) Analysis of the Reddit Dataset:

Reddit-based SCVI analysis shows that vulnerability drivers vary by risk level and dataset context. Experience and Frequency are prominent overall, while Behavioral factors characterize low-risk cases and Frequency dominates high-risk cases.

  • Monte Carlo analysis: SCVI outlier groups separate into low-score, stable cases and high-score cases with stronger ASI influence.Low SCVI values ranged from 1.039 to 1.098, while high SCVI values ranged from 2.182 to 2.211.
  • Comparative findings: Experience and Frequency were dominant SCVI drivers across both datasets.Their influence was especially associated with prior scam exposure and frequent threat encounters.
  • Comparative findings: Behavioral factors contributed most to low SCVI cases in Reddit, whereas Frequency contributed most to high SCVI cases.This pattern indicates different drivers of lower and elevated vulnerability within the Reddit data.
  • Comparative findings: The relative influence of SCVI components varied across datasets, supporting dataset-specific interpretation and further investigation of metric adaptability.iPoll emphasized Sophistication and Awareness, while Reddit emphasized Behavioral factors in low-score cases and Frequency in high-score cases.

D. SVI and CVSS Across Demographic Groups

SCVI diverges from SVI and CVSS while more clearly separating cyber-victim groups and revealing demographic and regional variation. Regional estimates remain less certain where samples are small, motivating cautious and tailored interpretation.

  • Metric comparison: SCVI has a moderate positive association with CVSS but a near-zero association with SVI.The reported Spearman correlations are ρ = 0.33 with CVSS and ρ = −0.01 with SVI.
  • Victim-class comparison: SCVI distributions show greater separation between victims and non-victims than CVSS and SVI.Non-victims cluster near SCVI 1, while victims show a prominent peak around 4.8.
  • Race-ethnicity groups: The Hispanic group has the highest reported racial-ethnic scores, including SCVI 4.06, while the White, non-Hispanic group has the lowest values.SCVI exceeds CVSS and SVI for the Hispanic group.
  • Age groups: Younger adults aged 18–24 and 25–29 exhibit the highest overall vulnerability, with SCVI exceeding CVSS and SVI.All three indices generally decrease with age, although a slight uptick appears among ages 45–54.
  • Regional disparities: Regional SCVI estimates are higher but less certain in several states with small samples, whereas populous states show lower and more stable scores.Alaska, Rhode Island, and Nevada have wide confidence intervals; California, Texas, and New York have larger samples and more reliable estimates.
  • Regional disparities: The analysis supports region-specific assessments and improved data collection to strengthen SCVI reliability.The paper links cautious interpretation to sparse data and recommends larger samples for regional and national strategies.

VII. DISCUSSIONS: KEY FINDINGS AND LIMITATIONS

SCVI combines individual susceptibility with attack severity to characterize socio-cyber risk across survey and social-media data. Findings show context-dependent vulnerability drivers, distinct coverage from CVSS and SVI, demographic and regional disparities, and important constraints from self-reported, cross-sectional, and unevenly sampled data.

  • Key Findings: Psychological factors and prior victimization are particularly influential IVI components, supporting resilience- and emotion-aware education alongside technical guidance.The finding is reported across iPoll and Reddit analyses.
  • Key Findings: SCVI correlates moderately with CVSS (r= 0.33) and is largely independent of SVI (r≈−0.01), enabling scam- and mechanism-specific intervention design.The comparison indicates that SCVI captures socio-technical signals distinct from the two existing indices.
  • Limitations: Self-report bias, sampling and self-selection effects, uneven state sample sizes, uniform weighting, cross-sectional data, and evolving threats constrain reliability and generalizability.The authors call for broader validation, adaptive weighting, longitudinal and cross-cultural studies, and periodic updates.
  • Key Findings: SCVI fuses IVI and ASI to bridge user susceptibility and attack severity across heterogeneous data modalities.IVI includes awareness, behavior, psychological factors, and experience; ASI includes frequency, consequence, and sophistication.
  • Key Findings: Sensitivity and weight-variability analyses show stable behavior under plausible weighting while identifying context-dependent drivers across iPoll and Reddit.Awareness consistently elevates SCVI; psychological effects are stronger in iPoll, while frequency is more influential in Reddit.

APPENDIX

The appendix documents how SCVI inputs are operationalized from iPoll survey data and organized into IVI, ASI, and state-wise summary tables.

  • IVI Feature Extraction: IVI survey responses are mapped to ordinal scores so larger values consistently represent higher individual vulnerability.The mapping covers lower familiarity or knowledge, riskier behaviors, and higher trust or impulsivity.
  • IVI Feature Extraction: “DON’T KNOW,” “SKIPPED,” and “REFUSED” responses are treated as missing and excluded from aggregation.This rule applies to the iPoll IVI feature extraction procedure.
  • IVI Feature Extraction: Table I summarizes feature extraction for IVI components from the iPoll dataset.
  • ASI Feature Extraction: Tables II and II (cont’d) summarize feature extraction for ASI components from the iPoll dataset.
  • State-Level Summary: Table III presents state-wise mean IVI, ASI, SCVI, and confidence intervals.
Loading 2608.26388v1…