PaperlayerView exact revision on arXiv

Source-linked AI summary

Social-HUMINT: An LLM-Augmented Unified Framework for Measuring Social Media Exposure

Pellegrino Casoria, Simon Pietro Romano

arXiv:2610.31573v1cs.CRcs.AIcs.SI

TL;DR

Social-media profiling can reveal undisclosed attributes, but existing models do not span the path from passive observation to active engagement. The paper unifies SOCMINT and Cyber-HUMINT in a 38-capability taxonomy and operationalizes its core with an LLM-augmented pipeline. On real users, profiles were accurate and anticipated personalized engagement preferences above chance.

  • Problem

    Existing models do not formalize the observation-to-engagement continuum or measure how profiles may support social-engineering susceptibility.

    Source: §1 · p. 1, §2.3 · p. 4

  • Method

    The authors unify SOCMINT and Cyber-HUMINT in an expert-validated 38-capability taxonomy and evaluate an LLM-augmented pipeline on participants’ public data.

    Source: §2.3 · p. 4, §8 · p. 15

  • Results

    Extraction reached F1 = 97.3%, inferred attributes scored 4.73/5, and profiles anticipated targets’ engagement preferences above chance.

    Source: §8 · p. 15

  • Takeaways & Limitations

    The framework and pipeline provide a way for individuals and organizations to assess exposure before an adversary does.

    Source: §8 · p. 15

    Active engagement was assessed by proxy rather than executed directly.

    Source: §7.2 · p. 15

Abstract

from arXiv · show

In the post-API landscape, social-media profiling has become inexpensive and automated: Large Language Models (LLMs) can extract structured information from unstructured content and deduce attributes users never disclosed. A person's true exposure is not what they post but what can be inferred from it. As these practices become routine, exposure to social engineering attacks becomes an operational certainty, demanding a shared framework for assessment and mitigation. We make this exposure measurable by quantifying how accurately a target can be profiled, and how far that profile enables social engineering attacks. Because no existing model fully spans this continuum, we unify Social Media Intelligence (SOCMINT) and Cyber Human Intelligence (Cyber-HUMINT), two individual-focused disciplines that evolved separately, into a single doctrine-grounded taxonomy of 38 capabilities across the Cyber Threat Intelligence lifecycle, where user profiling bridges passive collection and active engagement. Domain experts validated the taxonomy (Fleiss' $\kappa$ = 0.92). We operationalize its core path (extraction, inference, and susceptibility assessment) as an LLM-augmented multi-agent pipeline, evaluated on real participants' public data. Susceptibility is assessed indirectly: the pipeline predicts how each target would rank personalized engagement options, matching their actual choices significantly above chance. This ranking capacity captures the personalization on which tailored attacks depend. The underlying profiles are accurate: extraction reaches F1 = 97.3%, and inferred attributes score 4.73/5, both far above rule-based and NER-based baselines at negligible cost. The framework augments security analysts' tradecraft rather than replacing it and enables defenders to gauge their own exposure before an adversary does.

1. Introduction

Public social-media traces can support reconstruction of undisclosed attributes, yet existing models do not formalize the operational path from passive observation to active engagement. This work unifies that continuum and measures both profile accuracy and its relevance to social-engineering susceptibility.

1. Introduction

  • Existing models cover only parts of the passive-to-active path, leaving operational capabilities and the exposure they enable unformalized.The paper identifies this gap as including how far capabilities enable attacks and how much of the path LLMs can automate.Source: §1 · pp. 1–2

1.1. Major contributions of this work

  • The framework unifies 38 capabilities across five CTI phases, with user profiling bridging passive collection and active engagement.Experts validated the taxonomy with Fleiss’ κ = 0.92; no surveyed source spans more than about 68% of its capabilities.Source: §1.1 · p. 2
  • The LLM-augmented multi-agent system operationalizes cross-platform extraction, latent-attribute inference, and susceptibility assessment as analyst support.It produces traceable profiles and is designed to augment, not replace, established tooling.Source: §1.1 · p. 2
  • The study evaluates extraction and inference accuracy on participants’ data and whether profiles anticipate personalized engagement choices.It compares extraction with rule- and NER-based baselines.Source: §1.1 · p. 2

2.1. Social-Media Profiling and Intelligence

  • Social-media profiling spans disconnected research strands, while prior systems often leave profiles as opaque intermediates or infer isolated traits.Prior work includes CTI extraction and threat classification, attribute inference, and profiling or extraction tools.Source: §2.1 · pp. 2–3

2.2. Adversarial Use in Practice

  • Documented campaigns use social profiling and human engagement, while the kill chain compresses target profiling into one upstream step.The campaigns include fabricated recruiter and researcher personas, employee profiling, and voice phishing.Source: §2.2 · p. 3

2.3. SOCMINT and Cyber-HUMINT definitions

  • SOCMINT observes social-media data, Cyber-HUMINT engages people, and user profiling turns passive data into structured profiles that can support engagement.Prior work models this continuum conceptually but lacks a formal capability model.Source: §2.3 · p. 4

2.4. Tools for Social Media Analysis; 2.5. Security Constraints of Social Platforms

  • Existing tools leave analysts to fuse fragmented data, infer unstated attributes, and assess relevance; platform access constraints also shape collection.The study therefore collects only participant-authorized publicly visible data, while platform defenses and API restrictions bound access.Source: §2.4 · p. 4, §2.5 · pp. 4–5

3. Methodology

The study moves from doctrinal foundations and taxonomy derivation to expert validation, system construction, and human-subjects evaluation. Its empirical validation uses participants’ public Facebook and LinkedIn data and compares pipeline outputs with baselines and self-reports.

3. Methodology

  • The six-phase design spans domain foundations, framework derivation and validation, system architecture, empirical validation, and discussion.The phases are grounded in established methodologies.Source: §3 · pp. 5–6
  • The taxonomy derives 38 capabilities from eight doctrinal and operational sources through thematic analysis, grouping concepts into five CTI lifecycle phases.Each capability retains traceability to source passages.Source: §3 · p. 6
  • Thirteen cybersecurity professionals assessed capability relevance, CTI-phase assignment, framework structure, and missing or superfluous capabilities.The panel covered CTI, cyber defense, and strategy and risk functions.Source: §3 · p. 6
  • The system instantiates cross-platform extraction, latent-attribute inference, and susceptibility assessment in a multi-agent pipeline that distinguishes extracted facts from inferred attributes.The selected capabilities are those the authors considered amenable to LLM automation.Source: §3 · p. 6
  • Fourteen cybersecurity professionals with active public Facebook and LinkedIn profiles took part in an observational study using their public data.Participants reviewed generated reports; extraction was compared with rule- and NER-based baselines, inference with self-report, and susceptibility with Kendall’s τ.Source: §3 · p. 6

4. The Social-HUMINT Framework

Social-HUMINT organizes intelligence capabilities across a continuum from passive observation to active engagement, with user profiling as the bridge. Its five-phase framework provides the operational foundation for the paper’s LLM-augmented architecture.

4. The Social-HUMINT Framework

  • The taxonomy organizes 38 capabilities across five CTI phases and distinguishes SOCMINT, user profiling, and Cyber-HUMINT.The process is cyclical: disseminated intelligence informs subsequent planning.Source: §4 · p. 6
  • Three interaction levels differ by reachable sources and degree of target engagement, with higher intensity bringing greater traceability and operational exposure.Low is public passive monitoring; medium includes access-controlled sources and low-engagement actions; high involves active engagement and is exclusive to Cyber-HUMINT.Source: §4 · p. 6

4.1. Planning and Direction

  • Planning sets objectives, risk and legal boundaries, intelligence requirements, source strategy, and operational personas before downstream activity.Tool setup and engagement planning configure infrastructure and target-interaction workflows from structured profiles.Source: §4.1 · pp. 6–7

4.2. Collection

  • Collection spans passive observation, automated acquisition, direct engagement, multimedia, crowdsourced signals, and historical records.Active engagement can validate or refine inferred profile attributes.Source: §4.2 · p. 7

4.3. Processing

  • Processing normalizes and filters heterogeneous data, extracts entities and metadata, resolves identities, and maintains source attribution.Enrichment can add external sources to fill gaps not addressable through social-platform collection alone.Source: §4.3 · pp. 7–8

4.4. Analysis & Correlation

  • Analysis combines behavioral, media, network, temporal, intent, and threat analysis with attribute inference and profile construction.Profiles preserve the distinction between extracted entities and inferred attributes with associated confidence levels.Source: §4.4 · p. 8

4.5. Dissemination

  • Dissemination delivers intelligence to authorized stakeholders through sharing, reporting, recommendations, visualizations, liaison, and actionable guidance.Recommendations and lessons learned feed findings into subsequent planning and collection cycles.Source: §4.5 · p. 8

4.6. LLM Augmentation across CTI Phases

  • LLM augmentation is mapped across CTI phases, while the implemented core focuses on extraction, inference, profile construction, behavioral analysis, and personalized susceptibility assessment.The pipeline scrapes Facebook and LinkedIn, fuses their outputs using source-specific conflict-resolution rules, and produces a structured intelligence dossier.Source: §4.6 · pp. 8–9

5. Case Study: LLM-Augmented Social-HUMINT

The case study operationalizes profiling across collection, processing, and analysis, using a multi-agent pipeline to turn public social-media data into structured profiles and assess their use in personalized engagement.

5.1. System Architecture

  • The pipeline spans passive collection, profile construction, and Cyber-HUMINT engagement, with profiling bridging the observation-driven and interaction-driven stages.It collects from LinkedIn and Facebook, infers latent attributes, and produces targeted engagement artifacts and reporting outputs.Source: §5.1 · p. 9, §5 · p. 9

5.1.1. Scraping Strategy

  • Scrapers retrieve raw pages with minimal semantic parsing, while downstream agents handle interpretation; the target platforms are LinkedIn and Facebook.The study used three scroll iterations and captured up to ten recent posts per platform, with depth and post volume configurable.Source: §5.1.1 · pp. 9–10

5.1.2. Agentic Reasoning

  • Specialized agents extract and infer attributes, fuse platform outputs into a unified profile, and generate engagement topics; the Engagement Agent uses behavioral scenarios rather than direct engagement.Outputs use JSON schemas with confidence scores and reasoning chains to support auditability and human-in-the-loop validation.Source: §5.1.2 · p. 10

5.1.3. User Profile Data Model

  • The two-tier data model separates scraped, directly observable fields from LLM-derived attributes, each inference carrying a confidence score and reasoning explanation.Extracted fields are grouped into seven categories; inferred intelligence covers behavioral profiling, interests, target assessment, and engagement support.Source: §5.1.3 · p. 10

5.1.4. Evaluation Instruments

  • Participants self-report extraction correctness, rate inferred attributes, and rank profile-derived engagement options across career, communication, and interests.Extraction is evaluated with precision, recall, and F1 because true negatives are undefined; participant recall is treated as a lower bound.Source: §5.1.4 · pp. 10–11
  • The perception survey combines nine Likert items and three open-ended questions covering context, accuracy, operational value, and feedback.Its measures include perceived profile authenticity, usefulness, social-engineering potential, and professional adoption.Source: §5.1.4 · p. 11

6. Empirical Validation

The study validates both the capability framework and its operational pipeline, measuring profile quality and whether profiles anticipate participants’ choices among personalized engagement options. Experts rated the framework positively, while the pipeline’s extraction, inference, and ranking assessments showed strong performance in this participant sample.

6.1. Framework Validation

  • Experts rated all 38 capabilities relevant overall (mean 4.42/5); 31 exceeded the I-CVI content-validity threshold, and phase-classification agreement was Fleiss’ κ = 0.92.All 13 experts considered the framework suitable as a reference (mean 4.5); all seven capabilities below the I-CVI threshold were retained for completeness.Source: §6.1 · p. 11

6.2. Operational Validation

  • The operational validation evaluates extraction and inference against participants’ own data and tests whether resulting profiles can support credible social engineering.The framework assessment separately examines coherence and grounding in adversary behavior.Source: §6 · p. 11, §6.2 · p. 12

6.2.1. Pipeline Execution

  • Pipeline execution averaged 508 seconds per participant, with API cost of $0.14–$0.18 per profile; all runs completed successfully after retrying six JSON parsing errors.The study involved 14 participants and reported $2.23 total API cost.Source: §6.2.1 · p. 12

6.2.2. Extracted Data

  • F1 = 97.3% (precision = 97.9%, recall = 96.6%) for extracted data, with recall more sensitive than precision to profile complexity.The lowest participant F1 was 84.1% for the most data-rich profile; Certifications (F1 = 93.8%) and Places Visited (F1 = 93.2%) were the weaker field categories.Source: §6.2.2 · p. 12, Table 3 · p. 12

6.2.3. Inferred Intelligence

  • Inferred attributes received a mean participant rating of 4.73/5, with 79.1% of ratings at the maximum and a median of 5 in every category.Two participants accounted for 11 of the 13 ratings below 4, and model confidence was uniformly below participant ratings (overall mean difference −0.68).Source: §6.2.3 · pp. 12–13, Table 4 · p. 13

6.2.4. Social Engineering Engagement Assessment

  • Mean Kendall’s τ was 0.36, and agreement was significantly above chance (W = 105, p < 0.001, n = 14), with all 14 per-subject means positive.Top-1 accuracy was about 52%, double the random baseline; Communication ranked highest (τ = 0.54), ahead of Career (0.30) and Interests (0.23).Source: §6.2.4 · p. 13, Figure 6 · p. 13, Figure 7 · p. 14

6.2.5. Perception Survey

  • Participants rated extraction and inference accuracy at 4.4/5, and found engagement questions relevant to their profiles (4.1/5).They rated perceived social-engagement potential 4.3/5, while professional adoption received 3.4/5 with substantial disagreement.Source: §6.2.5 · p. 13, Figure 8 · p. 14
  • Open-ended feedback identified overfitting, outdated content, and incorrect collected data as weaknesses, each reported by 29% of participants.Participants most often requested additional data sources (64%), deeper reasoning (36%), and broader platform integration (21%).Source: §6.2.5 · pp. 13–14

7. Discussion

The discussion contrasts the pipeline with conventional extraction baselines, frames the framework’s dual-use implications, and identifies limits in study scope, validation, and operational testing.

7. Discussion

  • The same capabilities can support attackers’ targeting or defenders’ exposure assessment, early warning, counter-influence work, and adversary profiling.The framework is dual-use; the capabilities differ by intent and authorization.Source: §7 · p. 14
  • At about $0.16 per target without human intervention, profiling lowers barriers for both defensive self-assessment and low-resource attackers.The passage states that profiling at this cost makes exposure an operational certainty.Source: §7 · p. 14

7.1. Baseline comparison

  • 97.3% F1 for the pipeline exceeded regex at 28.0% and spaCy NER at 17.3% on the same raw scrapes; neither baseline produced inferred attributes.The baselines were scored against 736 validated ground-truth items, and matching was deliberately permissive, making their scores an upper bound.Source: §7.1 · p. 14
  • The LLM gap reflects semantic parsing of typed attributes, directly usable output, and inference of latent attributes across profile fields.Baseline outputs require extensive manual filtering, while neither baseline infers attributes across fields.Source: §7.1 · p. 14

7.2. Limitations and Future Directions

  • The 14-person feasibility pilot recruited cybersecurity professionals through a professional network, and its coverage is limited to Facebook/LinkedIn and one snapshot.The authors state that a larger, more diverse cohort would improve generalizability.Source: §7.2 · p. 14
  • Extending coverage to other platforms and longitudinal capture could broaden completeness and support deeper behavioral tracking.The paper names Instagram, GitHub, and breach databases as possible additional OSINT sources.Source: §7.2 · p. 15
  • Inferred-attribute agreement relies on participant self-report, residual hallucinations remain, and results depend on specific model versions.The paper reports 15 extraction false positives and says 29% of participants reported incorrect data; it suggests benchmarking across LLMs to separate framework effects from model-specific effects.Source: §7.2 · p. 15
  • Active engagement was assessed by proxy rather than executed, and the architecture exercises only a subset of the taxonomy’s 38 capabilities.The authors propose controlled engagement under ethics-board oversight and further exploration across all five CTI phases.Source: §7.2 · p. 15

8. Conclusions

The conclusion presents a unified, expert-validated framework and an LLM-augmented pipeline for measuring exposure from public social-media data. Profiles achieved high extraction and inference accuracy and anticipated personalized engagement preferences above chance.

8. Conclusions

  • The framework unifies SOCMINT and Cyber-HUMINT, operationalizing their core path as an LLM-augmented pipeline under human oversight.The conclusion emphasizes that exposure includes what a machine can infer, not only what a person posts.Source: §8 · p. 15
  • 97.3% F1 was achieved for extraction, far above rule- and NER-based methods.The conclusion also reports that inferred attack-relevant attributes not disclosed by targets were confirmed as accurate at 4.73/5.Source: §8 · p. 15
  • Profiles anticipated targets’ engagement preferences above chance, and targets judged them sufficient for a credible social-engineering approach.The conclusion characterizes the profiles as operationally relevant beyond descriptive profiling.Source: §8 · p. 15
  • The framework is presented as a reusable organizational self-assessment capability for gauging exposure and taking countermeasures before adversary exploitation.The authors expect mapping augmentation opportunities across all five CTI phases to support systematic study across the observation-to-engagement continuum.Source: §8 · p. 15

Ethical Considerations

The study reports ethics approval, informed consent, and safeguards for collecting and processing participants’ public social-media content.

Ethical Considerations

  • The University of Naples Federico II ethics committee approved the study, and all participants gave written informed consent.Participation was voluntary, uncompensated, and withdrawable without consequence.Source: Ethical Considerations · p. 15
  • The study used only content participants made publicly visible through authenticated sessions, with no deception, manipulation, or covert observation.Consent covered study procedures, data retention, pseudonymization, and GDPR rights.Source: Ethical Considerations · p. 15
  • Processing was GDPR-compliant, data was pseudonymized or anonymized, and raw captures and profiles were encrypted and access-controlled.Raw data was retained for twelve months for reproducibility.Source: Ethical Considerations · p. 15

Funding

The authors report that the research received no specific grant funding.

Funding

  • The research received no specific grant from public, commercial, or not-for-profit funding agencies.Source: Funding · p. 15

Data availability

The participant data contain sensitive, personally identifiable information, so raw data are not publicly available. De-identified aggregates and research materials may be requested, but full replication requires access and credentials that cannot be redistributed.

  • Raw participant data are not publicly available because they contain sensitive, personally identifiable information.Source: Data availability · p. 15
  • Aggregated, de-identified data can be requested from the corresponding author upon reasonable request.Source: Data availability · p. 15
  • The pipeline configuration and scripts can be shared for research, but full replication requires authenticated platform access and LLM API credentials that cannot be redistributed.Prompt details are provided as Supplementary Material.Source: Data availability · pp. 15–16

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

The authors used Claude for manuscript editing, revision, clarity, and coding support, and OpenAI models in the experimental methodology. They reviewed and edited the resulting content and code and take responsibility for the article.

CRediT authorship contribution statement

Pellegrino Casoria is credited across the study's conception, execution, analysis, and writing. Simon Pietro Romano is credited with project administration, supervision, and review and editing.

Loading 2610.31573v1…