PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Automated Cyber Threat Intelligence Elicitation in Underground Forums

Lorenzo Bossi, Federico Saccani, Francesco Panebianco, Antonio Maci, Stefano Zanero, Stefano Longari, Michele Carminati

arXiv:2608.23185v1cs.CR

TL;DR

Passive monitoring may miss intelligence that emerges through interaction in increasingly private underground forums. DarkBot uses specialized agents for gated, ATT&CK-guided conversational elicitation and recovered 72.8% of validated techniques in simulated discussions.

  • Problem

    Passive collection may miss threat intelligence that emerges through targeted questioning as relevant discussions move to more private or hidden forums.

  • Method

    DarkBot uses specialized agents for gated, ATT&CK-guided question generation and linguistic adaptation to actively elicit CTI through forum interaction.

  • Results

    72.8% of validated MITRE ATT&CK techniques were recovered from simulated CrimeBB discussions, while live deployment elicited CTI disclosures without observed suspensions, interventions, or accusations.

  • Takeaways & Limitations

    DarkBot demonstrates that active, specialized conversational engagement can elicit CTI from public underground forum discussions within the evaluated settings.

  • Takeaways & Limitations

    Effectiveness may vary across platforms and communities because live deployment covered specific forums, while linguistic drift and ethical constraints limited generalizability.

Abstract

from arXiv · show

Cyber threat intelligence from underground forums has traditionally relied on passive monitoring. However, as users have become more aware of large-scale data collection, valuable intelligence has become increasingly rare in open forums, often migrating instead to private or harder-to-reach spaces, making passive approaches inadequate. Building on the intuition that relevant information can be obtained through active elicitation, this paper presents DarkBot, to the best of our knowledge, the first multi-agent LLM-based system for active CTI elicitation in underground forums. DarkBot decomposes the interaction task across eleven specialized agents organized into three functional blocks: engagement gating for relevance and safety filtering, context-aware question generation driven by MITRE ATT&CK tactics, and linguistic style adaptation to better align with real forum users. In a controlled evaluation across 100 CrimeBB conversations, the system recovered 72.8% of the validated MITRE ATT&CK techniques present in the original discussions by observing only the initial post at the start of each interaction, and it consistently outperformed a monolithic baseline. The proposed layered safety design contained all injected jailbreak attempts at the pipeline level. These results were further supported by real-world experiments: in a prospective matched deployment, threads assigned to DarkBot accumulated an average of 3.85 more CTI entities than their controls over seven days, and across 104 live forum conversations, the system elicited CTI-relevant disclosures without observed account suspensions, moderator interventions, or explicit accusations of automated participation.

I. INTRODUCTION · II. BACKGROUND · III. RELATED WORK

The paper frames CTI collection as an increasingly interactive challenge because operationally relevant information is moving toward private or hidden spaces. It introduces DarkBot, a multi-agent LLM system for safe, context-aware active elicitation in underground forums, addressing a gap left by primarily offline or single-agent approaches.

  • II. BACKGROUND: CTI transforms heterogeneous, unstructured threat data into contextualized insights such as TTPs, motivations, and likely targets aligned with an organization’s risk profile.OSINT supplies information from public or semi-open sources, while CTI structures knowledge about threat actors, techniques, vulnerabilities, and malicious infrastructure for defensive decisions –.
  • I. INTRODUCTION: Open forums provide less operationally relevant information as users become more aware of large-scale data collection and shift toward private or hidden platforms.Because useful information often emerges through clarification and targeted questioning, passive monitoring is insufficient for this setting.
  • I. INTRODUCTION: DarkBot is presented as the first multi-agent LLM system for active CTI elicitation in public underground forums, coordinating eleven specialized agents for gating, ATT&CK-guided planning, question generation, adversarial-input handling, and linguistic adaptation.The system operates iteratively: it assesses whether to engage, generates context-consistent questions, analyzes replies, and decides whether to continue or terminate.
  • I. INTRODUCTION: DarkBot uses a layered safety architecture combining relevance filtering, reply analysis, jailbreak detection, and post-generation filtering to contain adversarial inputs and block most policy-violating outgoing messages.Safety is integrated with engagement gating rather than treated as a separate post hoc process.
  • I. INTRODUCTION: The system elicited CTI-relevant disclosures in 104 live conversations without observed account suspensions, moderator interventions, or explicit accusations of automated participation.This real-world evaluation complemented a prospective matched experiment in public forums.
  • III. RELATED WORK: Prior chatbot work ranges from conceptual dark-web intelligence gathering to implemented single-agent engagement with scammers that uncovers tactics, coordination, and payment infrastructure [25], [26].These lines of work establish active conversational infiltration but do not provide the multi-agent design described here.
  • III. RELATED WORK: Multi-agent LLM frameworks have decomposed security tasks such as passive forum analysis, classification, translation, and planning, often improving complex-pipeline performance,, but prior work had not investigated recurring interactive engagement with human users in underground forums.LLMs are also increasingly used in CTI pipelines for downstream analysis and earlier data-collection stages,.

IV. MOTIVATION AND PROBLEM STATEMENT · V. PROPOSED METHODOLOGY · A. Design Principles

DarkBot addresses declining CTI availability in public underground forums by shifting from passive monitoring to active, targeted elicitation through a multi-agent system. Its design combines safety gating, tactic-driven questioning, and linguistic adaptation for adversarial, implicit, and informal forum environments.

  • IV. MOTIVATION AND PROBLEM STATEMENT: Public underground forums, historically valuable CTI sources, now contain less information as relevant discussions migrate to harder-to-reach spaces.This motivates moving beyond passive monitoring toward active participation in underground discussions.
  • IV. MOTIVATION AND PROBLEM STATEMENT: The system must operate safely despite suspicious, adversarial, and technically sophisticated users, while avoiding over-refusal that would prevent discussion of malware, exploits, or illegal activity.Standard LLM safety mechanisms may miss conversational manipulation and are designed primarily for cooperative settings.
  • A. Design Principles: Because CTI is fragmented and implicit, DarkBot uses context-aware questioning to maximize relevant disclosures while minimizing detection risk.Generic conversational replies are inadequate for extracting intelligence from informal underground discussions.
  • V. PROPOSED METHODOLOGY: DarkBot shifts CTI collection toward iterative participation, deciding whether to engage, generating targeted questions, and continuing or terminating based on new replies.The final thread state is evaluated using human contributions while excluding the bot’s own messages.
  • A. Design Principles: DarkBot adopts a multi-agent architecture to decompose the complex elicitation task across specialized agents, following gains reported for complex security tasks,.This architecture is presented as a foundational design choice motivated by multiple parallel challenges.
  • A. Design Principles: An engagement-gating stage restricts interaction to relevant threads, detects jailbreak attempts or bot accusations, terminates risky conversations, and enforces ethical guidelines.The gating mechanism addresses the adversarial environment by limiting unnecessary or unsafe participation.
  • A. Design Principles: DarkBot frames questions around high-level MITRE ATT&CK tactics rather than individual techniques because tactics align more naturally with forum topics and attacker goals.The tactic-driven mechanism addresses sparse signals by matching conversational topics such as Initial Access and Privilege Escalation.
  • A. Design Principles: To appear credible, DarkBot matches each thread’s conversational tone and community-specific slang, leveraging perceived similarity that can favor information sharing,.This addresses the stylistic mismatch caused by aligned models that suppress informal or adversarial language [30]–.

B. Functional Blocks · VI. EXPERIMENTAL SETUP · A. Evaluation Framework

DarkBot organizes active CTI elicitation into three functional blocks for relevance and safety gating, MITRE ATT&CK-driven question generation, and stylistic adaptation. Its evaluation framework tests productive inquiry, safety, architectural contributions, intelligence yield, and operation in live underground forums.

  • B. Functional Blocks: DarkBot divides its agents into three functional blocks: engagement gating, context modeling and question generation, and linguistic style and credibility.The blocks define the pipeline’s goals, inputs, and outputs, with their specifications summarized in Table I.
  • B. Functional Blocks: Engagement gating filters threads and messages for CTI relevance and prohibited or high-risk content before any message is generated.THREADANALYZER produces a structured thread report, while RELEVANCEAGENT performs the subsequent gating decision using a prompt-based approach suited to hacking-related CTI discussions.
  • B. Functional Blocks: Context modeling extracts attack behaviors and MITRE tactics, after which TACTICSELECTOR ranks candidate lines of inquiry and QUESTIONAGENT generates safe follow-up questions.The tactic heuristic combines evidence strength, response probability, intelligence value, conversational flow, and applicability; the pipeline terminates when no tactic is sufficiently effective.
  • B. Functional Blocks: Linguistic adaptation uses conversation tone and stylistic patterns to produce natural single-question messages while avoiding AI-suspicious phrasing and unsafe inconsistencies.A reflection loop sends feedback to QUESTIONAGENT until a candidate satisfies stylistic and safety requirements.
  • VI. EXPERIMENTAL SETUP: The experimental setup uses open-weight models on offline hardware, assigning abliterated Gemma3-27B to uncensored attack-behavior reasoning and Qwen3-32B [52] to reasoning and classification.This setup avoids sharing potentially sensitive personal information with commercial model providers.
  • A. Evaluation Framework: The evaluation framework first tests whether DarkBot can identify productive lines of inquiry from an initial forum post and recover missing CTI knowledge in simulated real-conversation interactions.This is formalized as EQ1 and evaluates contextual understanding and coherent question formulation (§VI-B).
  • A. Evaluation Framework: The remaining evaluation questions assess safety under adversarial conditions, architectural contributions against a single-agent baseline with component ablations, intelligence yield in matched online experiments, and effective undetected operation in real forum threads.These tests cover jailbreak and policy-violation blocking, simulated extraction and stylistic credibility, participation-driven yield, and live operation without bans, moderator interventions, or AI detection (§VI-C, §VI-E, §VI-F).

B. Intelligence Probing Evaluation (EQ1) · C. Robustness Evaluation (EQ2)

DarkBot recovered 72.8% of MITRE ATT&CK techniques from complete CrimeBB discussions while seeing only initial posts, and its layered safeguards rejected all tested jailbreak prompts. Robustness evaluations further showed safe topic-focused engagement and substantial standalone detection and filtering performance under adversarial conditions.

  • B. Intelligence Probing Evaluation (EQ1): The controlled CrimeBB evaluation gave DarkBot only the initial post, while a constrained proxy returned original replies only when questions directly targeted their expressed information.The proxy could not add external knowledge, synthesize facts, or reveal unrelated conversation content.
  • B. Intelligence Probing Evaluation (EQ1): Recovery was measured under message caps, with each point reflecting replies obtained within the first x published questions and termination possible when no tactic met expected-effectiveness criteria.This setup characterized how technique recovery evolved as DarkBot published more questions.
  • B. Intelligence Probing Evaluation (EQ1): 72.8% of MITRE ATT&CK techniques were recovered from complete source conversations when DarkBot saw only each thread’s initial post.The system averaged 0.171 additional techniques per published question under the defined user-proxy policy.
  • C. Robustness Evaluation (EQ2): Among five relevant Arms, Porn, and Violence texts, FILTERAGENT blocked the Porn question, while four Arms and Violence cases elicited CTI-focused questions without addressing harmful content.The evaluation therefore showed topic-focused engagement rather than reinforcement of harmful aspects.
  • C. Robustness Evaluation (EQ2): 100% of tested jailbreak prompts received no response in three representative dark-web conversations after injection into the full pipeline.The scenarios covered malware, botnets, and initial access brokerage.
  • C. Robustness Evaluation (EQ2): The full pipeline rejected 100% of tested jailbreak prompts despite the DEFENSIVEAGENT alone misclassifying 7.2%, demonstrating layered robustness.The reported combined estimated failure rate was 1.04%, based on the stated component-failure assumptions.
  • C. Robustness Evaluation (EQ2): FILTERAGENT blocked 89.6% of harmful messages in isolation and 85.6% after slang rewriting, with most undetected questions classified as borderline content.It served as the final safeguard against policy-violating messages reaching publication.

D. Design Validation (EQ3)

Design validation shows that DarkBot’s multi-agent pipeline improves both technique recovery and probing efficiency over a single-agent baseline, while explicit tactic planning contributes to broader exploration. Component validation further shows higher extraction coverage than LADDER at comparable precision and a strong positive effect from SLANGCONVERTER.

  • Pipeline comparison: 72.8% technique recovery and 0.171 new techniques per question surpassed the single-agent baseline’s 51.4% and 0.131, respectively.The comparison used the same 100-thread CrimeBB cohort; the full pipeline also retrieved 122 unique external references, versus fewer for the baseline.
  • TTP Extraction Validation: 46.4% recall and 19% precision on 105 AnnoCTR reports exceeded LADDER’s 15.8% recall and 18.8% precision.The LLM system extracted 2,479 techniques, or 23.61 per report, compared with LADDER’s 796, or 7.58 per report.
  • TTP Extraction Validation: On 100 CrimeBB threads, the LLM extractor produced 1,996 techniques, or 19.96 per thread, versus LADDER’s 50, or 0.50 per thread.CrimeBB measures candidate extraction yield because it lacks expert technique labels, whereas AnnoCTR supports accuracy evaluation.
  • TACTICSELECTOR ablation: Ablating TACTICSELECTOR reduced final recovery by 5.4% and probing efficiency by 0.165 techniques per question.The result indicates that explicit tactic planning sustains broader exploration across interaction turns.
  • Stylistic validation: Among the evaluated stylistic components, SLANGCONVERTER had the strongest positive measured effect.The validation assessed both contextual question generation and the stylistic credibility of produced messages.

E. Intelligence Yield Effect Evaluation (EQ4) · F. Automated Forum Deployment (EQ5)

DarkBot increased CTI yield in a prospective matched deployment and, in 104 live forum conversations, elicited additional threat intelligence without observed moderation or account-enforcement incidents. Organic deployment also produced richer user activity, credible disclosures, and platform-specific interaction patterns, while some users moved sensitive details to private chat.

  • E. Intelligence Yield Effect Evaluation (EQ4): 3.85 more CTI entities were accumulated by DarkBot threads than matched controls across 20 topic pairs, with a 95% confidence interval of 0.37–7.38 entities.The paired deployment compared identical posts with and without bot activity across BHW and Dread; Figure 5 reports per-topic differences.
  • F. Automated Forum Deployment (EQ5): DarkBot elicited concrete disclosures in live interactions, including a user’s report of implementing and testing the recently published EDR-Freeze technique.Some users were reluctant to share publicly and preferred private chat, where the bot did not operate due to compliance constraints.
  • F. Automated Forum Deployment (EQ5): 9 versus 6 median user messages on Dread and 18 versus 7 on BHW were observed in bot-interacted versus passive threads, indicating substantially richer activity with active engagement.The comparison used relevant threads from the same forum sections and creation windows, but differing topics mean it describes association rather than fully isolated causality.
  • F. Automated Forum Deployment (EQ5): 103 questions on BHW received 112 qualifying replies, averaging 1.09 replies per question and 3.39 per thread, with 22 of 33 threads receiving at least one reply.On BHW, user replies averaged 221.5 characters versus 130.0 for system messages; on Dread, the corresponding averages were 141.1 and 152.4 characters.
  • F. Automated Forum Deployment (EQ5): Neutral emotion dominated both platforms, with limited negative sentiment and higher Anger and Fear proportions on Dread, particularly in discussions involving more sensitive topics.No analyzed interactions escalated into adversarial exchanges.
  • F. Automated Forum Deployment (EQ5): 2.8 h versus 7.2 h median time to first reply and 48.5 h versus 82.8 h median conversation duration showed faster interaction dynamics on BHW than Dread.Deployments lasted approximately three weeks on BHW and one month on Dread; durations measure from the bot’s first message to the last message receiving no further replies.
  • F. Automated Forum Deployment (EQ5): DarkBot messages averaged detection scores of 6.87% on BHW and 10.75% on Dread, compared with 27.58% and 13.09% for other forum users.Together with the absence of visible enforcement action, these results supported the system’s stylistic credibility, although the moderation observation was not a controlled comparison.
  • F. Automated Forum Deployment (EQ5): Across 104 live conversations and 213 published messages, DarkBot triggered no account suspensions, moderator interventions, or explicit AI accusations while eliciting CTI unavailable through passive monitoring.Disclosures included newly published attack techniques being tested in the wild and details about active database breaches.

VII. THREATS TO VALIDITY

The paper identifies internal and external validity threats involving simulated evaluation conditions, manual ground truth, benchmark coverage, participant expertise, and limited real-world deployment scope.

  • Internal Threats: The intelligence-probing evaluation uses simulated users and manually validated techniques, because extracting TTPs from unstructured forums remains difficult and the extractor produces false positives.Manual validation reduces closed-loop dependence on the extractor, but simulated behavior may not fully reflect real adversarial dynamics.
  • Internal Threats: Relevant questions may be penalized when their topics were absent from the original CrimeBB conversation, potentially underestimating elicitation capability.The robustness evaluation also relies on general-purpose jailbreak benchmarks, which may not capture design-aware, system-specific manipulations.
  • External Threats: The credibility survey uses computer-science participants who may lack familiarity with underground-forum linguistic conventions and slang.
  • External Threats: Live deployment covered specific forums, so effectiveness may vary across platforms and communities with different norms and moderation practices.

VIII. CONCLUSION · IX. ETHICAL CONSIDERATIONS

DarkBot is a multi-agent LLM system that actively elicits cyber threat intelligence from public underground forums by decomposing interaction across 11 specialized agents. The paper identifies future extensions while emphasizing ethical risks arising from automated interaction with potentially harmful communities and human-mimicking behavior.

  • VIII. CONCLUSION: DarkBot targets active CTI elicitation from public underground forums through targeted conversational engagement.It addresses a broader elicitation goal than approaches based on passive crawling or narrowly scoped single-agent designs.
  • VIII. CONCLUSION: The system decomposes interaction across 11 specialized agents for engagement gating, question generation, and linguistic style adaptation.
  • VIII. CONCLUSION: Future work will extend DarkBot to multilingual forums and broader platforms, use community feedback for adaptive elicitation, and evaluate sustained detection risk longitudinally.
  • VIII. CONCLUSION: Additional directions include targeting emerging threats such as new CVEs or threat actors, migrating conversations to private channels, and discovering hidden underground communities.
  • IX. ETHICAL CONSIDERATIONS: The study raises ethical concerns because it automates interaction with communities that may discuss or participate in harmful or illegal activities.
  • IX. ETHICAL CONSIDERATIONS: Key ethical risks include enabling criminal behavior, compromising interaction safety, and misusing automated human-mimicking conversation beyond defensive purposes.

A. Institutional Evaluation Process … F. Engagement Guidelines

The study received institutional privacy and ethics approval and adopted safeguards for consent, privacy, deception, participant safety, dual-use risks, and live engagement. DarkBot was limited to defensive research in publicly accessible forums under monitored, constrained interaction rules.

  • A. Institutional Evaluation Process: The study completed a privacy compliance review and received approval from the university’s Research Ethics Committee after detailed discussion of its methodology, ethics, and safeguards.The research team submitted required documentation and attended a dedicated Committee hearing before approval to proceed.
  • B. Human Subjects and Consent: Because individual consent was impractical and could compromise evaluation of human-like behavior, the study used publicly accessible pseudonymous forums under established observational-research guidelines.The interaction design sought to assess whether the conversational agent could operate indistinguishably from a human user.
  • C. Personal Data and Privacy: Although direct identifiers were not collected, pseudonyms and text were treated as GDPR personal data, with salted-hash pseudonymization, secure institutional storage, and restricted access.These measures addressed the possibility of reidentification through external data.
  • D. Deception and Interaction Risks: DarkBot concealed its automated nature without impersonating people or organizations, encouraging illegal activity, or pursuing sensitive topics beyond safeguarded cybersecurity questions.Interactions were automatically terminated when automation was suspected or unexpected interaction patterns emerged, and live experiments were manually monitored without requiring intervention.
  • E. Dual-Use and Misuse Concerns: To limit dual-use harms such as social engineering, surveillance, and disinformation, the work focused on defensive cybersecurity and restricted DarkBot’s codebase to approved scientific use.Access required author approval and a detailed request describing intended use and ethical justification.
  • F. Engagement Guidelines: Engagement guidelines limited the system to public forum threads, excluded private chats and access-restricted forums, and constrained automated interaction and data collection.Excluded spaces included forums requiring payment, invitation, or proof of affiliation, despite indications that private chats may yield richer CTI (§VI-F).

GENERATIVE AI USAGE … APPENDIX C STYLISTIC CREDIBILITY SURVEY

The paper documents its generative-AI use, artifact-access conditions, implementation infrastructure, and a survey evaluating stylistic credibility and tone alignment. Reproducibility is supported through available code, public models and datasets, while some data and artifacts remain restricted by ethical and data-protection constraints.

  • GENERATIVE AI USAGE: The authors used Claude to assist with writing, experimental code, and LaTeX visualization drafts, while manually reviewing text, validating code, and checking plots against underlying experiments.The supplied passage describes AI assistance as reviewed and revised by the authors.
  • APPENDIX A OPEN SCIENCE: The DarkBot codebase, including agent prompts, orchestration logic, and evaluation scripts, will be available upon request subject to intended-use and ethical review.Requests should be directed to the corresponding author.
  • APPENDIX A OPEN SCIENCE: All base models are publicly available open-weight models, and the fine-tuned SLANGCONVERTER checkpoint will be released with the codebase under the same access conditions.The passage gives Gemma3-27B and Qwen3-32B as examples and directs readers to Appendix B for full details.
  • APPENDIX A OPEN SCIENCE: CrimeBB and its derived synthetic slang-conversion dataset cannot be redistributed because of data-sharing and data-protection constraints.The public AnnoCTR, BeaverTails, CoDA, and jailbreak datasets are available through referenced sources.
  • APPENDIX C STYLISTIC CREDIBILITY SURVEY: The survey recruited uncompensated Computer Science participants through universities and social networks using Google Forms, with consent materials warning about potentially offensive or vulgar content.Participants received a task description and a visual example of the response format.
  • APPENDIX C STYLISTIC CREDIBILITY SURVEY: The survey’s Style and Credibility section compared which of two replies seemed more human-written, while Tone Alignment assessed which better followed the conversational criterion.“Equally Likely” was available as a third response option in the first section.

APPENDIX D DATASETS DETAILS · APPENDIX E LLAMAGUARD3 COMPARISON

The appendices document the datasets used to evaluate DarkBot’s relevance, safety, ATT&CK extraction, and style adaptation components, and compare its relevance filtering with LLaMA Guard 3. The comparison shows that default LLaMA Guard over-blocks hacking content, while a custom CTI policy improves hacking recall but increases false positives.

  • APPENDIX D DATASETS DETAILS: CoDA contains 10,000 dark-web documents across ten topic categories and serves as the best available proxy for evaluating CTI-relevance identification and engagement gating.Its labels represent high-level topics rather than CTI relevance, and the dataset is also used to assess robustness against prohibited topics.
  • APPENDIX D DATASETS DETAILS: CrimeBB contributes over 99 million posts from 34 multilingual forums, with evaluation using 100 relevant English threads from the September 2024 snapshot.Selected conversations contain 15–150 messages and at least one predefined relevant keyword, such as exploit, malware, DDoS, antivirus, or 0day.
  • APPENDIX D DATASETS DETAILS: AnnoCTR contains expert-annotated cyber threat reports with explicit and implicit MITRE ATT&CK tactics and techniques, including 105 reports with at least one technique annotation.It provides ground truth for evaluating the TTP EXTRACTOR.
  • APPENDIX D DATASETS DETAILS: The jailbreak evaluation contains 845 deduplicated prompts drawn from Do Anything Now and In-the-Wild Jailbreaks, while BeaverTails supplies harmful-content questions across eight safety categories.These datasets evaluate adversarial robustness and final-message filtering against harmful content bypassing earlier safety components.
  • APPENDIX D DATASETS DETAILS: The Slang Conversion Dataset pairs real CrimeBB slang messages with formal LLM-generated rewrites to fine-tune SlangConverter for producing forum-style questions.The dataset is used exclusively for SlangConverter fine-tuning.
  • APPENDIX E LLAMAGUARD3 COMPARISON: The LLaMA Guard 3 comparison evaluates an 8B single-model baseline against DarkBot’s specialized relevance and safety agents on the same CoDA subset.The comparison tests whether a general-purpose safety classifier can replace the structured exclusion rules and CTI indicators used by the agents.
  • APPENDIX E LLAMAGUARD3 COMPARISON: LLaMA Guard 3’s default policy blocks substantially more hacking documents than DarkBot, whereas a custom CTI category improves hacking recall but increases false positives.The default policy treats safe posts as relevant and unsafe posts as irrelevant; the custom policy marks posts relevant only when classified unsafe under S14: Cyber Threat Intelligence.

APPENDIX F STYLE EMBEDDINGS ANALYSIS · APPENDIX G CTI ENTITIES SUBSET

DarkBot’s style embeddings substantially overlap with real-user message clusters, while CTI-entity measurement uses STIX 2.1 object types and annotations aligned with the activity-measure population. The appendices therefore evaluate stylistic blending and define the matched-pair entity-counting procedure.

  • APPENDIX F STYLE EMBEDDINGS ANALYSIS: StyleDistance produced average cosine similarities of 0.73 on Dread and 0.68 on BHW between DarkBot and user messages.Messages from live interactions were embedded with Style-Embedding and styledistance, then compared using average cosine similarity to user messages.
  • APPENDIX F STYLE EMBEDDINGS ANALYSIS: The style analysis compared DarkBot messages with other users on Blackhatworld and Dread using two style-focused sentence transformers.The representations were Style-Embedding and styledistance [67].
  • APPENDIX F STYLE EMBEDDINGS ANALYSIS: DarkBot and user-message clusters substantially overlapped under both style representations, with no isolated cluster containing only system messages.The UMAP projections support the conclusion that DarkBot messages blend into user-message clusters and suggest indistinguishable style from regular users.
  • APPENDIX G CTI ENTITIES SUBSET: Matched-pair threat-intelligence yield was defined by annotating entity mentions according to STIX 2.1 object types rather than an evaluation-specific scheme.Table IV lists eight STIX Domain Objects and related entity types; evasive manoeuvres are annotated as Technique.
  • APPENDIX G CTI ENTITIES SUBSET: The annotation covered 174 human-authored messages from 20 matched pairs: 90 in DarkBot-assigned threads and 84 in controls.The seven-day observation window excluded initial posts, bot messages, and moderator or automated messages so the annotated population matched the activity measure.
  • APPENDIX G CTI ENTITIES SUBSET: The entity-annotation population excluded initial posts, bot messages, and moderator or automated messages, and used a pre-annotation source to reduce omissions.Only human-authored messages published in either arm during the seven-day observation window were annotated.

APPENDIX H WEEKLY TIMING PATTERNS

Appendix H presents weekly message-timing patterns on Dread and BHW during experiment VI-F, showing their message distribution by day of week.

  • APPENDIX H WEEKLY TIMING PATTERNS: During experiment VI-F, Figure 13 shows the number of messages received on each day of the week on Dread and BHW.

APPENDIX I INTERACTION EXAMPLES

The appendix presents anonymized DarkBot interactions from the Dread forum, showing the bot participating as a regular user without disclosing its automated nature. Examples cover malware propagation, EDR-evasion research, stolen personal data, CAPTCHA automation, and account-security bypassing.

  • Interaction presentation: The examples are anonymized Dread-forum exchanges in which DarkBot intervenes as a regular participant without revealing that it is automated.Initial posts appear above the exchanges; relevant information extracted by the bot is highlighted, and harmful language is censored.
  • Malware and automation: DarkBot engages with requests about Linux spreading methods and detecting Bash or curl commands for social-media automation.One exchange mentions binding malware with a crack, while another proposes server-based detection and outreach to inexperienced users.
  • EDR evasion: Another interaction concerns source-code analysis and recent methods for evading endpoint-detection-and-response behavioral analysis.The post requests malware source from repositories such as VX-Underground or MalwareBazaar, or alternative recent evasion methods.
  • Data and account abuse: Additional examples address selling approximately 29,000–30,000 US fullz, CAPTCHA-solving requirements for automation, and bypassing Microsoft-account security.The stolen-data post describes about 8,000 encrypted credit-card records, fresh private data, document files, and a price of $5 per unit.
Loading 2608.23185v1…