PaperlayerView exact revision on arXiv

Source-linked AI summary

TagZilla: Automated Owner and Abuse Type Tagging for Indicators of Compromise in Threat Reports

Gibran Gomez, Juan Caballero

arXiv:2608.28124v1cs.CR

TL;DR

CTI reports often list IoCs with little context, making it difficult to determine their owners and abuse types. TagZilla extracts and filters indicators, then applies open-world owner tagging and closed-world abuse tagging. It achieves F1 scores of 0.94 for owner tagging and 0.93 for abuse type tagging, and tags 15,583 IoCs across 765 reports.

  • Problem

    CTI reports commonly describe or list IoCs with little context about their owners and associated abuse types.

  • Method

    TagZilla extracts and filters candidate indicators, uses open-world LLM classification for owners, and uses closed-world classification for 29 abuse types.

  • Results

    0.94 F1 for owner tagging and 0.93 F1 for abuse type tagging were achieved, while deployment across 765 reports identified 15,583 IoCs.

  • Takeaways & Limitations

    TagZilla supports per-IoC threat intelligence and enables IoC profiles for threat groups and malware families across reports.

  • Takeaways & Limitations

    The entity database cannot be complete because new malware families and threat groups continue to appear.

Abstract

from arXiv · show

Cyber Threat Intelligence (CTI) reports often describe Indicators of Compromise (IoCs) such as IP addresses, URLs, file hashes, and cryptocurrency wallets involved in cyberattacks. Those IoCs are typically described in the unstructured report's text, or listed at the end of the report with little context, limiting their usefulness. This paper presents TagZilla, a platform that, given a threat report, automatically analyzes its text and tags the IoCs it describes with contextual information about the threat group and malware family that the IoC belongs to and the type of abuse associated with the IoC (e.g., phishing, sextortion, command-and-control). TagZilla provides a novel LLM-based approach to assign owner tags to IoCs using an open-world classification, and assigns 29 abuse type tags to IoCs using a closed-world classification. We evaluate TagZilla on a manually generated ground truth of 100 threat reports containing 1,534 indicators, where it achieves an F1 score of 0.94 for owner tagging and 0.93 for abuse type tagging. Then, we apply TagZilla to tag 765 threat reports, identifying 15,583 IoCs belonging to 637 malware families, 113 threat groups, and 162 other entities. The results show that TagZilla can tag IoCs even in reports describing multiple actors and malware families, enabling the generation of IoC profiles for those entities.

1 Introduction

TagZilla addresses the limited context and per-report ambiguity of IoCs by automatically assigning per-IoC owner and abuse-type tags. Its evaluation reports high tagging accuracy and broad deployment across threat reports.

  • CTI reports often contain IoCs with little contextual information about their owners or abuse types, limiting their actionability.
  • TagZilla uses open-world owner classification and closed-world classification for 29 abuse types.
  • 1,534 indicators from 100 manually annotated threat reports form the ground truth for evaluation.
  • 0.94 F1 for owner tagging and 0.93 F1 for abuse type tagging were achieved using gpt-oss-20b.

2 Problem Definition

The paper defines IoC ownership at the entity level that controls malicious use and motivates per-IoC tagging because reports can contain multiple entities and benign indicators. TagZilla therefore distinguishes owners, abuse types, and non-IoCs for individual indicators.

  • The goal is to convert an unstructured threat report into structured IoCs annotated with owner and abuse type tags.
  • An indicator is an indicator-type and indicator-value pair that may be benign or malicious across networking, contact, artifact-hash, and blockchain-address categories.
  • 2.1 Ownership: Ownership is assigned to the entity at the end of the leasing chain that controls how an indicator is abused.
  • 2.1 Ownership: Scams and extortions receive abuse-type tags but generally no owner tag because they are rarely associated with a threat group or malware family.
  • 2.3 Per-IoC Granularity: Reports may mention multiple malware families or threat groups, so report-level tags cannot be safely assigned to every indicator.
  • 2.3 Per-IoC Granularity: The example assigns different malware-family and abuse tags to two IoCs linked to the same threat group while marking Hunt.io as benign.

3 Approach Overview

TagZilla combines indicator extraction, benign-indicator filtering, abuse tagging, and owner tagging to annotate IoCs from heterogeneous threat-report documents. Its classifiers refine candidates and normalize entity names while using different label spaces for abuse and ownership.

  • TagZilla has four modules: Indicator Extraction, Generics Filtering, Abuse Tagging, and Owner Tagging.
  • Indicator Extraction: Indicator Extraction obtains report text from HTML, PDF, or Word files and identifies indicators with their positions and values.
  • Generics Filtering: Generics Filtering removes a subset of benign indicators without text analysis, reducing unnecessary LLM queries.
  • Abuse Tagging: Abuse Tagging classifies candidates as false positives, benign indicators, or IoCs before selecting abuse tags from predefined taxonomies.
  • Owner Tagging: Owner Tagging proposes unrestricted owner names and then normalizes aliases by searching for similar entities in an entity database.

4 Datasets & Tags

TagZilla’s datasets and tag design address aliasing, incomplete taxonomies, benign indicators, false positives, and reports containing multiple owners. The manually constructed ground truth labels 1,534 indicators across 100 reports for evaluation.

  • Entity Database: An entity database stores canonical names, aliases, entity types, descriptions, and related metadata for malware families and threat groups.Canonical names serve as unique entity tags, while aliases support normalization across reports.
  • Entity Database: The entity database is necessarily incomplete because new malware families and threat groups continue to appear.Previously unknown entities are added to the database and can be contributed back to MISP clusters.
  • Ground Truth: The ground truth contains 100 threat reports and 1,534 candidate indicators, reviewed by two analysts who labeled false positives, benign indicators, IoCs, and their tags.The reports came from security vendors, independent researchers, cybersecurity news outlets, and four blockchain-related blogs containing only benign indicators.
  • Tag Taxonomy: The abuse taxonomy accommodates uncovered abuses by assigning a parent tag when no existing subtype captures a reported abuse.This design reflects the incompleteness of abuse taxonomies as cybercriminals introduce new abuse types.
  • Tag Taxonomy: During ground-truth construction, analysts added 34 owners and 11 abuse types, including phishing, web_skimming, and command-and-control.They also generalized one abuse type and removed two tags judged to describe techniques rather than abuse types.
  • Ground Truth: 1,310 indicators (85.4%) were IoCs, 192 (12.5%) were benign, and 32 (2.1%) were iocsearcher false positives.False positives commonly included filenames or filepaths misclassified as domains or URLs.
  • Ground Truth: 18 of the 100 reports contain IoCs from multiple owners, supporting fine-grained per-IoC processing rather than assigning one owner to an entire report.Different IoCs within one report may belong to different entities.

5 TagZilla

TagZilla combines generic-indicator filtering with LLM-based abuse and owner tagging. Its abuse classifier uses hierarchical, indicator-type-specific taxonomies, while owner tagging proposes entities openly and then normalizes them against an entity database.

  • 5.1 Generics Filtering: The generics filtering module uses blocklists to tag selected benign indicators as notabuse before LLM processing.It improves efficiency but does not use report text and therefore misses some benign indicators.
  • 5.1 Generics Filtering: Generic detection applies indicator-specific rules to apex domains, FQDNs, URLs, IPv4 addresses, emails, hashes, and blockchain addresses.Hashes and blockchain addresses are always treated as non-generic by this module.
  • 5.2 Abuse Tagging: The abuse tagging module filters extracted indicators and assigns abuse types while identifying false indicators, benign indicators, and refined hash types.Its LLM classifier supports contact, hash, networking, payment, and blockchain indicators.
  • 5.2 Abuse Tagging: The abuse classifier traverses a hierarchical taxonomy from top to bottom, querying the LLM with report text, IoC details, candidate tags, and tag definitions at each level.The output includes both the assigned label and the model’s reasoning.
  • 5.2 Abuse Tagging: Indicator-type-specific abuse taxonomies remove inapplicable tags, reducing the LLM’s reasoning options and query size.Separate subsets are prepared for networking, hashes, contact, blockchain, and payment indicators.
  • 5.3 Owner Tagging: Owner tagging proposes owner names without restricting the possible labels, then matches each proposal to canonical names or aliases in the entity database.Known entities reuse canonical tags; unmatched entities receive new tags.
  • 5.3 Owner Tagging: Owner refinement separates proposals into threat groups, malware families, and other owners before searching canonical names and aliases.The refinement prompt includes the IoC, proposed label, owner-identification reasoning, and candidate entities.

6 Evaluation

TagZilla achieves high accuracy on ground-truth owner and abuse-type tagging, while performance varies by model and classification granularity. Large-scale tagging shows broad coverage of IoCs, abuse types, and entities, alongside identifiable error and scope patterns.

  • Accuracy Evaluation: 0.93 F1 is achieved for abuse type tagging with gpt-oss-20b, compared with 0.76 using gemma4-e4b.With gpt-oss-20b, precision is 0.94 and recall is 0.93; gemma4-e4b reaches precision 0.87 and recall 0.72.
  • Accuracy Evaluation: F1 declines from 0.99 at taxonomy level L1 to 0.93 across all four abuse-taxonomy levels because upper-level errors accumulate.The intermediate F1 scores are 0.95 at L2 and 0.95 at L3, and errors at upper levels cannot be corrected.
  • Accuracy Evaluation: 0.94 F1 is achieved for owner tagging with gpt-oss-20b on the manually annotated ground truth.Owner-tagging precision is 0.95 and recall is 0.94; owner tagging is evaluated only when the abuse type is correct and predicted and ground-truth owners intersect.
  • Report Tagging: Across 765 reports, TagZilla identifies 15,583 IoCs, 4,117 benign indicators, and 563 invalid indicators from 20,239 candidates.IoCs occur in 532 reports, while 708 reports contain at least one candidate indicator identified by iocsearcher.
  • Report Tagging: The most frequent abuse types are command_and_control at 26.9%, malware_distribution at 22.0%, and phishing at 12.6%.These three categories are assigned to networking IoCs; downloader, infostealer, and rat are also common malware classes.
  • Report Tagging: Owner tags are assigned to 10,961 IoCs spanning 912 entities, including 113 threat groups and 637 malware families.Of the entities, 439 were in the original entity database and 473 were new entities.

7 Discussion

The discussion identifies scope and design limitations involving indicator extraction, abuse-type coverage, model choice, image-only content, and human oversight. It also outlines corresponding directions for improvement.

  • Indicator extraction: Missed indicators by iocsearcher introduce false negatives, including URLs split across consecutive lines.Direct LLM extraction was preliminarily tested but had lower recall because the LLM was not thorough.
  • Multiple abuse types: TagZilla currently assigns only one abuse type per IoC, although an IoC may support multiple abuse types.Accumulating tags across reports is possible, but extracting multiple tags within one report may require changing the hierarchical taxonomy.
  • Open models: Commercial frontier models may improve accuracy but would make evaluation less replicable and more costly.The evaluation used a variety of open models run locally, and TagZilla is designed to be LLM-independent.
  • Images: IoCs and their context that appear only in report images are currently outside TagZilla’s text-only analysis.The authors plan to explore multimodal LLMs provided with complete reports containing text and images.
  • Human in the loop: Human supervision may help review newly proposed owner tags and inspect TagZilla’s operation.The authors specifically mention filtering overly broad tags and supervising later processing steps.

8 Related Work

Prior CTI work progressed from rule-based indicator extraction toward contextual, semantic, and LLM-based analysis. TagZilla differs by jointly classifying each IoC’s abuse type and controlling entity, including in multi-actor reports.

  • Indicator extraction: Earlier systems extracted structured indicators from unstructured text using regular expressions, while iocsearcher identifies 41 IoC types.GoodFATR evaluates indicator extraction tools without requiring a ground-truth dataset.
  • Contextual extraction: Later approaches extracted threat actions, attack behaviors, techniques, and ATT&CK tactics or techniques from report context.Other work transformed extracted information into knowledge graphs, temporal attack graphs, or STIX objects.
  • LLM-based methods: LLM-based methods have been applied to indicator extraction and classification using different prompting techniques.Prior work also used LLMs to identify abuse-report context and separate malicious indicators from benign ones.
  • Research gap: Existing approaches generally do not jointly determine fine-grained IoC abuse type and controlling entity, often assuming one actor or campaign per report.TagZilla instead performs per-IoC abuse classification and ownership attribution in reports involving multiple threat actors.

9 Conclusion

The conclusion presents TagZilla as an automated platform for enriching IoCs with owner and abuse-type tags. Its evaluation and large-scale deployment support generating entity-specific IoC profiles for several threat-intelligence uses.

  • Conclusion: TagZilla automatically annotates IoCs with owner entities and abuse types, answering who controls an IoC and how it is used.This per-IoC focus distinguishes the platform from work centered primarily on attack techniques and named entities.
  • Conclusion: F1 scores of 0.94 for owner tagging and 0.93 for abuse type tagging demonstrate accurate tag generation.The reported evaluation covers both tagging tasks.
  • Conclusion: TagZilla processed 765 threat reports, identifying 15,583 IoCs across 637 malware families, 113 threat groups, and 162 other entities.These results support generating IoC profiles for threat attribution, threat hunting, infrastructure tracking, and automated campaign correlation.

A Abuse Type Taxonomies

The paper directs readers to the abuse-type taxonomies used by TagZilla, with separate figures presenting taxonomy views for hashes and contacts.

  • Abuse Type Taxonomies: Figures 5, 6, 7, and 8 detail the taxonomies used by TagZilla.The cited figure references cover the platform’s taxonomy definitions.
  • Abuse Type Taxonomies: Figure 6 presents the hashes taxonomy.
  • Abuse Type Taxonomies: Figure 7 presents the contact taxonomy.

B Category Evaluation

This section presents figures describing TagZilla’s abuse-type confusion matrix and blockchain taxonomy.

  • Figure 9 presents the confusion matrix for abuse-type classification on the ground truth using TagZilla.
  • Figure 8 presents a blockchain taxonomy.
Loading 2608.28124v1…