PaperlayerView exact revision on arXiv

Source-linked AI summary

FISSION: Label Augmentation for Bot Detection

Sen Yang, Ignacy Nieweglowski, Aviv Yaish

arXiv:2609.26279v1cs.AI

TL;DR

Reliable labels for abusive accounts are scarce and can derive from collection heuristics. FISSION uses disjoint temporal views of an account’s history to learn representations before task labels are available, then fits a classifier with a small labeled sample. It improves classification across markedly different domains and retains most full-label accuracy when labels are scarce.

  • Problem

    Reliable labels for abusive accounts are scarce and may reflect collection artifacts, while account histories contain recurring patterns that could provide supervision without external annotation.

    Source: §1 · p. 1

  • Method

    FISSION aligns disjoint temporal views of the same account history without abuse labels, then uses a small labeled sample to fit the task classifier.

    Source: §1 · p. 1, §6 · pp. 11–12

  • Results

    FISSION improves classification across markedly different domains; with 1% of Wikipedia labels, it falls only 0.729 points below its full-label accuracy.

    Source: §6 · p. 12, §7 · p. 13

  • Takeaways & Limitations

    An account’s own history can supply pre-label supervision, with task labels selecting which recurring features matter for classification.

    Source: §6 · p. 11, §7 · p. 13

    Fission depends on behavior persisting across dense windows; short histories or abrupt strategy changes can produce weak or mismatched views.

    Source: §6 · p. 13

Abstract

from arXiv · show

Bot accounts and coordinated influence operations are often discovered via heuristic methods, leaving a dearth of reliable ground-truth labels for training detection systems. To address this challenge, we study a natural question: can we generate labels to assist in learning embeddings in which bots and accounts from the same coordinated operation are close? We present FISSION, a method to generate labels by splitting each account's activity into positively labeled sub-accounts. Given this label source, we train detection models which preserve behavioral regularities recurring across positive sub-accounts. We evaluate FISSION and show it outperforms prior methods in detecting Wikipedia sockpuppets and Twitter/X bots.

1 Introduction

Reliable abuse labels are scarce and may reflect collection artifacts, motivating the question of whether an account’s own history can supply supervision. FISSION aligns views of the same history without abuse labels, then uses a small labeled sample to fit a task classifier.

1 Introduction

  • Reliable labels can require expert investigation, while heuristic-derived labels may encode collection artifacts and fail to transfer across platforms or time [24].Coordinated-abuse accounts can be constructed to obscure shared control and behavioral similarity.Source: §1 · p. 1
  • FISSION aligns views of the same account history without an abuse label, then fits a classifier on a small labeled sample.This approach uses an account’s own history as supervision when labels are scarce.Source: §1 · p. 1
  • With 1% of Wikipedia training accounts, FISSION is more than 6% above raw features; 5% of Cresci15 labels brings it within 1% of full-label results.The improvement is greatest when labels are scarce.Source: §1 · p. 2
  • Wikipedia results are 98.684% accuracy, 81.957% sockpuppet F1, and 94.543% ROC-AUC; an RBF SVM reaches 99.227% on Cresci15.The evaluation uses the Wikipedia sockpuppet and Cresci15 datasets.Source: §1 · pp. 1–2

2 Related Work

Prior account-abuse detection uses learned account, graph, and multimodal representations, while sockpuppet systems have moved from stylometry toward behavioral and contextual models. FISSION instead learns from structure within each account history and reserves investigation labels for classification.

2 Related Work

  • Social-bot detection includes direct account embeddings such as SATAR and graph- or modality-based models such as BotRGCN and BotMoE [17] [18] [35].The sockpuppet literature has moved from stylometry toward behavioral and contextual models.Source: §2 · p. 2
  • Wikipedia sockpuppet systems commonly use labels from platform investigations [41], but supervised accuracy can reward collection shortcuts that fail across datasets [24].FISSION learns its encoder from within-history structure and uses investigation labels only for the final classifier.Source: §2 · pp. 2–3
  • Contrastive methods define representations through selected shared information [6]; FISSION uses disjoint, activity-adaptive history views rather than a new objective.Related temporal and user-representation work studies recurrent contexts or observations [51] and identity continuity [21].Source: §2 · p. 3
  • Coordination methods compare accounts for shared-controller evidence [38] [47], whereas FISSION starts from the common source of two periods within one account.Source: §2 · p. 3

3 Problem Setting and FISSION

FISSION learns account representations from unlabeled histories of disjoint training accounts, then uses a small labeled subset to fit task classifiers. It constructs positives from contiguous temporal views, encodes behavioral and lexical information, and trains contrastively to preserve recurring within-account regularities.

3.1 Account Histories and Task Labels

  • Only training accounts supply histories for representation learning, while a labeled subset determines the task boundary and test labels are reserved for evaluation.The training and test account sets are disjoint.Source: §3.1 · pp. 3–4

3.2 Temporal Fission

  • Temporal fission partitions a history into contiguous views that cover every event once, so paired views share an origin without sharing events.The number of views depends on activity volume; the rule bounds the count between two and ten and caps pairs at 45.Source: §3.2 · p. 4

3.3 Behavioral and Lexical Views

  • Each view is represented using behavioral features and, for both datasets, 50 lexical dimensions reduced from TF-IDF features by truncated SVD.Cresci15 uses 39 behavioral dimensions and Wikipedia uses 30.Source: §3.3 · pp. 4–5

3.4 Contrastive Representation Learning

  • An MLP encoder maps views to normalized representations, and a temporary projection network is trained with NT-Xent on paired views and other minibatch views.The projection network is discarded after pretraining.Source: §3.4 · p. 5
  • The ensemble averages class probabilities from three independently fitted encoders, with each classifier receiving a 64-dimensional representation.Probability averaging avoids increasing the representation dimension seen by each classifier.Source: §3.4 · p. 5

3.5 Why the Relation Can Help

  • Recurring behavior can aid classification when it supports the task, and pretraining can use eligible histories before labels are needed for the downstream classifier.The view-count rule keeps sparse histories denser while limiting the contribution of highly active accounts.Source: §3.2 · p. 4, §3.5 · p. 6
  • Training caps each account at 45 materialized pairs and uses minibatch negatives rather than enumerating account pairs; inference produces a 64-dimensional representation.The three-encoder model repeats encoding and averages classifier probabilities.Source: §3.5 · p. 6

4 Experimental Design

The experiments test whether common-history pretraining improves matched account classification as task labels shrink, using account-disjoint evaluations on Wikipedia and Cresci15. Comparisons vary the representation and label budget while controlling the accounts, labeled subsets, and evaluation procedure.

4.1 Questions and Datasets

  • The study tests whether common-history pretraining improves matched account classification as labels shrink, using Wikipedia’s 161,604-account active cohort and all 5,301 Cresci15 profiles.Wikipedia’s active-cohort filter raises sockpuppet prevalence from 4.0% to 41.0%.Source: §4.1 · pp. 6–7

4.2 Account-Disjoint Model Selection

  • Nested account-disjoint evaluation excludes each test account’s events from fitting its feature transform, representation, classifier, calibration map, and threshold.Repeated outer cross-validation uses inner folds for classifier and threshold selection, separating model selection from evaluation.Source: §4.2 · pp. 7–8

4.3 Controls and Component Tests

  • Matched raw-feature controls share accounts, training-fold transforms, labeled subsets, selection, and thresholds; component tests vary views, encoder averaging, classifier family, and feature groups.Fixed ten-way fissioning gives eligible accounts 45 pairs, separating the central comparison from activity-dependent pair counts.Source: §4.2 · p. 8, §4.3 · p. 8

4.4 Performance and Label Efficiency

  • Model comparisons use paired account-level metric contrasts, with accuracy confidence intervals bootstrapping accounts and keeping each account’s repeated predictions together.Accuracy is primary; positive-class F1 measures the designated abuse class, while ROC-AUC evaluates ranking independently of threshold.Source: §4.4 · pp. 8–9
  • Label-efficiency tests vary labeled accounts from 1% to 100% while keeping fitted representations and outer splits fixed, and give raw features and FISSION identical subsets.Pretraining includes every eligible outer-training history but uses no labels.Source: §4.4 · p. 9
  • The 0.01-label threshold is the smallest tested budget after which accuracy remains within one percentage point of the full-label result.The threshold uses ε = 0.01.Source: §4.4 · p. 9

5 Results

FISSION improves matched account-classification accuracy on both evaluated platforms, with a particularly strong advantage when task labels are scarce. The results also show that feature and component contributions differ across datasets.

5.1 Does Fission Improve Account Classification?

  • FISSION-Distill scores all 3,312,087 nonempty Wikipedia identifiers, reaching 98.684 ± 0.002% accuracy, 81.957 ± 0.025% sockpuppet F1, and 94.543 ± 0.005% ROC-AUC.The one-edit route contains 73.939% of the full cohort.Source: §5.1 · p. 10
  • Wikipedia logistic accuracy improves by 4.037%; Cresci15 logistic accuracy gains 0.355 points, while its RBF-SVM representation gains 0.686 points.On Wikipedia, a single fixed-view encoder improves accuracy by 3.181 points; the full adaptive model adds a further 0.857 points.Source: §5.1 · p. 10

5.2 How Many Task Labels Are Needed?

  • At 1% of Wikipedia labels, accuracy is 94.577%, 6.114 points above raw features; 5% of Cresci15 labels brings FISSION within one point of its full-label accuracy.FISSION has higher mean accuracy at every evaluated budget.Source: §5.2 · pp. 10–11, Figure 3 · p. 11

5.3 What Does the Representation Use?

  • Removing page-focus and inter-edit-rhythm features lowers Wikipedia accuracy by 10.413 points, while removing Cresci15 lexical dimensions lowers accuracy by 1.946 points.Behavioral-group intervals on Cresci15 include zero; component gains also differ between platforms and classifier families.Source: §5.3 · p. 11

6 Discussion

FISSION’s representation improves Cresci15 classification, with benefits most apparent under scarce labels. The discussion attributes its gains to recurring within-account behavior while noting limits for sparse or unstable histories and component comparisons.

6 Discussion

  • Temporal fission aligns disjoint periods from one account so the encoder learns recurring behavior, while task labels let the classifier select useful dimensions.The relation is weaker than an abuse label and broader than a hand-designed activity proxy; disjoint views cannot align through event duplication.Source: §6 · pp. 11–12
  • At one-percent label budgets, FISSION is 0.729 points below its full-label accuracy on Wikipedia, while raw features are 2.805 points below their own.The Cresci15 one-percent fold contains 41 labeled accounts, and the label-sufficiency threshold is five percent; unlabeled histories remain fixed as the task-label budget changes.Source: §6 · p. 12
  • Cresci15’s raw baseline is nearly saturated, yet FISSION leads the matched model at every label budget, with its widest gap at one percent.The benchmarks apply the same common-history relation and training protocol to collaborative editing and social posting.Source: §6 · p. 12
  • Editing rhythm carries the Wikipedia ablation, whereas text carries Cresci15; the authors argue this cross-domain pattern is not a generic activity proxy.The common-history relation rewards recurring behavior available on each platform.Source: §6 · p. 12
  • A single fixed-view encoder explains most of Wikipedia’s difference, but Cresci15’s gain appears only with adaptive views and the ensemble, which the comparison does not isolate individually.Adaptive segmentation’s effect varies by dataset and classifier; fixed-view gains on Wikipedia also persist despite activity-dependent pair counts.Source: §6 · p. 12
  • 0.356 points with logistic regression and 0.686 with the RBF SVM are Cresci15 representation gains; the latter is measured above a 98.537% raw baseline on the same held-out accounts.The paired analysis bootstraps differences at the account level.Source: §6 · p. 13
  • Fission needs behavior to persist across several dense windows; short histories and handoffs or abrupt strategy changes can make positive views unreliable.The paper identifies learning useful representations from the first few events as distinct from reducing labels for established histories.Source: §6 · p. 13
  • Unlabeled histories can refresh the encoder while a smaller current sample refits the classifier and threshold; available operator identities could also define positive pairs.The proposed operator-based pairing would require holding out entire operators during evaluation.Source: §6 · p. 13

7 Conclusion

FISSION learns from the common origin of disjoint temporal views before task labels are assigned, and the conclusion reports benefits across Wikipedia and Twitter/X, including sparse-label settings. The appendices specify the evaluated cohorts, view and feature construction, model training, and scoring protocols.

7 Conclusion

  • FISSION uses disjoint temporal views with a common observed origin to learn representations before task labels, and FISSION-Distill transfers the active-history signal to sparse prefixes.The conclusion reports improved classification across the two domains while retaining most full-label accuracy when labels are scarce.Source: §7 · p. 13

A Reproducibility

  • The paper uses existing public datasets and says its pipeline and table-and-figure aggregates will be open-sourced upon publication.Appendices provide cohort, feature, training, and evaluation details.Source: A Reproducibility · p. 20

B.1 Cresci15

  • Cresci15 evaluates 5,301 profile accounts; training accounts with at least 20 released events form fission pairs, while shorter histories are still embedded and evaluated.The stored retweet indicator is uniformly zero and is reconstructed from the “RT @” text prefix; the authors treat the dataset as an in-cohort comparison because collection sources are closely associated with class.Source: §B.1 · p. 20

B.2 Cresci15 Profiles Without Released Posts

  • Combining FISSION with the fixed-threshold profile-only fallback across all 5,301 profiles yields 99.227 ± 0.061% accuracy and 99.389 ± 0.048% bot-F1; the fallback correctly classifies 152 of 153 fallback accounts.Source: §B.2 · p. 20

B.3 Wikipedia

  • Wikipedia’s main experiment uses 161,604 accounts with at least ten edits, and account-level folds cannot be verified as disjoint by investigation or operator.The active cohort is 4.9% of supplied identifiers; the main Cresci15 and Wikipedia experiments use stratified five-fold account-level cross-validation.Source: §B.3 · p. 21, §B.4 · p. 21

C.1 Behavioral and Lexical Inputs

  • Cresci15 uses 39 behavioral dimensions and Wikipedia uses 30; both datasets also receive account-level TF-IDF and 50-dimensional SVD lexical features.The TF-IDF transform is fitted on training accounts and applied unchanged to training views, histories, and outer-test histories.Source: §C.1 · p. 22

C.2 View Construction and Pair Weighting

  • Adaptive fission forms contiguous, near-equal views and materializes every unordered pair; below the ten-view cap, eligible Cresci15 accounts contribute 6–36 pairs versus 45 under fixed ten-way fissioning.Figure 5 shows five denser adaptive views for a 25-event history, with both rules reaching ten views at 50 events.Source: §C.2 · p. 22, Figure 5 · p. 23

C.3 Encoder and Contrastive Objective

  • The encoder maps each segment to a feature vector and treats same-account segment pairs as NT-Xent positives; each outer fold refits data-dependent preprocessing, encoder, classifier selection, calibration, and thresholds within its training partition.Inference embeds each complete history once; the raw-feature controls share the outer accounts, labels, fitted lexical transforms, and threshold rule.Source: §C.3 · p. 23, D Complete Experimental Protocol · p. 23, §D.1 · p. 23

D.2 Compute Environment

The compute-environment subsection records the hardware and software used for encoder fitting and downstream analysis.

D.2 Compute Environment

  • Encoder fitting used an NVIDIA RTX 4080 GPU; feature extraction, downstream classification, and analysis used the CPU.The environment included Ubuntu 24.04.4 LTS, an Intel Xeon w5-2445 CPU, and 256 GB RAM.Source: §D.2 · p. 24

E Metrics and Statistical Summaries

This section defines the reported performance measures and pooled out-of-fold summaries, then describes paired account-level uncertainty and significance procedures.

E Metrics and Statistical Summaries

  • Precision and recall use thresholded predictions, while ROC-AUC uses independent positive- and negative-class scores; the positive class is bot on Cresci15 and sockpuppet on Wikipedia.The supplied definition names classic F-measure and probability-of-ranking interpretations of ROC-AUC.Source: E Metrics and Statistical Summaries · p. 24
  • Average precision for the full dataset is computed with sklearn.metrics.average_precision_s.Source: E Metrics and Statistical Summaries · p. 24
  • Pooled out-of-fold metrics concatenate the five outer-test folds so every account contributes one tuple; pooling weights accounts equally and avoids averaging folds of slightly different sizes.Variation across outer repetitions is also reported.Source: E Metrics and Statistical Summaries · pp. 24–25
  • Model accuracy differences are bootstrapped by resampling paired account-level differences; centering approximates the bootstrap null, adding one prevents a zero Monte Carlo value, and Holm adjustment is used for related feature-removal tests.The reported interval uses the 2.5th and 97.5th percentiles.Source: E Metrics and Statistical Summaries · p. 25

F Retrospective Wikipedia Full-Dataset Coverage

The full-dataset extension transfers frozen FISSION scores to sparse Wikipedia histories, while component analyses show that useful signals and the benefit of adaptive views vary across datasets and classifiers.

F Retrospective Wikipedia Full-Dataset Coverage

  • The full-dataset output has the highest sockpuppet F1, accuracy, macro-F1, and sockpuppet precision among evaluated final outputs.Positive-class F1 is the headline operating-point objective; balanced accuracy is 87.104±0.012% and macro-F1 is 90.637±0.013%.Source: F Retrospective Wikipedia Full-Dataset… · pp. 25–26
  • The one-edit route is the main bottleneck, covering 2,448,931 accounts (73.939%) with 68.228% ROC-AUC and 5.962% AP.The two-edit route reaches 93.726% ROC-AUC and 66.017% AP; routes with three through nine edits range from 97.046% to 98.997% ROC-AUC and 86.257% to 97.264% AP.Source: F Retrospective Wikipedia Full-Dataset… · p. 26

G.1 Count-Adaptive Views and Classifier

  • Adaptive views improve accuracy more for logistic regression than for the RBF SVM, while the kernel contributes more under fixed views.On the separate 30,000-account Wikipedia cohort, adaptive views add 0.299 points (95% CI [0.188, 0.411]) over fixed 10-way views.Source: §G.1 · p. 26

G.2 Cresci15 Feature Groups

  • Removing text lowers Cresci15 accuracy by 1.9 points, while the other feature-group removal intervals include zero.The leave-one-group-out experiment preserves encoder width by replacing the removed input group with zeros.Source: §G.2 · p. 26

G.3 Wikipedia Feature Blocks

  • On Wikipedia, removing page-focus and inter-edit-rhythm features produces the largest observed loss, while removing lexical edit-comment features causes a smaller but consistent loss.The full-scale analysis retrains the complete model after removing one input block.Source: §G.3 · p. 26

H Classifier-Family Robustness

The classifier-family comparison spans linear, nonlinear, tree-based, neighbor, neural, and pretrained-tabular approaches. The accompanying figure tracks positive-class F1 and ROC-AUC across label budgets for two datasets.

H Classifier-Family Robustness

  • The classifier comparison spans linear log-odds, nonlinear maximum-margin, tree, cosine-neighbor, shallow-MLP, and pretrained-tabular approaches.The compared families include logistic regression, RBF SVM, bagged and boosted trees, cosine-neighbor voting, a shallow MLP, and TabPFN.Source: H Classifier-Family Robustness · p. 26
  • TabPFN and RBF SVM tie, and RBF SVM is retained because it needs no extra pretrained tabular model and supports CPU inference.The comparison evaluates averaged per-encoder probabilities and, where applicable, one model on the concatenated 192-dimensional representation.Source: H Classifier-Family Robustness · p. 26
  • Figure 6 plots positive-class F1 and ROC-AUC across label budgets for Cresci15 and Wikipedia, contrasting raw features with FISSION.Bars show sample standard deviations.Source: Figure 6 · p. 27

I Complete Label-Efficiency Results

The section reports label-budget performance, error patterns, feature contrasts, and representation geometry, then qualifies the scope of the resulting estimates.

I Complete Label-Efficiency Results

  • At the one-percent label budget, on identical stratified subsets, FISSION improves over raw features in accuracy, positive-class F1, and ROC-AUC by 2.804, 2.239, and 2.867 points on Cresci15, and by 6.114, 7.114, and 3.423 points on Wikipedia.Source: I Complete Label-Efficiency Results · p. 27

J.1 Confusion Matrices

  • Recall remains high for both classes: 98.66% for humans and 99.56% for bots on Cresci15, and 95.63% for benign Wikipedia accounts and 94.83% for sockpuppets.Figure 7 shows row-normalized out-of-fold confusion matrices averaged over the main repeated evaluations.Source: §J.1 · p. 28, Figure 7 · p. 28

J.2 Cresci15 Residual-Error Activity

  • Of 20 accounts misclassified in every Cresci15 evaluation, 60% have fewer than 20 events, versus 33% of ever-misclassified accounts and 23% of the cohort.Activity count is intertwined with class and collection source.Source: §J.2 · p. 28, Figure 8 · p. 29

J.3 Directly Computed Feature Contrasts

  • Cliff’s δ summarizes univariate class separation; Cresci15 differs on several activity and interaction statistics, while Wikipedia’s largest differences involve edits per page and inter-edit timing.Figure 9 plots δ for directly computed scalar features on balanced samples of 1,000 accounts per class, with positive values indicating larger values for bots or sockpuppets.Source: §J.3 · pp. 28–29, Figure 9 · p. 30

J.4 Input and Representation Geometry

  • Two-dimensional PCA projections retain class overlap in both datasets, most visibly for Wikipedia raw features.Figure 10 projects raw inputs and concatenated three-encoder representations for one prespecified outer-test fold per dataset.Source: §J.4 · p. 29, Figure 10 · p. 36

K Scope

  • The datasets identify accounts but not common operators or investigations, so the estimates describe retrospective performance within these cohorts.Wikipedia folds are account-disjoint but not operator-disjoint; the full-dataset extension also uses other folds’ target labels for route calibration and threshold selection, and its extension choices were made after aggregate target results were inspected.Source: K Scope · pp. 29–30

Related papers

Loading 2609.26279v1…