Source-linked AI summary
Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours
Karishma Sharma, Yizhou Zhang, Emilio Ferrara, Yan Liu
TL;DR
Existing coordinated-account detection can depend on strict behavioural assumptions or require known malicious accounts. AMDN-HAGE jointly models activity traces and hidden groups, and experiments identify coordinated patterns in Twitter campaigns, including COVID-19 conspiracies. The authors also report that influence between coordinated accounts is higher and decays faster than between non-coordinated pairs.
Problem
Existing approaches to coordinated-account detection can rely on strict behavioural assumptions or require some malicious accounts to be identified beforehand.
Source: §1 · pp. 1–2Method
AMDN-HAGE is an unsupervised generative model that jointly models account activities and hidden groups using temporal point processes and a Gaussian Mixture Model.
Source: §1 · p. 2Results
Analysis of Russian Interference and COVID-19 datasets found higher influence between coordinated accounts, with influence decreasing faster over time than for non-coordinated pairs.
Source: §6 · p. 9Takeaways & Limitations
The method detects coordinated accounts from social-media activity without relying on linguistic, metadata, or platform-specific features.
Source: §6 · p. 9The basic input excludes action type, post content, and account metadata, though additional features can be incorporated.
Source: §3.1 · p. 3
Abstract
from arXiv · showhide
Disinformation campaigns on social media, involving coordinated activities from malicious accounts towards manipulating public opinion, have become increasingly prevalent. Existing approaches to detect coordinated accounts either make very strict assumptions about coordinated behaviours, or require part of the malicious accounts in the coordinated group to be revealed in order to detect the rest. To address these drawbacks, we propose a generative model, AMDN-HAGE (Attentive Mixture Density Network with Hidden Account Group Estimation) which jointly models account activities and hidden group behaviours based on Temporal Point Processes (TPP) and Gaussian Mixture Model (GMM), to capture inherent characteristics of coordination which is, accounts that coordinate must strongly influence each other's activities, and collectively appear anomalous from normal accounts. To address the challenges of optimizing the proposed model, we provide a bilevel optimization algorithm with theoretical guarantee on convergence. We verified the effectiveness of the proposed method and training algorithm on real-world social network data collected from Twitter related to coordinated campaigns from Russia's Internet Research Agency targeting the 2016 U.S. Presidential Elections, and to identify coordinated campaigns related to the COVID-19 pandemic. Leveraging the learned model, we find that the average influence between coordinated account pairs is the highest.On COVID-19, we found coordinated group spreading anti-vaccination, anti-masks conspiracies that suggest the pandemic is a hoax and political scam.
1 INTRODUCTION
Disinformation campaigns use coordinated accounts to influence public opinion, but existing detection approaches can rely on narrow behavioural signatures or require some malicious accounts to be known. AMDN-HAGE instead models latent influence and collectively anomalous group behaviour from account activities.
1 INTRODUCTION
- Existing group-detection methods rely on signatures such as synchronized timing or similar hashtags, although observed coordination gaps range from under six hours to more than half a week.Hand-crafted signatures can be inconsistent and limit generalization to unseen accounts.Source: §1 · p. 2
- AMDN-HAGE models account activities and hidden groups jointly with Neural Temporal Point Processes and a Gaussian Mixture Model, using bilevel optimization to learn from observed traces.The model represents strong hidden influence between accounts and collective anomaly relative to normal accounts.Source: §1 · p. 2
- AMDN-HAGE learns account representations and hidden groups from activity traces without requiring known coordinated accounts or predefined individual features.Additional features can still be incorporated if available.Source: §1 · p. 2
- Experiments on known Twitter campaigns and COVID-19 data identify coordinated groups, including groups promoting anti-mask and anti-vaccine claims that COVID-19 is a hoax or political scam.The learned model also finds that influence between coordinated accounts is highest and decays fastest.Source: §1 · p. 2
2 RELATED WORK
Prior work detects suspicious accounts through individual features or group-level activity patterns. The paper contrasts these approaches with learning latent account interactions directly from observed activities.
2 RELATED WORK
- Individual-account approaches use participation in disinformation spreading, account characteristics, or activity traces to infer malicious behaviour.Examples include suspiciousness scores from fake-news cascades and features such as language, shared links, hashtags, devices, and cross-platform activity.Source: §2 · p. 2
- Collective approaches cluster accounts by synchronized actions or similarity in hand-crafted features such as shared hashtags or articles.These approaches assume synchronization or selected features reliably indicate coordination.Source: §2 · p. 3
- The paper identifies synchronization and hand-crafted coordination features as limitations, and proposes learning latent account interactions from observed activities instead.Source: §2 · p. 3
3 TASK DEFINITION AND PRELIMINARIES
The task is to identify coordinated accounts from activity traces and hidden group behaviour. The preliminaries define temporal point processes as a way to model sequences of account events over continuous time.
3.1 Task Definition
- Coordinated campaigns are orchestrated efforts in which accounts collude to spread narratives and manipulate opinion, and the task is to identify those accounts.AMDN-HAGE models activity traces and latent account groups to detect collectively anomalous behaviour.Source: §3.1 · p. 3
- The method uses activity traces—events ordered by time—and excludes action type, post content, and account metadata from its basic input.Its coordination assumptions are that coordinated accounts are few relative to normal accounts and have highly concerted activity patterns.Source: §3.1 · p. 3
3.2 Temporal Point Process
- A temporal point process models discrete events occurring in continuous time, with event history and conditional intensity describing the past and instantaneous event rate.In social-network data, a Hawkes process represents event influence through mutually triggering intensities that decay over time.Source: §3.2 · p. 3
4 COORDINATION DETECTION METHOD
AMDN-HAGE jointly models account activity traces and latent account groups to identify coordination, treating coordinated groups as collectively anomalous. It combines an attention-based temporal point process with a Gaussian mixture over account embeddings and trains them through bilevel optimization.
4 COORDINATION DETECTION METHOD
- AMDN-HAGE jointly models activity traces and hidden account groups, identifying coordination through latent groups with anomalous distributions.Its two components share account embeddings.Source: §4 · p. 3, Figure 2 · p. 4
4.1 Modeling Account Activities
- AMDN predicts each activity from prior-event history and the known account set using an encoder-decoder architecture.The history encoder forms context vectors; the event decoder models the conditional density of the next event.Source: §4.1 · p. 4
4.1.1 AMDN architecture and training.
- Masked self-attention with temporal encoding represents event influence, while a log-normal mixture models the next-event time distribution.The mixture provides a flexible conditional density for nonnegative inter-event times.Source: §4.1.1 · pp. 4–5
4.2 Modeling Hidden Groups
- HAGE fits a Gaussian mixture to learnable account embeddings, treating groups with smaller variance or size than other accounts as anomalous.The hidden groups are inferred from activity traces rather than observed account features.Source: §4 · p. 3, §4.2 · p. 5
4.3 Jointly Learning
- Bilevel training alternates EM updates for GMM parameters with SGD updates for embeddings and AMDN, addressing constrained mixture optimization and converging to a local minimum or saddle point under stated conditions.The method pre-trains embeddings and AMDN on observed sequences before jointly optimizing both likelihood terms.Source: §4.3 · p. 5
5 EXPERIMENT RESULTS
AMDN-HAGE is evaluated on Twitter campaigns from the IRA and COVID-19, against existing detection approaches and model variants. On IRA, it outperforms other methods in both supervised and unsupervised detection.
5.1.1 Russia’s Internet Research Agency (IRA) coordinated campaign.
- The IRA dataset contains 312 coordinated trolls with 1.2 million tweets and 1,713 normal accounts; activity traces use posts and subsequent engagements.The account set includes 2,025 accounts, with 15% of sequence subsets held out for validation.Source: §5.1.1 · p. 6
5.1.2 COVID-19 pandemic.
- COVID-19 data has no labeled coordinated groups, and Twitter suspensions cannot estimate detection precision or recall because suspensions have multiple causes.The analysis instead examines detected groups and their overlap with suspended accounts.Source: §5.1.2 · p. 6
5.2 Baselines and Model Variant
- Baselines include feature-based clustering, inverse-reinforcement-learning methods, and Hawkes Processes, alongside AMDN-HAGE variants that alter group estimation or classification.The variants include AMDN without hidden-group estimation and embedding-based K-means or neural classifiers.Source: §5.2 · p. 6
5.3 Results of Coordination Detection
- AMDN-HAGE outperforms other methods in both unsupervised and supervised IRA detection, with a larger margin in the unsupervised setting.Results are averaged over five-fold stratified cross-validation and five random seeds.Source: §5.3 · p. 7, Table 2 · p. 7
- AMDN-HAGE consistently captures better coordination behavior than AMDN, indicating that jointly modeling group behavior helps beyond modeling account-pair influence alone.AMDN models activity traces without hidden-group estimation.Source: §5.3 · p. 7
- Bilevel training's validation loss converges through pretraining and joint training, while direct Adam optimization reaches an invalid covariance matrix and NaN likelihood.The validation loss is negative log-likelihood on held-out activity traces.Source: §5.3 · p. 7, Figure 3 · p. 7
5.4 Analysis of Coordination Detection
The analysis examines learned influence patterns in IRA accounts and suspicious groups in COVID-19 data. Coordinated IRA pairs have the strongest influence ties, while identified COVID-19 groups show elevated representation of some account types and conspiratorial narratives.
5.4.1 Uncovering characteristic behaviours from influence structure.
- IRA coordinated-account pairs have the strongest aggregate influence ties, while normal–coordinated pairs have the weakest; coordinated accounts also form a distinct anomalous cluster.The influence comparison averages attention weights over five random seeds.Source: §5.4.1 · p. 8, Figure 4 · p. 8
- Influence is higher at shorter time differences and decreases faster for coordinated than normal pairs; a small fraction of coordinated activity persists beyond 2,000 hours.Figure 5 plots influence against time difference, with green for normal pairs and blue for coordinated pairs.Source: §5.4.1 · p. 8, Figure 5 · p. 8
5.4.2 Uncovering coordinated groups in COVID-19 data.
- In the unlabeled COVID-19 data, AMDN-HAGE identifies two anomalous clusters for inspection, so the analysis examines their features rather than labeled coordination outcomes.The clusters are based on clustering silhouette scores.Source: §5.4.2 · p. 8
- State-backed and coupled accounts appeared 1.5–2 times more often than by random chance in identified anomalous clusters, even when few such accounts were found in the collected-account set.Targets were more uniformly distributed because they capture only unidirectional engagement attempts from state-backed accounts, unlike coupled accounts, which engage bidirectionally.Source: §5.4.2 · pp. 8–9
- Hashtags and topic clusters in identified groups include anti-mask, anti-vaccine, anti-science, QAnon, and claims that COVID-19 is a hoax or political scam.The reported narratives appear in both Spanish and English.Source: Table 4 · p. 8, §5.4.2 · p. 9, Figure 6 · p. 9
6 CONCLUSIONS
AMDN-HAGE detects coordinated accounts from collective social-media activity without depending on linguistic, metadata, or platform-specific features. Analysis of Russian Interference and COVID-19 data found higher influence between coordinated accounts and identified COVID-19 groups spreading anti-vaccine and anti-mask narratives.
6 CONCLUSIONS
- The method detects coordinated accounts from collective behavior inferred directly from social-media activity, without linguistic, metadata, or platform-specific features.The authors state that such features can also be incorporated into the model.Source: §6 · p. 9
- Influence between coordinated accounts was higher, and influence between coordinated pairs decreased faster over time than between non-coordinated pairs.The analysis used Russian Interference and COVID-19 datasets.Source: §6 · p. 9
- The identified COVID-19 coordinated groups spread anti-vaccine and anti-mask posts framing the pandemic as a hoax and political scam.The paper reports this as the main narrative in the groups.Source: §6 · p. 9
A.1 Details of the Baselines
The baselines span clustering on co-activity, clickstream behavior, and learned motives, alongside Hawkes-process activity models. They include both supervised and unsupervised detection approaches.
A.1 Details of the Baselines
- Co-activity clustering uses SVD-derived account features from a binary account–post event matrix, then clusters accounts that repeatedly share content.The event types include tweets, retweets, and replies.Source: §A.1 · p. 10
- Clickstream clustering hierarchically groups accounts with similar post, reply, and re-share activity patterns.Source: §A.1 · p. 10
- IRL (S) estimates motives from activity-trace rewards and trains AdaBoost on labeled coordinated and normal accounts.The cited work reported AdaBoost outperforming its other classifiers on those features.Source: §A.1 · p. 10
- The unsupervised IRL baseline clusters estimated rewards with GMM or K-means to identify coordinated groups.Source: §A.1 · p. 10
- The Hawkes-process baselines factorize account-pair influence using learnable embeddings and include clustering and supervised detection variants.Source: §A.1 · p. 10
A.2 Details of Experiments
The experiments use activity sequences and select the number of HAGE clusters by silhouette scores. COVID-19 analyses examine account-creation years and cluster hashtags, including evidence that coordinated activity may extend beyond disinformation.
A.2 Details of Experiments
- HAGE uses two components for IRA and three for COVID-19, selected using silhouette scores on embeddings learned in pretraining.The shared covariance matrix is constrained to be diagonal.Source: §A.2 · p. 10, Figure 7 · p. 10
- The two anomalous COVID-19 clusters have higher ratios of recently created accounts, consistent with the authors' observation that coordinated campaigns have arisen in recent years.Figure 8 shows account-creation years by identified group or cluster.Source: §A.2 · p. 10, Figure 8 · p. 10
- C1's top hashtags include “Resist” and “Resistance,” political-movement tags not linked to disinformation or conspiracies, suggesting coordination may also occur outside disinformation spreading.The hashtag distribution for C1 is shown in Figure 9.Source: §A.2 · p. 11, Figure 9 · p. 11
A.3 Details of Neural Point Process Models
Prior point-process methods trade off flexible intensity models, closed-form likelihoods, and the ability to examine event influence. The paper uses interpretable neural parameterizations of conditional density to learn and inspect influence while retaining a closed-form likelihood.
A.3 Details of Neural Point Process Models
- SAHP and THP require Monte Carlo likelihood-integral estimates, which produce noisier gradient approximations during training.Source: §A.3 · p. 11
- HP and RMTPP use specified intensity forms such as exponentials for closed-form likelihoods, but their chosen forms limit flexibility.Source: §A.3 · p. 11
- LogNormMix and FullyNN model conditional density and cumulative density for flexible, closed-form likelihoods, but their recurrent parameterizations cannot directly examine event and event-type influence.Source: §A.3 · p. 11
- The proposed approach models conditional density with interpretable neural parameterizations to learn and examine influence while retaining the point-process conditional PDF form used by LogNormMix.Source: §A.3 · p. 11
A.4 Proof of Convergence
The proof establishes that the training loss converges, then bounds its endpoint to a local minimum or saddle point.
A.4 Proof of Convergence
- EM steps increase the mixture-model likelihood, which the proof uses to show that the loss decreases monotonically.Source: §A.4 · p. 11
- Constraining mixture-model variances above a constant ε bounds the loss on a given activity-trace set.Source: §A.4 · p. 11
- The bounded, monotonically decreasing loss converges as the iteration count increases.Source: §A.4 · p. 11
- The proof places the converged loss at a local minimum or saddle point, invoking EM's known convergence to a local minimum.Source: §A.4 · p. 11