Source-linked AI summary
Still out there: Modeling and Identifying Russian Troll Accounts on Twitter
Jane Im, Eshwar Chandrasekharan, Jackson Sargent, Paige Lighthammer, Taylor Denby, Ankit Bhargava, Libby Hemphill, David Jurgens, Eric Gilbert
TL;DR
The paper asks whether models trained on older Russian troll accounts can identify currently active trolls, a question tied to suspected election interference through fake Twitter accounts. The authors train machine-learning classifiers using profile, behavioral, and linguistic features from known troll accounts and control accounts, then apply logistic regression to unseen accounts mentioning journalists. A logistic regression achieved precision of 78.5% and an AUC of 98.9%; applied to late-2018 accounts mentioning high-profile journalists, it flagged 3.7%, with trolls estimated at 2.6% of mentions.
Problem
The paper asks whether models trained on older Russian troll accounts can identify currently active trolls, a question tied to suspected election interference through fake Twitter accounts.
Source: §1 · p. 1Method
The authors train machine-learning classifiers using profile, behavioral, and linguistic features from known troll accounts and control accounts, then apply logistic regression to unseen accounts mentioning journalists.
Source: §1 · p. 1, §3.2 · p. 3, Out-of-sample Twitter accounts · p. 5Results
A logistic regression achieved precision of 78.5% and an AUC of 98.9%; applied to late-2018 accounts mentioning high-profile journalists, it flagged 3.7%, with trolls estimated at 2.6% of mentions.
Source: §1 · p. 1Takeaways & Limitations
The authors argue that platforms with access to private internal signals should identify Russian troll accounts at similar or better rates than models using externally accessible data.
Source: §1 · p. 2The evidence is Twitter-specific; adapting the model to Facebook or Reddit and testing whether qualitative findings transfer remain open questions.
Source: §8.1 · p. 9
Abstract
from arXiv · showhide
There is evidence that Russia's Internet Research Agency attempted to interfere with the 2016 U.S. election by running fake accounts on Twitter - often referred to as "Russian trolls". In this work, we: 1) develop machine learning models that predict whether a Twitter account is a Russian troll within a set of 170K control accounts; and, 2) demonstrate that it is possible to use this model to find active accounts on Twitter still likely acting on behalf of the Russian state. Using both behavioral and linguistic features, we show that it is possible to distinguish between a troll and a non-troll with a precision of 78.5% and an AUC of 98.9%, under cross-validation. Applying the model to out-of-sample accounts still active today, we find that up to 2.6% of top journalists' mentions are occupied by Russian trolls. These findings imply that the Russian trolls are very likely still active today. Additional analysis shows that they are not merely software-controlled bots, and manage their online identities in various complex ways. Finally, we argue that if it is possible to discover these accounts using externally - accessible data, then the platforms - with access to a variety of private internal signals - should succeed at similar or better rates.
1 Introduction
The paper tests whether models trained on 2016 Russian troll accounts can identify newer trolls, then applies a model to accounts contacting high-profile journalists. It reports strong cross-validation performance and evidence that suspected trolls remain active, while describing their varied identity strategies.
1 Introduction
- The study asks whether models trained on old trolls can identify new ones, using profile, behavioral, and linguistic features against a control set of 170K accounts.The troll dataset contains 2.2K accounts, or 1.4% of the sample.Source: §1 · p. 1
- 78.5% precision and 98.9% AUC were achieved by a simple logistic regression distinguishing trolls from randomly selected controls.The model was also applied to unseen accounts mentioning high-profile journalists.Source: §1 · p. 1
- Roughly 70% of 50 model-flagged accounts reviewed by humans were judged highly likely to be trolls, and suspected trolls were estimated to occupy 2.6% of journalists’ mentions.The flagged accounts were still active, and they did not score highly on Botometer.Source: §1 · p. 1
- Open coding found varied identity-deception strategies, including accounts presenting as American mothers or middle-aged white men while posting unusually often about politics alone.The authors characterize these observations as exploratory.Source: §1 · pp. 1–2
- The authors make their model available to researchers and argue that platforms with private signals should identify trolls at similar or better rates than externally accessible data allows.They also conclude that flagged accounts are not merely bots and manage online identities in diverse ways.Source: §1 · p. 2
2 Related Work
The related work situates Russian troll activity within broader election interference, research on troll behavior and influence, bot detection, and online identity deception. It also describes prior evidence on campaigns and the users and platforms they reached.
2 Related Work
- The related-work section reviews Russian interference, research on the 2016 trolls, social bots, and theories of online deception informing the paper’s quantitative approach.Source: §2 · p. 2
2.1 Russia’s Interference on Political Campaigns
- Prior work frames Russian troll activity as one part of state-level online interference in democratic processes beyond the United States.Examples include the Brexit referendum, MacronLeaks, and the Catalonian referendum.Source: §2.1 · p. 2
2.2 Emerging Work on the 2016 Russian Trolls
- Studies of the 2016 trolls describe distinct account categories and political engagement patterns, while reporting both behavioral commonalities and changing tactics.Research also reports unique linguistic behavior relative to a baseline cohort.Source: §2.2 · p. 2
Users Who Interact with the Trolls.
- Research on users who interact with trolls reports that conservatives shared their misinformation more often than liberals and that spreaders can be predicted from ideology, bot likelihood, and account metadata.Source: Users Who Interact with the Trolls · p. 2
Measuring the Propaganda’s Effect.
- Prior studies measured propaganda exposure and influence across platforms using ads, tweets, browser logs, and Hawkes Processes.One estimate found 1 in 40,000 internet users were exposed to IRA ads on a given day.Source: Measuring the Propaganda’s Effect · p. 3
2.3 Bots
- Bot-detection research spans network, crowdsourcing, and machine-learning approaches, while prior findings note increasingly sophisticated bot behavior.Bots were reported to have generated millions of tweets before the 2016 election.Source: §2.3 · p. 3
2.4 Deception and Identity Online
- Online deception research treats profiles and conversations as identity signals, with users crafting self-presentation differently across online communities and platforms.Prior Twitter research describes identity management through both profiles and ongoing conversations.Source: §2.4 · p. 3
3 Data
The study combines known Russian troll accounts, a control cohort of typical U.S. users, and unseen accounts contacting high-profile political journalists. The cohorts are assembled to support model construction and later out-of-sample application.
3 Data
- The study constructs a dataset of known Russian trolls and a control cohort of regular users for modeling potential Russian trolls on Twitter.Source: §3 · p. 3
3.1 Russian Trolls
- 2,286 of Twitter’s 3,841 IRA-linked accounts were selected because users chose English; the study used each troll account’s most recent 200 tweets.The troll linguistic sample contained 346,711 tweets.Source: §3.1 · p. 3
3.2 Control Accounts
- 171,291 randomly sampled U.S.-located controls had at least five tweets during 2012–2017, yielding a 98.6% control and 1.4% troll class imbalance.The most recent 200 tweets per control account were collected, totaling 29,960,070 tweets.Source: §3.2 · pp. 3–4
3.3 Journalists’ Mentions
- The out-of-sample case study collected 47,426 mentions of 57 high-profile political journalists, representing 39,103 unique accounts for model application.Source: §3.3 · p. 4
4 Method
The method combines account-profile, behavioral, and linguistic signals to classify Russian trolls, then tests the classifier on unseen accounts mentioning political journalists. Human raters independently assess a sample of flagged and unflagged accounts.
4.1 Constructing Classifiers
- Profile features capture account age, description length, followers, following, and their ratio; behavioral features cover hashtags, mentions, links, and tweet volume and length.Behavioral measures include per-day tweet averages and variability, retweet rate, and link ratios.Source: §4.1 · p. 4
- Stop-word features measure each of 179 words’ share of an account’s tweet words, using function-word patterns as potential cues to non-native English.The list comes from sklearn and includes common content words as well as function words.Source: §4.1 · p. 4
- Language features encode each account’s timeline distribution across 82 languages, while bag-of-words features represent tweet unigram and bigram frequencies from the top 5,000 n-grams.Tweets are lowercased and stopwords are removed before n-gram extraction.Source: §4.1 · pp. 4–5
Classifiers.
- The authors test Logistic Regression, depth-10 Decision Tree, and Adaptive Boosted Decision Tree models using all five feature categories, plus one classifier per category.The category-specific models assess the contribution of each feature group.Source: Classifiers · p. 5
Out-of-sample Twitter accounts.
- For unseen data, the authors apply the all-feature Logistic Regression model to 39,103 accounts that mentioned 57 high-profile political journalists, treating its predictions as unverified flags.They chose Logistic Regression because its simplicity seemed most likely to generalize out of sample.Source: Out-of-sample Twitter accounts · p. 5
Validation through human evaluation.
- Human validation randomly samples 50 flagged and 50 unflagged accounts, hides flag status from three trained raters, and asks them to judge troll likelihood independently.Raters trained for over 12 hours by examining previously released troll tweets.Source: Validation through human evaluation · p. 5
- Raters inspect profiles and 20 recent tweets, score troll likelihood from 1 to 5, and the authors calculate inter-rater agreement and average account ratings.Inspection considers activity levels, profile details, political-only content, personal-life evidence, and inflammatory statements or hashtags.Source: Validation through human evaluation · p. 5
5 Results
Cross-validation shows strong but differently balanced performance across the full-feature classifiers, while feature-category and SAGE analyses identify linguistic signals. On unseen journalist mentions, the model flags accounts that human raters often judge suspicious.
5.1 Cross-validation
- 78.5% precision and 98.9% AUC for Logistic Regression compare with 94.2% precision and 91.2% AUC for Adaptive Boosted Decision Trees in 10-fold cross-validation.The authors prioritize precision because false positives have a large downside in this imbalanced dataset.Source: §5.1 · p. 5, Table 2 · p. 6
- 98% AUC and 58% precision make bag-of-words the best-performing single feature category; language-distribution and stop-word classifiers each exceed 80% AUC.Source: §5.1 · p. 5, Table 3 · p. 6
5.2 SAGE Analysis
- SAGE highlights chaos- and distrust-related terms, political figures, MAGA, and Fukushima-related keywords as distinguishing troll tweets from control tweets.Examples include “danger,” “disaster,” “crisis,” “emergency,” and “attacks.”Source: §5.2 · p. 6
5.3 Testing on Unseen Data & Human evaluation
- 1,466 of 39,103 journalist-mentioning accounts (3.7%) were flagged, averaging 25.7 per journalist; every journalist had at least one flagged account.Source: §5.3 · p. 6, Table 4 · p. 6
- 70% of flagged accounts had at least two raters score them above 3, and 84% had at least one such rating, supporting out-of-sample generalization.Rater agreement was moderate (Krippendorff’s α = 0.512); flagged accounts’ median rating was 3.12 versus 2.5 for all sampled accounts.Source: §5.3 · pp. 6–7, Figure 3 · p. 6
6 Analysis of the Flagged Accounts
The analysis examines whether model-flagged accounts resemble bots and compares their behavior with unflagged accounts. Flagged accounts show some bot-like traits, alongside account and tweeting patterns associated with Russian trolls.
6.1 Botometer Scores
- Both flagged and unflagged accounts scored below 1 on Botometer’s 5-point scale, although flagged accounts scored significantly higher.The authors interpret this as evidence that bot-like characteristics are not dominant among flagged accounts.Source: §6.1 · p. 7, Figure 4 · p. 7
- All six Botometer category scores were higher for flagged accounts than unflagged accounts, but remained below 2; the largest difference was 0.49 for user-based features.The comparison covers six feature categories and all-category scores.Source: §6.1 · p. 7, Table 6 · p. 8
6.2 Flagged Accounts’ Behavioral Characteristics
- By January 13, 2019, four flagged accounts were suspended and eight no longer existed, although those accounts existed when the model was run.Source: §6.2 · p. 7
- Flagged accounts averaged 41.99 tweets per day versus 26.83 for unflagged accounts, and were relatively newer with more followers and following.Flagged accounts also had more tweets in Russian than unflagged accounts; both groups were likely to use English most frequently in this journalist-mention sample.Source: §6.2 · p. 8, Table 7 · p. 8, Table 8 · p. 8
- Flagged and unflagged accounts shared many political hashtags, but flagged accounts used more hashtags overall, including #MAGA more than twice as often despite only slightly more tweets.Source: §6.2 · p. 8, Table 9 · p. 8
7 Discussion
Raters observed mismatches between flagged accounts’ biographies and tweets, varied profile-picture strategies, aggressive exchanges, and unusually high political retweet activity. The authors argue that platforms could identify such accounts using their broader data access.
7 Discussion
- Raters observed mismatches between some bios and tweets, varied profile-picture strategies, and aggressive posts about controversial issues.One account claimed to live in Brooklyn but used a restaurant image from Iceland; these observations were anecdotal.Source: §7 · pp. 8–9, Figure 5 · p. 9
- Flagged accounts often showed high retweet rates concentrated on political content; one examined account tweeted roughly 65 times daily and another posted over 44,200 times since January 2017.Source: §7 · p. 9
7.1 Implications for Social Platforms
- The authors argue that platforms with complete user, content, and interaction data can identify Russian trolls, while current bot-detection efforts are unlikely to capture most of them.Source: §7.1 · p. 9
8 Future Work
The authors suggest testing whether their model and qualitative findings adapt to other platforms, where Russian troll activity has differed. They also release their models to support further research on troll accounts and their networks.
8.1 Social Platforms Other than Twitter
- Whether the model and qualitative findings transfer to Facebook or Reddit remains an open question, because Russian troll activity has differed across platforms.Source: §8.1 · p. 9
8.2 Publicly Available Models
- The publicly available models may help researchers study troll networks, which cannot be examined using the 2016 election dataset because Twitter deleted those accounts.The authors also hope researchers will use the models to investigate account objectives and possible effects on users’ political beliefs.Source: §8.2 · p. 9
9 Conclusion
The conclusion reports that the model identifies suspected trolls among accounts contacting prominent journalists, with human review supporting many flags. The authors argue that platforms with internal signals could achieve similar or better detection rates.
9 Conclusion
- The model achieved an AUC of 98.9% in cross-validation and flagged 3.7% of accounts contacting prominent journalists as suspected Russian trolls.Source: §9 · p. 9
- Human evaluation suggested that roughly 70% of the flagged journalist-contacting accounts were valid suspected trolls.Source: §9 · p. 9
- The authors argue that platforms with internal user signals can find and deactivate similar adversarial actors at similar or better rates.Source: §9 · p. 9