Source-linked AI summary
Towards Detecting Compromised Accounts on Social Networks
Manuel Egele, Gianluca Stringhini, Christopher Kruegel, Giovanni Vigna
TL;DR
Compromised social-network accounts can exploit trusted audiences to distribute malicious or false messages, motivating detection beyond systems aimed at fake accounts. The paper presents COMPA, which learns behavioral profiles and uses anomaly detection, reporting reliable detection of high-profile and large-scale compromises while recognizing a staged compromise as legitimate.
Problem
Attackers can hijack trusted high-profile accounts to spread malicious or false information, while prior systems mainly target fake accounts rather than compromised legitimate accounts.
Method
COMPA learns statistical behavioral profiles from account messages and flags anomalous messages, grouping similar messages for regular accounts with more variable behavior.
Results
COMPA reliably detected high-profile compromises and large-scale compromises, including four high-profile incidents and hundreds of thousands of compromised accounts across Twitter and Facebook.
Takeaways & Limitations
Consistent behavior in high-profile accounts supports low-false-positive compromise detection, while message aggregation extends COMPA to regular accounts.
Takeaways & Limitations
Attackers may evade COMPA by aligning messages with victims’ behavioral profiles or by avoiding the similarity measures used to group messages.
Abstract
from arXiv · showhide
Compromising social network accounts has become a profitable course of action for cybercriminals. By hijacking control of a popular media or business account, attackers can distribute their malicious messages or disseminate fake information to a large user base. The impacts of these incidents range from a tarnished reputation to multi-billion dollar monetary losses on financial markets. In our previous work, we demonstrated how we can detect large-scale compromises (i.e., so-called campaigns) of regular online social network users. In this work, we show how we can use similar techniques to identify compromises of individual high-profile accounts. High-profile accounts frequently have one characteristic that makes this detection reliable -- they show consistent behavior over time. We show that our system, were it deployed, would have been able to detect and prevent three real-world attacks against popular companies and news agencies. Furthermore, our system, in contrast to popular media, would not have fallen for a staged compromise instigated by a US restaurant chain for publicity reasons.
1 Introduction
Compromised high-profile accounts can exploit established trust to spread harmful information before manual detection. COMPA uses stable account behavior to identify such compromises and extends detection to large-scale attacks by grouping similar messages.
- Motivation: Compromised high-profile accounts can exploit trusted relationships and large audiences to spread malicious or false information with serious reputational and financial consequences.An AP false-news message preceded a 1% S&P 500 drop and temporarily erased US$136B.
- Research gap: Existing detection systems mainly target fake accounts, leaving legitimate accounts that attackers have taken over insufficiently addressed.The paper distinguishes compromised legitimate accounts from accounts created specifically for malicious activity.
- Approach: COMPA builds behavioral profiles from past messages and flags new messages that significantly deviate from the learned profile.The approach models users’ stable habits, such as posting times, devices, and interaction patterns.
- Evaluation: The evaluation reports detection of 383,613 compromised Twitter accounts and 11,087 compromised Facebook accounts, alongside four detected high-profile compromises.COMPA also correctly identified a staged Chipotle compromise as legitimate.
- High-profile accounts: High-profile accounts often have consistent behavior, enabling COMPA to detect compromises with minimal false positives.The paper presents this stability as the basis for reliable detection of isolated high-profile compromises.
- Large-scale compromises: COMPA also groups similar messages to detect large-scale compromises affecting more variable regular accounts while keeping false positives low.This grouping accounts for the greater behavioral variability of regular users.
2 Background: Social Network Compromises
The paper illustrates how compromises of prominent Twitter accounts were used to disseminate false information. It positions behavioral anomaly detection as a way to identify and prevent such incidents.
- Case studies: The Associated Press Twitter account was compromised and used to distribute false information about an explosion at the White House.The case study treats the incident as an example of the critical impact of high-profile account compromise.
- Case studies: Yahoo! News also suffered a compromise in which its account disseminated false information about an Ebola outbreak in Atlanta.The incident is presented as another high-profile compromise involving a news organization.
- Detection approach: COMPA learns typical user behavior and flags messages that do not match that behavior as possible compromises.The paper later analyzes anomalies from four high-profile incidents using this approach.
3 Behavioral Profiles
COMPA represents normal account activity through statistical models trained on historical message streams. It combines message characteristics into anomaly scores to identify behavior that departs from each account’s established profile.
- Profile construction: A behavioral profile uses a user’s historical message stream to characterize normal activity, including posts on Twitter timelines or Facebook walls.Facebook streams include posts on the user’s wall and on friends’ walls.
- Profile construction: Accounts need at least S = 10 messages for COMPA to build a representative behavioral profile.Shorter streams may not capture the breadth and variety of a user’s activity, causing legitimate behavior to appear anomalous.
- Feature models: COMPA extracts feature values from messages and trains a statistical model for each feature, such as posting time or message source.The resulting models capture characteristic aspects of an account’s message behavior.
- Anomaly scoring: Each feature model produces a score from 0 to 1, and COMPA composes these scores into a message anomaly score.A score of 0 denotes perfectly normal behavior for a feature, while 1 indicates highly anomalous behavior.
- Modeled characteristics: The profile models activity timing, message sources, topics, links, and message language to detect deviations from established behavior.Examples include unusual posting hours, new applications, unfamiliar domains, unrelated topics, and language changes.
- Scope: The system does not use more sophisticated natural-language processing techniques for extracting message topics because they are outside the work’s scope.Hashtags are identified as an available source of topic information when platforms provide them.
- Modeled characteristics: For links, COMPA models domains rather than full URLs because domains remain stable across pages a user may share.Previously unseen domains can indicate a behavioral change, while the model also considers link frequency and consistency.
4 Training and Evaluation of the Models
COMPA builds per-feature behavioral profiles from users’ message histories, scores new messages for anomalies, and combines those scores to detect profile violations. The method requires sufficient history and distinguishes mandatory from optional features when evaluating messages.
- Method: COMPA retrieves past messages, extracts features, models each feature separately, and combines feature anomaly scores into a global message score.The global score indicates whether the account has likely been compromised.
- Feature models: Feature models store feature values, their message counts, and the total training-message count N.Mandatory models always have one value per message, whereas optional models may have multiple or no values.
- Training: Time-of-day profiles average each hour with adjacent hours to reduce false anomalies caused by discrete one-hour intervals.This adjustment treats messages near a user’s usual posting hours more smoothly.
- Training: Training is aborted for message streams shorter than S = 10, and messages from those users are not evaluated.The system therefore assumes that a sufficiently long history is available for reliable behavioral profiling.
- Final anomaly score: The final anomaly score is a weighted sum learned with Sequential Minimal Optimization, and a message violates the profile when the score exceeds a threshold.Feature weights can differ across social networks.
5 Behavioral Profile Stability
The evaluation tests whether popular accounts maintain stable behavioral profiles and compares their violations with regular-account behavior. Most popular accounts are highly consistent, but multi-actor accounts can produce many false positives, while regular accounts require campaign-level grouping because individual violations are common.
- 5.1 Popular Accounts: Most popular accounts showed little behavioral variability, allowing COMPA to protect them while reliably detecting compromises without fearing false positives.The threshold was tuned using four previously described high-profile incidents.
- 5.1 Popular Accounts: 47 out of 100 messages from The Guardian’s Twitter account would have been flagged as malicious, illustrating the false-positive risk of highly variable accounts.The authors suspect these accounts are managed by multiple actors with different client and editing preferences.
- 5.2 Regular Accounts: 2,606 of 64,368 messages violated regular users’ profiles, producing a 4% base false discovery rate.Because this rate is too high for practical individual-account detection, COMPA groups accounts by message similarity into large-scale campaigns.
- 5.2 Regular Accounts: Message similarity supplements profile violations because a single deviation may reflect normal behavioral change rather than a compromise.The approach assumes attackers spread malicious messages to a larger victim population.
6 Detecting Large-scale Social Network Compromises
COMPA detects large-scale compromises by grouping similar messages and checking them against account behavioral profiles. Its design addresses false positives from coincidental similarities and bulk applications, while remaining extensible to additional similarity measures.
- Message grouping: COMPA can group messages either before profile-violation analysis or by clustering only messages that violate behavioral profiles.The latter approach supports grouping by social-graph connections or user demographics while reducing the messages requiring examination.
- Message grouping: Expanding shortened URLs would improve grouping of messages linking to the same page, but experiments omitted expansion because daily URL volume exceeded service request limits.Twitter produced several million URLs per day, most of them shortened.
- Message grouping: COMPA’s two message-similarity measures perform well in practice and can be extended with additional similarity measures.The authors do not claim that the selected measures are the only possible grouping methods.
- Compromise detection: The system groups similar messages generated within an observation interval and labels a group suspicious when profile-violating messages exceed a size-dependent threshold.The threshold is higher for small groups because coincidental similarities could otherwise produce false positives.
- Compromise detection: COMPA requires at least 10 percent of messages in large groups to violate behavioral profiles before flagging the group’s users as compromised.The reported parameters k = −0.005 and d = 0.82 were empirically selected, and small modifications did not affect result quality.
- Bulk applications: The system distinguishes bulk applications from regular clients and excludes suspicious groups attributed to popular bulk applications.It estimates popularity from prior distinct-account usage and an age factor; a score above 1 million qualifies as popular.
7 Evaluation
COMPA was evaluated on Twitter and Facebook datasets, real-world high-profile compromises, and a staged compromise. It detected compromised accounts with few false positives and identified behavioral deviations in case studies.
- Evaluation setup: COMPA processed Twitter and Facebook data using behavioral profiles, message similarity, and grouping to identify compromised accounts.The evaluation included real-time Twitter collection, a crawled Facebook dataset, and profile-based comparison of new messages.
- Twitter evaluation: 343,229 Twitter accounts were identified as compromised from 9,362 groups, with 12,382 accounts estimated as false positives.The text-similarity evaluation produced 374,920 groups overall; 9,362 were flagged as compromised.
- Twitter evaluation: 4% of the 9,362 Twitter groups flagged as compromised were labeled false positives, equivalent to 3.6% of flagged users.The false-negative analysis estimated roughly 4% false negatives for compromised regular Twitter accounts.
- Facebook evaluation: 11,499 Facebook accounts were flagged as compromised, while 412 users in legitimate groups were incorrectly classified, producing 3.6% false positives.COMPA generated 206,876 profiles in 48,586 groups and flagged 671 groups as compromised.
- High-profile case studies: COMPA detected behavioral deviations in high-profile cases through features including posting application, links, hashtags, mentions, timing, and language.The AP, Fox News Politics, Skype, and Yahoo News cases showed deviations from account-specific behavior; timing and language sometimes remained consistent.
- High-profile case studies: COMPA did not raise an alert for the staged Chipotle compromise because the account’s observed behavior largely matched its behavioral profile.The account was almost exclusively managed through SocialEngage, making a stealthy compromise require compromising that client account as well.
8 Limitations
COMPA faces evasion attempts, but attackers must either reproduce victims’ behavioral profiles or avoid its similarity measures. The former demands substantial resources, while the latter can be addressed with broader similarity measures.
- Profile imitation requires attackers to invest significant time and computational resources gathering victims’ behavioral information.Social-network anti-crawling mechanisms further slow this information-gathering process.
- Social-network mechanisms that prevent automated crawling make behavioral-profile reconstruction more difficult for attackers.
- Attackers may evade COMPA’s similarity measures with messages that violate behavioral profiles, but additional measures can counter this strategy.One proposed extension compares landing pages instead of message URLs.
9 Related Work
Related work studies social-network populations, spam propagation, fake accounts, malicious messages, worms, and botnet access. These approaches generally address adjacent abuse categories, whereas COMPA distinguishes compromised accounts and can detect isolated hijacked accounts.
- Prior studies characterized Facebook and Twitter users and examined differences between Twitter and traditional social networks.
- Research on spam examined how popular topics and trending hashtags help spammers reach victims.
- Early systems detected fake accounts and spam using profile structure, URLs, message similarity, and other spam-oriented features.These techniques were designed around accounts created to spread malicious content rather than legitimate accounts later compromised.
- Some approaches infer compromise from benign contacts or prior spam, but can misclassify legitimate accounts and require spam involvement.COMPA is described as detecting compromised accounts even when they are not involved in spam campaigns.
- Other systems detect malicious URLs, redirection chains, worms, botnet access, or synchronized activity without clearly distinguishing compromised from legitimate accounts.COMPA additionally identifies hijacked accounts used in isolation and can detect scams or worms without malicious URLs.
10 Conclusions
COMPA models social-network users’ behavior and detects sudden changes through anomaly detection. The paper reports reliable detection of high-profile compromises and detection of regular-account compromises by aggregating similar malicious messages.
- COMPA uses statistical models to characterize user behavior and anomaly detection to identify sudden behavioral changes.
- COMPA reliably detects compromises affecting high-profile social-network accounts.
- COMPA detects compromises of more variable regular accounts by aggregating similar malicious messages.