Source-linked AI summary
Home Location Identification of Twitter Users
Jalal Mahmud, Jeffrey Nichols, Clemens Drews
TL;DR
The paper addresses sparse and unreliable Twitter location information by inferring users’ home locations from tweet content and tweeting behavior. It combines statistical and heuristic classifiers with external geographic knowledge in a hierarchical ensemble, and finds higher prediction performance than previous algorithms, with further gains after identifying traveling users.
Problem
Twitter location information is sparse: less than 1% of tweets are geo-tagged, and profile location fields are unreliable.
Method
The algorithm combines statistical and heuristic classifiers, external gazetteer knowledge, tweet content, tweeting behavior, and hierarchical prediction across geographic granularities.
Results
The evaluation shows higher location-prediction performance than previous algorithms, with best accuracies of 64% for cities, 66% for states, 78% for time zones, and 71% for regions.
Takeaways & Limitations
Identifying traveling users further improves accuracy to 68% for cities, 70% for states, 80% for time zones, and 73% for regions.
Takeaways & Limitations
Performance drops as the number of tweets per user decreases, because the classifiers have less information available.
Abstract
from arXiv · showhide
We present a new algorithm for inferring the home location of Twitter users at different granularities, including city, state, time zone or geographic region, using the content of users tweets and their tweeting behavior. Unlike existing approaches, our algorithm uses an ensemble of statistical and heuristic classifiers to predict locations and makes use of a geographic gazetteer dictionary to identify place-name entities. We find that a hierarchical classification approach, where time zone, state or geographic region is predicted first and city is predicted next, can improve prediction accuracy. We have also analyzed movement variations of Twitter users, built a classifier to predict whether a user was travelling in a certain period of time and use that to further improve the location detection accuracy. Experimental evidence suggests that our algorithm works well in practice and outperforms the best existing algorithms for predicting the home location of Twitter users.
1. INTRODUCTION
Twitter location data is sparse, motivating algorithms that infer users’ home locations from tweet content and behavior. This paper proposes hierarchical ensemble classifiers and reports higher accuracy than existing methods across multiple geographic granularities.
- Motivation: Less than 1% of tweets are geo-tagged, while profile location information is unreliable, making user-location inference necessary.Prior work found only 42% of users reported valid city-level profile locations, and 0.77% of tweets were geo-tagged.
- Approach: The paper predicts users’ home locations from tweet content, tweeting behavior, and external location knowledge such as gazetteers and location-based services.The approach uses explicit place references when available, but can still operate with reduced accuracy without them.
- Approach: The algorithm predicts location hierarchically, using time zone, state, or geographic region before city-level classification.The hierarchy is intended to support more granular location predictions in the future.
- Evaluation: 64% city, 66% state, 78% time-zone, and 71% region accuracies were achieved on 1.52 million tweets from 9551 users in the top 100 US cities.The evaluation also reports that the method outperformed Cheng et al.’s method for city-level home-location prediction.
- Movement analysis: 68% city, 70% state, 80% time-zone, and 73% region accuracies were achieved after eliminating users identified as traveling.The paper analyzes movement variation, detects traveling users, and uses that result to improve location prediction accuracy.
2. RELATED WORK
Prior work estimates Twitter users’ locations from tweet content, social connections, profiles, geotagged activity, and other location-based social media. This paper builds on those approaches while adding multi-granularity classification and time-zone prediction.
- Content-based methods commonly build probabilistic models from tweet content to estimate users’ locations.
- City-level estimation is more difficult than state- or country-level estimation because datasets typically contain many more cities.
- Earlier city-level methods identified local words or used conversational relationships, but several reported relaxed accuracy within 100 miles of the actual city.
- Kinsela et al. used an exact accuracy metric for multiple granularities, unlike earlier city-level studies using approximate distance-based metrics.
- The paper extends prior work with an ensemble, hierarchical classification across granularities, time-zone prediction, and a comparison against Cheng et al.’s city-level method.
- Location inference also uses profiles, geotagged tweets, social networks, check-ins, and location extraction, but these approaches can depend on available geo-active friends or popular venues.
3. DATASET
The dataset comprises geotagged Twitter users discovered across the 100 most populous US cities, with up to 200 recent tweets collected per user. The authors use the discovery city as ground-truth home location and evaluate with cross-validation.
- Tweets were collected from bounding boxes covering the top 100 US cities, stopping after 100 unique users were obtained per location.
- The discovery city was assumed to be each user’s ground-truth home location, and the paper later discusses the validity of this assumption.
- The final dataset contains 1,524,522 tweets from 9551 users after private-profile accounts were removed.
- 100599 tweets (6.6%) came from Foursquare, while 289650 tweets (19%) contained gazetteer references to cities or states, including ambiguous matches.
- The entire dataset was divided into 90% training and 10% testing sets for 10-fold cross-validation.
4. LOCATION ESTIMATION – PROBLEM STATEMENT
The paper formulates user-location estimation as predicting a user’s location at a chosen granularity from tweets, tweet times, and optional external location knowledge. It evaluates city, time-zone, state, and geographic-region estimation.
- For user u and granularity g, the location L_g(u) is modeled as a function of tweet set S_u, tweet creation times T_u, and external knowledge E.
- External knowledge E may come from services such as Foursquare or dictionaries such as the USGS gazetteer, and is optional.
- The framework supports granularities including country, state, geographic region, time zone, city, street, and landmark.
- Tweet content S_u is mandatory for every granularity except time zone, which can be estimated using tweet times T_u alone.
5. LOCATION CLASSIFICATION APPROACHES
The location classifiers combine tweet-content features, place-name extraction, heuristic signals, and time-dependent tweeting behavior to predict locations at multiple granularities.
- Content-based Statistical Classifiers: Three statistical classifiers use words, hashtags, and gazetteer-identified city or state place names as alternative feature sources.Place names are matched against uni-, bi-, and tri-grams from tweets using the USGS gazetteer.
- Content-based Statistical Classifiers: Local terms are selected using location-specific frequency and conditional-probability thresholds, with K empirically set to 5%.The filtering process removes noisy terms and retains terms that discriminate among locations.
- Heuristic Classifiers: Heuristic classifiers estimate home locations from the frequency of mentioned places and users’ visit histories retrieved from Foursquare check-ins.The local-place heuristic targets city or state classification, while visit history applies at all granularities.
- Behavior-based Time Zone Classifier: Tweeting behavior provides time-zone features because activity patterns exhibit temporal offsets across time zones.The classifier uses normalized tweet counts in one-minute time slots, weighted by cross-time-zone variation.
6. ENSEMBLE OF LOCATION CLASSIFIERS
The paper combines statistical and heuristic location classifiers in an ensemble whose weights adapt to the information available for each user.
- Dynamic Weighting: The dynamically weighted ensemble combines statistical and heuristic classifiers according to each classifier’s instance-specific discriminative strength.The behavior-based time-zone classifier instead uses its classification confidence as the weight.
- Classification Strength: Classification strength decreases as a user’s matching location set contains more possible locations.For five matching cities, the example classification strength is 1/5 = 0.2.
- Ensemble Alternatives: The evaluation compares a combined statistical classifier, a statistical-classifier ensemble, and an ensemble including statistical and heuristic classifiers.Majority voting, AdaBoost, and dynamic weighting are also compared.
7. HIERARCHICAL ENSEMBLE OF CLASSIFIERS
A two-level hierarchy first predicts a broader geographic category and then predicts city within the selected category.
- Hierarchical Design: The hierarchical approach divides city prediction into smaller classification problems by using a high-level location classifier before city classification.The first level can represent time zone, state, or geographic region.
- Time-zone Hierarchy: Time-zone hierarchy uses four US time zones at the first level and trains separate city classifiers within each time zone.The first-level ensemble combines content-based and behavior-based classifiers.
- State Hierarchy: State hierarchy uses states as the first level and builds city classifiers for states containing more than one city in the dataset.Its first-level ensemble contains only content-based classifiers.
- Regional Hierarchies: Regional hierarchies use either four Census regions or ten standard Federal Regions before city classification.Both regional variants follow the basic state-hierarchy approach.
8. EXPERIMENTS
Experiments evaluate individual and ensemble classifiers, hierarchical prediction, comparison with prior work, explicit location references, and performance under reduced tweet availability.
- Individual Classifier Performance: The Place Name classifier performs best among individual classifiers, while sparse Foursquare URLs limit visit-history performance.Only 6.6% of dataset tweets contained Foursquare URLs, and some could not be resolved.
- Ensemble Classifier Performance: Statistical-classifier ensembles outperform a single classifier using the same features because feature categories have distinct discriminative abilities.The remaining experiments use the ensemble design selected from these comparisons.
- Time-zone Prediction: Weighting behavior-based time-zone features and reducing time-slot size improves performance, with results leveling off at one-minute slots.Combining the behavior-based classifier with content-based classifiers produces the best time-zone classification.
- Hierarchical City Prediction: All hierarchical city classifiers outperform the single-level ensemble, with the time-zone hierarchy performing best.Its advantage is attributed largely to more accurate time-zone prediction than state or region prediction.
- Comparison with Existing Approach: The proposed city predictor outperforms Cheng et al.’s algorithm across every tested number of cities in the comparison dataset.The comparison uses exact and distance-based relaxed accuracy metrics.
- Explicit Location References: City-location accuracy remains predictable without place names or Foursquare URLs, but explicit-reference removal reduces performance.The experiment uses only word and hashtag statistical classifiers.
- Real World Usage Issues: Performance drops as the number of tweets per user decreases because fewer tweets provide fewer predictive features.The tested setting is city classification with a time-zone-based hierarchical ensemble.
9. MOVEMENT AND LOCATION PREDICTION
The paper examines how user movement affects home-location prediction and evaluates a traveling-user pre-filter. Most users stayed near their detected location, while removing classified travelers improved location-prediction accuracy.
- Effect of Movement on Location Prediction: Most users stayed within 10 miles of the location where they were initially identified across their historical 200 tweets.This supports the assumption that the detected location represents users’ home location for most of the dataset.
- Effect of Movement on Location Prediction: Location prediction accuracy was higher for users who stayed near their initially identified location.
- Traveling-User Classification: Traveling users were labeled using a maximum inter-tweet geo-distance threshold above 100 miles.Words, place names, and hashtags were used as classifier features.
- Traveling-User Classification: SMO was selected from several WEKA classifiers and achieved 75% F1 with 10-fold cross-validation.
- Effect of Traveling-User Filtering: Eliminating users classified as traveling improved location-prediction accuracy.The improvement was reported for both the location-prediction results and hierarchical location-prediction results.
10. CONCLUSION
The paper concludes that a hierarchical ensemble algorithm predicts Twitter users’ home locations at multiple granularities and reports higher performance than previous algorithms. Future work targets finer-grained and message-level location prediction, improved travel detection, and incremental model updates.
- Conclusion: The paper presents a hierarchical ensemble algorithm for predicting Twitter users’ home locations at different granularities.The approach combines varied features, domain knowledge, statistical classification, and heuristic classification.
- Conclusion: Experimental performance demonstrates higher location-prediction performance than previous algorithms.
- Future Work: Future work includes predicting locations at smaller granularities such as the neighborhood level.The authors plan to incorporate additional domain knowledge, including a landmark database, and explore message-level location prediction.
- Future Work: The authors plan to improve traveling-user detection and use it to improve location-classifier accuracy.
- Future Work: The authors also plan to support incremental model updates for better integration.
16 http://poidirectory.com/poifiles/united_states/
This section contains bibliographic references to prior work on geolocation, Twitter, social-media analysis, classification, and assigning place names to geography-related web pages.
- Related Work: The references include studies on geolocating blogs and inferring locations from Twitter content.
- Related Work: Several cited works address home-location inference or location dynamics for Twitter users.
- Related Work: The bibliography includes research on event detection, epidemic tracking, and topical recommendation using Twitter.
- Classification Methods: The cited methods literature covers boosting and dynamically weighted ensemble neural networks for classification.
- Geographic Knowledge: One reference concerns assigning place names to geography-related web pages.