Source-linked AI summary
A survey of location inference techniques on Twitter
Oluwaseun Ajao, Jun Hong, Weiru Liu
TL;DR
Inferring Twitter users’ locations is valuable for crisis response, public-health monitoring, advertising, and profiling, but remains difficult because available data can be noisy, incomplete, or misleading. The paper surveys location-inference techniques and finds that algorithmic refinements and additional spatial features have improved accuracy and granularity over time.
Problem
Accurately identifying where a Twitter message originated or where its author is located remains challenging despite the value of location information for crisis response, public health, advertising, and profiling.
Method
The paper surveys Twitter location-inference techniques spanning natural language processing, machine learning, probabilistic methods, gazetteers, location databases, and diverse spatial features.
Results
60% accuracy within 10km was achieved in later work, compared with 51% accuracy over a 160km radius in earlier work, alongside broader improvements in accuracy and granularity.
Takeaways & Limitations
Improved algorithms and expanded spatial features have enabled increasingly fine-grained Twitter location inference, from country and city levels toward postal-code-level estimation.
Takeaways & Limitations
Gazetteer-based methods can be slow and may miss unconventional or non-matching location names, producing false negatives.
Abstract
from arXiv · showhide
The increasing popularity of the social networking service, Twitter, has made it more involved in day-to-day communications, strengthening social relationships and information dissemination. Conversations on Twitter are now being explored as indicators within early warning systems to alert of imminent natural disasters such earthquakes and aid prompt emergency responses to crime. Producers are privileged to have limitless access to market perception from consumer comments on social media and microblogs. Targeted advertising can be made more effective based on user profile information such as demography, interests and location. While these applications have proven beneficial, the ability to effectively infer the location of Twitter users has even more immense value. However, accurately identifying where a message originated from or author's location remains a challenge thus essentially driving research in that regard. In this paper, we survey a range of techniques applied to infer the location of Twitter users from inception to state-of-the-art. We find significant improvements over time in the granularity levels and better accuracy with results driven by refinements to algorithms and inclusion of more spatial features.
1. Introduction
Twitter location inference matters for crisis response, profiling, and advertising, but identifying users’ locations is difficult because Twitter data are often incomplete, informal, and unstructured. The survey reviews diverse techniques and reports improving accuracy and spatial granularity over time.
- Twitter’s brevity, abbreviations, typographical errors, emoticons, irony, sarcasm, and hashtags create noisy text for location inference.
- Location inference can support cyberbullying investigations, disease and disaster detection, crisis management, and targeted advertising.
- Researchers use metadata, tweet content, social relationships, account information, hybrid feature sets, and location-based social networks as spatial indicators.
- Methods include natural language processing, machine learning, probabilistic techniques, gazetteers, and location databases.
- Results have reached average error distances below 1 km while the survey identifies itself as the first review of techniques in this field.
2. Types of location on Twitter
Twitter location inference distinguishes several meanings of location, including places mentioned in messages, contextual locations, current locations, and user location profiles.
- Earlier studies sometimes treated a tweet’s geotagged location as the user’s location without distinguishing current location from home residence.
- Four location types are identified: directly mentioned locations, focused locations described by message context, current locations, and location profiles.
3. Spatial features and indicators
Twitter location inference combines message context, links, social networks, account profiles, geotags, geosocial data, and metadata. These indicators differ in coverage, precision, and susceptibility to misleading or missing information.
- 3.1. Message Context: Message text is central to location inference, but abbreviations and other unstructured writing make extraction difficult.
- 3.1. Message Context: URL domain and server IP information can support country-level inference, while combining message content with additional features produces more robust outputs.
- 3.2. Social Networks: Follower relationships, reciprocal ties, message exchanges, and name mentions provide evidence about users’ home residences and shared cities.
- 3.3. User Profiles: Account fields and websites can provide location clues, but free-text expressions and geographically mismatched web hosting can mislead inference.
- 3.4. Geotags: GPS metadata can pinpoint geographic coordinates, but fewer than 0.5% of Twitter users enable location services.
- 3.5. Location-Based Social Networks: Foursquare and Yelp provide geographic directories and check-in information that can connect users or content with places of interest.
- 3.6. Time Zones: Tweet timestamps and time zones support inference at least to country-level granularity when message text contains sparse location information.
- Webpage snippets and Wikipedia place information have been used to supplement sparse tweets when locating points of interest.
4. Methods of inferring locations on Twitter
Location inference methods span natural language processing, gazetteers, machine learning, probabilistic models, and social-network approaches. Their effectiveness depends on handling noisy language, ambiguous place names, and unconventional location expressions.
- Methods draw from machine learning, statistics, probability, natural language processing, geographical information systems, and surveying.
- Challenges: Twitter’s informal language creates challenges involving emoticons, sarcasm, hashtags, abbreviations, typographical errors, and contextual meaning.Place names can also be ambiguous, requiring toponym resolution.
- Natural Language Processing: Natural language processing uses named entity recognition, part-of-speech tagging, probabilistic tools, and classifiers such as CRF.Retrained StanfordNER achieved an F-measure of 0.9 on disaster-related tweets.
- Gazetteers: Gazetteers and geographic databases resolve place names, but abbreviation mismatches and toponym ambiguity can produce false negatives.Gazetteers are easy to implement and do not require training data, but processing can be slow on very large databases.
- Machine Learning: Machine-learning and probabilistic approaches include k-nearest neighbour, fuzzy matching, Naive Bayes, probabilistic clustering, and Markov models.
- Network and Hybrid Methods: Other approaches infer location from friends’ relationships, graph-based label propagation, geographic and topic models, location-based services, and language models.
5. Tweet Gathering and Analysis
Twitter data can be gathered through online retrieval and real-time streaming interfaces, enabling collection of both historical user timelines and live updates. Studies have used datasets ranging from tens of thousands to hundreds of millions of tweets over periods of weeks to months.
- The Twitter REST API retrieves public tweets and user timelines, while the streaming API provides real-time updates.The REST API can backtrack a user’s timeline to collect up to 3,200 recent tweets.
- Tweet corpora range from under 62,000 tweets to 615 million tweets, with collection periods usually spanning weeks to months.
5.2. Results and Metrics
Location-inference results improved over time in both accuracy and spatial granularity, driven by algorithm refinements and additional spatial features. Evaluation uses accuracy ranges, error distances, cross-validation, precision, recall, and F-measure.
- Accuracy and granularity improved over time as algorithms were refined and more spatial features were included.Finer targets such as postal codes require more accurate prediction than country-level inference.
- Evaluation metrics include accuracy within a specified radius, error distance, average error distance, median error distance, precision, recall, and F-measure.K-fold cross-validation is used to compare methods against baselines.
- 60% accuracy within 10 km was reported in later work, compared with 51% accuracy within a 160 km radius in earlier work.
- Table 2 documents a trend toward finer-grained location inference over the past five years.
6. Conclusion and Future Work
Location inference supports applications including marketing, public-health surveillance, emergency response, and addressing cyberbullying. The survey concludes that future work should improve location granularity while recognizing privacy and data-availability constraints.
- Location inference can support marketing, consumer profiling, and location-aware analysis of social-media content.
- More precise location inference could strengthen epidemic surveillance by locating patterns in disease-symptom mentions.
- Inferring locations from tweet content, relationships, and account information raises privacy concerns for anonymous users facing stalking or cyberbullying.
- Mobile-device battery limitations can reduce how often users enable GPS location services.Users may enable location services during disasters to support emergency rescue efforts.
- Future work is expected to improve location granularity and reduce the friend-network and information required for accurate inference.