Source-linked AI summary
TweetNLP: Cutting-Edge Natural Language Processing for Social Media
Jose Camacho-Collados, Kiamehr Rezaee, Talayeh Riahi, Asahi Ushio, Daniel Loureiro, Dimosthenis Antypas, Joanne Boisson, Luis Espinosa-Anke, Fangyu Liu, Eugenio Martínez-Cámara, Gonzalo Medina, Thomas Buhrmann, Leonardo Neves, Francesco Barbieri
TL;DR
NLP for social media must address noisy, multilingual, fast-changing text that can undermine traditional task performance. TweetNLP packages Twitter-specialized, relatively lightweight Transformer models with a Python library, online demo, and tutorial across many tasks. Its public tools support model use and inspection, while the authors note ongoing scope and bias limitations.
Problem
Social-media NLP must account for noise, multilinguality, slang, immediacy, limited context, and platform-specific constraints that make traditional NLP tasks suboptimal.
Method
TweetNLP continually pre-trains base-sized RoBERTa- and XLM-R-based models on Twitter data, fine-tunes them for social-media tasks, and distributes them through a Python library and online demo.
Results
TweetNLP provides a broad, accessible platform spanning popular social-media NLP tasks, with models designed to run on standard computers or free cloud services.
Takeaways & Limitations
The Python library, online demo, and tutorial enable both programmers and non-programmers to use and inspect Twitter NLP models.
Takeaways & Limitations
Models trained on social-media data may amplify existing biases, and future work targets additional platforms, languages, and tasks.
Abstract
from arXiv · showhide
In this paper we present TweetNLP, an integrated platform for Natural Language Processing (NLP) in social media. TweetNLP supports a diverse set of NLP tasks, including generic focus areas such as sentiment analysis and named entity recognition, as well as social media-specific tasks such as emoji prediction and offensive language identification. Task-specific systems are powered by reasonably-sized Transformer-based language models specialized on social media text (in particular, Twitter) which can be run without the need for dedicated hardware or cloud services. The main contributions of TweetNLP are: (1) an integrated Python library for a modern toolkit supporting social media analysis using our various task-specific models adapted to the social domain; (2) an interactive online demo for codeless experimentation using our models; and (3) a tutorial covering a wide variety of typical social media applications.
1 Introduction
Social media provides valuable behavioral data, but its noisy and fast-changing communication creates challenges for NLP. TweetNLP addresses this gap with Twitter-specialized models, accessible tools, and real-time analysis.
- Social media text includes noise, multilinguality, slang, immediacy, limited context, length restrictions, and emojis that standard NLP must handle.
- TweetNLP integrates Twitter-trained Transformer models fine-tuned for social-media NLP tasks.
- The platform provides a Python API, an interactive online demo, and real-time Twitter analysis.
- Over 15M downloads made TweetNLP’s sentiment model the most downloaded model in Hugging Face’s model hub in January 2022.
2 Related Work
Existing NLP libraries are broad but generally not specialized for social media, while social-media-specific tools have focused mainly on lower-level linguistic tasks. TweetNLP is positioned as a broader Twitter-oriented alternative.
- General-purpose libraries such as CoreNLP, Stanza, spaCy, and Hugging Face Transformers are not specialized for social media or Twitter.
- Earlier social-media libraries mainly addressed tokenization, part-of-speech tagging, and dependency parsing.
- TweebankNLP provides Twitter-specific models for tokenization, lemmatization, NER, part-of-speech tagging, and dependency parsing.
3 Models and Functionalities
TweetNLP combines Twitter-adapted Transformer models, task-specific fine-tuning, and embeddings across a broad set of social-media NLP functionalities. Its models use relatively small base architectures intended for standard hardware and free cloud tiers.
- 3.1 Language models: TweetNLP uses RoBERTa and XLM-R checkpoints that are continually pre-trained on Twitter-specific corpora.
- 3.1 Language models: RoBERTa-base and XLM-R-base architectures support efficient execution on standard hardware and free cloud-computing tiers, including without GPU support.
- 3.1 Language models: The model family includes TweetEval, continually updated TimeLMs, and multilingual XLM-T trained on over thirty languages.
- 3.2 Supported tasks: TweetNLP fine-tunes its models for classification tasks and uses T-NER for named entity recognition.
- 3.2 Supported tasks: Supported tasks include sentiment, emotion, emoji, irony, hate-speech, offensive-language, stance, topic classification, and named entity recognition.
- 3.2 Supported tasks: Sentiment analysis predicts positive, neutral, or negative labels, while topic classification assigns one or more of 19 topics.
- 3.3 Embeddings: TweetNLP also releases word and tweet embeddings for applications including similarity analysis and tweet retrieval.
- 3.2 Supported tasks: Topic-classification and named-entity-recognition datasets were internally annotated and were still being prepared for public release.
4 TweetNLP Python library
The TweetNLP Python library provides pip-installable access to task-specific models through simple loading and prediction functions. It also supports custom fine-tuned models.
- TweetNLP is distributed through PyPI and can be installed with pip.
- The library loads task-specific models in the background and exposes predefined functions for predictions on tweets or sentences.
- Users can load sentiment, tweet-embedding, and masked-language models through the same simple API pattern.
- Custom loading of existing fine-tuned language models not included in TweetNLP is also supported.
5 Tutorials
TweetNLP provides instructive Google Colab tutorials that guide users through social-media NLP workflows, from using specialized models to retrieving data, fine-tuning, embeddings, and temporal language models.
- Tutorial platform: The tutorials provide starter code and examples for users with varying experience in NLP and social-media processing.They expose the underlying Python code alongside the TweetNLP library.
- Tutorials: The introduction tutorial demonstrates specialized social-media models across tasks including sentiment analysis and named entity recognition.
- Data collection: The data tutorial explains Twitter API interaction and retrieving tweets using hashtags or keywords.
- Model adaptation: The custom fine-tuning tutorial covers adapting language models to specific tasks and evaluating them on TweetEval.It uses TweetEval data and its unified format.
- Representations: The word-embeddings tutorial covers training embeddings on custom data and obtaining similarity scores or nearest-neighbor words.
- Temporal models: The language-models-over-time tutorial uses TimeLMs to access models trained during short periods from 2019 onward.
- Representations: The tweet-embeddings tutorial shows how vector representations support tweet similarity and retrieval.
6 Demo
TweetNLP offers a web-based demo for codeless experimentation with its social-media models. Users can analyze tweets and hashtags, predict masked words, compare tweet similarity, and identify named entities.
- Web demo: The comprehensive web demo integrates TweetNLP models so users can test them and obtain predictions without writing code.
- Tweet classification: Tweet classification accepts sentences, tweets, or tweet URLs and displays prediction confidence, including English tasks and multilingual sentiment analysis.
- Hashtag analysis: Hashtag analysis retrieves tweets for a chosen keyword, date interval, task, and language, then computes aggregated results.
- Word prediction: Word prediction ranks likely replacements for a masked word using a masked language model.
- Tweet similarity: Tweet similarity displays a 0-100 cosine similarity score for two short text pieces using the default tweet-embedding model.
- Named Entity Recognition: Named Entity Recognition locates named entities in a tweet or sentence and infers their types.
7 Evaluation
TweetNLP is evaluated across nine Twitter-based tasks using default Twitter-specific language models and established comparison systems. Results show strong cross-task performance, multilingual gains from in-domain training, and mixed but competitive embedding behavior on standard benchmarks.
- Experimental setting: Evaluation uses TweetEval datasets for tweet classification tasks except topic classification, alongside the train/validation/test splits described for the supported tasks.
- Experimental setting: Default models use TimeLMs-21 for English and XLM-T for other languages and multilingual tasks, fine-tuned to each corresponding task.
- Experimental setting: Comparison systems include frequency-based SVM, fastText, BiLSTM, RoBERTa variants, the original TweetEval model, and BERTweet.
- Results: The default TimeLMs-21 model achieves the overall results on most of the nine Twitter-based tasks, especially versus a comparable general-purpose RoBERTa model.
- Caveat: BERTweet’s Irony result overlaps with its pre-training corpus and the Irony dataset, which was constructed using distant supervision.
- Results: XLM-T achieves the best overall multilingual sentiment-analysis results in all evaluated languages against XLM-R and fastText.
- Results: 0.77 (WS-Sim), 0.72 (WS-Rel), 0.69 (SemEval), and 0.79 (MEN) are the Spearman correlations for the Twitter word embeddings, which outperform standard fastText on two datasets.
- Results: Tweet embeddings significantly outperform all-mpnet-base-v2 on tweet-reply retrieval, while all-mpnet-base-v2 performs best on the STS-Benchmark.
8 Conclusion and Future Work
TweetNLP is an all-round social-media NLP platform powered by relatively lightweight Twitter-trained models adapted to multiple tasks. It provides an online demo, Python library, and tutorial, while future work targets new tasks, datasets, platforms, and languages.
- TweetNLP provides relatively lightweight language models trained on Twitter and adapted to popular social-media NLP tasks.
- The platform combines an online demo, Python library, and tutorial for users with different levels of expertise.
- TweetNLP supports inspection by non-programmers to help identify harmful biases or errors and improve models in the future.
- Future work will add models and tasks, develop social-media datasets, extend beyond tweet classification, and support additional platforms and languages.
9 Impact Statement
TweetNLP uses social-media data, particularly Twitter, while addressing privacy and content risks through anonymization and content removal efforts. The authors caution that models trained on this data may amplify existing platform biases.
- Models trained on social-media data may amplify existing biases present on the platform.
- Dataset-related tweets were anonymized with URLs removed, and creators often tried to remove offensive or harmful content.
A Languages supported
The sentiment-analysis demo, including hashtag analysis, supports English plus a broad set of additional languages. These languages use an XLM-T multilingual model and the Twitter API.
- The sentiment-analysis demo supports English and numerous additional languages, including Arabic, Chinese, French, German, Hindi, Japanese, Korean, Persian, and Vietnamese.
- The supported languages use both an XLM-T multilingual model and the Twitter API.