NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis

Motivation

• Sentiment analysis research largely focuses on high-resource languages, while languages with limited data remain poorly represented.The introduction notes that lack of training data leaves many Nigerian languages underserved by digital technology.Source: §1 · p. 1

Main contributions

• Language-specific models and language-adaptive fine-tuning perform best on average in the benchmark experiments.The paper reports strong performance from pre-trained multilingual models; its best models use African-language pretraining or language-adaptive fine-tuning.Source: §1 · p. 1
• NaijaSenti is presented as the first publicly available large-scale manually annotated Twitter sentiment dataset for Hausa, Igbo, Yorùbá, and Nigerian Pidgin.The authors report around 30,000 annotated tweets in Hausa, Igbo, and Yorùbá, and Nigerian Pidgin.Source: §1 · p. 1
• The paper introduces a manually annotated sentiment lexicon for Hausa, Igbo, and Yorùbá.It also reports automatically developing translated emotion and sentiment lexicons in these languages.Source: §1 · pp. 1–2

SA for low-resource languages

• Earlier low-resource sentiment corpora were built both automatically, including through distant supervision, and through manual annotation by native speakers or experts.Examples include Go et al. (2009), Pak and Paroubek (2010), Wicaksono et al. (2014), Kwaik et al. (2020), and Refaee and Rieser (2014a).Source: §2 · p. 2

Nigerian languages

• Most prior work on Nigerian languages relied on automatically generated data, and only two datasets were freely available.The section cites Hausa, Igbo, Yorùbá, and Pidgin corpus and lexicon efforts, including Abubakar et al. (2021), Ogbuju and Onyesolu (2019), and Oyewusi et al. (2020).Source: §2 · p. 2
• NaijaSenti is described as the first publicly available large-scale manually annotated sentiment dataset for Hausa, Igbo, Yorùbá, and Nigerian Pidgin.Earlier work related to its collection combines emoji-based tweet extraction and translated sentiment lexicons for filtering (Al-Twairesh et al., 2017; Kwaik et al., 2020; Nakov et al., 2019).Source: §2 · p. 2

3.1. Data Collection

• Because the Twitter API does not support the target languages, the authors used heuristic approaches to crawl tweets.The API supports language parameters for 70 languages, with Amharic the only African language supported in the described setting.Source: §3.1 · pp. 2–3
• Tweets were collected with stopword queries combined with happy and sad emojis, and translated Affin lexicons filtered for sentiment-bearing tweets in Hausa, Igbo, and Yorùbá.The authors also crawled trending hashtags and frequently used language-specific news handles, limiting tweets per conversation to mitigate domination by popular conversations.Source: §3.1 · p. 3

3.2. Language Detection and Data Cleaning

• To address stopword overlap across languages, the authors used geographic locations and language-identification tools, including an n-gram detector for Pidgin.They cleaned tweets by removing retweets, duplicates, URLs, mentions, excess whitespace, and tweets under three words, and converted text to lowercase.Source: §3.2 · p. 3

4.1. Annotation Guidelines

• The guidelines distinguish positive, negative, neutral, mixed, and indeterminate tweets, with indeterminate also covering guesses and tweets in another non-code-mixed language.Three annotators per language labeled batches of 1,000; disagreements were adjudicated, and ambiguous tweets were excluded. The project also created sentiment lexicons and translated lexicons for Hausa, Igbo, and Yorùbá.Source: §4.1 · pp. 3–4, §4.2 · p. 4

4.3. Inter-Annotator Agreement

• Mixed sentiment had the lowest class-level agreement; adjusted 3-class agreement retained positive, negative, and neutral labels and changed invalid labels when two annotators agreed.For example, POS, POS, MIX became POS, while POS, POS, NEU stayed unchanged.Source: §4.3 · p. 5, Table 3 · p. 5
• Across 30 batches, overall agreement deteriorated; Igbo had the smallest drop, while only Yorùbá annotators maintained consistency above 0.5.The authors recommend monitoring agreement and using random quality checks during annotation.Source: §4.3 · p. 5, Figure 1 · p. 5

4.4. Human Evaluations

• Re-annotation of 200 random tweets by three different annotators used majority vote, and the authors report that human performance validates corpus reliability.The evaluation reports micro-F1 and Matthew’s correlation coefficient.Source: §4.4 · p. 5, Table 5 · p. 5

4.5. NaijaSenti Statistics

• Code-mixing was more prevalent in Igbo (43%) than Hausa (23%) or Yorùbá (19%); missing diacritics can make Yorùbá and Igbo tweets indeterminate.Pidgin had the most indeterminate cases because some tweets judged to be Nigerian English were labeled indeterminate.Source: §4.5 · p. 6, Table 6 · p. 6
• The benchmark uses negative, neutral, and positive tweets split 70%/10%/20% into TRAIN, DEV, and TEST.Source: §4.5 · p. 6, Table 7 · p. 6

5.1. Sentiment Classification Models

• Multilingual pretrained models were fine-tuned separately on each language, providing the baseline sentiment-classification setup.The compared models differ in language coverage and pretraining; for example, XLM-R includes Hausa, while AfriBERTa includes all four NaijaSenti languages.Source: §5.1 · pp. 6–7

5.2. Language Adaptive Fine-tuning

• Language-adaptive fine-tuning continued masked-language-model training on unlabeled target-language text, with vocabulary augmentation using 99 frequent wordpieces.The authors tested small Twitter collections and general-domain data, including released checkpoints.Source: §5.2 · p. 7

5.3. Multi-task Sentiment Classification

• A joint model aggregated training data from all four languages to classify sentiment across them, with the stated practical aim of supporting one deployable model.The authors also note that related-language knowledge may benefit transfer.Source: §5.3 · p. 7

5.4. Cross-Lingual Transfer

• The cross-lingual transfer experiments evaluated an English-trained SemEval-2017 classifier zero-shot on each Nigerian language and assessed target-language sample needs.They examined how many target-language tweets would match English-transfer performance and exceed a majority classifier’s accuracy.Source: §5.4 · p. 7

6.1. In-language Training

• At least 70% F1 is the minimum reported for PLMs, and AfriBERTa is the best baseline across languages; majority baselines score 16–45% weighted F1 and 33–56% Micro F1.Source: §6.1 · p. 7, Table 8 · p. 8
• General-domain LAFT improves mBERT and XLM-R by 2–3% on average for Hausa and Yorùbá and less than 1% for Igbo; Nigerian-Pidgin improves only with mBERT (+1.2%).Twitter-domain LAFT does not improve performance; the authors attribute this to the smaller, shorter Twitter corpus. AfriBERTa is competitive with or better than LAFT except for Nigerian-Pidgin.Source: §6.1 · pp. 7–8, Table 8 · p. 8
• A joint model has the same AfriBERTa performance and only a 0.3% drop with mDeBERTa compared with separate models.Source: §6.1 · p. 8

6.2. Zero-shot Cross-Lingual Transfer

• AfriBERTa gives the best overall zero-shot transfer from English SemEval; it beats the majority classifier by 41.8% for Hausa, 20.8% for Igbo, 4% for Nigerian-Pidgin, and 19.1% for Yorùbá.Nigerian-Pidgin has the best zero-shot performance across models, while Yorùbá results are lower at 36–43% F1.Source: §6.2 · p. 8, Table 9 · p. 8

6.3. Sample Efficiency in Transfer

• 38–40% F1 with 10 examples exceeds majority voting, and at least 1,000 examples reaches 70% F1; 100 examples exceed English zero-shot transfer.The authors identify potential to adapt a multitask model to other African languages with 100 or 1,000 annotated samples.Source: §6.3 · p. 8, Figure 2 · p. 9

• NaijaSenti is presented as the first publicly available large-scale manually annotated Twitter sentiment dataset for Hausa, Igbo, Nigerian-Pidgin, and Yorùbá.Source: §7 · p. 9
• The paper introduces a manually annotated sentiment lexicon for Hausa, Igbo, and Yorùbá and benchmarks Twitter sentiment models and transfer learning.Source: §7 · p. 9
• The reported results indicate that language-specific models and language-adaptive fine-tuning perform best on average.Source: §7 · p. 9
• Future work includes benchmarking the sentiment lexicon and extending NaijaSenti to other African languages through AfriSenti.Source: §7 · p. 9

• Hausa is a Chadic language with Eastern, Western, and Northern dialects; it uses Boko or Ajami scripts, and its tones are usually unmarked in everyday writing.The passage reports approximately 77 million speakers worldwide, mostly in West Africa.Source: A. Overview of the Languages · p. 12
• Igbo is a Benue-Congo language with about 30 dialects; it is tonal, and tone varies by dialect.The overview describes the O. nwu. orthography and typical subject-verb-object order.Source: A. Overview of the Languages · pp. 12–13
• Yorùbá is a Yoruboid language spoken mainly in Nigeria, with tones described as high, mid, and low.The passage places it in the Volta-Niger branch of the Niger-Congo family and notes speakers in Togo and Benin.Source: A. Overview of the Languages · p. 13
• Nigerian-Pidgin, also called Naija, is an English-based creole used as a lingua franca across Nigerian regions, without a standardized orthography.The overview estimates about 40 million first-language and 80 million second-language speakers.Source: A. Overview of the Languages · p. 13

• Models use batch size 32, sequence length 128, 20 epochs, and learning rate 5e −5, except XLM-R and RemBERT use 2e −5; experiments ran on Nvidia V100 and RTX 2080 GPUs.Source: B. Model Hyper-parameters for… · p. 13