SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypes

1 Introduction

• Multilingual model safety and fairness evaluations have largely been restricted to English and Western viewpoints, despite models’ growing multilingual use.The paper notes that multilingual adversarial attacks can also break model safeguards.Source: §1 · p. 1
• English-only resources and translations can miss stereotypes salient in other language communities because simple translation loses cultural relevance.The paper argues that stereotype propagation and potential downstream harms are tied to geo-cultural factors.Source: §1 · p. 1
• SeeGULL Multilingual applies the SeeGULL approach to create a resource covering 20 languages, 23 regions, 25,861 stereotypes, and 1,190 identity groups.The dataset also records differences in offensiveness across global regions and is used to test model safeguards.Source: §1 · p. 2

2 Dataset Creation Methodology

• The methodology identifies relevant identity terms, generates candidate associations for them with a few-shot language-model prompt, and obtains socially situated human validation.Stereotypes are represented as associations between an identity and an attribute.Source: §2 · p. 2

2.1 Identifying Salient Identity Terms

• The study treats each language-country pair separately and scopes collection to national and local regional identities.Nationality demonyms are translated from 179 English terms, with gendered forms included where applicable; regional demonyms are sourced in the target language.Source: §2 · p. 2, §2.1 · p. 2

2.2 Generating Associations

• PaLM-2 generates candidate (identity, attribute) tuples in different languages from few-shot examples of stereotypes in existing datasets.The prompt asks the model to complete translated identity-attribute pairs, yielding candidate associations.Source: §2.2 · p. 2

2.3 Culturally Situated Human Annotations

• Annotators validate whether candidate tuples are stereotypes in their respective language and country, with three stereotype annotations per tuple.Each attribute also receives three globally situated in-language offensiveness ratings on a Likert scale.Source: §2.3 · pp. 2–3

3 Dataset: SeeGULL Multilingual

• 25,861 stereotypes span 20 languages and 23 countries, covering 1,190 identities and 7,159 attributes.The dataset records stereotype frequency and mean offensiveness ratings for its identity-attribute tuples.Source: §3 · p. 3
• Only 949 of SGM’s 9,251 nationality-based stereotypes overlap with English SeeGULL, while 10,292 regional-demonym stereotypes are newly introduced.Overall overlap is about 5%, based on exact matches of translated stereotypes.Source: §3 · p. 3
• 45.4% of 1,138 common Portuguese tuples were marked stereotypical by at least two annotators in Portugal, compared with 74.6% in Brazil.The paper reports this country-level difference as part of a consistent trend across Bengali, Spanish, and Portuguese.Source: §3 · p. 3, Figure 2 · p. 3
• In Portuguese, Spanish, Italian, and Dutch, gendered demonyms capture gendered and sometimes intersectional stereotypes.Examples include Bragantinos associated with “party-goers” and Bragantinas with “conservative” in Portugal.Source: §3 · p. 3

4.1 Offensive Stereotypes in SGM

• Across nationalities, Albania and Rwanda had some of the most offensive stereotypes associated with them, while Singapore and Canada had the least.Figure 3 maps average offensiveness by country, with darker red indicating higher average scores.Source: §4.1 · p. 3, Figure 3 · p. 4
• Figure 4 gives examples of highly offensive stereotypes for national and regional identities, with the country column identifying where annotation occurred.Source: §4.1 · p. 3, Figure 4 · p. 4
• About 22% of Italian and Swahili stereotypes were marked “Somewhat Offensive” or more, versus 1.83% in Hindi and 2.66% in Korean.These were respectively the languages with the most and least offensive stereotypes by this measure.Source: §4.1 · p. 4

4.2 Foundation Model Evaluations with SGM

• All evaluated models endorsed SGM stereotypes, but endorsement varied by language; English-translated queries missed a significant fraction of endorsements for three of four models.PaLM 2 had the highest endorsement rate and Mixtral the lowest.Source: Table 1 · p. 4, §4.2 · p. 4
• 4,600 evaluation questions each contain one stereotypical answer, with 100 samples drawn across each language, region, and demonym type.The example prompt is shown in Bengali alongside its English translation.Source: §4.2 · p. 4, Figure 5 · p. 4

• English-only resources and translations are insufficient for holistic safety evaluations of multilingual models.Source: §5 · p. 4
• Stereotypes may percolate unchecked into system output because coverage is lacking.Source: §5 · pp. 4–5
• Stereotype endorsements vary by language and place when aggregated across models.Source: Figure 6 · p. 5
• The conclusion calls for cross-cultural consideration of stereotypes, their offensiveness, and potential harms, and for exploring SGM in application-specific contexts.Source: §5 · p. 5

• The dataset covers 20 languages and annotations from 23 countries, with expansion constrained by model generation quality, annotator availability, and annotation cost.Source: Limitations · p. 5
• Recorded identity terms are limited to demonyms for nationalities and regions, and regional coverage does not include regions within every country.Source: Limitations · p. 5
• Other stereotypes at different geographic granularities and about other identities are not covered.The authors describe these scope choices as supporting reliable collection grounded in local knowledge.Source: Limitations · p. 5

Ethical Considerations

• The dataset does not capture all stereotypes about any identity or all geo-cultural identities.Users should not rely on it alone to categorize a model or its output as completely devoid of stereotypes.Source: Ethical Considerations · p. 5
• Dataset users should make careful choices based on their intended application and use the resource for evaluation.The paper identifies evaluation as the dataset’s intended use in its Data Card.Source: Ethical Considerations · p. 5
• The authors caution against unintentional or malicious misuse of the dataset.They note that its large number of stereotypes can help build model safeguards.Source: Ethical Considerations · p. 5

A.1 Dataset

• At least one of three annotations marking an association as stereotypical is sufficient for a tuple to enter the published dataset.Data filtering for use is left to users.Source: §A.1 · p. 8
• 19,543 of 25,861 annotated tuples have unique English translations, with differences arising from cross-country annotation and gendered identity terms translated identically.The dataset covers 23 language-country annotation combinations.Source: §A.1 · p. 8
• Table 3 organizes unique stereotypes and identities across nationality and regional axes; 492 nationality demonyms map to 179 countries.The data card and dataset are also made available online.Source: Table 3 · p. 8, §A.1 · p. 8

A.2 Related Stereotype Resources

• Many stereotype resources are limited to English or specific contexts, while English-only global resources can miss stereotypes salient in other languages.Simply translating resources can lose cultural relevance.Source: §A.2 · p. 8
• The cost of curation often constrains resources in both size and global coverage.Prior efforts have expanded coverage across languages and cultures, but these limits remain.Source: §A.2 · p. 8
• The paper addresses these challenges by leveraging social information captured and generated by multilingual models and globally situated annotations.The passage connects broad identity coverage with enabling models across modalities and languages to be safe and beneficial for all.Source: §A.2 · p. 8

A.3 Annotation Details

• Annotations use two tasks: identifying stereotype tuples and rating attribute offensiveness; annotators were recruited for target-language and regional context.The vendor handled recruitment and consent; each data point included at least one nonmale-identifying annotator, and pay exceeded prevalent market rates while respecting local minimum-wage laws.Source: §A.3 · pp. 8–9

A.3.1 Task: Stereotype Annotation / A.3.2 Task: Offensiveness Annotation

• Stereotype labels are Non-Stereotype, Stereotype, or Unsure; offensiveness uses a regional Likert rating from Not Offensive (-1) to Extremely Offensive (+4), plus Unsure.Annotators were proficient in English and the target language; ratings were collected from region-specific users, with three annotators of diverse gender identities per attribute for offensiveness.Source: §A.3.1 · p. 9, §A.3.2 · p. 9

A.4 Offensiveness

• Table 4 reports the per-region share rated somewhat offensive or higher, while Tables 5–6 list countries or regions with the highest and lowest mean offensiveness scores.The top- and bottom-score analyses exclude countries with fewer than 30 stereotypes.Source: §A.4 · pp. 9–10, Table 4 · p. 9, Table 5 · p. 10

A.5 Overlap with English SeeGULL

• About 5% of unique SeeGULL Multilingual stereotypes overlap with English SeeGULL: 2,370 of 25,861 tuples, representing 949 unique stereotypes.Table 7 reports overlap by language-country combination.Source: §A.5 · p. 10, Table 7 · p. 10

A.6 Stereotypes in a Language across Countries

• Annotations for the same language can differ across countries with distinct socio-cultural backgrounds.Table 8 quantifies annotation differences across two regions sharing a language.Source: §A.6 · p. 10, Table 8 · p. 11

A.7.1 Creating the Evaluation set

• The evaluation set balances country, language, and regional or international demonyms, then builds prompts in target languages and parallel English.Attributes are paired with three distracting demonyms without the same association; prompts combine translated instructions with demonyms and attributes from the dataset.Source: §A.7.1 · pp. 10–11

A.7.2 Multilingual capabilities of Models / A.7.3 Evaluation protocol

• The authors evaluate four foundation-model families across all included languages using one temperature-zero sample per model, with safety guardrails disabled where configurable.System versions were fixed and inference ran in January and February 2024; a heuristic parsed responses to identify endorsement of the stereotype.Source: §A.7.2 · p. 11, §A.7.3 · p. 11, Table 9 · p. 11

A.8 Regional Demonyms / A.9 Licenses of models and data used

• Regional demonyms were sourced online and manually validated; when no particular demonym was found, the default was “People from [name of the region].”The paper also states that SeeGULL data were released under CC-BY-4.0 and that model terms of use were followed.Source: §A.8 · p. 11, §A.9 · p. 12