Harm Is Behavioural, Moderation Is Lexical: How Language and Culture Decide Whom Automated Safety Systems Protect and Whom They Silence

1 Introduction

• Language and culture shape how speech is read: a term or teasing style can signal solidarity or closeness in one community but appear hostile to an outsider.The paper notes that systems have seen the most examples of harm in the languages of their builders.Source: §1 · p. 1
• The paper argues that over-flagging and under-protection share a lexical root and should be evaluated together, while noting that the link across communities is supported directly in only a few places.The evidence for dialect penalties and low-resource-language under-protection largely comes from different populations.Source: §1 · p. 2
• The paper argues that harmful behaviours may be more stable across languages than the words expressing them, and illustrates the distinction with published test cases.It states the limits of this argument and presents it as a claim to evaluate.Source: §1 · p. 2
• The paper organises published evidence into seven failure modes in which surface-form decisions diverge from decisions based on what a message does.The taxonomy is presented as a distinct contribution.Source: §1 · p. 2
• The proposed protocol makes the best-to-worst-served language-community gap a headline result and specifies how to report it and size test sets.The paper demonstrates the protocol on five open systems and public test suites in eleven languages.Source: §1 · p. 2
• The empirical study tests whether existing systems decide harm from form, but does not test whether behaviour-based recognition would serve language communities more evenly.That second claim remains a hypothesis because the required test sets do not yet exist for the harms in question.Source: §1 · p. 2

2 Two failures with one cause

• Both over-flagging and under-protection stem from judging surface features rather than what a message does.Systems penalize ordinary speech that resembles abuse and miss harmful behaviour expressed without marked vocabulary or in less-trained languages, dialects, or scripts.Source: §2 · pp. 2–3
• Lowering sensitivity to reduce false positives for a dialect can also reduce protection for its speakers.Conversely, raising sensitivity in a low-resource language without enough data to distinguish harm from ordinary speech can silence more speakers.Source: §2 · p. 3
• African American English is a revealing case: lexical systems can misread a dialect of the field’s best-resourced language, not just languages lacking data.The paper argues that the issue also concerns what training data teaches systems to treat as harmful.Source: §2 · p. 3

3.1 Dialect and identity are penalised

• Tweets in African American English were up to two times more likely to be labelled offensive, and annotator identities and beliefs also shaped toxicity judgments (Sap et al., 2022).Classifiers trained on five annotated Twitter datasets also predicted African American English as abusive more often than Standard American English (Davidson, Bhattacharya and Weber, 2019).Source: §3.1 · p. 3

3.2 Most languages are barely served

• Multilingual models are trained on far more English than other languages, while native-speaker resources show performance depends substantially on language (Nicholas and Bhatia, 2023; Muhammad et al., 2025).Code-switched text and languages written in another script remain poorly represented in safety evaluations.Source: §3.2 · pp. 3–4

3.3 Safety behaviour does not transfer across languages

• Translating unsafe English prompts into low-resource languages raised safeguard bypass rates from under 1% to 79%, and independent evaluations report multilingual and implicit-harm weaknesses (Yong, Menghini and Bach, 2023; Ng et al., 2026).Other studies report greater harmful-content generation in low-resource languages and unsafe behaviour on code-switched queries.Source: §3.3 · p. 4

3.4 The consequences are not hypothetical

• Human-rights assessments reported failures to address harmful content affecting communities in Myanmar and northern Ethiopia, whose languages systems served least (BSR, 2018; Amnesty International, 2023).Table 1 reproduces contrastive hateful and non-hateful pairs from the HateCheck suite (Röttger et al., 2021).Source: §3.4 · p. 4, Table 1 · p. 5

4 Why behaviour travels better than vocabulary

• Harmful online conduct often unfolds through acts such as trust-building, secrecy, isolation, and compliance testing rather than distinctive harmful words.Models of grooming describe overlapping strategies, including praise and small talk, that keyword lists would not identify as harmful.Source: §4 · p. 5
• Harmful functions such as requesting secrecy, extracting payment, isolating, and threatening recur across languages, while the words expressing them do not.The paper argues that recognizing those functions in conversation offers a target that is in principle shared across languages, not that resources are unnecessary.Source: §4 · p. 5
• In HateCheck, models often misclassified reclaimed-slur cases while handling non-hateful profanity almost perfectly, suggesting scores followed word choice rather than function (Röttger et al., 2021).The test suite contrasts hateful and non-hateful cases that share a slur or profanity but differ in what it is doing.Source: §4 · pp. 5–6
• A constructed secrecy pair shows why context matters: identical words can perform grooming-related isolation or refer to a harmless surprise, depending on sender, recipient, and conversation.The paper reports no known public dataset containing such contrastive pairs for grooming, coercion, or scams in any language.Source: §4 · p. 6
• H1 tests whether systems rank harmful cases above harmless look-alikes across languages, comparing system performance with native-speaker balanced accuracy before choosing a threshold.A system falls short when the bootstrap interval for the difference lies entirely above zero; the paper says this is a conservative finding.Source: §4 · p. 6
• H2 predicts smaller cross-language discrimination differences for systems recognizing function; coupling instead diagnoses how false positives and misses are distributed across languages.A system can discriminate poorly while spreading errors evenly, so coupling alone does not show whether it reads function.Source: §4 · p. 6
• The paper cannot test whether function-based systems meet H1 or whether H2 holds because relevant test sets for grooming, coercion, and scams do not yet exist.Behavioural framing also does not remove the need for language resources, annotator judgments, or evaluation of privacy risks from analysing more conversation context.Source: §4 · pp. 6–7

5 A taxonomy of failure modes

• The seven failure modes describe distinct divergences between decisions based on surface form and decisions based on what a message does.The modes can co-occur, so speakers of under-served varieties may face both dialect penalties and blindness to harms expressed in unfamiliar forms.Source: §5 · p. 7

6 An evaluation protocol: make the gap the headline

• The protocol calls for native-authored parallel cases, contrastive examples by function, both error directions and threshold-free ranking, plus the best–worst gap at a stated operating point.It also recommends reporting the equal-service expected gap, its excess and permutation p-value, and the worst-served variety’s error rate.Source: §6 · p. 7

6.1 Measuring the gap

• With equal 10% error rates, ten varieties of 100 cases each still produce an expected 9.2-point best-minus-worst gap; at 400 cases each, it remains 4.6 points.The observed gap is an extreme-value statistic, so a league table can show a bottom variety even when service is equal.Source: §6.1 · p. 9
• The stratified percentile bootstrap fails for the gap: it never covers a true zero gap and covers a 10-point gap in only 64–76% of simulations.The nominal interval is 95%.Source: §6.1 · p. 9

6.2 A worked example on public data

• The demonstration applies five open systems to public functional test suites in eleven languages; it is a measurement example, not an audit of deployed products.The suites include harmless contrast cases and vocabulary-free implicit hate, while excluding abuse aimed at non-protected targets.Source: §6.2 · pp. 9–10
• 51 of 55 system-language pairs fall reliably short of native-speaker performance at separating implicit hate from harmless contrast cases.Without English, 46 of 50 combinations still fall short; the four exceptions include three French guard-model results, where annotators also disagreed on many cases.Source: Table 5 · p. 11, §6.2 · pp. 11–12
• The larger guard models improve discrimination in nine languages but barely move in Arabic and romanised Hindi, widening the cross-language spread.The spread rises from 0.20 for the small Qwen3Guard to 0.29 for the 8B model and 0.33 for PolyGuard.Source: §6.2 · p. 13
• PolyGuard over-flags Arabic most (69.0% of harmless contrast cases) and misses 15.9% of implicit hate, while romanised Hindi guard models miss more: 72.1%, 77.1% and 40.0%.The Hindi lexicon flags nothing (0.0% false positives) alongside 100% missed implicit hate; this is distinct from the guard-model results.Source: §6.2 · p. 13
• The example uses synthetic, templated suites, covers no languages classed as low-resource by the paper, and tests each system at one version and operating point.The authors state these limits likely understate the disparities at issue; per-language differences also reflect how cases were written.Source: §6.2 · p. 14

For builders of safety systems

• Define harm by function, test it with contrastive pairs before deployment, and report per-variety errors in both directions alongside the gap.Source: §7 · p. 14
• Treat a false-positive fix as unfinished until its effect on that community’s protection is measured, and vice versa.Source: §7 · p. 14

For platforms deploying them

• Ask vendors for per-language and per-dialect error rates, route uncertain under-served-language cases to fluent human reviewers, and do not treat system silence as proof that nothing harmful occurred.Source: §7 · p. 14

For builders of safety systems

• Require risk assessments to report performance by language for the populations a service actually serves, and treat the inter-community gap as a measure of protection.Source: §7 · p. 14

For researchers and funders

• Fund native-speaker safety datasets beyond hate speech and build shared, community-owned evaluation sets that no single vendor controls.Source: §7 · pp. 14–15

• Apart from the worked example, the paper reports no new measurements; it tests H1 in one direction and does not test H2.Source: §8 · p. 15
• Most evidence concerns hate speech and English dialects; failure modes 6 and 7 beyond hate speech are argued from problem structure, not measured.Source: §8 · p. 15
• Section 6.1 assumes independent cases, while clustered test sets widen every interval.Source: §8 · p. 15
• Some cited studies describe systems as they were when studied, and the Table 1 pairs and Section 4 secrecy pair are test cases, not measurements.Source: §8 · p. 15

Code and data availability

• The worked example uses public test suites, a lexicon, frequency lists, and four openly licensed models; code, simulations, and per-case outputs are available in a reproduction package.Source: Statements · p. 15

Ethical considerations

• The worked example uses synthetic public test suites written or validated by native-speaking annotators, with no human subjects, user data, or deployed product.Source: Statements · p. 15

Generative AI usage

• Claude provided drafting and language-editing support and wrote and ran validation, test, and analysis code; the authors reviewed the work and take responsibility.Source: Statements · p. 15

• The founder, chief executive and chief technology officer, a co-founder and chief scientific officer, and a senior advisor are affiliated with Tuteliq AB and have an interest in adoption of the paper’s framing.Source: Conflict of interest · p. 15
• The worked example evaluates a keyword lexicon and four openly licensed models on public test suites, not a Tuteliq product or commercial service.Source: Conflict of interest · p. 15
• The authors propose that Tuteliq products should also be held to the evaluation protocol.Source: Conflict of interest · p. 15