BiasFinder: Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systems

1 INTRODUCTION

• Prior SA bias tests used a small number of short, handcrafted templates, which may miss biases across more complex scenarios.The introduction describes EEC as generating test cases from 11 handcrafted templates.Source: §1 · p. 2
• BiasFinder curates corpus text into templates, fills placeholders with values from different demographic classes, and checks whether paired mutants receive different sentiments.A fair system should predict the same sentiment for the pair, which differs only in words associated with the target characteristic.Source: §1 · p. 2
• NLP techniques including coreference resolution and named-entity recognition help identify references that must be changed consistently to keep a text coherent.The example requires updating a person’s name and its references together when changing gender-associated wording.Source: §1 · p. 2, Figure 1 · p. 2
• BiasFinder uncovered more bias-uncovering test cases than EEC and MT-NLP on IMDB and Twitter, while annotators judged its mutants more fluent than MT-NLP’s.The evaluation covered gender, occupation, and country-of-origin bias; the reported fluency comparison is with MT-NLP.Source: §1 · p. 2
• The framework is presented as a way to generate many realistic test cases across demographic characteristics, addressing prior work’s narrower template coverage.The introduction identifies gender, occupation, and country-of-origin as the characteristics examined.Source: §1 · p. 2

2.1 Metamorphic Testing for Fairness

• Fairness is tested by pairing texts that share non-protected features but differ in protected features; unequal predictions are bias-uncovering test cases.The EEC contains 8,640 sentences designed for gender and race bias; its scope is limited to those characteristics, and its templates are described as short and simplistic.Source: §2.1 · p. 3

2.2.1 Part-of-speech Tagging

• Part-of-speech tagging assigns each word in a text a grammatical category, such as noun, verb, or pronoun.In the example, “Maria” is tagged as a proper noun, while “She” and “him” are tagged as pronouns.Source: §2.2.1 · p. 3

2.2.2 Named Entity Recognition

• Named entity recognition identifies named entities and groups them into categories such as people, occupations, organizations, and locations.BiasFinder is mainly interested in person entities for gender and country-of-origin bias, and occupations for occupation bias.Source: §2.2.2 · p. 3

2.2.3 Coreference Resolution

• Coreference resolution groups expressions referring to the same entity into lists; unlike named entity recognition, it links mentions without identifying entity types.Figure 2 links “Maria” with “She” and “a friend” with “him.”Source: §2.2.2 · p. 3, §2.2.3 · p. 4, Figure 2 · p. 4

2.2.4 Dependency Parsing

• Dependency parsing assigns grammatical structure and encodes relationships between words, allowing phrase roots to be identified.In “That guy from Blade Runner,” “guy” is the root of the phrase.Source: §2.2.4 · p. 4, Figure 3 · p. 4

3 BIASFINDER

• BiasFinder takes texts and a sentiment analysis system through template generation, mutant generation, and failure detection to produce bias-uncovering test cases.Mutants from a shared template differ in the target characteristic but are expected to retain the same sentiment.Source: §3 · p. 4

3.1 Template Generation Engine

• The template engine identifies characteristic-related entities in input texts and replaces their references with placeholders to create bias-targeting templates.A template is a text unit with placeholders that can be filled to produce texts expected to have the same sentiment.Source: §3.1 · pp. 4–5

3.2 Mutant Generation Engine

• The mutant engine fills related placeholders using values from one class, keeping each generated text valid and class-associated.For example, it avoids mixing male- and female-related values within a mutant.Source: §3.2 · p. 5

3.3 Failure Detection Engine

• The failure detector flags mutants from the same template when different classes receive different sentiment predictions.Such pairs are output as bias-uncovering test cases.Source: §3.3 · p. 5

3.4 Instantiating BiasFinder to Different Biases

• BiasFinder customizes its three engines to investigate gender, occupation, and country-of-origin bias.Each target characteristic requires corresponding changes to template generation, mutant generation, and failure detection.Source: §3.4 · p. 5

4 GENDERBIASFINDER

• GenderBiasFinder flags differing sentiments for texts that vary only in gender-associated words, focusing on male and female classes.The approach is described as extensible to non-binary genders.Source: §4 · p. 5

4.1 Template Generation Engine

• Templates replace person names, gender pronouns, and gender nouns with placeholders after filtering coreference lists to avoid unsound templates.Pronoun placeholders distinguish grammatical forms such as subjective, objective, possessive, and reflexive pronouns.Source: §4.1 · p. 6

4.2 Mutant Generation Engine

• Mutants fill placeholders with values from one gender class; name values are globally gender-unique, USA-origin, and default to 30 names per class.Gender-specific pronouns and gender nouns also supply placeholder values.Source: §4.2 · p. 7

4.3 Failure Detection Engine

• The failure detector pairs same-template mutants when their gender classes and predicted sentiments differ.These pairs are the gender-bias test cases output by GenderBiasFinder.Source: §4.3 · p. 7

5.1 Occupation Bias

• Occupation templates remove adjectives from occupation phrases and replace determiners as needed to keep generated text semantically and grammatically suitable.The method removes modifiers such as “race car” from “race car driver” before substituting an occupation.Source: §5.1 · pp. 7–8
• Occupation mutants use 79 honest, gender-neutral occupation names; differing sentiments among same-template mutants flag occupation bias.Determiners are linked to the substituted occupation, such as “a” for “teacher” and “an” for “engineer.”Source: §5.1 · p. 8

5.2 Country-of-Origin Bias

• Country-of-origin bias compares same-template texts differing only in country; names are replaced only with consistent-gender coreferences, and 52 names from 26 countries supply mutants.The names are selected to be unique in both country of origin and gender, with one male and one female name per country.Source: §5.2 · pp. 8–9

6.1 Dataset and Experimental Settings

• The study fine-tuned five Transformer models on IMDB and Twitter sentiment data, using 25,000 train and 25,000 test reviews for IMDB and 400,000 train and 100,000 test texts for Twitter.The IMDB reviews were split evenly after non-natural-language text was removed; the Twitter train and test samples were mutually exclusive.Source: §6.1 · p. 9, Table 4 · p. 9

6.2 Research Questions

• BiasFinder was compared with EEC and MT-NLP using BTC counts, and an annotation study rated mutant fluency on a 1–3 scale with passable fluency set at an average of at least 1.5.Two native English-speaking non-authors rated randomly sampled mutants; each participant rated 384 mutants from each of four populations, with Cohen’s kappa of 0.55.Source: §6.1 · p. 9, §6.2 · p. 10

6.3 Results

• 42,349 gender BTCs on IMDB and 124,417 on Twitter: BiasFinder found the most on each dataset, versus 22,942 and 4,028 for EEC and 4,530 and 2,110 for MT-NLP, respectively.The comparison covers five sentiment-analysis models; the Twitter baseline counts were two orders of magnitude lower than BiasFinder’s.Source: Table 6 · p. 10, §6.3 · p. 10
• BiasFinder found 16,816 country-of-origin BTCs on IMDB and 25,832 on Twitter, plus 723,416 and 95,106 occupation BTCs, respectively.Tables 7 and 8 report the country-of-origin and occupation counts.Source: §6.3 · p. 10, Table 7 · p. 10, Table 8 · p. 11
• Mutant fluency ratings ranged from 1.77 to 3; gender mutants averaged 2.61 for BiasFinder versus 2.03 for MT-NLP, while country-of-origin mutants received 3 from both annotators.For gender mutants, the reported BiasFinder average was 28.57% higher; occupation ratings are also reported in Table 9.Source: Table 9 · p. 11, §6.3 · p. 11

6.4 Threats to Validity

• Results are limited to models fine-tuned from five Transformer architectures and templates generated from IMDB and Twitter; some BTCs may reflect model errors rather than bias, though high-accuracy models were used to mitigate this threat.The authors also checked that the selected 30 male and 30 female names were first names because the source database included gender-independent surnames.Source: §6.4 · pp. 11–12

6.5 Potential Usage

• The authors propose using BiasFinder mutants for training, runtime bias detection, and repair, and suggest its approach may extend to other text-classification tasks.They report prior downstream uses in runtime verification and automatic healing of bias.Source: §6.5 · p. 12

7.1 Bias in AI Systems

• Prior bias studies look for behavioral differences when demographic characteristics change, while Themis, Aequitas, and FairTest target numerical or image inputs rather than NLP text.Fairway instead includes strategies for mitigating bias in training data.Source: §7.1 · p. 12
• CheckList tests NLP capabilities, including fairness, but relies on a small set of predefined templates; BiasFinder is presented as complementary because it generates cases without that restriction.Source: §7.1 · p. 12
• For sentiment analysis, Diaz et al. manually replace age-encoding words; EEC compares texts differing by a gender- or race-associated word, while the approach dynamically generates varied, realistic templates and mutates other-characteristic words.Source: §7.1 · p. 12
• Compared with prior approaches, BiasFinder is fully automated rather than requiring extensive manual template creation, and it covers multiple fairness issues rather than one alone.The paper names gender, country-of-origin, and occupation bias as its demonstrated targets.Source: §7.1 · p. 12

7.2 AI Testing

• Other metamorphic-testing work targets bugs in machine translation or autonomous driving, whereas BiasFinder targets fairness bugs in sentiment analysis; NLP robustness work instead tests correctness under noise or adversarial changes.Source: §7.2 · p. 13

• The work contributes to fairness testing and motivates automatic fairness-testing research across machine-learning tasks, including sentiment analysis.Source: §8 · p. 13
• BiasFinder curates corpus texts into templates and mutates them with class-associated words to test for demographic bias in SA systems.Source: §8 · p. 13
• Unlike prior work’s limited templates or single-bias focus, BiasFinder covers gender, occupation, and country-of-origin, expanding the scenarios represented.Source: §8 · p. 13
• BiasFinder uncovers more gender BTCs than EEC and MT-NLP across all studied SA models, finds additional occupation and country-of-origin BTCs, and yields mutants annotators judge more fluent than MT-NLP’s.Source: §8 · p. 13
• Future work will add biases and text corpora and evaluate whether BiasFinder generalizes beyond sentiment analysis to tasks such as general text classification.Source: §8 · p. 13