PaperlayerView exact revision on arXiv

Source-linked AI summary

Fair is Better than Sensational:Man is to Doctor as Woman is to Doctor

Malvina Nissim, Rik van Noord, Rob van der Goot

arXiv:1905.09866v2cs.CL

TL;DR

The paper addresses whether analogy tasks provide a sound way to diagnose bias in word embeddings, given concerns about their intrinsic validity and common usage. It examines analogy definitions, algorithms, implementation constraints, and subjective query and reporting choices. It concludes that analogies can distort the apparent presence of bias, although embeddings do encode human biases and those biases require attention.

  • Problem

    Analogy tasks are widely used to expose bias in embeddings, but their definitions, implementation constraints, and query choices may provide an unreliable diagnostic.

    Source: §1 · p. 1, §2.2 · p. 3, §3 · p. 5

  • Method

    The paper analyzes analogy requirements, compares established scoring formulations, and examines how input restrictions, parameter choices, and reporting practices affect results.

    Source: §1 · p. 2, §3 · p. 5, §4.2 · p. 7

  • Results

    The analysis shows that analogy use can yield a distorted picture of embedding bias, including apparent bias caused by forbidding expected answers or selectively reporting outputs.

    Source: §4.1 · pp. 5–6, §4.2 · p. 7

  • Takeaways & Limitations

    Analogies are probably not the right tool for diagnosing embedding bias, so their construction and interpretation require greater transparency while bias itself remains a serious concern.

    Source: §1 · p. 2

    Reported analogy findings can be misleading because selected outputs may be cherry-picked, and some tested analogies cannot be reproduced even within the top ten results.

    Source: §4.2 · p. 8

Abstract

from arXiv · show

Analogies such as "man is to king as woman is to X" are often used to illustrate the amazing power of word embeddings. Concurrently, they have also been used to expose how strongly human biases are encoded in vector spaces built on natural language, like "man is to computer programmer as woman is to homemaker". Recent work has shown that analogies are in fact not such a diagnostic for bias, and other methods have been proven to be more apt to the task. However, beside the intrinsic problems with the analogy task as a bias detection tool, in this paper we show that a series of issues related to how analogies have been implemented and used might have yielded a distorted picture of bias in word embeddings. Human biases are present in word embeddings and need to be addressed. Analogies, though, are probably not the right tool to do so. Also, the way they have been most often used has exacerbated some possibly non-existing biases and perhaps hid others. Because they are still widely popular, and some of them have become classics within and outside the NLP community, we deem it important to provide a series of clarifications that should put well-known, and potentially new cases into the right perspective.

1. Introduction

The paper argues that analogies are flawed tools for diagnosing bias and that implementation and query choices can distort apparent embedding biases. It clarifies analogy constraints, comparison methods, and the role of human judgment while affirming that embeddings encode harmful biases.

1. Introduction

  • Analogies are not reliable bias diagnostics because observed relations may reflect irrelevant neighborhood structure, and prior work has shown they can mislead debiasing efforts.The paper notes that contextualized embeddings no longer use analogies to evaluate soundness or detect bias.Source: §1 · pp. 1–2
  • The paper aims to clarify analogy requirements, implementation choices, and reported results so established and new examples are interpreted more appropriately.Source: §1 · p. 2
  • Classic analogy implementations force all four terms to differ, creating artifacts when the correct answer should equal an input term.The restriction is enforced explicitly in the original proportional analogy implementation.Source: §1 · p. 2
  • The study compares 3COSADD, 3COSMUL, and the Bolukbasi et al. (2016) formulation, which reduces the influence of subjective query choices.Source: §1 · p. 2
  • Human biases influence which analogies are searched and reported, while parameter tuning can also affect results even when input subjectivity is reduced.Source: §1 · p. 2
  • The paper does not deny embedding bias: it states that embeddings encode human biases and that the issue deserves sustained attention.Transparency about analogy construction and use is presented as important for addressing this issue.Source: §1 · p. 2

2. What counts as analogy?

The paper distinguishes formal analogies from looser semantic relations and shows that analogy queries differ in whether they have a unique expected answer. These distinctions determine whether repeated terms and alleged bias conclusions are methodologically appropriate.

2. What counts as analogy?

  • Analogies include morpho-syntactic and semantic categories, but semantic relations are often looser and more interpretation-dependent than strict proportional analogies.Source: §2 · p. 3
  • Analogy analysis must distinguish four-term queries from cases with repeated terms and unique expected answers from cases admitting multiple plausible completions.Source: §2 · p. 3

2.1 Should all terms be different?

  • Homographs, hypernyms, ordered concepts, and extended test examples can require repeated terms, so forbidding input terms is not universally appropriate.Examples include split:split, Singapore:Singapore, cat:animal :: dog:animal, and country:language relations.Source: §2.1 · p. 3

2.2 Is there a correct answer?

  • Traditional analogy datasets assume one factual fourth term, but semantic queries such as man:computer_programmer :: woman:X may lack a predefined unbiased answer.The comparison he:actor :: she:X is more straightforward because actress supplies a morphosyntactic completion.Source: §2.2 · p. 3
  • For man:doctor :: woman:X, the morphosyntactic answer may be doctor, whereas a semantic interpretation offers no single clearly correct completion.Source: §2.2 · p. 4
  • If a query expects an input term as the unbiased answer, it should not be posed under an algorithm that forbids returning input terms, and failure to return it should not be labeled bias.Source: §2.2 · p. 4

3. Algorithms

The paper reviews three analogy-scoring strategies and explains how their search spaces and constraints shape outputs. It emphasizes that implementations commonly exclude input vectors, a choice that materially affects analogy evaluation.

3. Algorithms

  • The study considers 3COSADD, 3COSMUL, and the Bolukbasi et al. (2016) method for identifying analogy relations.3COSMUL was introduced to reduce domination by a single large cosine term.Source: §3 · p. 4
  • The Bolukbasi et al. (2016) method ranks B:D pairs without requiring B in advance, reducing subjective input choices.Its threshold δ is set to 1.0, requiring B and D to be closer than two random embedding vectors.Source: §3 · p. 4
  • When B is known, the Bolukbasi formulation can be transformed to search only for D; the resulting scores remain equivalent to Equation 3.Source: §3 · p. 5
  • Most implementations ignore one or more input vectors even though the equations do not require this, reflecting the traditional assumption that all analogy terms differ.The paper argues this constraint is strong in both morpho-syntactic and semantic analogies and is not widely understood outside the field.Source: §3 · p. 5

4. Is the bias in the models, in the implementation, or in the queries?

The section shows that apparent bias in analogy outputs depends substantially on implementation constraints, query formulation, reporting choices, and parameter settings. These choices can change which answers appear and how confidently they support bias claims.

4.1 Ignoring or allowing the input words

  • On the Google Analogy test set, allowing input terms in the unconstrained setting causes a large performance drop for 3COSADD and 3COSMUL, often returning B or C.Unconstrained evaluation includes input vectors; the second or third analogy term is returned as the answer in many cases.Source: §4.1 · p. 5, Table 1 · p. 6
  • For gender-bias analogies, unconstrained outputs can return doctor, showing that the classic man:doctor :: woman:nurse example is not fairly interpreted when doctor is prohibited.The original code cannot produce doctor as D because it enforces distinct terms, so failure to return doctor does not by itself establish bias.Source: §4.1 · p. 6, Table 2 · p. 6

4.2 Subjective factors

  • Subjective query direction, expected answers, and selective reporting can substantially alter apparent bias, making reported examples arbitrary or misleading.The authors describe inverse-query mistakes, non-top-ranked reported terms, and unclear algorithm choices in successive versions of Manzini et al. (2019a,b,c).Source: §4.2 · pp. 7–8

4.3 Other constraints

  • Vocabulary cutoffs, preprocessing, and the Bolukbasi threshold δ materially affect outputs, with the same man:doctor :: woman:X query ranging from nurse to doctors.Table 4 reports that extreme δ values yield doctor at ≤0.5 and she at ≥1.5 when query words may be returned.Source: §4.3 · pp. 8–9, Table 4 · p. 9

5. Final remarks

Analogies may be an inappropriate bias diagnostic, and their common use can amplify misleading conclusions. Transparent reporting of implementation and subjective choices is therefore necessary for interpreting embedding bias.

  • Analogies may not be the most appropriate tool for capturing certain relations, and their use can worsen the resulting picture.Source: §5 · p. 9
  • Sensational analogy examples can spread through research, other fields, and popular articles where readers cannot readily verify their reliability or significance.Source: §5 · p. 9
  • Analogy-based bias searches may mislead both the identification of embedding bias and the application of debiasing strategies.Source: §5 · p. 9
  • Clear transparency about what analogies can diagnose, and about subjective and implementation choices, is required to understand how embeddings reflect societal biases.Source: §5 · p. 9
Loading 1905.09866v2…