PaperlayerView exact revision on arXiv

Source-linked AI summary

Linear Adversarial Concept Erasure

Shauli Ravfogel, Michael Twiton, Yoav Goldberg, Ryan Cotterell

arXiv:2201.12091v8cs.LGcs.CL

TL;DR

The paper asks how to remove concept information from fixed, pretrained representations while preserving their other information. It formulates linear concept removal as a constrained maximin game and develops closed-form solutions and a convex relaxation; empirically, removing a 1-dimensional subspace mitigates linearly encoded concepts.

  • Problem

    The paper asks whether a post-hoc transformation can make fixed, pretrained representations uninformative about concept labels while preserving as much of their other information as possible.

    Source: §1 · p. 1

  • Method

    It models linear concept removal as a constrained maximin game over fixed-rank orthogonal projections, deriving closed-form solutions for some objectives and a convex relaxation, RLACE, for classification loss.

    Source: §1 · p. 2, §2.3 · p. 3

  • Results

    Removing a 1-dimensional subspace is enough to mitigate linearly encoded concepts, according to the paper’s empirical results.

    Source: §7 · p. 9

  • Takeaways & Limitations

    The linear projection provides an interpretable, minimally invasive approach to removing linearly recoverable concepts, but protects only against linear classifiers.

    Source: §1 · p. 1, §7 · p. 9

    The gradient descent–ascent procedure is not guaranteed to converge.

    Source: §3.3 · p. 4

Abstract

from arXiv · show

Modern neural models trained on textual data rely on pre-trained representations that emerge without direct supervision. As these representations are increasingly being used in real-world applications, the inability to \emph{control} their content becomes an increasingly important problem. We formulate the problem of identifying and erasing a linear subspace that corresponds to a given concept, in order to prevent linear predictors from recovering the concept. We model this problem as a constrained, linear maximin game, and show that existing solutions are generally not optimal for this task. We derive a closed-form solution for certain objectives, and propose a convex relaxation, \method, that works well for others. When evaluated in the context of binary gender removal, the method recovers a low-dimensional subspace whose removal mitigates bias by intrinsic and extrinsic evaluation. We show that the method is highly expressive, effectively mitigating bias in deep nonlinear classifiers while maintaining tractability and interpretability.

1. Introduction

The paper asks how to remove concept information from fixed representations while preserving other information, focusing on linear projections that prevent linear classifiers from recovering the concept. It analyzes this as a maximin problem and develops solutions for several objectives, with binary gender removal as its experimental example.

1. Introduction

  • The task is to post-hoc transform fixed representations so they are not predictive of concept labels while preserving as much of the original information as possible.The paper focuses on real-valued representations from neural language encoders and frames gender removal as one example.Source: §1 · p. 1
  • The proposed linear approach removes a concept subspace with an orthogonal projection, preventing linear classifiers from recovering the concept while minimally changing inputs.The linear constraint is presented as practical and interpretable.Source: §1 · p. 1
  • Prior gender-subspace methods include PCA-based removal, which later work showed can leave gender recoverable by a linear classifier, and INLP, which iteratively projects onto learned classifier-weight nullspaces.INLP was reported as competitive, while the paper questions whether it is optimal for the stated objective.Source: §1 · pp. 1–2
  • The paper formulates linear concept removal as a maximin game, derives closed-form solutions for some objectives, and proposes the RLACE convex relaxation for classification loss.Experiments focus on binary gender and report bias mitigation in static and contextualized representations.Source: §1 · p. 2

2. Linear Maximin Games

The section formulates concept removal as a constrained maximin game: a predictor seeks to recover concept labels while an adversary projects representations into a fixed-dimensional subspace. Its linear projection structure supports interpretable interventions and principled solutions for several generalized linear objectives.

2.1. Notation and Generalized Linear Modeling

  • Generalized linear models combine a linear predictor with a link function and loss, providing a framework for learning predictors of concept labels.The notation treats inputs as D-dimensional real vectors and responses as binary labels.Source: §2.1 · p. 2

2.2. The Linear Bias Subspace Hypothesis

  • The linear bias subspace hypothesis assumes concept information lies in a subspace B, which can be removed by projecting representations onto its orthogonal complement.The projection matrix has B as its kernel and the non-bias subspace as its range.Source: §2.2 · p. 2

2.3. Linear Maximin Games

  • The adversary chooses a rank-(D−K) orthogonal projection to hinder a predictor, forming a sequential maximin game that is generally NP-hard to solve.The projection constrains how the adversary can change inputs, enabling principled solutions while minimally changing them.Source: §2.3 · p. 3
  • The framework instantiates linear, partial-least-squares, and logistic regression objectives; partial least squares adds a parameter constraint outside the general linear-model formalism.For the game, the max over projections precedes the min over predictors; the paper notes the formulation is convex–convex, not convex–concave.Source: §2.3 · p. 3

3. Solving the Linear Maximin Game

The paper solves the linear maximin game in closed form for linear regression and partial least squares, and uses a convex relaxation for logistic regression. The solutions specialize the removed subspace to each objective, while the practical logistic procedure lacks a convergence guarantee.

3. Solving the Linear Maximin Game

  • Closed-form solutions apply to linear regression and Rayleigh-quotient objectives, while logistic regression uses a practical convex relaxation.The relaxation is optimized by alternating gradient steps over the predictor and projection, then projecting onto the Fantope.Source: §3 · p. 3, §3.3 · pp. 4–5

3.1. Linear Regression

  • For K = 1, linear regression has an optimal solution at (P*, 0), and the projection's null space spans the representation–response covariance direction X⊤y.Because linear regression explains covariance, the method needs only a one-dimensional bias subspace.Source: §3.1 · p. 4

3.2. Partial Least Squares

  • Partial least squares is a Rayleigh-quotient special case whose solution follows from an eigendecomposition, with objective value λK+1.The objective value is stated for the general maximin game in this Rayleigh-quotient setting.Source: §3.2 · p. 4

3.3. Logistic Regression

  • The logistic-regression relaxation is optimized by alternating gradient descent and ascent, but this procedure is not guaranteed to converge.The authors did not explore more complex algorithms with convergence guarantees, citing promising empirical results.Source: §3.3 · pp. 4–5

4. Relation to Interative Nullspace Projection

The section tests when INLP matches the paper’s maximin objective and finds that its behavior depends on the loss: it is not optimal for linear regression but is exact for partial least squares. In logistic regression experiments, RLACE neutralizes gender prediction with one removed dimension, while INLP requires more than one.

4. Relation to Interative Nullspace Projection

  • The section asks whether INLP exactly solves the paper’s maximin objective for different loss and link functions.INLP iteratively trains a generalized linear model and projects onto the complement of its parameter-vector subspace.Source: §4 · p. 5

4.1. Linear Regression

  • For linear regression, INLP does not return the optimal projection after one iteration and may remove more dimensions than necessary.The proposition states that its projection differs from the projection in Proposition 3.1; INLP eventually damages linear-regression prediction ability.Source: §4.1 · p. 5

4.2. Partial Least Squares

  • For partial least squares, INLP returns the projection in Proposition 3.3 after one iteration.The section relates this exact result to the iterative solution of Rayleigh quotient problems by singular value decomposition.Source: §4.2 · p. 5

4.3. Logistic Regression

  • Across all experiments, optimizing Eq. (13) identified a 1-dimensional subspace whose removal completely neutralized the target concept, whereas INLP required more than one direction.Source: §4.3 · p. 5

5. Experiments

The experiments test RLACE on static word representations, contextualized profession classification, and image pixels. Across these settings, results include gender-prediction reduction with a rank-1 projection, profession-classifier fairness effects, and concept removal from images.

5.1. Static Word Representations

  • A rank-1 RLACE projection reduces gender prediction on GloVe representations to nearly chance, while INLP does not reach majority-class accuracy after removing 20 dimensions.Nonlinear classifiers still predict gender above 90% after projection.Source: §5.1 · p. 6
  • RLACE is the most effective method on the reported WEAT bias association tests, and rank-1 projection leaves SimLex-999 correlations near the original level.Pearson correlations are 0.399 for original representations, 0.392 after RLACE, and 0.395 after one INLP iteration.Source: Table 1 · p. 6, §5.1 · p. 7

5.2. Profession Classification

  • For profession classification, RLACE and INLP mitigate TPR-gap bias similarly, but INLP does not fully remove gender prediction from finetuned BERT, even after 100 iterations.On frozen BERT, INLP also degrades profession accuracy from 79.91% to 71.27%, while RLACE does not significantly damage that task.Source: §5.2 · p. 7, Figure 3 · p. 7, Table 2 · p. 8
  • In the finetuned model, the decrease in correlation between profession TPR gaps and women’s representation is moderate.Adversarially finetuned models show somewhat decreased TPR-gap bias but do not hinder gender prediction; the link between gender predictability and TPR-gap bias is not clear-cut.Source: §5.2 · p. 8

5.3. Erasing Concepts in Image Data

  • On CelebA images, a rank-1 RLACE projection brings concept-classification accuracy to less than 1% above majority-class accuracy.The projection manipulates expected visual features, but its projection constraint makes removal easier than adding features.Source: §5.3 · p. 8

6. Related Work

Prior concept-erasure work includes training-time adversarial methods and post-hoc linear methods. This paper’s framing differs from subspace clustering by targeting linear predictability without assuming the data lies in a linear subspace.

6. Related Work

  • Training-time adversarial concept-erasure methods have been described as unstable and not completely removing concepts from representations.The cited discussion attributes the latter finding to Elazar and Goldberg (2018).Source: §6 · p. 8
  • Bolukbasi et al. (2016) used PCA to identify a gender subspace, while later linear formulations include Dev and Phillips (2019), Ravfogel et al. (2020), Dev et al. (2021), and Kaiser et al. (2021).Sadeghi et al. (2019) study a related linear adversarial formulation for linear regression, with an unconstrained general linear adversary.Source: §6 · p. 9
  • Unlike subspace clustering, linear concept erasure targets whether a linear classifier can predict the concept and does not assume the data lies in a linear subspace.Concept subspaces have also been used for interpretability, causal analysis, and studying representation geometry.Source: §6 · p. 9

7. Conclusion

The paper formulates concept erasure as a constrained maximin game, deriving closed-form solutions for some objectives and a convex relaxation for others. Its experiments focus on binary gender removal, and the authors caution that this is preliminary work whose effectiveness must be assessed in each deployment context.

7. Conclusion

  • The constrained maximin formulation yields closed-form solutions for some objectives and a convex relaxation that works well in practice for others.The relaxed optimization recovers a 1-dimensional subspace whose removal mitigates linearly encoded concepts.Source: §7 · p. 9
  • The proposed method protects against linear classifiers; effectively removing nonlinear information while retaining the constrained linear approach remains an open challenge.The authors report that removing a 1-dimensional subspace is enough to mitigate linearly encoded concepts.Source: §7 · p. 9

A. Ethical Considerations

  • Experiments address binary gender, and the authors note that gender is non-binary and their datasets may miss subtle or implicit forms of gender bias.They caution practitioners to assess effectiveness using the deployment data, fairness metrics, and application, and not to treat RLACE as a solution to representation bias.Source: A. Ethical Considerations · p. 13
  • The authors characterize the work as preliminary and warn that different forms of bias may remain after applying RLACE.They urge care in deployment and evaluation in the specific application context.Source: A. Ethical Considerations · p. 13

B.1. Proof of Proposition 3.1

The appendix proves a special-case maximin result by bounding the objective, then characterizes an eigenspace solution and describes alternating optimization of the relaxed objective using Fantope projection.

B.1. Proof of Proposition 3.1

  • The proof of Proposition 3.1 establishes a solution for K = 1 by lower- and upper-bounding the maximin game.The proof notes that the minimizing player can choose θ⋆ = 0 for the upper bound.Source: §B.1 · pp. 13–14

B.2. Proof of Lemma 3.2

  • For the maximin game in Lemma 3.2, the objective value is λK+1, with P chosen to span eigenvectors associated with the K smallest eigenvalues.The proof states that the solution uses vK+1 and bounds the objective at λK+1.Source: §B.2 · pp. 14–15

C.1. Alternate Optimization with Projected Gradient Descent

  • The relaxed objective is optimized by alternating descent updates for θ and projected gradient ascent updates for P.The projection step maps P onto the Fantope, using an eigendecomposition and bisection to find γ.Source: §C.1 · p. 15
  • After optimization, the procedure returns the closest Fantope vertex as an orthogonal projection whose range spans the first D − K eigenvectors.Source: §C.1 · p. 15

C.2. Experimental Setup

The static-word experiments use GloVe vectors and approximate binary gender labels induced from a he–she direction. The setup specifies the data splits, optimization procedure, and evaluation across subspace dimensions.

C.2. Experimental Setup

  • Experiments use 300-dimensional uncased GloVe vectors, labeling the 7,500 most male- and female-biased words via projection on the he–she direction.That direction induces approximate gender labels for RLACE training; the authors distinguish this use from Bolukbasi et al. (2016).Source: §C.2 · p. 16
  • The binary classification split contains 7,350 training, 3,150 evaluation, and 4,500 test examples, with 50,000 alternating adversary and classifier updates.Gender-neutral words are discarded, following the train–dev–test split used by Ravfogel et al. (2020).Source: §C.2 · p. 16
  • RLACE and INLP are evaluated at K = 1–20 using logistic regression gender classifiers, with 10 runs reported as mean ± standard deviation.The adversary uses SGD with learning rate 0.005 and batch size 128; the selected adversary yields the highest classification loss.Source: §C.2 · p. 16

D. Experimental Setting: Deep Classification

The deep-classification setting uses the biographies dataset and evaluates RLACE and INLP across subspace dimensions for frozen and fine-tuned BERT. It also describes profession-classifier training and the TPR-GAP fairness measure.

D. Experimental Setting: Deep Classification

  • The biographies dataset is split into 255,710 training, 39,369 evaluation, and 98,344 test examples; RLACE and INLP vary K up to 50 for BERT-frozen and 100 for BERT-finetuned.The pipeline is run three times, with results reported as mean ± standard deviation.Source: D. Experimental Setting: Deep… · p. 16
  • The setting trains profession classifiers with BERT-frozen, BERT-finetuned, and BERT-adv; the adversarial model adds a gradient-reversed gender head and converges below 55% gender-classification accuracy.The passage says this low accuracy is commonly interpreted as successful removal.Source: D. Experimental Setting: Deep… · p. 16
  • TPR-GAP is described as tightly related to equal opportunity: task-label prediction success should be similar across populations conditioned on the true class.Source: D. Experimental Setting: Deep… · p. 16

E. V -Measure

In GloVe, rank-1 relaxed projection sharply reduces cluster–gender association; the section also describes a before-and-after neighbor analysis as a qualitative check.

E. V -Measure

  • The evaluation varies cluster count on the x-axis, and lower V-measure indicates weaker association between gender labels and cluster identity.Source: Figure 5 · p. 17
  • After rank-1 relaxed projection, the V-measure drops drastically, while INLP and PCA-based removal have smaller effects.The original representations have V-measure 1.0, indicating strong alignment between cluster identity and gender labels.Source: E. V -Measure · pp. 16–17

F. Influence on Neighbors in Embedding Space

  • The neighbor analysis compares closest neighbors for 15 randomly sampled vocabulary words before and after the intervention.It is presented as a qualitative demonstration of whether the intervention damages general semantics.Source: F. Influence on Neighbors in Embedding… · p. 17

F.1. Additional results on the CelebsA dataset

The relaxed solution’s spectrum is reported to remain close to that of a valid rank-K projection. The appendix also presents randomly sampled outputs for six concepts on CelebA.

F.1. Additional results on the CelebsA dataset

  • Randomly sampled outputs are presented for six CelebA concepts: GLASSES, SMILE, MUSTACHE, BEARD, BALD, and HAT.Source: §F.1 · p. 17

F.2. Relaxation Quality

  • The relaxed solution’s spectrum has leading eigenvalues near 1 and the rest near 0, supporting that it is close to a valid rank-K projection.The passage describes this approximation as tight.Source: §F.2 · p. 17, Figure 12 · p. 19
  • Orthogonal projections have eigenvalues of 0 or 1, and their sum gives the matrix rank, the criterion used to assess relaxation quality.Source: §F.2 · p. 17

Related papers

Loading 2201.12091v8…