PaperlayerView exact revision on arXiv

Source-linked AI summary

Confounder Identification-free Causal Visual Feature Learning

Xin Li, Zhizheng Zhang, Guoqiang Wei, Cuiling Lan, Wenjun Zeng, Xin Jin, Zhibo Chen

arXiv:2111.13420v3cs.LGcs.CV

TL;DR

Confounders can bias learned features and hinder generalization, while back-door approaches depend on identifying confounders. CICF uses front-door-based sample interventions and approximates their global effects; the authors report state-of-the-art domain-generalization performance.

  • Problem

    Back-door causal-learning approaches require explicit confounder identification, although confounders can be diverse, implicit, or difficult to model.

    Source: §1 · p. 2

  • Method

    CICF models interventions among samples using the front-door criterion and approximates global intervening effects through optimization.

    Source: §5 · p. 9

  • Results

    CICF achieves state-of-the-art performance across domain-generalization benchmarks, including PACS, Digits-DG, Office-Home, and VLCS.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The experiments support that CICF can improve domain-generalization capability by learning causal features without identifying confounders.

    Source: §5 · p. 10

    The stated setup assumes X and Z have no common causes, so the effect of X on Z is identifiable as P(Z|X).

    Source: §3.1 · p. 4

Abstract

from arXiv · show

Confounders in deep learning are in general detrimental to model's generalization where they infiltrate feature representations. Therefore, learning causal features that are free of interference from confounders is important. Most previous causal learning based approaches employ back-door criterion to mitigate the adverse effect of certain specific confounder, which require the explicit identification of confounder. However, in real scenarios, confounders are typically diverse and difficult to be identified. In this paper, we propose a novel Confounder Identification-free Causal Visual Feature Learning (CICF) method, which obviates the need for identifying confounders. CICF models the interventions among different samples based on front-door criterion, and then approximates the global-scope intervening effect upon the instance-level interventions from the perspective of optimization. In this way, we aim to find a reliable optimization direction, which avoids the intervening effects of confounders, to learn causal features. Furthermore, we uncover the relation between CICF and the popular meta-learning strategy MAML, and provide an interpretation of why MAML works from the theoretical perspective of causal learning for the first time. Thanks to the effective learning of causal features, our CICF enables models to have superior generalization capability. Extensive experiments on domain generalization benchmark datasets demonstrate the effectiveness of our CICF, which achieves the state-of-the-art performance.

1 INTRODUCTION

Confounders can bias learned visual features and undermine generalization under distribution shift, while common back-door approaches require confounders to be identified. CICF uses front-door-based sample interventions and optimization to approximate global effects, and reports state-of-the-art domain-generalization performance.

1 INTRODUCTION

  • Confounders can infiltrate learned representations and impair generalization when test data shift from the training distribution.For example, tree branches associated with birds can be mistaken for bird features.Source: §1 · p. 1
  • Prior back-door approaches adjust for identified confounders, but leave unmodeled confounders when they are inaccessible or difficult to identify.The paper notes that confounders may be unidentifiable or hard to model in real scenarios.Source: §1 · p. 2
  • CICF models interventions among samples using the front-door criterion and approximates global intervening effects from instance-level interventions to find an optimization direction for causal feature learning.The approach is designed to avoid requiring explicit confounder identification.Source: §1 · p. 2
  • A gradient-based approach models how training samples intervene through parameter updates, while cluster-then-sample approximates global effects more efficiently than traversing the entire training set.The authors also interpret MAML as tending to learn causal features.Source: §1 · p. 2
  • CICF achieves state-of-the-art performance on domain-generalization benchmarks, including PACS, Digits-DG, Office-Home, and VLCS.The experiments cover all four datasets.Source: §1 · pp. 2–3

2 RELATED WORK

Domain-generalization work spans invariant representations, data or feature manipulation, and meta-learning. Prior causal approaches commonly use back-door adjustment, whereas this work applies the front-door criterion to causal visual feature learning by considering interventions among samples.

2 RELATED WORK

  • Many causal-learning approaches use the back-door criterion and therefore require identifying and modeling confounder distributions, which can be difficult when confounders are implicit or diverse.A prior front-door vision-language method still models intervention at test time.Source: §2 · p. 3
  • This work applies the front-door criterion to causal visual feature learning by modeling interventions among samples, without identifying confounders.The authors characterize it as the first such attempt and approach generalization from an optimization perspective.Source: §2 · p. 3
  • Domain-generalization methods include invariant representation learning, data or feature manipulation, and meta-learning.MAML seeks parameters that adapt quickly to new tasks.Source: §2 · p. 3

3 PROPOSED METHOD

CICF uses the front-door criterion to learn causal visual features without identifying confounders, estimating global interventions through optimization. The method also relates this global-intervention perspective to MAML.

3.1 PROBLEM DEFINITION AND ANALYSIS

  • Front-door criterion avoids explicit confounder identification by using intermediate features Z to assess the effect of X on Y, addressing a gap in visual feature learning.The paper notes that front-door methods remain under-explored for visual feature learning.Source: §3.1 · p. 4

3.2 CONFOUNDER IDENTIFICATION-FREE CAUSAL VISUAL FEATURE LEARNING

  • CICF models another sample’s intervention through its gradient update, then approximates the global effect with clustering-then-sampling rather than traversing all training samples.The paper states that clustering-then-sampling estimates the data distribution and global gradient more accurately than random sampling.Source: §3.2 · pp. 5–6
  • The estimated global-scope gradient updates f, producing a model output that incorporates interventions from other training samples before cross-entropy training.CICF approximates the causal conditional probability without identifying confounders.Source: §3.2 · pp. 5–6

3.3 DISCUSSION

  • CICF interprets MAML through causal learning: prior methods model local interventions, while the paper argues reliable causal features require modeling interventions across all samples.The paper presents its efficient global-scope intervention method as a practical way to model and eliminate those interventions.Source: §3.3 · pp. 6–7

4 EXPERIMENTS

The experiments evaluate CICF for domain generalization, examining its components, comparing it with prior methods across four benchmarks, and visualizing learned features.

4.1 DATASETS AND IMPLEMENTATION DETAILS

  • The study uses four domain-generalization benchmarks and leave-one-domain-out evaluation, with pretrained ResNet or AlexNet backbones and results averaged over six runs.PACS, Office-Home, and VLCS use the stated pretrained backbones; Digits-DG follows architectures from prior work.Source: §4 · p. 7, §4.1 · p. 8

4.2 ABLATION STUDY

  • On PACS, CICF improves ERM by 2.4% and ERM∗ by 3.2%; on Digits-DG, it improves ERM∗ by 2.0%, while MAML is not robust across baselines.MAML improves ERM by 0.1% with known domain labels but does not work for ERM∗.Source: §4.2 · p. 8

4.3 COMPARISON WITH STATE-OF-THE-ARTS

  • CICF achieves state-of-the-art performance on PACS, Digits-DG, Office-Home, and VLCS, including a 0.7% margin over MixStyle on Office-Home.On Digits-DG, it beats MAML by 1.4% and MixStyle by 5.2%; Office-Home accuracy is 66.2%, and VLCS exceeds MASF by 0.6%.Source: Table 2 · p. 8, §4.3 · p. 9

4.2 ABLATION STUDY

  • Clustering-then-sampling beats random sampling by 0.7% with lower memory use at batch-size 256; performance saturates at 21 PACS clusters.The authors attribute the gains to more accurate global-intervention estimates; three clusters per class suffice.Source: §4.2 · p. 9
  • Shallower choices of intermediate variable Z produce better accuracy than deeper feature choices in the reported PACS experiments.The authors suggest shallower Z leaves a larger downstream model f with more capacity to learn the conditional relationship.Source: §4.2 · p. 9

4.4 FEATURE VISUALIZATION

  • Grad-CAM visualizations show ERM focusing on object-irrelevant regions, while ERM+CICF tends to focus more on foreground objects.The figure compares Art and sketch examples from PACS.Source: Figure 4 · p. 8, §4.4 · p. 9

5 CONCLUSION

The conclusion presents CICF as a way to learn causal visual features without identifying confounders. Domain-generalization experiments support its generalization benefits; application to NLP remains future work.

5 CONCLUSION

  • CICF learns causal visual features without explicitly identifying or exploiting confounders.The method models interventions among samples and approximates global-scope intervening effects.Source: §5 · p. 9
  • Domain-generalization experiments support CICF's ability to improve model generalization through causal feature learning; applying it to NLP is left for future work.Source: §5 · p. 10

A.1.1 STATISTICAL ANALYSIS ON SAMPLING ALGORITHMS

The analysis compares clustering-then-sampling with random mini-batch sampling for estimating the global-scope gradient. It derives a smaller standard error for the clustering strategy and explains how clustering reduces gradient variance.

A.1.1 STATISTICAL ANALYSIS ON SAMPLING ALGORITHMS

  • Clustering-then-sampling has a smaller standard error than random mini-batch sampling when estimating the global-scope gradient.The analysis states that increasing the number of clusters can further reduce the maximum within-cluster variance and the resulting standard error.Source: §A.1.1 · pp. 15–16
  • Computing the global-scope gradient directly requires traversing the entire training set, which is intractable.The clustering analysis motivates estimating it through sampling instead.Source: §A.1.1 · p. 15
  • Random mini-batch sampling estimates the global-scope gradient from a randomly selected batch, with its standard error derived under M ≪ N.The gradient variance σ^2 is defined over all training samples.Source: §A.1.1 · p. 15
  • Clustering-then-sampling partitions each class into K-means clusters and samples from each cluster in proportion to its share of the training data.The resulting batch has size M, with M = Σ_k Nk and Nk = MP(Kk).Source: §A.1.1 · p. 15

A.1.2 EXPERIMENTAL EVIDENCE FOR THE SIGNIFICANT DIFFERENCE BETWEEN TWO SAMPLING STRATEGIES.

On PACS, clustering-then-sampling selected substantially different numbers of instances per cluster than random mini-batch sampling, including when random sampling accounted for class priors.

A.1.2 EXPERIMENTAL EVIDENCE FOR THE SIGNIFICANT DIFFERENCE BETWEEN TWO SAMPLING STRATEGIES.

  • 55 of 256 samples differed between the strategies on PACS with 21 clusters, a difference ratio of 21.5%.The setup clustered three clusters for each of PACS’s seven classes.Source: §A.1.2 · p. 16
  • 46 of 256 samples differed when random mini-batch sampling was weighted by class size, a difference ratio of 18.0%.The authors conclude that clustering-then-sampling differs significantly from random mini-batch sampling.Source: §A.1.2 · p. 16

A.1.3 MORE DETAILS ON THE SETTING OF SAMPLING ALGORITHM

CICF uses separate mini-batches to estimate the global-scope gradient and compute its loss. The sampling allocation determines whether cluster imbalance is modeled in the loss batch.

A.1.3 MORE DETAILS ON THE SETTING OF SAMPLING ALGORITHM

  • CICF forms a batch for estimating the global-scope gradient by sampling Nk instances from each cluster, using clustering-then-sampling.The method then updates θ to θ† using the estimated gradient and computes LCICF with a second mini-batch.Source: §A.1.3 · p. 17
  • For the gradient batch, CICF can set Nk to MP(Kk) or M/K†, where P(Kk) is the cluster’s share of all training data.M is the total batch size for computing the global-scope gradient, and K† is the total number of clusters.Source: §A.1.3 · p. 17
  • For the loss batch, equal sampling from every cluster uses Nk = M_l/K†, while random sampling from all training data uses Nk = M_lP(Kk).The latter allocation models the intervention caused by cluster imbalance.Source: §A.1.3 · p. 17

A.2 MORE DETAILS ON BACK-DOOR/FRONT-DOOR CRITERION

The back-door criterion removes a spurious path by intervening on X, while the front-door criterion introduces an intermediate variable Z to avoid explicitly identifying confounders. The front-door account decomposes the causal effect into components involving Z and X.

A.2 MORE DETAILS ON BACK-DOOR/FRONT-DOOR CRITERION

  • The back-door criterion targets the spurious path X ← C → Y by using do(X) to cut the connection from C to X.Figure 5 contrasts the observed association P(Y|X), which includes both the spurious and causal paths, with the intervened quantity P(Y|do(X)).Source: §A.2 · p. 17, Figure 5 · p. 18
  • The front-door criterion introduces intermediate variable Z and shifts modeling from confounder effects on X → Y to effects of X on Z → Y.It decomposes P(Y|do(X)) into P(Y|do(Z)) and P(Z|do(X)).Source: §A.2 · p. 18, Figure 6 · p. 18
  • Because X and Z have no common causes, P(Z=z|do(X)) equals P(Z=z|X).The account blocks the spurious path into Z → Y by cutting X → Z with the back-door criterion, without identifying C.Source: §A.2 · p. 18

A.3 USING h(X) AS Z SATISFIES FRONT-DOOR CRITERION

The appendix argues that Z = h(X) satisfies the front-door criterion and examines intermediate-output choices, computational cost, and learned-feature visualizations. It reports a 2.4% gain alongside 4.8% extra training time on PACS.

A.3 USING h(X) AS Z SATISFIES FRONT-DOOR CRITERION

  • Z = h(X) satisfies the front-door criterion: it intercepts directed X-to-Y paths, has no unblocked X-to-Z back-door path, and conditioning on X blocks Z-to-Y back-door paths.The appendix also states that P(X, Z) > 0 because Z is deterministic given X at inference.Source: §A.3 · p. 19

A.4 DIFFERENT CHOICES OF Z

  • Shallower choices of Z yield better accuracy in the PACS ablation, and the paper selects Z0 to give f more parameters for learning causal features.Figure 7 shows the different choices of Z; corresponding results are cited as Fig. 3(c).Source: §A.4 · p. 19

A.5 MORE IMPLEMENTATION DETAILS

  • A 2.4% gain accompanies 164 seconds of clustering, 4.8% extra training time, and 7.151 GiB of GPU memory cost on PACS.The reported whole-training time is 3432 seconds.Source: §A.5 · p. 20

A.6 FEATURE VISUALIZATION

  • Grad-CAM visualizations show CICF focusing more on foreground regions while ERM attends to misleading regions; t-SNE features from CICF are more compact within categories.The Grad-CAM comparison uses PACS cartoon/photo images, and the t-SNE visualization is on Digits-DG.Source: §A.6 · pp. 20–21, Figure 8 · p. 20, Figure 9 · p. 20

Related papers

Loading 2111.13420v3…