Source-linked AI summary
Semi-Supervised Domain Generalization with Stochastic StyleMatch
Kaiyang Zhou, Chen Change Loy, Ziwei Liu
TL;DR
This paper studies how to learn models that generalize to unseen domains while using limited labels, a combination treated separately in prior work. It benchmarks existing methods and proposes StyleMatch, which extends FixMatch for SSDG; StyleMatch shows clear advantages over competitors, with reported gains over the vanilla model in low-label settings.
Problem
Method
Results
StyleMatch shows clear advantages over competitors; its average gain over the vanilla model rises from around 17% to 25% on PACS when moving from 10-labels to 5-labels.
Source: §5.1 · p. 8, §6 · p. 9Takeaways & Limitations
The authors conclude that DG or SSL methods alone, and naive combinations of them, do not suffice for SSDG; StyleMatch offers a strong baseline for future research.
Source: §6 · p. 9The formulation considers shifts in P(X) while assuming P(Y) remains the same across domains.
Source: §3 · p. 4
Abstract
from arXiv · showhide
Ideally, visual learning algorithms should be generalizable, for dealing with any unseen domain shift when deployed in a new target environment; and data-efficient, for reducing development costs by using as little labels as possible. To this end, we study semi-supervised domain generalization (SSDG), which aims to learn a domain-generalizable model using multi-source, partially-labeled training data. We design two benchmarks that cover state-of-the-art methods developed in two related fields, i.e., domain generalization (DG) and semi-supervised learning (SSL). We find that the DG methods, which by design are unable to handle unlabeled data, perform poorly with limited labels in SSDG; the SSL methods, especially FixMatch, obtain much better results but are still far away from the basic vanilla model trained using full labels. We propose StyleMatch, a simple approach that extends FixMatch with a couple of new ingredients tailored for SSDG: 1) stochastic modeling for reducing overfitting in scarce labels, and 2) multi-view consistency learning for enhancing domain generalization. Despite the concise designs, StyleMatch achieves significant improvements in SSDG. We hope our approach and the comprehensive benchmarks can pave the way for future research on generalizable and data-efficient learning systems. The source code is released at \url{https://github.com/KaiyangZhou/ssdg-benchmark}.
1 Introduction
The introduction frames semi-supervised domain generalization (SSDG) as combining generalization to unseen domains with learning from limited labels. Benchmarks reveal shortcomings in existing DG and SSL approaches, motivating StyleMatch’s targeted extensions to FixMatch.
1 Introduction
- SSDG combines domain generalization across unseen targets with SSL’s use of abundant unlabeled data and only a handful of labeled images per source.The paper presents generalization and data-efficiency as important goals that prior research treated separately.Source: §1 · pp. 1–2
- DG methods perform weakly with limited labels, while SSL methods such as FixMatch do better but remain noticeably behind full-label training.The introduction also reports that a naive combination of DG and SSL methods does not fare well.Source: §1 · p. 2
- Two benchmarks based on widely used DG datasets evaluate state-of-the-art methods from DG and SSL under the SSDG setting.The introduction uses these benchmarks to examine the limits of applying either field’s methods to SSDG.Source: §1 · pp. 2–3
- StyleMatch extends FixMatch for SSDG, addressing its multi-source performance deterioration, which the authors hypothesize stems from source shifts and scarce labels.The paper identifies these challenges as distinct to SSDG.Source: §1 · p. 2
- StyleMatch models classifier weights with Gaussian distributions to reduce overfitting on scarce labels and adds style-transfer consistency to FixMatch’s two-view learning.The stochastic classifier implicitly learns an ensemble; the third view compares images with their style-transferred counterparts.Source: §1 · p. 2
2 Related Work
The related work reviews DG, consistency and pseudo-labeling methods in SSL, and stochastic neural networks. It positions SSDG as requiring methods that can use unlabeled data across heterogeneous sources.
2.1 Domain Generalization (DG)
- DG methods include feature alignment, meta-learning, and diverse data augmentations, but most cannot use unlabeled data except some self-supervised approaches.The authors report that their approach fits SSDG better than previous DG methods.Source: §2.1 · p. 3
2.2 Semi-Supervised Learning (SSL)
- SSL consistency learning matches predictions across views, while pseudo-labeling assigns labels to unlabeled examples and can use strong input or parameter noise.The paper notes that heterogeneous sources make SSDG unlabeled data more challenging than SSL data, and simply applying SSL methods is suboptimal.Source: §2.2 · p. 3
2.3 Stochastic Neural Networks
- Stochastic neural networks model weights as probability distributions to drive parameter-space exploration, and the authors report being first to apply SNNs successfully to SSDG.They distinguish this from prior SNN applications in domain adaptation, segmentation, and person re-identification.Source: §2.3 · p. 4
3 Problem Definition
SSDG trains a model for an unseen target using labeled and unlabeled data from multiple related but distinct source domains. This formulation assumes domains share a label space while their input distributions may differ.
3 Problem Definition
- A domain is a joint distribution over inputs and labels, and this work considers shifts in the input marginal while keeping the label marginal unchanged.Thus, all domains share the same label space.Source: §3 · p. 4
- Each of K distinct, related source domains has only a small labeled subset, while most source data are unlabeled.The labeled and unlabeled subsets are drawn from their source domain’s joint and input distributions, respectively.Source: §3 · p. 4
- The goal is to learn from labeled and unlabeled source data, then deploy directly in a target domain whose joint distribution differs from every source.The target data are unseen during training.Source: §3 · p. 4
4 Methodology
StyleMatch combines FixMatch pseudo-labeling with a stochastic classifier and multi-view consistency learning for multi-source, partially labeled training.
4 Methodology
- StyleMatch assigns pseudo-labels to unlabeled data and combines uncertainty with data augmentation to address domain shifts.Source: §4 · p. 4
4.1 Stochastic Classifier
- Each class prototype is sampled from a learned Gaussian during training, implicitly optimizing an ensemble; test-time classification uses the mean parameters.Source: §4.1 · p. 5
4.2 Multi-view Consistency Learning
- Training uses weakly augmented labeled images and weak-view pseudo-labels to supervise strongly augmented images only above confidence threshold π=0.95.Source: §4.2 · p. 5
- A third style-transfer view enforces prediction consistency between an image and its counterpart from another source domain, complementing geometric and color-based strong augmentation.Source: §4.2 · pp. 5–6
- The final objective adds labeled, strong-augmentation, and style-consistency losses without balancing weights.Source: §4.2 · pp. 5–6
5 Experiments
The experiments evaluate StyleMatch on PACS and OfficeHome under leave-one-domain-out settings with 10 or 5 labeled images per class in each source domain. StyleMatch outperforms the competing methods, while component analyses examine its stochastic classifier, multi-view consistency, augmentations, and behavior with fewer source domains.
5.1 Main Results
- StyleMatch outperforms all competitors, including FixMatch; its average gain over the vanilla model rises from around 17% to 25% on PACS and around 2% to 4% on OfficeHome when labels fall from 10 to 5 per class.On PACS art-painting and sketch targets, StyleMatch also surpasses Full-Labels; on OfficeHome, it remains below that pseudo-oracle.Source: Table 1 · p. 7, Table 2 · p. 7, §5.1 · p. 8
- DG methods using only limited labels hardly beat the vanilla model, SSL methods generally do better, and naively combining FixMatch with RSC adds no significant improvement.FixMatch outperforms MeanTeacher and Entropy-Minimization; OfficeHome is more challenging than PACS, so small gains there can be significant.Source: §5.1 · pp. 7–8
5.2 Ablation Study and Analysis
- The SNN classifier adds around 2% on PACS and 1% on OfficeHome, while multi-view consistency further improves performance, especially with fewer labels.The ablation compares StyleMatch's two core components against FixMatch, and reports that their combination brings significant improvements.Source: §5.2 · p. 8, Figure 5 · p. 8
- Without SNN, overconfidence overtakes pseudo-label accuracy during training; with SNN, overconfidence converges toward pseudo-label accuracy instead.For FixMatch without SNN, overconfidence outpaces accuracy early and pseudo-label accuracy then stops improving.Source: §5.2 · p. 8, Figure 6 · p. 9
- Tstrong is more suitable than Tstyle alone for two-view consistency, while combining them yields better PACS performance.The comparison evaluates Tstrong-only, Tstyle-only, and the final Tstrong+Tstyle model.Source: §5.2 · p. 9, Table 3 · p. 9
- StyleMatch beats FixMatch with one, two, or three source domains; increasing sources from two to three yields larger gains for StyleMatch than for FixMatch in both label settings.With one source domain, StyleMatch mixes image styles between random instances from that same domain.Source: §5.2 · p. 9, Table 4 · p. 9
6 Conclusion
The conclusion frames SSDG as a unification of model generalization and data-efficiency, which had been studied separately. It reports that DG and SSL methods, as well as their naive combination, do not suffice in the proposed benchmarks, while StyleMatch shows significant experimental gains.
6 Conclusion
- Model generalization and data-efficiency are two critical problems for real-world vision applications that have been studied separately.The paper explores their unification as semi-supervised domain generalization.Source: §6 · p. 9
- In the two proposed benchmarks, state-of-the-art DG or SSL methods alone, and a naive combination of them, do not suffice to solve SSDG.Source: §6 · p. 9
- StyleMatch demonstrates significant gains in the experiments and is presented as a strong baseline for future research.The conclusion characterizes its designs as concise.Source: §6 · p. 9