Unsupervised Object Segmentation by Redrawing

• Image segmentation divides an image into non-overlapping regions; prior unsupervised methods used region priors, while deep networks renewed interest in fully supervised semantic segmentation [5] [58].Source: §1 · p. 1
• Pixel-level annotations required by supervised segmentation limit its use where large-scale labeled datasets are unavailable.The paper frames unsupervised segmentation as a way to address this supervision constraint.Source: §1 · p. 1
• ReDO uses a discriminator to distinguish real images from redrawn images, learning segmentation by testing whether replacing object contents yields realistic images.The approach assumes independence between image regions and calls the resulting model ReDO (ReDrawing of Objects).Source: §1 · p. 2

• Weakly supervised pipelines use image-level labels to derive saliency maps, then apply spatial priors or region proposals for segmentation [59].WILDCAT post-processes saliency maps with a CRF, while PRM selects regions from proposals using saliency peaks.Source: §2 · p. 2
• Co-segmentation seeks objects shared across images through recurring patterns, but feature-similarity approaches are not easily extended to class-agnostic segmentation.ReDO instead relies on independence between the contents of image regions.Source: §2 · p. 2
• Earlier unsupervised methods used handcrafted objectness features or energy functions, with later approaches learning representations or pixel distances, but not entirely from data [55].Depth maps can yield strong results, while designing energy functions for RGB-only images is harder [44].Source: §2 · p. 2
• Scene-decomposition methods use attention and VAE reconstruction or recursive generation; iterative composition can handle arbitrary object counts but is harder to train, so ReDO begins with a fixed count [4] [14] [18] [56].Some prior recomposition methods also require object-free backgrounds [42] [45].Source: §2 · pp. 2–3
• Disentanglement methods use adversarial learning to control latent information, but their aims are unrelated to segmentation [9] [32].InfoGAN encourages interpretable generator features, while Fader Networks remove class information from a vector.Source: §2 · p. 3

3.1 Overview

• ReDO addresses segmentation by learning image-generation components adversarially so generated images match the dataset distribution, rather than optimizing region properties for each image.The segmentation function assigns each pixel to one of n regions.Source: §3.1 · p. 3

3.2 Generative Process

• The generative process chooses masks together, draws each region’s pixels independently given its mask, and assembles the regions into an image.It assumes object colors and textures are independent, a deliberately naive but useful prior.Source: §3.2 · pp. 3–4

3.3 From Generative Process to Object Segmentation

• ReDO extracts masks from an input image, generates pixel values for each region, and learns the segmentation and region generators adversarially to match the training-image distribution.The generator composes the new image from the region-wise outputs.Source: §3.3 · p. 4
• Redrawing only one region at a time ties masks to the input, while latent-code reconstruction discourages empty regions and the trivial all-image-in-one-region solution.The information-conservation constraint is necessary for non-trivial solutions.Source: §3.3 · p. 5

4 Implementation

• The mask generator F uses a PSPNet-inspired fully convolutional architecture with pyramid pooling to aggregate local, regional, and global information.Pooling layers encourage each pixel representation to incorporate information at multiple scales.Source: §4 · p. 6
• Region generators, the discriminator, and latent-code reconstruction network draw on SAGAN; self-attention is used in G_k and D, while conditional batch normalization in G_k promotes stochasticity and style encoding.Spectral normalization is used for all networks except the mask provider F.Source: §4 · p. 6
• Training uses ADAM with batch size 25 and learning rates of 10^-4, except 10^-5 for F; early empty-mask collapse triggers an automatic restart.The experiments use a single Tesla P100 GPU, and λ_z and initialization were identified as critical hyperparameters.Source: §4 · p. 7

5.1 Datasets

• The benchmark includes Flowers, LFW, and CUB-200-2011, plus colored-2-MNIST sanity-check data and a fused Flowers+LFW set.Flowers masks and annotated LFW images are used for evaluation; training uses unlabeled LFW images.Source: §5.1 · p. 7

5.2 Results

• On LFW, ReDO reaches about 92% accuracy and 79% IoU; supervised training needs 50–60 labeled examples to reach similar performance.Across datasets, unsupervised results are in the range of supervised results and usually outperform supervised models trained with fewer than around 50 or 100 examples, depending on the dataset.Source: §5.2 · p. 8
• ReDO produces good-quality masks in many cases, although its redrawn images are less realistic than state-of-the-art GAN outputs.Figure 2 juxtaposes inputs, ground-truth masks, inferred masks, and redraws of two objects.Source: §5.2 · p. 8, Figure 2 · p. 8
• 0.856 accuracy and 0.691 IoU are preliminary results on fused Flowers+LFW data without labels; fixed z produces class-dependent textures in Figure 3.The Flowers dataset hyperparameters were reused without further tuning.Source: §5.2 · p. 8, Figure 3 · p. 8
• ReDO achieves reasonable performance on three real-world datasets, with Table 1 reporting accuracy and IoU for retrieved masks.The table reports means and standard deviations over five runs with fixed hyperparameters.Source: §5.2 · p. 7, Table 1 · p. 9

6 Conclusion

• ReDO casts unsupervised segmentation as adversarial learning based on the assumption that replacing a segmented object with a random one can still yield a realistic image.The method's experimental results on three datasets support this principle.Source: §6 · p. 9
• ReDO is competitive with supervised approaches trained on a few hundred labeled examples.This is the conclusion's stated comparison for the segmentation model.Source: §6 · p. 9
• Handling more complex and diverse scenes remains future work because training GANs in those settings is currently difficult.The authors also identify semi- or weakly supervised use as future work.Source: §6 · p. 9

Supplementary Material for Unsupervised Object Segmentation by Redrawing

• Supplementary material details architectures for mask network f, discriminator D, and encoder δ; f uses fewer residual blocks than Cycle-GAN-like translation networks and adds pyramid pooling.Source: Table 2 · p. 13, Table 3 · p. 13, Supplementary Material for… · p. 13

Hyperparameters

• The authors identify λz, initialization, and generator channel count as important choices, with λz and initialization critical to their search.Source: Hyperparameters · p. 14
• Learning rates were 10−4 except 10−5 for the mask network, with batch sizes of 20–25 and latent sizes of 16 for c2-MNIST and 32 elsewhere.Source: Hyperparameters · p. 14
• Generator channel counts were 64 for LFW and colored-2-MNIST but 32 for CUB and Flowers; spectral normalization was omitted for the mask network.Source: Hyperparameters · p. 14
• Pyramid pooling produced significantly better masks than a standard residual network.Source: Hyperparameters · p. 14
• Whether self-attention is necessary in both discriminator and generator remains unresolved pending further experiments.Source: Hyperparameters · p. 14
• Changing the ratio of discriminator to generator updates did not help.Source: Hyperparameters · p. 14

Additional output masks

• Additional masks include accuracy- and IoU-ranked and random outputs for Flowers and CUB; in Flowers’ lowest-agreement cases, ReDO sometimes better matches objects than the automated ground truth.Source: Figure 5 · p. 15, Figure 6 · p. 16, Figure 7 · p. 17