Source-linked AI summary
Tagger: Deep Unsupervised Perceptual Grouping
Klaus Greff, Antti Rasmus, Mathias Berglund, Tele Hotloo Hao, Jürgen Schmidhuber, Harri Valpola
TL;DR
The paper addresses perceptual inference for complex multi-object inputs by learning how to group inputs and internal representations rather than imposing a task-specific segmentation. TAG amortizes iterative refinement of group assignments and representations, achieving fast convergence and improved performance on cluttered classification tasks, including semi-supervised learning. Its current formulation assumes one level of global groups, leaving hierarchical grouping as future work.
Problem
Complex inputs require grouping objects and their features, but the correct grouping can be dynamic, ambiguous, and task dependent.
Method
TAG learns unsupervised iterative grouping and amortized inference over group assignments and group representations, using a Ladder network within a domain-agnostic framework.
Results
TAG converges within five iterations, outperforms a state-of-the-art convolutional network on a heavily cluttered classification task, and greatly improves semi-supervised learning over conventional Ladder networks.
Takeaways & Limitations
Grouping input elements and internal representations can improve handling of complex multi-object inputs and support sample-efficient learning.
Takeaways & Limitations
TAG assumes one level of global groups, which does not reflect the hierarchical structure of the world.
Abstract
from arXiv · showhide
We present a framework for efficient perceptual inference that explicitly reasons about the segmentation of its inputs and features. Rather than being trained for any specific segmentation, our framework learns the grouping process in an unsupervised manner or alongside any supervised task. By enriching the representations of a neural network, we enable it to group the representations of different objects in an iterative manner. By allowing the system to amortize the iterative inference of the groupings, we achieve very fast convergence. In contrast to many other recently proposed methods for addressing multi-object scenes, our system does not assume the inputs to be images and can therefore directly handle other modalities. For multi-digit classification of very cluttered images that require texture segmentation, our method offers improved classification performance over convolutional networks despite being fully connected. Furthermore, we observe that our system greatly improves on the semi-supervised result of a baseline Ladder network on our dataset, indicating that segmentation can also improve sample efficiency.
1 Introduction
Perceptual grouping structures inputs and representations into coherent groups, and TAG learns this grouping process iteratively without assuming a particular segmentation structure. The framework is domain agnostic and improves cluttered multi-digit classification, including in semi-supervised settings.
- Motivation: Perceptual grouping organizes inputs, object properties, and relations into coherent structures that support tasks such as manipulation and driving.The paper notes that analogous grouping applies across visual, audio, tactile, and abstract feature modalities.
- TAG framework: TAG learns efficient iterative inference that splits inputs and internal representations into groups without assuming a predefined segmentation structure.The model discovers relevant features and how to perform the splitting through end-to-end training.
- TAG framework: The framework uses an auxiliary denoising task to amortize posterior inference over object features and their groupings.TAG can operate unsupervised or alongside supervised classification and segmentation.
- Relation to attention: TAG structures inputs without discarding irrelevant information, and its grouping mechanism could complement attention-based processing.The paper proposes that group structure may help determine what to focus on while attention simplifies the task.
- Results: TAG outperforms a strong convolutional baseline on two-digit textured-MNIST classification despite using a fully connected network, with larger gains in semi-supervised learning using 1,000 labels.The results suggest grouping can improve sample efficiency on the evaluated dataset.
2 Iterative Amortized Grouping (TAG)
TAG learns to iteratively infer coherent input groups and their representations without predefined segmentations. It amortizes this inference through a shared parametric mapping trained with denoising, while supporting unsupervised or supervised objectives.
- Grouping: TAG splits inputs and internal representations into K groups, leaving the network to learn how to use the grouping for a given task.Latent binary variables gk,j indicate whether input element xj is assigned to group k.
- Iterative inference: The model jointly refines group assignments and group representations through iterative inference, analogous to alternating estimation in mixture models.Improving estimates over successive steps is intended to produce convergence toward a final solution.
- Amortization: Amortized inference trains a parametric mapping to approximate the inference result efficiently rather than deriving and repeatedly running a separate inference algorithm.The denoising objective trains the network from corrupted inputs without specifying the underlying generative model.
- Iterative update: At each iteration, TAG uses previous group estimates together with likelihood and modeling-error information to produce updated estimates.The network updates both group assignment probabilities and group reconstructions from quantities computed using the corrupted input.
- Parametric mapping: The same parametric mapping is reused across groups and iterations, with each group maintaining representations zk and assignment probabilities mk.For continuous inputs, zk represents the mean of a Gaussian distribution with learned constant variance v.
- Training objective: TAG can be trained completely unsupervised with denoising, while additional supervised terms can support tasks such as classification or segmentation.The framework’s denoising task encourages coherent groups that can be modeled efficiently.
3 Experiments and results
Tagger is evaluated on unsupervised grouping, denoising, segmentation, classification, and semi-supervised learning across Shapes and textured MNIST datasets. It learns useful groupings, converges quickly, generalizes across object and group counts, and improves classification relative to the reported baselines.
- Datasets and evaluation: The experiments use Shapes and TextureMNIST2, with evaluations covering denoising, AMI segmentation quality, classification, and semi-supervised learning.Tagger is trained unsupervised from raw inputs without ground-truth masks or labels, using four groups and three iterations.
- Unsupervised perceptual grouping: Tagger converges rapidly on Shapes and achieves significantly better AMI segmentation than Greff et al.Table 1 reports median performance over 20 random seeds and compares convergence across inference iterations.
- Unsupervised perceptual grouping: Tagger learns instance-level groupings for Shapes and textured MNIST, including separating identical objects and leaving an excess trained group empty.For textured MNIST, group representations encode textures and masks encode texture segments; iterative inference progressively resolves background, visible, and occluded digits.
- Unsupervised perceptual grouping: The Shapes examples show robustness when evaluation changes the number of objects or uses fewer groups than training.The examples include a two-group model tested on three objects, a four-group model tested on four objects, and the same model tested on two objects.
- Classification: Tagger significantly outperforms fully connected baselines on textured-MNIST classification and is on par with or better than the convolutional baseline.The advantage is more pronounced for overlapping two-digit inputs, where grouping is especially important.
- Semi-supervised learning: With 1,000 labels, Tagger achieves 10.5 % and 24.9 % error on the two reported variants, while fully supervised methods fail and the FC Ladder reaches 30.5 % and 68.5 %.For the two-digit case, Tagger with 1,000 labels even outperforms the Ladder baseline trained with all 50k labels.
4 Related work
TAG relates perceptual grouping to attention, synchronization-based binding, multi-causal models, and recurrent architectures. Its distinguishing contribution is an explicit mechanism for splitting inputs into multiple representations and learning grouping efficiently.
- Attention: Attention restricts processing to selected input regions, whereas TAG structures the entire input without discarding irrelevant information.The paper presents the approaches as complementary: grouping may help determine what attention should focus on.
- Synchronization-based grouping: TAG abstracts synchronization-based binding into discretized synchronization slots that represent groups.The work builds on neuronal synchronization models and related grouping architectures.
- Multi-causal models: Multi-causal models represent each object with a separate latent variable and approximate otherwise intractable posterior inference using EM-like procedures or sampling.TAG shares similar assumptions but uses a different trainable grouping mechanism.
- Recurrent architectures: RNNs can in principle implement binding, segmentation, attention, and metalearning, but general recurrence does not ensure that grouping is discovered easily.The paper argues that TAG adds an explicit mechanism rather than relying on recurrent networks to learn grouping as a byproduct.
- Recurrent architectures: Compared with the Neural Abstraction Pyramid, TAG explicitly splits inputs into multiple representations to learn grouping quickly and efficiently.The cited recurrent architecture uses lateral and feedback connections for resolving ambiguities and incorporating high-level information.
5 Future Work
The proposed future work addresses dependencies between groups, hierarchical grouping, and temporal information flow. These extensions target more realistic structure and possible scaling to video.
- Dependencies between groups: Assuming groups are independent objects or events is unrealistic, so conditional independence could be modeled by sharing the Ladder network’s top layer across groups.This would relax the current independence assumption while retaining separate group processing.
- Hierarchical grouping: A single level of global groups does not reflect hierarchical structure, motivating nested local groupings built from TAG components.The proposed construction collapses groups at a hidden layer and feeds the resulting representation into another Tagger.
- Temporal extension: Temporal information could flow forward between higher layers so grouping iterations proceed alongside changing inputs, potentially enabling video applications.The paper identifies movement as a strong segmentation cue.
6 Conclusion
The conclusion presents TAG as an efficient, unsupervised framework for learning grouping and amortized iterative inference. It reports benefits on cluttered classification, semi-supervised learning, and modality-general applicability.
- Contribution: TAG enables networks to learn grouping and corresponding amortized iterative inference in an unsupervised manner.The framework groups input elements and internal representations to handle complex multi-object inputs.
- Efficiency: The resulting iterative inference converges within five iterations.This is the conclusion’s headline efficiency claim.
- Classification: A fully connected Tagger significantly outperformed a state-of-the-art convolutional network on a heavily cluttered classification task.The comparison is presented as evidence that grouping benefits complex multi-object inputs.
- Semi-supervised learning: TAG greatly improved semi-supervised learning and exceeded conventional Ladder networks by a large margin.The conclusion emphasizes this result as especially notable.
- Scope: The method makes minimal assumptions about the data and can be applied to any modality.The conclusion frames modality generality as a central scope of the framework.
A.1 Notation
The notation defines dimensions, inputs, group assignments, model parameters, activations, and probabilistic quantities used to describe TAG’s grouping and denoising formulation.
- Dimensions and indices: N denotes input dimensionality, K the total number of groups, H the input and output dimension of the parametric mapping, and i the iteration index.The notation also uses j for input elements and k for group indices.
- Inputs and outputs: x is the input vector, ˜x the corrupted input, z_k the predicted input mean for group k, m_k group-assignment probabilities, and δz_k the modeling error.C(x) denotes the training loss for input x.
- Parameters: v is the input-estimate variance, W_h and W_u are projection weights, and Θ contains all Ladder parameters.The variance is used only in the continuous case.
- Functions: f() is the rectified linear activation, g() the logistic sigmoid, and softmax() the elementwise softmax over groups.These symbols specify the principal nonlinearities used in the model.
- Latent variables and probabilities: G_j is the latent variable encoding which group contains x_j, while p(x | ˜x), q(x), and q(x_j | g_k,j) denote posterior and learned conditional quantities.g_k,j abbreviates the event G_j = k.
A.2 Input
Tagger corrupts each input with modality-appropriate noise and learns to remove that noise through iterative, group-based reconstruction.
- A.2 Input: Binary inputs are corrupted with bitflip noise, while real-valued inputs receive Gaussian noise before denoising.The Shapes experiments use β = 0.2 for bitflip noise, and real-valued inputs use σinput = 0.2.
- A.2 Input: During training, several groups iteratively maintain reconstructions and masks derived from the corrupted input.The shared network updates each group’s reconstruction zi and mask mi across iterations.
A.3 Group Assignments
TAG represents group assignments as normalized per-element probabilities, initialized randomly through a softmax so each input element is distributed across groups.
- A.3 Group Assignments: Each group k has a vector mk whose entries assign every input element or pixel a probability over K groups.For each input element xj, mk,j = q(gk,j).
- A.3 Group Assignments: The assignment probabilities for every input element sum to one across groups.This normalization is the condition enforced by Equation 7.
- A.3 Group Assignments: Assignments are initialized by sampling auxiliary values from a standard Gaussian distribution and normalizing them with a softmax.The random initialization is analogous to expectation maximization while satisfying the per-element normalization constraint.
A.4 Predicted Inputs
TAG maintains one predicted input reconstruction per group and converts each reconstruction into a likelihood model appropriate to binary or continuous data.
- A.4 Predicted Inputs: Tagger maintains an input reconstruction zk for every group k.
- A.4 Predicted Inputs: For binary inputs, zk is passed through a sigmoid and interpreted as the reconstruction probability.
- A.4 Predicted Inputs: The binary reconstruction model combines the predicted probability with the bitflip corruption process to compute ˜zk.
- A.4 Predicted Inputs: For continuous inputs, zk represents the mean of an isotropic Gaussian with learned variance.
- A.4 Predicted Inputs: All reconstructions are initialized to the data expectation, using 0.5 for TextureMNIST and 0.26 for Shapes.
A.5 Modeling Error
TAG derives modeling-error signals from the corrupted input and iteratively updates group reconstructions and assignments through a shared parametric network.
- A.5 Modeling Error: The modeling error δzk is proportional to the negative derivative of the corrupted-input cost with respect to zk.Using the corrupted input prevents clean-input information from entering the denoising network.
- A.5 Modeling Error: For continuous inputs, the error signal is proportional to the residual between the corrupted input and reconstruction, weighted by group assignment and likelihood.
- A.5 Modeling Error: The binary case defines its modeling-error computation using the bitflip probability β.
- A.5 Modeling Error: The framework can be implemented with recurrent iterations that update reconstructions and assignments using shared network transformations.The pseudocode runs for T iterations, and assignments are projected through softmax while binary reconstructions use a logistic sigmoid.
- A.5 Modeling Error: Training computes gradients through time and uses ADAM with batch-size 100.
- A.5 Modeling Error: Real-valued and binary inputs use Gaussian and bitflip corruption, respectively, before iterative processing.