PaperlayerView exact revision on arXiv

Source-linked AI summary

OrthoRF: Exploring Orthogonality in Object-Centric Representations

Despoina Touska, Bastiaan Onne Fagginger Auer, Alexandru Onose, Tejaswi Kasarla, Luis Armando Pérez Rey, Maximilian Lipp, Lyubov Amitonova, Martin R. Oswald, Pascal Cerfontaine

arXiv:2610.35203v1cs.CVcs.AIcs.LG

TL;DR

Synchrony-based models encode object affiliation in activation phases, but distributed outputs can require post-hoc clustering and become uncertain in overlap regions. OrthoRF adds competitive binding and an orthogonality loss to RF; it matches or surpasses relevant models and recovers occluded parts.

  • Problem

    Synchrony-based models produce distributed representations that require post-hoc clustering to recover objects and become uncertain in overlap regions.

    Source: §1 · p. 2

  • Method

    OrthoRF extends RF with competitive binding and an inner-product orthogonality loss that aligns each object to a distinct phase axis.

    Source: §1 · p. 3, §6 · p. 10

  • Results

    OrthoRF matches or surpasses relevant models in unsupervised object discovery and recovers occluded parts.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    Orthogonality provides an effective inductive bias for turning distributed representations into discrete, more directly usable object encodings.

    Source: §6 · p. 10

    OrthoRF converges more slowly than RF, at approximately 200k versus 100k steps, and training can rarely stall in suboptimal states.

    Source: §6 · p. 10

Abstract

from arXiv · show

Neural synchrony is hypothesized to help the brain organize visual scenes into structured multi-object representations. In machine learning, synchrony-based models analogously learn object-centric representations by storing binding in the phase of complex-valued features. Rotating Features (RF) instantiate this idea with vector-valued activations, encoding object presence in magnitudes and affiliation in orientations. We propose Orthogonal Rotating Features (OrthoRF), which enforces orthogonality in RF's orientation space via an inner-product loss and architectural modifications. This yields sharper phase alignment and more reliable grouping. In evaluations of unsupervised object discovery, including settings with overlapping objects, noise, and out-of-distribution tests, OrthoRF matches or outperforms current models while producing more interpretable representations, and it eliminates the post-hoc clustering required by many synchrony-based approaches. Unlike current models, OrthoRF also recovers occluded object parts, indicating stronger grouping under occlusion. Overall, orthogonality emerges as a simple, effective inductive bias for synchrony-based object-centric learning.

1 INTRODUCTION

Object-centric learning decomposes visual scenes into modular object representations, addressing the binding problem of integrating features into coherent objects. OrthoRF extends Rotating Features with orthogonal phase-space encoding to improve interpretability and grouping, including in overlap regions.

1 INTRODUCTION

  • Object-centric learning discovers modular, compositional object representations that support generalization and relational reasoning across visual tasks.The broader goal of scene decomposition has long been pursued in computer vision.Source: §1 · p. 1
  • Figure 2 contrasts RF's distributed orientation-space content with OrthoRF's distinct latent axes and cleaner separation of overlap regions.The proposal is motivated by the goal of improving interpretability and representational capacity in synchrony-based models.Source: Figure 2 · p. 2, §1 · p. 2
  • Synchrony-based models use relative phases to bind features, but their distributed representations require post-hoc clustering and become uncertain in overlap regions.Evaluations often exclude overlaps, potentially underrepresenting settings where robust binding is most needed.Source: Figure 1 · p. 1, §1 · p. 2
  • OrthoRF extends RF by enforcing orthogonal object encoding in phase space through competitive binding and an inner-product loss.This directs objects toward distinct phase axes and one-hot-like encodings.Source: §1 · pp. 2–3
  • OrthoRF matches or surpasses current methods across object-discovery settings, removes post-hoc clustering, and recovers occluded object parts.Its orthogonal encoding yields sharper phase alignment and more reliable grouping.Source: §1 · p. 3

2 BACKGROUND: ROTATING FEATURES

Rotating Features lifts scalar activations into vector-valued features, using magnitudes for feature presence and orientations for object affiliation. Its shared-weight layers gate and rescale orientations, while reconstruction training and vector addition encourage alignment for features of the same object.

2 BACKGROUND: ROTATING FEATURES

  • RF lifts each feature into an n-dimensional vector, whose magnitude encodes feature presence and orientation encodes object affiliation.The lifting applies to input images and activations at any layer.Source: §2 · p. 3
  • RF applies shared weights to vector inputs and their per-feature magnitudes, then combines the outputs to form gated magnitudes.The gated magnitude is passed through ReLU and used to rescale the output orientation.Source: §2 · p. 3
  • The decoder reconstructs the image by scaling the per-pixel magnitude of final activations and applying a sigmoid.Training uses mean squared error between the input and reconstructed images; vector additions produce constructive or destructive interference.Source: §2 · p. 3
  • The paper notes an inconsistency between its description and code for calculating the bound magnitude, and uses the code formula because it performs better.The discrepancy concerns the calculation of mbind.Source: §2 · p. 3

3 METHOD

OrthoRF addresses RF’s distributed object representations and need for post-hoc clustering through competitive assignment and orthogonality in orientation space. Its design also uses magnitude gating to distinguish visible regions from occluded shape content.

3.1 MOTIVATION

  • OrthoRF targets distributed object representations and the post-hoc clustering they require by constraining objects to distinct components in orientation space.The motivation also identifies overlap-driven uncertainty as a cue for occlusion recovery.Source: §3.1 · p. 4

3.2 ORTHOGONAL ROTATING FEATURES

  • Per-layer softmax competition promotes object-wise specialization across orientation components, with encoder-output centering used to prevent one component from dominating.The softmax produces winner-take-most assignments.Source: §3.2 · p. 4
  • The orthogonality loss penalizes squared off-diagonal inner products between centered component vectors, encouraging their cross-component similarities toward zero.The loss averages over samples and component pairs, and its contribution to the full objective is weighted by λ.Source: §3.2 · pp. 4–5
  • Weight sharing across orientation components gives the autoencoder permutation equivariance along the orientation axis.Permuting the input orientation components permutes the output in the same way.Source: §3.2 · p. 5
  • At the final binding step, magnitude gates visibility while pre-gated content ψ contains occlusion-complete shapes.The text offers learned shape priors under reconstruction as a plausible explanation and says clean gating depends on softmax over orientation channels.Source: §3.2 · p. 5

4 EXPERIMENTS

Experiments evaluate OrthoRF on object-discovery datasets spanning variable object counts, overlap, and noisy SEM images. Results show improvements over RF in several settings, including SEM robustness and shape completion, alongside tighter phase-space grouping.

4.1 EXPERIMENTAL SETTINGS

  • The evaluation follows prior protocols across 4Shapes, MNIST&Shapes, Shapes, and synthetic clean/noisy SEM, using ARI and MBO metrics for object-discovery comparisons.The SEM images contain four laterally displaced, occluding layers; its noisy variant adds Gaussian blur and noise. Models were trained for 100–200k steps with Adam.Source: §4.1 · pp. 5–6

4.2 RESULTS

  • For 4Shapes shape completion, thresholded ψfinal reaches approximately 0.98 MBOOV_i at n = 5, versus approximately 0.80 for zout from both RF and OrthoRF.Thresholding ψfinal permits multi-label pixels in overlaps, unlike k-means assignments that give each pixel one label.Source: Table 1 · p. 6, §4.2 · p. 7
  • On SEM, OrthoRF beats RF on noise-free ARI-BG (0.9908 vs 0.9551) and noisy ARI-BG (0.8356 vs 0.7043).The results include severe occlusion and noise; Table 2 also compares non-neural baselines.Source: Table 2 · p. 7, §4.2 · p. 8
  • Clean-trained OrthoRF retains ARI-BG 0.9836 when tested on noisy SEM, while noisy-trained OrthoRF falls to 0.7610 when tested on clean SEM.The corresponding same-condition values are 0.9908 on clean data and 0.8356 on noisy data; the authors suggest smoothed boundaries may underfit sharp clean edges.Source: §4.2 · p. 8
  • On variable Shapes, OrthoRF achieves the best performance among compared models when object counts and shape combinations vary; on MNIST&Shapes, it matches RF and CAE on most metrics and has the highest ARI-BG.DBM and SA do not reliably discover objects on MNIST&Shapes.Source: Table 4 · p. 8, §4.2 · p. 8, Table 3 · p. 8
  • On 4Shapes, softmax with centering and orthogonality loss together yield near-perfect ARI and the highest MBOi for n = 5, whereas neither alone performs strongly.Table 5 ablates softmax with centering and orthogonality loss, with λ = 0 disabling the loss.Source: Table 5 · p. 8, §4.2 · p. 9
  • OrthoRF forms tighter output clusters than RF; its 4Shapes inter-cluster angles are near-orthogonal with substantially lower variance, while RF's higher mean angle is more variable.OrthoRF's 4Shapes angle standard deviation is 4.39 versus 13.91 for RF; on SEM, OrthoRF averages 80° and RF 82.12°.Source: Table 6 · p. 9, §4.2 · p. 9, Figure 6 · p. 9

5 RELATED WORK

Related work covers object-centric learning through slot-based representations and synchrony-based phase binding, alongside broader uses of orthogonality in deep learning.

Object-centric learning

  • Object-centric learning includes unsupervised scene decomposition methods such as MONet, IODINE, and GENESIS, while Slot Attention uses competing slots to bind distinct objects.Later work extends slot-based approaches to Transformer encoders and decoders, 3D scenes, and video.Source: §5 · p. 10

Synchrony-based learning

  • Synchrony-based approaches encode binding through phase, including CAE, its contrastive extension CtCAE, and higher-dimensional Rotating Features; other models explicitly synchronize phases with oscillators.The passage distinguishes synchrony that emerges dynamically from synchronization imposed by mechanisms such as Kuramoto oscillators.Source: §5 · p. 10

Orthogonality

  • Prior deep-learning work uses orthogonality for initialization, training stability and generalization, discriminative features, disentanglement, and incremental object learning.The cited examples include orthogonality at multiple training levels to mitigate catastrophic interference in open-world object detection.Source: §5 · p. 10

6 CONCLUSION

OrthoRF addresses overlap-related uncertainty in RF’s distributed representations through competitive binding and orthogonality, producing sharper alignment and removing post-hoc clustering. It matches or surpasses relevant models, recovers occluded parts, and uses orthogonality to make distributed representations more discrete and usable.

6 CONCLUSION

  • Across unsupervised object discovery, OrthoRF matches or surpasses relevant models and recovers occluded parts.The authors describe orthogonality as an effective inductive bias that regularizes distributed representations into discrete ones, resolving overlap ambiguity and improving downstream usability.Source: §6 · p. 10
  • OrthoRF couples competitive binding with an inner-product orthogonality loss to align each object to a distinct phase axis, sharpen alignment, and remove clustering.The conclusion identifies overlap-related uncertainty in RF’s distributed representations as the limitation being addressed.Source: §6 · p. 10
  • OrthoRF converges more slowly than RF, at approximately 200k versus 100k steps, and training can rarely stall in incomplete separation or multi-object collapse.Increasing λ or lowering the learning rate stabilizes training.Source: §6 · p. 10

A REPRODUCIBILITY STATEMENT

The reproducibility statement describes how OrthoRF can be reproduced from an RF implementation, despite code not yet being publicly available. It identifies the required orthogonality-loss and competitive-binding changes and points to the model configuration and training setup.

A REPRODUCIBILITY STATEMENT

  • Code is not yet publicly available, but the authors say OrthoRF can be reproduced from an RF implementation with the orthogonality loss and competitive-binding modifications.The latter modifications are described in the main paper, while Appendix B provides experiment details.Source: A REPRODUCIBILITY STATEMENT · p. 15
  • The orthogonality-loss procedure takes batched rotation-encoder outputs and returns a scalar loss based on the mean squared off-diagonal pairwise inner products.The pseudocode centers the outputs, computes pairwise inner products, and excludes diagonal entries from the loss.Source: A REPRODUCIBILITY STATEMENT · p. 15

B.1 ARCHITECTURE AND HYPERPARAMETERS

  • OrthoRF uses a convolutional autoencoder with fully connected layers, following RF; Appendix B specifies dataset architectures and training hyperparameters.Tables 9 and 10 give architectures, and Table 11 lists training hyperparameters.Source: §B.1 · p. 15

B.2 DATASET DETAILS

The appendix describes the datasets and evaluation procedures, then examines thresholding, OrthoRF’s output, and the effect of orthogonality strength across orientation-space dimensions. The supplementary results show object separation and recovery of occluded parts, while smaller slot counts depend more on orthogonality regularization.

B.2 DATASET DETAILS

  • The evaluation covers 4Shapes, MNIST&Shape, Shapes, and a synthetic SEM dataset; 4Shapes contains four potentially overlapping shared-color shapes, and SEM has noisy and noise-free variants with layered occlusion.The SEM dataset is designed to test layer separation and robustness to acquisition noise; it is not publicly released but can be reproduced from its description.Source: §B.2 · pp. 15–16

B.3 EVALUATION DETAILS

  • Evaluation includes visible-only discovery, which excludes occluded regions, and shape completion, which uses per-shape 4Shapes labels to assess recovery under occlusion.The authors consider visible-only evaluation less fair, particularly for boundaries and overlaps.Source: Figure 8 · p. 15, §B.3 · p. 16, Figure 9 · p. 16

B.4 THRESHOLDING

  • For metric computation, ψ-thresholding discards low-variance background channels and applies a fixed threshold to the remaining maps; a wide threshold range yields nearly identical binarizations.The appendix gives 0.1 as a convenient default; Figure 10 shows object-related dimensions with higher variance and separated object and background pixels.Source: §B.4 · p. 16, Figure 10 · p. 17

C.1 EFFECT OF ORTHOGONALITY MECHANISM

  • On 4Shapes, OrthoRF’s final-layer outputs show object separation and reconstruct occluded parts, while its final output can still contain holes at occlusions.The comparison figure contrasts distributed object content in RF’s output with clearer separation in OrthoRF’s ψ output.Source: §C.1 · pp. 16, 18

C.2 VARYING λ

  • For smaller slot counts, especially n = 5 or 6, low λ substantially worsens ARI and MBO; larger n keeps performance stable across a wide λ range.The authors interpret this as higher-dimensional spaces naturally supporting object separation, while orthogonality becomes important when representational capacity is limited.Source: §C.2 · p. 18, Table 8 · p. 18, Figure 11 · p. 19

C.3 QUANTITATIVE RESULTS FOR THE 4SHAPES DATASET

The authors reran RF on 4Shapes for a fair comparison with OrthoRF and report that their results closely match the official RF numbers.

C.3 QUANTITATIVE RESULTS FOR THE 4SHAPES DATASET

  • On 4Shapes, the replicated RF experiments closely match the official results published by L¨owe et al. (2024a), supporting a fair comparison between the rerun and reported numbers.Table 1 reports the published RF results, while Table 7 shows the replicated outcomes.Source: §C.3 · p. 19

C.4 QUALITATIVE RESULTS FOR THE 4SHAPES DATASET

On 4Shapes, OrthoRF performance remains largely stable across orientation-space dimensionalities, and visualizations show that only necessary dimensions are used.

C.4 QUALITATIVE RESULTS FOR THE 4SHAPES DATASET

  • Performance remains largely stable as OrthoRF's orientation-space dimensionality varies.The section reports this comparison in Table 1 for the 4Shapes dataset.Source: §C.4 · p. 19
  • For orientation-space dimensionalities n ∈ {6, 7, 8, 9}, visualizations show that only necessary dimensions are used.The qualitative results are shown in Figs. 13–16.Source: §C.4 · p. 19
  • The remaining orientation-space dimensions stay empty in the visualizations.Source: §C.4 · p. 19

C.5 QUALITATIVE RESULTS FOR THE SEM DATASET

Qualitative results show OrthoRF separating SEM layers into distinct dimensions on both noise-free and noisy data, sparse activations on Shapes, and ψ information that supports reconstruction of missing deeper-layer parts.

C.5 QUALITATIVE RESULTS FOR THE SEM DATASET

  • On noise-free and noisy SEM images, OrthoRF separates layers into distinct dimensions, while RF distributes layer content across orientations.The section compares the models in Figs. 17 and 19; Fig. 18 shows OrthoRF output alongside its thresholded version for metric binarization.Source: §C.5 · p. 19

C.6 QUALITATIVE RESULTS FOR THE SHAPES DATASET

  • On Shapes at n = 8, OrthoRF activates only necessary dimensions, leaving the rest unused and yielding sparse object representations.The visualizations are shown in Fig. 21.Source: §C.6 · p. 20

C.7 EFFECT OF MAGNITUDE

  • Although zout has gaps in deeper-layer shapes such as the ellipsoid, ψ provides information about the missing parts for more complete reconstruction.The outputs are shown in Fig. 20.Source: §C.7 · p. 20
Loading 2610.35203v1…