Source-linked AI summary
Weakly supervised concept Bottleneck Learning for Robust Two stage Object centric visual reasoning
Sparsh Tiwari, Gesina Schwalbe, Bettina Finzel
TL;DR
Weakly supervised object-centric concept bottlenecks must ground interpretable predicates despite expensive labels and instability from sparse supervision. D-OCB combines stabilized variational learning, orthogonal concept subspaces, and dynamic capacity allocation, achieving accurate concept alignment and downstream reasoning with minimal supervision. The approach is limited by manually specified slot counts and information loss from latent compression on highly textured data.
Problem
Object-centric concept bottlenecks remain underexplored under weak supervision, while sparse labels create representation collapse, entanglement, and fixed-capacity allocation problems.
Method
D-OCB is a weakly supervised object-centric slot-VAE bottleneck using calibrated noise floors, orthogonal concept subspaces, dynamic dimensionality allocation, and independent symbolic reasoning.
Results
D-OCB achieves near-oracle downstream performance with frozen symbolic reasoners across CLEVR, CLEVR-Hans, and HAM10000 under 1%–15% supervision.
Takeaways & Limitations
D-OCB provides human-aligned predicates that can drive diverse off-the-shelf symbolic reasoners without joint backpropagation.
Takeaways & Limitations
D-OCB requires manually defining the maximum number of object slots, and latent compression can discard fine-grained details needed by highly textured datasets.
Abstract
from arXiv · showhide
Two-stage neuro-symbolic architectures provide an elegant paradigm for visual problem solving by cleanly separating connectionist perception of predefined symbols from possibly later defined relational reasoning thereon. However, anchoring high-level predicates into visual frames typically necessitates annotations that are expensive to acquire. In this work, we introduce the Dynamic Orthogonal Concept Bottleneck (D-OCB), an object-centric slot- VAE framework designed to extract human-aligned symbolic predicates under extremely weak supervision. D-OCB eliminates the arduous manual tuning of loss-balancing coef- ficients by dynamically learning optimal hyperparameter allocations during training. To infuse prior knowledge on independence of concept categories, in addition to standard re- construction self-supervision we penalize correlation across concept subspaces. Crucially, to combat the instability of very low supervision regimes, D-OCB incorporates a dynamic di- mensionality allocation mechanism; this adaptive formulation allows well-represented con- cepts to yield latent dimensions to underperforming concepts that are lagging behind, effectively preventing representation collapse and significantly improving overall concept accuracy. Through an extensive empirical evaluation, we demonstrate that our framework achieves high concept alignment and downstream visual reasoning accuracy using minimal label budgets, matching or outperforming end-to-end paradigms.
1. Introduction
Two-stage neuro-symbolic systems separate visual perception from symbolic reasoning, but weakly supervised object-centric concept grounding remains difficult because concepts require costly labels and unstable representations. D-OCB addresses these challenges with a weakly supervised perceptual bottleneck that stabilizes and reallocates concept representations.
- Two-stage neuro-symbolic architectures map raw visual inputs to explicit concepts before symbolic engines reason over discovered predicates.
- Weakly supervised object-centric concept bottlenecks remain underexplored because traditional approaches require dense concept labels or high-variance reinforcement learning.
- Sparse labels can cause representation collapse, concept entanglement, and inefficient fixed-dimensional allocation across concepts.
- D-OCB grounds human-aligned predicates with minimal annotations while stabilizing concepts through calibrated noise floors, orthogonal subspaces, and dynamic dimensionality allocation.The allocation mechanism redistributes feature-space capacity according to concept convergence before Hungarian matching.
2. Related Work
The related work connects D-OCB to concept bottlenecks, object-centric slot representations, and two-stage neuro-symbolic reasoning. These traditions provide interpretability and compositionality but leave challenges in weak supervision, concept disentanglement, and stable slot-based learning.
- CBMs and Generative Realignment: Concept bottleneck models constrain intermediate dimensions to human-understandable concepts but typically use holistic encodings and dense supervision.
- Object-Centric Tokenization and Slot-Based Bottlenecks: Object-centric tokenization decomposes scenes into distinct slot tokens, supporting spatial specialization and compositional reasoning in multi-object settings.
- Object-Centric Tokenization and Slot-Based Bottlenecks: Extending slot attention to interpretable concept layers introduces training instabilities that D-OCB targets with a block-diagonal concept representation.
- Neuro-Symbolic Reasoning and Two-Stage Pipelines: Neuro-symbolic systems combine neural perception with symbolic deduction, while end-to-end differentiable frameworks jointly optimize both stages.
3. Approach: Dynamic Orthogonal Concept Bottleneck
D-OCB maps images into object-centric slots and partitioned concept subspaces, then predicts human-interpretable predicates through independently trained heads. Its objective stabilizes weakly supervised learning and dynamically reallocates latent capacity across concepts.
- Architecture: D-OCB maps raw visual scenes to object-slot concept valuations that a standalone symbolic solver uses for downstream deduction.
- Architecture: A feature extractor and slot-attention module produce N object slots, with hidden slot dimension D typically set to 128.
- Concept Representation: The latent concept space is partitioned into independent concept-specific subspaces whose dimensionalities are dynamically adjusted to support struggling concepts.
- Concept Representation: Independent prediction heads convert each concept subspace into class probabilities and then discrete predicate outputs.
- Architecture and Optimization Objectives: The training objective combines reconstruction, concept prediction, and cross-correlation losses, with scaling coefficients updated from moving averages of gradient variances.
- Generative Bottleneck: Calibrated Fixed Noise Floors: D-OCB replaces learned per-sample variance with concept-specific fixed noise floors calibrated once on labeled data, preventing reconstruction noise from entering variance.
- Dynamic Dimensionality Allocation: Hungarian matching aligns latent dimensions with concepts, while LDA and Johnson-Lindenstrauss projections dynamically transfer capacity to underperforming concepts without manual tuning.
4. Experiments and Results
Experiments evaluate D-OCB for weakly supervised concept extraction, frozen symbolic reasoning, and robustness to confounders across CLEVR, CLEVR-Hans variants, and HAM10000. The results report accurate concept alignment, compatibility with multiple symbolic solvers, and robust out-of-distribution performance under limited annotation budgets.
- Experimental setup: D-OCB is evaluated on CLEVR, CLEVR-Hans3, CLEVR-Hans7, and HAM10000 using weak supervision and largely unannotated training pools.The experiments cover visual concept extraction and logical scene classification, with sample budgets ranging from 1% to 15% in the described configurations.
- Experimental setup: The evaluation compares D-OCB with concept bottleneck, object-centric, Slot Attention, and SlotVAE baselines.Baselines include CEM, ProbCBM, SlotFormer, Vanilla+SA, SlotVAE-NO-KL, and SlotVAE-KL.
- Evaluation metrics: Concept accuracy and Macro-F1 assess perceptual grounding, while rule-evaluation accuracy and F1 assess downstream task execution.Predicted object slots are aligned with ground-truth annotations using Hungarian matching.
- RQ1: Visual Concept Extraction and Predicate Accuracy: 73.4% mean accuracy is achieved on HAM10000 with a DINO backbone at 15% supervision.The reported result accompanies improvements attributed to VAE reconstruction, dynamic dimension assignment, and learned variance relative to specified ablations.
- RQ2: Framework-Agnostic Logical Reasoning Evaluation: D-OCB extracts accurate predicates that remain stable for frozen symbolic reasoners without joint end-to-end logic backpropagation.The evaluation exports frozen valuation tensors to DILP, decision trees, Bayesian networks, and NS-CL.
- RQ3: Confounder Resistance and Out-of-Distribution Reasoning: At 1%–5% supervision, D-OCB paired with DILP achieves robust OOD accuracy with marginal generalization gaps on CLEVR-Hans3 and CLEVR-Hans7.The reported gaps are 1.1±0.8 on CLEVR-Hans3 and 3.00±0.56 on CLEVR-Hans7, matching densely supervised baselines and resisting confounders without explicit annotations.
5. Conclusion
D-OCB is presented as a weakly supervised visual feature extractor that supports symbolic reasoning with minimal annotations and strong downstream generalization. The architecture remains constrained by manually specified slot capacity and possible loss of fine-grained detail.
- Conclusion: Table 5 compares D-OCB with Slot-VAE on HAM10000 reasoning accuracy across diagnostic rules, backbones, supervision levels, and reasoning frameworks.Results are reported as mean ± standard deviation over 5 folds, with baseline F1 scores converted to percentages.
- Conclusion: D-OCB provides weakly supervised human-aligned predicates for downstream symbolic reasoning, using supervision ranging from 1% to 15%.The framework is evaluated as a perceptual module for frozen symbolic reasoners rather than as a jointly trained end-to-end architecture.
- Conclusion: D-OCB’s extracted predicates drive diverse off-the-shelf symbolic reasoners to near-oracle performance without joint backpropagation.Evaluated reasoning frontends include statistical, rule-based, and differentiable logic methods.
- Conclusion: Structural decoupling of perception and logical induction supports out-of-distribution generalization against confounding factors.The paper reports confounder-resistance evaluation on CLEVR-Hans3 and CLEVR-Hans7.
- Conclusion: The architecture requires manual, a priori definition of the maximum number of object slots and can lose fine-grained detail during bottleneck projection.The latter limitation may degrade performance on highly textured datasets.