PaperlayerView exact revision on arXiv

Source-linked AI summary

Overlapping Visual Grouping Without Semantic Priors

Teemu Saukkio, Hashem Haghbayan, Juha Plosila

arXiv:2609.27423v1cs.CV

TL;DR

Many vision systems organize images toward predefined interpretations, while open-ended use calls for retaining other sensor-supported structure. DPG groups measurements across complementary domains, allowing overlapping groups and reprocessing. On BSDS500, it retained substantial correspondence with human-annotated regions while preserving additional organization.

  • Problem

    When future relevance is unknown, category-, prompt-, or partition-based representations may not retain other organization present in sensor input.

    Source: §1 · p. 2

  • Method

    DPG forms sensor-grounded spatial groups across complementary measurement domains and preserves overlapping parent and localized groups for reprocessing.

    Source: §1 · p. 2

  • Results

    Across BSDS500, DPG retained substantial correspondence with human-annotated regions while preserving additional measurement-supported organization in the same observation.

    Source: §4 · p. 29, §5 · p. 33

  • Takeaways & Limitations

    The results support an intermediate representation that retains multiple sensor-supported organizations without semantic labels or reduction to one mutually exclusive partition.

    Source: §5 · p. 33

    The evaluation covers static RGB images and three implemented domains, and BSDS500 annotations do not exhaustively label additional overlapping or measurement-supported structures.

    Source: §4.5 · p. 32

Abstract

from arXiv · show

Most computer-vision systems organize visual input toward a predefined interpretation, such as semantic categories, prompted regions, learned object-like representations, or a single spatial partition. This work considers an earlier stage of visual organization: the formation of candidate perceptual units directly from sensor measurements before their identity, meaning, or task relevance is known. We introduce Domain Parent Grouping (DPG), a sensor-grounded grouping method in which complementary measurement relationships are represented in separate processing domains. Spatially connected groups formed within these domains are related through cross-domain overlap, yielding a non-exclusive grouping representation rather than a single mutually exclusive segmentation. This representation retains broader and more localized groups, as well as alternative grouping boundaries over the same image locations, simultaneously available. DPG also includes a native mechanism for reprocessing selected group content, in which input-relative measurement ranges allow the observational resolution to change while preserving previously formed groups. DPG is implemented using three domains representing locally contextualized luminance, direct chromatic relationships, and contextual chromatic relationships. Experiments on the BSDS500 dataset demonstrate the benefit of combining the three domains. The results further show that DPG forms measurement-supported groups corresponding to low-level image structure, and that these groups exhibit measurable correspondence with human-annotated regions and boundaries. This demonstrates that structured visual organization can emerge directly from relationships among sensor measurements.

1 Introduction

Many vision systems organize images around predefined categories, prompts, learned representations, or partitioning criteria. DPG instead forms sensor-grounded candidate groups before their identity or task relevance is known, preserving multiple organizations and supporting reprocessing.

1 Introduction

  • Existing approaches generally determine their intended output through categories, prompts, learned representations, predefined grouping criteria, or a single spatial partition.Promptable methods still require an external prompt to specify the region, while category-independent methods use predefined similarity, connectivity, or boundary criteria.Source: §1 · pp. 1–2
  • For open-ended environments, retaining only structures defined by existing categories, prompts, or one partitioning criterion may leave other sensor-input organization unrepresented.The stated goal is an intermediate representation for later learning and task-dependent interpretation, not a replacement for recognition or segmentation.Source: §1 · p. 2
  • DPG forms spatially connected, sensor-supported candidate groups without requiring their identity or later relevance to be specified during grouping.Broader groups retain direct spatial relations to smaller groups produced in other domains.Source: §1 · p. 2
  • Three complementary domains represent locally contextualized luminance, direct chromatic relationships, and contextual chromatic relationships.Selected group content can be reprocessed to expose internal measurement variation as additional groups while retaining the broader group.Source: §1 · p. 2

2.1 Problem Formulation

An image supplies normalized RGB measurements at spatial positions, but not which positions form a structure. The formulation treats grouping as identifying measurement-supported spatial units, including overlapping alternatives, that can be reprocessed within a selected extent.

2.1 Problem Formulation

  • The formulation seeks measurement-supported spatial groups rather than assigning object identity, surface class, shape category, or target boundaries in advance.Pixel measurements alone specify neither which positions belong together nor where structures begin and end.Source: §2.1 · p. 3
  • Illumination, shading, texture, and reflection can vary measurements within one structure, while different structures may be only weakly separated.Thus, similar measurements need not belong together, and differing measurements need not imply separate groups.Source: §2.1 · p. 3
  • Different measurement relationships can support distinct groups that share pixel positions, so the output need not be one mutually exclusive image partition.Groups are represented as masks of spatially connected positions, with boundaries following each mask's extent.Source: §2.1 · pp. 3–4
  • DPG evaluates measurement relationships in separate luminance, vector-colour, and colour-consistency domains, then retains groups with sufficient spatial support across domains.A retained group can define a restricted processing extent, where selected measurements and spatial context may produce further groups alongside the original.Source: §2.1 · p. 4

2.2 Method

The method derives separate measurement-based class maps using context-adaptive luminance and direct and compensated chromatic processing, then forms spatial groups within each domain. It relates groups across domains by spatial overlap and permits selected content to be reprocessed at a different observational resolution while preserving existing groups.

2.2 Method

  • DPG’s processing domains and transformations were derived from controlled visual experiments that examined how measurement relationships affect distinguishable spatial structure.Parameters were selected through iterative qualitative inspection of heterogeneous development images, with evaluation images excluded from parameter selection.Source: §2.2 · pp. 4–5

2.2.1 Luminance-Domain Processing

  • Luminance processing adapts its deviation threshold to local variation, assimilating pixels below threshold toward the local mean and strengthening locally distinct deviations.Input-relative luminance classes are recalculated for each image or selected group, allowing a restricted range to expose previously unresolved internal structure.Source: §2.2.1 · pp. 6–7

2.2.2 Colour-Domain Processing

  • The colour domains use direct RGB channel relationships for chromatic classes; the colour-consistency domain first applies a global compensation opposite the strongest direction-specific mean chromatic displacement.This compensation makes residual chromatic relationships available for classification; the direct relative channel-difference measure is invariant to common positive scaling except near zero intensity or under channel clipping.Source: §2.2.2 · p. 8

2.2.3 Spatial Grouping and Cross-Domain Parent Relations

  • Connected components are maximal connected subsets of pixels sharing a class, formed independently within each domain; components below a domain-specific minimum area are left unassigned.Spatially separated regions remain distinct even if they share a measurement class.Source: §2.2.3 · p. 11
  • Across domains, DPG relates retained connected groups by sufficient overlap relative to the smaller group, assigning the larger as parent; relations are non-exclusive, and only parent-designated groups enter the final output.Equal-area groups do not form parent-child relations; a group may have multiple parents or children.Source: §2.2.3 · p. 12

2.2.4 Reprocessing of Selected Group Content

  • DPG reprocesses selected parent-mask pixels within their bounding rectangle using the same domains and fixed parameters, retaining new groups alongside the original for progressive resolution adjustment.Input-relative ranges and contextual relationships are recalculated from the selected content, potentially revealing groups that remained internal to the original parent.Source: §2.2.4 · pp. 12–13

2.3 Experimental Methods

The experiments combine qualitative analysis of DPG group structure with quantitative comparisons on BSDS500, using measures of correspondence to human annotations and proposal-set size. The protocol evaluates domain combinations and compares complete DPG with three category-independent grouping methods.

2.3.1 Dataset and Experimental Protocol

  • All 200 BSDS500 test images were evaluated against multiple human annotations, which capture principal regions but do not exhaustively enumerate overlapping or illumination-dependent structures.The dataset contains 500 images split into 200 training, 100 validation, and 200 test images.Source: §2.3.1 · p. 13

2.3.2 Implementation and Execution Environment

  • DPG and comparison methods ran in the same software environment without GPU acceleration, with parameters fixed before test evaluation and luminance discretization recalculated for each input.Reported runtimes include the required input preparation, domain calculations, parent-group construction, and proposal-mask formation; file loading, visualization, and metric calculation are excluded.Source: §2.3.2 · p. 14

2.3.3 Comparison Methods and Parameter Selection

  • The comparison set comprised Felzenszwalb, Quickshift, and watershed-RAG, with parameters selected using the validation set and then held fixed for test evaluation.Felzenszwalb parameters were calibrated toward published BSDS500 targets; Quickshift and watershed-RAG parameters were selected by validation-set covering.Source: §2.3.3 · pp. 15–16

2.3.4 Evaluation Metrics

  • Because DPG yields potentially overlapping masks, evaluation compares proposals with annotations using region covering, region recall, boundary precision, boundary recall, boundary F1, group count, and runtime.Region covering weights each annotated region’s best proposal IoU by region area; region recall counts regions matched above IoU thresholds 0.50 and 0.75.Source: §2.3.4 · pp. 16–17

2.3.5 Paired Statistical Analysis

  • Paired comparisons use two-sided Wilcoxon signed-rank tests, bootstrap confidence intervals, and rank-biserial effect sizes, with Holm-adjusted significance assessed at α = 0.05.The complete three-domain configuration is compared separately with single- and two-domain configurations, and with each conventional method.Source: §2.3.5 · p. 18

2.3.6 Qualitative Experimental Protocol

  • Qualitative analysis examines two captures of the same scene under different natural illumination, focusing on parent-child and overlapping groups and the effects of reprocessing selected content.Figure 2 presents the brighter capture, with conventional mutually exclusive partitions above the simplified complete DPG group set and input image.Source: §2.3.6 · p. 18, Figure 2 · p. 19

3 Results

Qualitative examples show DPG retaining overlapping groups and exposing finer organization through reprocessing; on BSDS500, the complete configuration retains region correspondence while producing higher boundary recall.

3.1 Qualitative Results

  • The bucket-associated parent remains one group despite variation from illumination, shading, its rim, and local surface structure, while child groups represent localized regions.Other examples show broader and more restricted tree-associated parents coexisting, with reprocessing adding finer foliage groups without replacing the originals.Source: §3.1 · p. 20
  • An illuminated-foreground group crosses detailed vegetation rather than following one semantic object, and can overlap groups supported by other measurement relationships.DPG’s visualization is simplified because one color cannot display every group at the same image location.Source: §3.1 · pp. 19, 22
  • In the lower-luminance examples, reprocessing makes tree- and container-associated groups available from a broad parent, including most of a weakly separated black container’s visible extent.Across the two captures, the same fence is less fragmented under comparatively uniform illumination and more fragmented under brighter but less uniform illumination.Source: Figure 7 · p. 24, Figure 8 · p. 25, §3.1 · p. 25

3.2 Domain Evaluation

  • The complete three-domain configuration had the highest region covering, recall at IoU 0.50, and recall at IoU 0.75 among DPG configurations, significantly exceeding every single- and pairwise-domain configuration.Relative to L+VC, its paired mean differences were 0.0573 for covering, 0.0809 for recall at IoU 0.50, and 0.0232 for recall at IoU 0.75.Source: Figure 9 · p. 26, §3.2 · p. 27
  • The complete configuration also had the highest DPG boundary recall and lowest boundary precision; its boundary F1 was lower than L+VC and VC+CC.These differences were significant for boundary recall and precision versus every individual- and pairwise-domain configuration; F1 did not differ significantly from L, VC, CC, or L+CC.Source: §3.2 · p. 27

3.3 Comparison with Conventional Methods

  • 0.5251±0.1095 (DPG) and 0.5260±0.1255 (Felzenszwalb) region covering were similar; DPG’s 0.2729 ± 0.1190 recall at IoU 0.50 led the comparison.DPG covering was significantly higher than watershed-RAG and Quickshift, but not significantly different from Felzenszwalb.Source: §3.3 · p. 27, Figure 10 · p. 28
  • 0.1064 ± 0.0121 seconds per image was DPG’s lowest mean runtime, significantly below Felzenszwalb, watershed-RAG, and Quickshift.The means were 0.1504 ± 0.0145, 0.2749 ± 0.2868, and 4.0096 ± 0.0922 seconds per image, respectively.Source: Figure 10 · p. 28, §3.3 · p. 29
  • 0.9772±0.0357 boundary recall was significantly higher for DPG than all three comparison methods, while its lowest boundary precision and lower F1 marked a trade-off.DPG boundary precision was 0.3965 ± 0.1213; its boundary F1 was 0.5530±0.1219, significantly below Felzenszwalb and Quickshift but not watershed-RAG.Source: §3.3 · p. 29

4 Discussion

The discussion finds that DPG’s domains contribute complementary grouping information, with trade-offs against conventional methods across region and boundary measures. Qualitative examples show how overlapping groups and reprocessing preserve structures beyond annotated regions, while the evaluation leaves their perceptual relevance and automatic selection unresolved.

4 Discussion

  • Across BSDS500, DPG retained substantial correspondence with human-annotated regions while qualitative examples showed domain-specific overlapping groups available simultaneously.Figure 11 illustrates one selected training-region match; its displayed IoU and B-F1 are not image-level test metrics.Source: §4 · p. 29, Figure 11 · p. 30

4.1 Complementary Contributions of the Domains

  • Combining all three domains significantly increased each region measure relative to every individual and pairwise configuration and produced the highest boundary recall, but lower precision and no F1 gain over the strongest components.VC had the strongest individual-domain region correspondence, L the highest individual-domain boundary recall, and CC the highest individual-domain boundary precision and F1.Source: §4.1 · p. 29

4.2 Relation to Conventional Segmentation Methods

  • At IoU 0.75, Felzenszwalb achieved significantly higher recall than DPG, although they had similar region covering and did not differ significantly in recall at IoU 0.50.At the lower threshold, DPG retained approximately corresponding groups at a frequency comparable to the graph-based partition.Source: §4.2 · p. 30
  • DPG recovered a larger proportion of annotated boundaries than comparison methods but had lower precision because its evaluated boundary map unions boundaries from overlapping parent groups.Boundaries without nearby annotations reduce precision whether they follow a distinguishable unannotated structure or an unsupported division.Source: §4.2 · p. 30

4.3 Interpretation of the DPG Representation

  • Human annotations do not exhaustively label additional structures, while qualitative examples show overlapping domain-supported groups and localized reprocessing without replacing broader groups.Figure 12 shows a proposal following the illuminated part of a tiger within a broader annotated region.Source: §4.3 · pp. 30–31
  • Reprocessing experiments used manually selected parent groups; automatic selection remains outside the work’s scope.Figure 13 illustrates a broad group retaining a complete white building.Source: §4.3 · p. 31

4.4 Computational Characteristics

  • The complete DPG configuration had the lowest measured runtime, with a mean group count similar to Quickshift, below watershed-RAG, and above Felzenszwalb.The additional representational structure was obtained without a corresponding runtime increase relative to comparison methods.Source: §4.4 · pp. 31–32

4.5 Limitations and Future Work

  • The evaluation is limited to static RGB images and three domains; parameters were selected qualitatively by one evaluator, and temporal association is proposed as future work.The groups are formed independently from individual images; temporal association could provide evidence about persistence and relationships between groups.Source: §4.5 · p. 32

5 Conclusions

DPG retains measurement-supported groupings that correspond with human-annotated regions while preserving additional organization in the same observation. It does so without semantic labels or a single mutually exclusive partition, leaving an intermediate representation for later interpretation.

5 Conclusions

  • DPG retains substantial correspondence with human-annotated scene regions while preserving additional measurement-supported organization in the same observation.The conclusion describes this as a non-exclusive, multi-domain grouping representation.Source: §5 · p. 33
  • DPG forms groups without assigning semantic labels or object categories and without reducing an image to one mutually exclusive partition.Source: §5 · p. 33
  • Retained parent groups, their cross-domain spatial relations, and reprocessing provide an intermediate representation for later interpretation and learning of perceptual-unit relevance.Source: §5 · p. 33

A Complete paired statistical results

The appendix reports paired statistical comparisons of complete DPG with component configurations and conventional grouping methods on the same 200 BSDS500 test images. It also notes public availability of the source and analysis code and the dataset.

A Complete paired statistical results

  • The paired statistical results compare complete DPG with six component configurations and three conventional grouping methods using the same 200 BSDS500 test images.Reported statistics include image-level means and sample standard deviations, paired differences, 95% paired-bootstrap confidence intervals, adjusted Wilcoxon p-values, and rank-biserial correlations.Source: A Complete paired statistical results · p. 34, Table 1 · p. 35, Table 2 · p. 37
  • Each reported paired difference is complete DPG minus the comparison configuration or method; negative runtime differences indicate faster DPG execution.Group count describes proposal-set size, not accuracy.Source: A Complete paired statistical results · p. 34

Code, Data, and Materials Availability

Related papers

Loading 2609.27423v1…