UniCounting: Instance-Aware Proposal Consolidation for Image-Query-Free Multi-Category Counting

• Image-query-free counting uses a fixed global vocabulary but supplies no per-image target category, exemplar, or restricted class list.Given an RGB image, the model returns a complete category–count vector and the proposal groups supporting nonzero entries.Source: §1 · p. 1
• Figure 1 traces representative proposals to category/count outputs and illustrates one predicted hot-dog component merging proposals from at least five ground-truth instances.The caption notes that the gap from 11 ground-truth instances to one predicted component may also include upstream proposal misses.Source: Figure 1 · p. 1
• Generic segmentation proposals can duplicate or partially depict objects and include neighboring instances, so semantic category scores alone cannot resolve object identity.The paper identifies instance consolidation as necessary before selecting one representative and label for each surviving group.Source: §1 · pp. 1–2
• UniCounting turns frozen foundation-model proposals into counts by grouping masks, selecting and labeling a representative, and admitting each component with replayable evidence.Only a 3,267-parameter category-shared relation head is trained, using instance-mask-derived supervision and no count or density-map targets.Source: §1 · p. 2
• Learned instance relations reduce vector error and absent-class false mass relative to tested edge-score alternatives, while exposing a fragmentation–merge trade-off.The evaluation framework separates system-level counting from proposal-consolidation quality using calibrated complete-vector references, matched-decoder edge comparisons, partition diagnostics, and decoder ablations.Source: §1 · p. 2

Queried counting

• Few-shot and text-conditioned counters require external target selection, while recent text-guided zero-shot methods improve grounding and exemplar construction but retain that interface.PseCo combines conditioning with proposals and SAM masks [Huang et al. 2024].Source: §2 · p. 2

Closed-set and discovered categories

• Supervised multi-class counters predict density channels for trained classes, while SIMCO clusters emergent foreground types; fixed-channel output alone is not the paper’s novelty claim.These precedents are described by Xu et al. 2021; Gao, Zhao, and Li 2024, and Godi et al. 2021.Source: §2 · p. 2

Detection, grounding, and count vectors

• Summing retained detections by class provides a direct complete-vector baseline, while grounding detectors use category names or referring expressions for open-set detection.The passage names Grounding DINO and OWLv2 as examples [Liu et al. 2024; Minderer, Gritsenko, and Houlsby 2023].Source: §2 · p. 2

Multi-category counting

• Reference-less counters often return one anonymous count; ABC123 and OCCAM discover image-specific types, whereas OmniCount names user-specified categories.PrACo and PrACo++ test absent-label rejection, and MUCCA provides real multi-category scenes [Ciampi et al. 2025; Pacini et al. 2026].Source: §2 · p. 2

Proposal-based recognition

• Foundation models support region discovery and semantic matching, but their proposal sets contain duplicates and incomplete views.UniCounting focuses on converting those proposals into complete named count vectors with group-level evidence.Source: §2 · p. 2

3.1 Task Definition

• For each run, the fixed global vocabulary defines the categories; from an RGB image alone, the model returns counts as the numbers of admitted proposal components per category.Each component retains a designated representative, and the output includes group evidence.Source: §3.1 · pp. 2–3

3.2 Fixed Category Head

• The fixed category head scores proposals using masked- and box-view OpenCLIP embeddings against fixed category prototypes, without trainable parameters or a learned rejection gate.A separate fixed bank of background prototypes is reserved for the admission rule.Source: §3.2 · p. 3

3.3 Relation Head and Same-Instance Score

• A 3,267-parameter relation head is the only trained module, mapping directed pair features—including visual similarities, area ratios, and SAM scores—to semantic, instance, and completeness branches.Primary inference uses the symmetric same-instance score; the other branches support diagnostic variants.Source: §3.3 · pp. 3–4

3.4 Instance Grouping, Representative Selection, and Labeling

• Each candidate selects up to two eligible neighbors with the largest instance scores; OR selection and union-find form connected components, whose maximum-category-margin representative supplies the component label.The direct neighbor limit does not constrain component size after transitive closure.Source: §3.4 · p. 4

3.5 Background-Margin Admission

• Each component receives a background-margin admission test using fixed background concepts; rejected components contribute zero counts and are not relabeled or redistributed.The background margin is a raw-cosine margin, not a probability.Source: §3.5 · p. 4

3.6 Supervision and Optimization

• The relation head is trained on 210 COCO train2017 images with instance-mask-derived pair supervision, while disjoint inner45 selects decoder and threshold settings without using target-test data.Selection minimizes complete-vector ℓ1 for graph construction, representative selection, and labeling, then selects admission by micro presence F1, with vector ℓ1 breaking ties.Source: §3.6 · p. 4

4.1 Protocols and Metrics

• The matched-decoder study changes only the symmetric edge score while holding the candidate pool, graph construction, representative and labeling rules, and admission fixed.OWLv2-All80 receives the same 80 raw COCO category names for every image; Faster R-CNN is a supervised reference using COCO box supervision.Source: §4.1 · p. 5
• COCO clean500 is a deterministic, annotation-blind 500-image subset of val2017, disjoint from relation-head training and inner45; it averages 2.894 present categories and 7.018 objects per image.The evaluation reports complete-vector errors, absent-class false mass, and macro and micro presence F1; transfer results are context only, not evidence for grouping.Source: §4.1 · p. 5, §4.3 · p. 6
• OmniCount-sub is a project holdout rather than an official benchmark, while FSC-147 and CARPK provide single-coordinate transfer diagnostics; FSC-147 and OmniCount-sub share 26 source-image hashes.CARPK transfer is zero-shot without fine-tuning, and the transfer results are reported separately rather than treated as independent confirmations.Source: §4.1 · p. 5

4.2 Implementation Details

• Frozen SAM 2.1 supplies proposals, DINOv2 relation features, and OpenCLIP category features; relation scoring evaluates all N(N − 1)/2 proposal pairs before graph sparsification.Decoder choices and thresholds are fixed using COCO inner-validation before target-test evaluation.Source: §4.2 · p. 5, §4.1 · p. 5

4.3 How does UniCounting perform under complete-vector counting?

• 6.485 versus 6.860 vector ℓ1 and 0.325 versus 0.474 absent-class false mass favor UniCounting over OWLv2-All80; micro-F1 is nearly identical, while OWLv2 has lower total-count MAE and higher macro-F1.The vector-ℓ1 gap is a point-estimate difference, with no significance claim.Source: §4.3 · pp. 5–6, Table 1 · p. 6

4.4 Does the relation-head score improve proposal consolidation under a matched decoder?

• Under a shared downstream decoder, the relation-head score has the lowest vector ℓ1 and false mass among tested scores, though DINO has higher micro-F1 and the overlap guard lower total-count MAE.Paired intervals for vector ℓ1 and false mass against containment, mask IoU, CLIP, and DINO exclude zero.Source: §4.4 · p. 6

4.5 Why does it help, and what does it sacrifice?

• The relation-head score yields the lowest fragmented-instance rate and vector ℓ1 among edge scores, accepting a moderate merge-error increase versus conservative containment and mask-IoU rules.It also attains Pair F1 0.694, purity 0.925, and representative IoU 0.215.Source: §4.5 · p. 6, Figure 3 · p. 7

4.6 Decoder Component Ablation and Selection Analysis

• Representative selection and raw-cosine labeling improve vector ℓ1 and micro-F1, with labeling having the largest effect; background-margin admission adds a smaller gain.Top2-OR sparsification is metric- and dataset-dependent, and the semantic constraint consistently degrades performance.Source: §4.6 · p. 7

Limitations

• The complete-vector evidence covers only the 500-image COCO evaluation and OmniCount-sub; MUCCA and PrACo++ are not evaluated.clean500 is a subset of COCO val2017, with unknown foundation-model pretraining overlap.Source: §5 · p. 7

Discussion

• Representative selection and labeling provide the largest consistent decoder gains, while top2-OR sparsification is metric- and dataset-dependent.Figure 4 audits choices selected on inner45 against clean500 and OmniCount-sub.Source: Figure 4 · p. 7

Limitations

• The candidate pool recalls 62.4% of non-ignore instances at mask IoU 0.50, and changing vocabularies may impair calibration of COCO-selected thresholds.Representative selection and admission maximize over the vocabulary, shifting score distributions.Source: §5 · p. 7
• The relation-head score reduces vector error and false mass, but its merge error exceeds mask IoU and containment while remaining below CLIP and DINO.The controlled ablation isolates five edge scores and two grouping controls under one decoder.Source: §5 · p. 7
• Inference still runs frozen SAM 2.1, DINOv2, and OpenCLIP components with quadratic pair scoring, despite only 3,267 trainable parameters.Deployment should also address privacy, ontology omissions, and systematic category errors.Source: §5 · p. 7

Conclusion

• UniCounting explicitly models proposal identity, forms instance-level components, and produces category-count outputs with traceable group evidence.The framework approaches image-query-free multi-category counting through proposal consolidation rather than directly mapping redundant segmentation proposals to counts.Source: §6 · p. 7
• A 3,267-parameter category-shared relation head improves proposal partition quality under a controlled decoder comparison, reducing vector-level counting error relative to several similarity-based alternatives.Source: §6 · p. 7
• More aggressive grouping reduces fragmentation but may increase instance-merging errors.The conclusion identifies explicit control over instance identity as important alongside semantic recognition.Source: §6 · p. 7
• Future work should improve proposal recall and develop more adaptive ways to balance merge and fragmentation errors.Source: §6 · p. 7

Supplementary Materials

• The listed authors are Jinshi Liu, Pan Liu, Lei He, Weichao Luo, and Rui Qian.Source: Supplementary Materials · p. 10
• The supplement specifies implementation choices, evaluator definitions, and supporting experiments for the method in the main paper.Source: Supplementary Materials · p. 10

Reproducibility Protocol

• Table S1 lists reproducibility settings for training, selection, and evaluation.Source: Table S1 · p. 10
• The training code uses fixed COCO train2017 assignments of 210 relation-training images and 45 inner-validation images, with no overlap.The split provenance lists 11,341 train candidates, 3,262 reliable proposals, and 45,508 reliable unordered pairs; inner45 covers 60 COCO categories.Source: Reproducibility Protocol · p. 10
• clean60 is a fixed 60-image COCO val2017 diagnostic set, not used for relation-head training or selection, and its construction supports no population-representativeness claim.clean500 excludes clean60 identities; the clean60 COCO80 targets include 340 non-ignore instances from 54 categories and 155 positive image–category cells.Source: Reproducibility Protocol · p. 10
• Table S2 covers automatic mask generation and filtering.Source: Table S2 · p. 10
• UniCounting receives no per-image category cue, and FSC-147 target coordinates are extracted only after the full fixed-vocabulary prediction.FSC-147 specifies one benchmark target, not every countable category in a scene.Source: Reproducibility Protocol · p. 10

Candidate Generation

• Every image follows the same 24-point grid, zero crop layers, and no-tiling path, with candidates greedily deduplicated by predicted IoU.The formal decoder has no density-, resolution-, count-bin-, or ground-truth-dependent router.Source: Candidate Generation · p. 11

Computing Environment

• Experiments used an RTX 4090 with approximately 24 GB memory, an Intel Core i9-14900KF, and Ubuntu 24.04.4 LTS under WSL2.The environment description distinguishes driver-supported CUDA from the CUDA version used to compile PyTorch.Source: Computing Environment · p. 10, Category and Relation Heads · p. 12

Category and Relation Heads

• Relation features use normalized DINOv2 and OpenCLIP embeddings for masked, box, and context views, alongside mask area and SAM 2.1 predicted-IoU scores.Reverse-direction features exchange proposal indices, with Feature 9 changing sign before clipping; standardized Feature 16 is inactive.Source: Category and Relation Heads · p. 12

Pseudo-Label Construction

• A proposal is reliable when its best non-ignore instance purity is at least 0.8, its second-best purity at most 0.1, and ignore/crowd overlap at most 0.1 of its area.Completeness positives additionally require a shared matched instance, at least 0.9 containment, and a coverage advantage of at least 0.2 for the larger-coverage proposal.Source: Pseudo-Label Construction · p. 12

Training and Decoding

• The relation head is trained on directed proposal-pair rows with binary cross-entropy before directional averaging, using 80 deterministic full-batch AdamW epochs.The learning rate is 3×10^-3, weight decay is 10^-4, and dropout is 0.1.Source: Training and Decoding · pp. 12–13
• The decoder forms components from same-instance affinities, selects a representative, labels it by raw category cosine, and admits each component using background margins.Each admitted component adds one count; rejected components add none.Source: Training and Decoding · p. 13

Evaluation Splits and Detector Protocols

• Clean500 contains 500 hash-selected val2017 images, excludes clean60, and remains untouched until thresholds are frozen on inner45.The described confidence intervals use paired image-bootstrap replicates with the same resampled images across methods and relation seeds.Source: Evaluation Splits and Detector Protocols · pp. 13, 15
• OWLv2-All80 is calibrated on inner45 using all 80 COCO thing-category names; the Faster R-CNN reference uses full COCO box supervision.The two detector protocols select score thresholds on inner45, and the Faster R-CNN model is explicitly a supervised reference, not a matched-training baseline.Source: Evaluation Splits and Detector Protocols · p. 15

Controlled Edge and Grouping Ablation

• The controlled edge study fixes the candidate pool and decoder, replacing only pair score and threshold; selection uses inner45, not target tests.The study includes singleton and overlap-guard controls, and Table 2 reports end-task metrics while Table S8 gives partition diagnostics.Source: Controlled Edge and Grouping Ablation · p. 15
• Candidate recall is 62.4% at IoU 0.50 and 48.8% at IoU 0.75, measuring the proposal-generation ceiling before later decoding stages.Recall pools non-ignore ground-truth instances across clean500 and considers all retained pre-grouping proposals.Source: Controlled Edge and Grouping Ablation · p. 16

Complete-Vector Transfer Diagnostic

• On OmniCount-sub, the zero vector has lower vector ℓ1 than UniCounting, which recovers nonzero category presence; the diagnostic calls for considering presence F1 jointly.The passage characterizes OmniCount-sub as a project holdout transfer diagnostic, not an official benchmark.Source: Complete-Vector Transfer Diagnostic · p. 16