PaperlayerView exact revision on arXiv

Source-linked AI summary

Count Anything

Mengqi Lei, Shuokun Cheng, Wei Bao, Shaoyi Du, Jun-Hai Yong, Siqi Li, Yue Gao

arXiv:2605.30846v1cs.CV

TL;DR

Object counting remains fragmented across specialized datasets and settings, motivating a text-guided, instance-grounded formulation that generalizes across domains. The paper constructs CLOC and proposes Count Anything, which uses discrete points and complementary counters; experiments report strong accuracy and multi-domain generalization, outperforming existing open-world counting methods.

  • Problem

    Object counting lacks a unified text-guided, instance-grounded, cross-domain setting spanning varied categories, visual domains, object scales, and density distributions.

  • Method

    The paper constructs CLOC and uses Count Anything with discrete instance points, dual-granularity counters, point-centric supervision, and Complementary Count Fusion.

  • Results

    Count Anything achieves strong counting accuracy and multi-domain generalization, substantially outperforming existing open-world counting methods.

  • Takeaways & Limitations

    CLOC provides a unified benchmark for training and evaluating generalist counting models across categories, domains, and density distributions.

  • Takeaways & Limitations

    The model’s cross-domain generalization is constrained by its pretrained text-conditioned visual encoder, especially for ambiguous categories and rare domain-specific terms.

Abstract

from arXiv · show

Object counting remains fragmented across domain-specific datasets and task formulations, despite rapid progress in generalist vision models. Existing counting models are often tailored to scenarios such as crowds, vehicles, cells, crops, or remote-sensing objects, and thus struggle to generalize across categories, visual domains, object scales, and density distributions. In this paper, we study text-guided object counting across domains, where a model takes an image and a natural-language query as input and returns an instance-grounded set of target points whose cardinality gives the count. This formulation unifies category-conditioned counting with interpretable spatial localization. To support this setting, we construct CLOC, a Cross-domain Large-scale Object Counting dataset that reorganizes diverse public data sources into a unified benchmark. CLOC covers six visual domains: General Scene, Remote Sensing, Histopathology, Cellular Microscopy, Agriculture, and Microbiology, with about 220K images, 619 categories, and 15M object instances. Based on CLOC, we propose Count Anything, a generalist model for text-guided object counting. Unlike density-map-based methods, which dominate counting models, Count Anything adopts discrete instance points and performs dual-granularity instance enumeration. A Region-level Sparse Counter provides object-level anchors for large and sparse targets, while a Pixel-level Dense Counter handles small, crowded, and weakly bounded targets via dense point prediction. A point-centric supervision strategy enables learning from heterogeneous annotations, and Complementary Count Fusion combines both counters in a parameter-free manner. Extensive experiments show that Count Anything achieves strong accuracy and multi-domain generalization, outperforming existing open-world counting methods. Code is available at: https://github.com/Mengqi-Lei/count-anything.

1 Introduction

Count Anything addresses the fragmentation of object counting by defining a text-guided, instance-grounded, cross-domain task and combining complementary prediction representations. The paper pairs the CLOC benchmark with dual-granularity instance enumeration to handle varied object scales and density distributions.

  • Motivation: Existing counting datasets and models are mostly specialized for scenarios such as crowds, vehicles, and remote-sensing objects.These settings bind counting systems to particular visual domains and target types.
  • Task formulation: A generalist counting model should accept natural-language targets, output localized instance predictions, and operate across six diverse visual domains.The desired domains include General Scene, Remote Sensing, Histopathology, Cellular Microscopy, Agriculture, and Microbiology.
  • Data challenge: Heterogeneous annotations include boxes, points, masks, polygons, label maps, group labels, and ignore-region annotations, creating a data-unification challenge.These protocols were developed for different original tasks and are not directly uniform for generalist counting.
  • Contributions: CLOC reorganizes multiple public sources into a unified text-guided counting benchmark with about 220K images, 619 categories, and 15M instances.It covers six visual domains and provides a data foundation for generalist counting.
  • Model design: Discrete instance points preserve localized counting evidence, while dual-granularity enumeration combines region-level anchoring for large sparse targets with pixel-level recall for small dense targets.This design avoids forcing all targets into a single representation and addresses limitations of both detection and density-map approaches.
  • Model design: Count Anything combines a Text-Conditioned Encoder with Region-level Sparse and Pixel-level Dense Counters to produce target-conditioned instance points.The final count is obtained from the cardinality of the predicted point set.

2 CLOC Dataset

CLOC reorganizes public datasets into a unified text-guided counting benchmark. It uses category names as textual queries to support evaluation across visual domains, target categories, scales, and density distributions.

  • Dataset motivation: CLOC addresses the limited cross-domain support of datasets built around specific scenarios or target types.The benchmark is intended to support generalist counting across categories, domains, object scales, and density distributions.
  • Dataset construction: CLOC reorganizes images, categories, and instance annotations from multiple public datasets around a text-guided counting task.Category names serve as textual queries, so category-specified counting becomes a concrete text-guided protocol.

3 Count Anything Model

Count Anything formulates text-guided counting as predicting an instance-grounded set of target points, whose cardinality is the count. Its dual-granularity architecture combines region-level sparse enumeration with pixel-level dense point prediction, using heterogeneous annotation supervision and one-to-one matching.

  • Point-set formulation: Count Anything predicts target instance points from an image and text query, with the final count equal to the point-set cardinality.Each predicted point has a location and confidence score, providing spatial evidence for counting.
  • Text-conditioned encoding: The framework uses a text-conditioned encoder to produce target-conditioned object representations for both counters.The encoder fuses multi-scale visual features with the query representation before counter-specific processing.
  • Complementary enumeration: The two branches are complementary because sparse region queries provide object extent but may miss dense small instances, whereas dense points improve coverage but can duplicate bounded targets.Count Anything merges their outputs with Complementary Count Fusion during inference.
  • Region-level Sparse Counter: The Region-level Sparse Counter predicts candidate target regions and uses their centers as counting points for large, sparse, or clearly bounded objects.Its region queries predict foreground logits and bounding boxes, while box centers provide the counting points.
  • Pixel-level Dense Counter: The Pixel-level Dense Counter generates discrete point candidates for small, densely distributed, and weakly bounded objects.It fuses target-conditioned features with higher-resolution visual features, adapts them into counting features, and predicts offsets and confidence relative to dense-grid anchors.
  • Supervision: Branch-specific one-to-one matching supervises dense candidates locally around each ground-truth point, reducing irrelevant competition while preserving instance supervision.Sparse local matching avoids constructing a full matching matrix for the much larger set of dense candidates.

4 Experimental Results

Experiments evaluate Count Anything on CLOC and related counting settings, showing strong cross-domain performance and complementary benefits from its design choices.

  • Experimental Setup: Count Anything is evaluated on CLOC using MAE, RMSE, and NAE, with comparisons against open-world counting, detection, and segmentation foundation models.The CLOC comparison uses officially released checkpoints and standard inference pipelines for competing methods.
  • Main Results: 9.34 MAE, 33.34 RMSE, and 0.75 NAE are achieved on CLOC, outperforming compared methods across six visual domains.Existing methods show larger errors in specialized domains, while Count Anything provides more stable cross-domain counting.
  • Main Results: Count Anything maintains stable counting results across substantial differences in target scale and density distribution.Qualitative predictions are reported as closer to ground truth across different scenarios.
  • Ablation Studies: The reported advantage is attributed to dual-granularity counting and CLOC’s broader coverage of categories, domains, and target-density distributions.The paper further examines these factors through ablation studies.
  • Dense Crowd Counting: The model is additionally evaluated on the ShanghaiTech Part A dense crowd-counting subset within the CLOC test split.This evaluation targets extremely dense distributions and complex occlusion relationships.

RSC PDC DU CCF MAE RMSE

Ablations examine the region and pixel counters, fusion rules, supervision choices, and data scale, highlighting complementary branches and the value of broader training coverage.

  • Aligned Training Data: 19.01 MAE and 100.12 RMSE are achieved when Count Anything is trained under the FSC-147 setting and evaluated on CLOC.This comparison isolates effects of training-data differences using aligned training settings.
  • RSC PDC DU CCF: Using only RSC or PDC is inferior to the full model because the two counters are complementary across object scales and density distributions.The dual-counter design combines region-level sparse counting with pixel-level dense counting.
  • RSC PDC DU CCF: Direct union of RSC and PDC outputs produces duplicate predictions and larger counting errors, whereas CCF achieves the best performance.CCF removes local duplicate predictions between the two branches.
  • Point-centric Supervision: GT-only box targets and soft classification targets support point-centric supervision, while pseudo boxes and hard foreground targets worsen results.All instances receive point supervision, but only reliable ground-truth boxes supervise geometry.
  • Data Scaling: MAE and RMSE consistently decrease as the CLOC training-data ratio increases from 10% to 100%.The full training set provides broader category, domain, and density coverage.

5 Conclusion

The paper frames generalist counting as a cross-domain, text-guided task requiring interpretable instance points, then combines complementary counting paradigms to address varied target scales and densities.

  • Density maps support dense counting but lack one-to-one instance correspondence, confidence scores, and localization-level error diagnosis.
  • Detection-based counting provides region-level evidence for large, sparse, or clearly bounded targets but struggles with small, crowded, and weakly bounded objects.
  • Point regression supplies discrete locations for dense or weakly bounded targets but lacks object extent and region-level geometric constraints.
  • The paper motivates combining detection-based region anchoring with point-regression-based dense enumeration for generalist text-guided counting.
  • Existing generalist methods still struggle to jointly support text-guided specification, instance-grounded counting, cross-domain generalization, and dense-target recall.

A.3 Counting Datasets

Counting datasets are typically organized around individual application domains, while a generalist benchmark requires unified handling of heterogeneous sources and annotations.

  • Domain-specific datasets usually cover a single visual domain, limited categories, or a specific density range, restricting cross-category and cross-domain generalist counting.
  • Category-conditioned benchmarks enlarge the target space but remain mainly based on general-scene images or limited visual sources.
  • Building a generalist dataset requires auditing and reorganizing multi-source annotations around one-instance-one-count supervision.
  • CLOC handles source-specific labels such as crowd, group, ignore, NegativeROA, and AMBIGUOUS while converting boxes, points, polygons, masks, rotated boxes, and label maps into a unified counting format.
  • The implementation trains Count Anything with frozen pretrained encoder parameters and trainable LoRA adapters, using a dense grid whose adjacent anchors are spaced by 7 pixels.

B.2 Quantitative Experiments

Quantitative experiments examine model efficiency and the contributions of supervision, feature adaptation, duplicate handling, and dense-candidate design.

  • Computational efficiency: Count Anything contains 848.62M parameters, requires 5.44 TFLOPs per forward pass, and reaches 68.51 ms latency on an RTX 5090.
  • Point-centric supervision: Removing RSC point, box, or GIoU losses weakens point localization or region-level geometric supervision.
  • PDC feature adapter: The complete 3-block residual PDC feature adapter achieves the best performance, improving pixel-level dense counting over shallower or absent adapters.
  • CCF duplicate handling: The default CCF setting uses Ks = 1, grouping the nearest PDC candidate with each retained RSC candidate to manage duplicate predictions.
  • IoM threshold: An IoM threshold of 0.5 achieves the best performance in the analyzed duplicate-removal experiments.
  • PDC candidate density: One PDC candidate per spatial location is retained because increasing candidate density destabilizes optimization and increases counting errors.

B.3 Qualitative Experiments

Qualitative experiments visualize Complementary Count Fusion and provide additional comparisons across all six CLOC domains and diverse natural-language targets.

  • Complementary Count Fusion: RSC supplies region-level anchors for clearly bounded instances, while PDC recovers dense point-level predictions for the fused output.
  • Complementary Count Fusion: CCF suppresses duplicated predictions and retains complementary predictions from both branches to form the final point set.
  • Domain-wise comparisons: Additional qualitative comparisons cover General Scene, Remote Sensing, Histopathology, Cellular Microscopy, Agriculture, and Microbiology.
  • Compared methods: The comparisons include CountSE and GroundingDINO alongside Count Anything and other listed methods.
  • Target queries: Example queries include people’s heads, eggs, cows, nuclei, epithelial nuclei, lymphocyte nuclei, and neutrophil nuclei.
  • Dataset scope: A general-purpose counting dataset should span multiple visual domains and provide sufficient images, categories, and instances for generalist-model training.

C.1.2 Overview of CLOC

CLOC unifies large-scale, text-guided object counting across six visual domains, broad category and count distributions, and heterogeneous real-world imagery. Its structure supports evaluation of cross-domain, cross-category, and density-generalization capabilities.

  • Cross-domain coverage: CLOC reorganizes public data into a unified category-specified counting benchmark spanning six visual domains.The domains are General Scene, Remote Sensing, Histopathology, Cellular Microscopy, Agriculture, and Microbiology.
  • Scale: CLOC contains about 220K images, 619 categories, and 15.356M instances, exceeding existing general counting datasets in category and instance scale.
  • Target-count distribution: Its higher average instances per image and larger standard deviation broaden evaluation across different counting difficulties and density levels.
  • Cross-domain coverage: CLOC preserves substantial variation in image sources, imaging styles, target appearances, and visual contexts across domains.Specialized domains contribute visual heterogeneity despite having fewer samples than General Scene and Remote Sensing.
  • Category distribution: The 619 categories form a long-tailed distribution, while high-frequency categories include targets from multiple visual domains.This combination supports evaluation from common to low-frequency categories without restricting frequent targets to General Scene imagery.
  • Target-count distribution: CLOC covers target counts from 1-5 to 1001+, retaining both common low- and medium-count scenes and dense target scenes.As target counts increase, sample numbers decrease, but high-count and extremely dense scenes remain represented.
  • Category distribution: Seen and unseen category groups are constructed to evaluate category generalization under a stricter semantic split.

C.3.1 Multi-source Raw Datasets

CLOC begins with public datasets from six visual domains and converts their incompatible annotations into unified counting supervision. The pipeline audits source-specific semantics before preserving only countable instances and representing them in a common structure.

  • Multi-source collection: The raw data pool collects public datasets across General Scene, Remote Sensing, Histopathology, Cellular Microscopy, Agriculture, and Microbiology.These sources were originally designed for diverse tasks including detection, segmentation, crowd counting, medical analysis, and phenotyping.
  • Annotation heterogeneity: Source annotations cannot be directly merged because their formats, category definitions, instance granularities, and semantics are inconsistent.
  • Countability audit: Countability auditing handles crowd, group, ignore, difficult, NegativeROA, invalid-category, and excluded-region annotations according to source-specific protocols.Only annotations that can be stably associated with independent instances proceed to unified conversion.
  • Countability audit: Objects365 crowd and Cityscapes group annotations are excluded for the corresponding category-image pairs because they may represent inseparable or incompletely annotated instances.
  • Countability audit: JHU-CROWD++ images with distractor = 1 are retained because the label denotes low-count or near-negative scenes rather than inseparable annotations.
  • Countability audit: VisDrone images containing ignored regions are skipped because targets inside those regions are not annotated reliably.
  • Unified conversion: Different source protocols, including JSON, XML, CSV, and other formats, are converted into image-organized, category-indexed counting data.Valid instances are unified through point and bounding-box representations.

C.3.4 Category Consolidation and Derived Sample Generation

CLOC consolidates heterogeneous category spaces while preserving annotation reliability, then generates targeted cropped and stitched training samples to expand medium- and high-count coverage.

  • Category consolidation: Category names are normalized and synonymous or near-synonymous expressions are merged into consistent global forms.
  • Category consolidation: Categories with ambiguous boundaries or region-like meanings are removed when they do not support stable instance-level counting.Examples include harbor, Expressway-Service-area, and overpass in remote-sensing data.
  • Category consolidation: Semantic category groups preserve relationships among fine-grained classes without promoting subclasses directly to parent categories.This avoids assuming that all parent-class instances are completely annotated and supports seen/unseen splitting.
  • Image normalization: High-resolution images are normalized to reduce memory, loading, and rendering costs in the unified pipeline.
  • Derived sample generation: Cropped and stitched samples supplement predefined medium- and high-count ranges during training.Cropping extracts dense local regions, while stitching combines same-domain patches with the same counting category.
  • Derived sample generation: Derived samples add coverage particularly in the 51-100, 101-200, and 201-500 target-count ranges, while original samples remain the main distribution source.

C.3.5 Task-aware Splitting

CLOC uses task-aware splitting to support leakage-resistant, cross-domain, and stricter category-generalization evaluation. It also defines image-category pairs as counting samples and evaluates predictions with complementary error metrics.

  • Task-aware splitting: CLOC isolates derived samples by generating cropped and stitched images only from training originals.Validation and test sets contain no samples generated by cropping or stitching.
  • Task-aware splitting: All dataset splits cover six visual domains while matching their overall image proportions as closely as possible.This constraint is applied alongside derived-sample isolation and category splitting.
  • Task-aware splitting: Seen/unseen evaluation groups semantically related categories together rather than relying only on category-name disjointness.This reduces implicit leakage from neighboring categories while preserving fine-grained categories.
  • Split statistics: 207.3K training images, 4.7K validation images, and 8.1K test images are distributed across six domains.The splits contain 586, 77, and 78 categories, respectively, with 14.70M, 278.4K, and 382.3K instances.
  • Evaluation metrics: MAE, RMSE, and NAE measure average counting error, sensitivity to large errors, and relative error across count scales.RMSE highlights severe errors on difficult or high-density samples, while NAE uses ϵ = 10^-6 to avoid division by zero.
  • Evaluation task: Each image-category pair forms an independent text-guided counting sample rather than counting all targets in an image jointly.The model predicts instances corresponding to the specified category query.

D.2 Discussion of the Dataset

CLOC provides a unified, large-scale benchmark assembled from six visual domains, while retaining practical limitations from heterogeneous sources, imbalanced coverage, and difficult annotations.

  • Dataset contribution: CLOC unifies multi-source counting data into a benchmark spanning six visual domains and diverse target densities.Its construction includes countability auditing, heterogeneous annotation conversion, category consolidation, derived-sample generation, and task-aware splitting.
  • Dataset contribution: About 220K images, 619 categories, and 15.356M instances support evaluation across cross-domain, long-tail, and dense-target scenarios.The benchmark is intended to assess both conventional counting and generalization beyond single application settings.
  • Limitations: CLOC may retain source-inherited noise because collection conditions, annotation standards, and category granularity differ across public datasets.Examples include imprecise boundaries, missing instances, and inconsistent category definitions.
  • Limitations: Domain coverage is imbalanced, with General Scene and Remote Sensing larger than the four specialized domains.The imbalance is attributed mainly to the availability of stable instance-level annotations in specialized-domain datasets.
  • Limitations: Large-scale counting annotation remains difficult when targets are dense, blurry, severely occluded, or have unclear boundaries.These conditions constrain the construction of comprehensive instance-level annotations.
Loading 2605.30846v1…