PaperlayerView exact revision on arXiv

Source-linked AI summary

Gaussian Density Splatting Network

Miao Shang, Yabin Wang, Xiaopeng Hong

arXiv:2610.10396v2cs.CV

TL;DR

Crowd counting must estimate both the number and spatial distribution of people, but grid-based maps and point detections have representational limits. GDSNet uses structured control points to predict continuous Gaussian density primitives. It reports consistent benchmark improvements and robustness to test-time sampling variation.

  • Problem

    Crowd counting requires estimating both people’s number and spatial distribution, while grid maps impose fixed-resolution structure and point detections omit spatial extent and can be unstable in dense scenes.

    Source: §1 · p. 1

  • Method

    GDSNet uses control-point-guided geometric constraints to predict continuous Gaussian primitives and trains by differentiably rendering and matching density maps.

    Source: §1 · pp. 1–2

  • Results

    GDSNet reports consistent improvements on challenging benchmarks and robustness to test-time sampling variation, with UCF-QNRF MAE below 80 for test strides under 15.

    Source: §1 · p. 2, §5 · p. 10, §4.3 · p. 10

  • Takeaways & Limitations

    The authors conclude that structured constraints enable stable continuous Gaussian learning while preserving spatial continuity and flexibility.

    Source: §5 · p. 10

    In extremely congested scenes, GDSNet can estimate total count reasonably while producing smooth regions rather than sharply localized peaks, because supervision lacks a strong individual-head localization constraint.

    Source: F Failure Cases and Limitations · p. 21

Abstract

from arXiv · show

This paper proposes a novel crowd counting approach, the Gaussian Density Splatting Network (GDSNet). Unlike methods that rely on conventional, grid-based density maps and are sensitive to spatial resolution, GDSNet represents a crowd as a superposition of continuous 2D Gaussian primitives. Our approach is built upon two key contributions. First, a control-point-based fitting mechanism is introduced to structure the prediction of Gaussian parameters. A set of control points is adaptively allocated to define local regions, from which features are pooled to regress each primitive's parameters. Second, we adapt a differentiable Gaussian Splatting framework to the counting task by parameterizing each primitive with geometric parameters and a scalar density mass. This formulation allows the network to be trained end-to-end via spatial matching of differentiably rendered density maps, naturally providing both local density supervision and global count optimization. Extensive evaluations on four standard benchmarks show GDSNet consistently outperforms the state of the art. The code is available at https://github.com/infinite0522/GDSNet-Gaussian-Density-Splatting-Network.

1 Introduction

GDSNet represents crowd density with continuous Gaussian primitives and structures their prediction through control points. This approach addresses the representational limits of grid and point methods and the instability of unconstrained Gaussian fitting.

1 Introduction

  • Grid-based maps impose fixed-resolution, piecewise-constant structure, while point-based methods omit spatial extent and orientation and can be unstable in dense regions.The paper also identifies quantization artifacts and difficulty modeling perspective-driven scale variation in grid maps.Source: §1 · p. 1
  • The method represents crowd density as learnable 2D Gaussians whose location, anisotropic covariance, and mass capture geometry and local population.This formulation unifies localization and counting.Source: §1 · p. 1
  • Unconstrained Gaussian fitting can waste capacity in sparse areas, produce degenerate or overlapping primitives, and ignore correlations between neighboring people.These issues make the learned density field locally incoherent.Source: §1 · p. 2
  • Structural-complexity-guided control points partition the image into coherent regions; triangulated simplices constrain centers, and neighboring-simplex features guide stable Gaussian regression.The allocation strategy places more points in structurally salient regions, while topology-aware feature aggregation addresses relational consistency.Source: §1 · p. 2
  • GDSNet reports state-of-the-art results on four challenging benchmarks, with consistent performance improvements across benchmarks.Its framework predicts Gaussian parameters from local simplex geometry and image features, then trains by differentiably rendering and matching density maps, with count-consistency and shape-regularization losses.Source: §1 · p. 2

2 Related Work

Prior crowd-counting work uses grid-based maps or point detections, while GDSNet instead predicts continuous, mass-carrying Gaussian bases. It adapts Gaussian representation to task-oriented density estimation rather than appearance reconstruction.

2.1 Crowd Density Representations

  • Grid-map methods use multiscale architectures and alternative supervision, while point methods build on bipartite matching; GDSNet directly predicts mass-carrying Gaussians, unlike GST’s offline appearance Gaussians used as transport priors.Point proposals remain zero-dimensional and matching can be fragile in ultra-dense scenes.Source: §2.1 · p. 3

2.2 Image Representation and Perception via Continuous Gaussians

  • Prior Gaussian-splatting work targets appearance reconstruction, using iterative optimization or feedforward parameter prediction; GDSNet adapts task-specific Gaussian fields to crowd density estimation.It uses topology-aware primitive interaction and task-specific constraints; the cited iterative optimization methods are computationally expensive.Source: §2.2 · p. 3

3 Method

GDSNet models crowd density as a continuous field of learnable Gaussian primitives, then structures their parameterization with adaptively allocated control points and differentiable rendering. Its method combines a Gaussian basis, triangulated local supports, topology-aware feature aggregation, and joint density, count, and geometry supervision.

3.1 A Unified Mathematical Perspective on Crowd Representation

  • Anisotropic Gaussian bases represent continuous density, with each primitive’s learnable mass contributing directly to the estimated count.The representation supports smooth density variation, decouples count estimation from the rasterization grid, and retains geometry for spatial supervision and constraints.Source: §3.1 · p. 4

3.2 Structural Complexity-Guided Control-Point Allocation

  • Structural-complexity-guided sampling allocates control points adaptively, bounds their number by the stride-controlled grid, and triangulates them into local supports.Delaunay simplices constrain primitive centers to convex regions, support stable barycentric regression, and bound the number of faces to O(|V|).Source: §3.2 · p. 5

3.3 Topology-Aware Feature Aggregation and Constrained Parameterization

  • GDSNet aggregates intra-simplex image and geometry features with neighboring-simplex context, then predicts Gaussian parameters relative to a simplex-based ellipse prior.Barycentric weights constrain each Gaussian center to the convex hull of its control points, preventing erratic migration.Source: §3.3 · p. 6

3.4 Continuous Density Rendering and Optimization

  • Differentiable Gaussian rendering is trained with spatial density alignment, count-conservation supervision, and shape regularization against excessively elongated primitives.The combined objective promotes density reconstruction, count preservation, and Gaussian geometry fitting jointly.Source: §3.4 · p. 7

4 Experiment Results and Discussion

GDSNet achieves state-of-the-art performance across the evaluated benchmarks, while ablations support its geometric constraints, adaptive control-point allocation, and loss design. Its continuous Gaussian representation remains robust to moderate sampling changes, with performance degrading at coarser test strides.

4.1 Evaluation Results

  • Figure 3 shows adaptive triangulation and smooth mass transitions between neighboring simplices, alongside the rasterized continuous density field.The simplex heatmap assigns each simplex the mass predicted by its associated Gaussian primitive.Source: §4.1 · p. 7, Figure 3 · p. 8
  • MAE 5.4 and MSE 8.0 on ShanghaiTech Part B; GDSNet achieves state-of-the-art performance across all four benchmarks.The evaluation uses MAE and MSE and compares GDSNet with representative map-based and point-based methods.Source: §4.1 · p. 7, §4 · p. 7, Table 2 · p. 8

4.2 Ablation Study

  • Removing topology-aware aggregation lowers performance; simplex-constrained geometry is critical, and structural complexity-guided sampling outperforms uniform sampling.Unconstrained variants fit training samples but generalize more weakly on the test set.Source: §4.2 · p. 8, Table 3 · p. 9
  • Gaussian bases produce smooth rather than piecewise-constant density, while learnable anisotropic geometry adapts support to local crowd structure.Anisotropic geometry captures directional variations and elongated density patterns that isotropic support cannot represent.Source: §4.2 · p. 9, Table 6 · p. 9
  • Rasterization, count, and geometric regularization losses contribute complementary constraints for spatial fidelity, total-mass estimation, and stable primitive shapes.The paper identifies rasterization loss as the primary driver and geometric regularization as a stabilizer against extreme anisotropies.Source: §4.2 · p. 9, Table 4 · p. 9
  • Structural complexity-guided sampling performs best; uniform sampling has the highest errors, while Sobel sampling over-allocates primitives around sharp boundaries and fine textures.The paper attributes Sobel sampling's weaker performance to allocation in semantically irrelevant regions.Source: §4.2 · p. 9, Table 5 · p. 9

4.3 Discussion on Sampling Stride Sensitivity

  • MAE remains below 80 for test strides under 15 across training strides 8, 10, and 12, but all settings degrade at test strides 15 and 16.Stride 12 achieves a matched-stride MAE of 77.23; stride 10 performs best in most cases and can exceed matched-stride performance at neighboring strides.Source: §4.3 · p. 10, Figure 5 · p. 10
  • Control-point stride bounds primitive allocation, and topology construction and primitive processing have quasi-linear O(N log N) complexity.For V control points and N simplex faces, Delaunay construction takes O(V log V) time and feature aggregation takes O(N).Source: §4.3 · p. 10

5 Conclusion

GDSNet formulates crowd counting as continuous density reconstruction with Gaussian basis functions, moving beyond grid- and point-based representations. Its control-point-guided parameterization structures Gaussian geometry, and experiments show consistent benchmark improvements and robustness to test-time sampling-density variation.

5 Conclusion

  • GDSNet represents crowd counting as continuous density reconstruction using Gaussian basis functions rather than grid- or point-based representations.Source: §5 · p. 10
  • Control-point-guided parameterization imposes structured geometric constraints on Gaussian primitives to support stable learning while preserving spatial continuity and flexibility.Source: §5 · p. 10
  • Experiments show consistent improvements on challenging benchmarks and robustness to variations in test-time sampling density.Source: §5 · p. 10
  • Further acceleration of Delaunay triangulation remains future work.Source: §5 · p. 10

A Discussion on Mathematical Presentation of Different Counting Paradigms

The section frames crowd counting as continuous density approximation and compares grid, point, and Gaussian basis representations. It presents GDSNet’s Gaussian formulation as spatially continuous and geometrically explicit.

A Discussion on Mathematical Presentation of Different Counting Paradigms

  • The section unifies counting paradigms as weighted basis functions over the image domain, with the global density integral yielding the estimated count.Grid methods use box bases, point methods use Dirac bases, and GDSNet introduces Gaussian bases.Source: A Discussion on Mathematical… · p. 14
  • Grid-based methods model density with fixed-size, grid-aligned box supports, limiting flexibility for sub-pixel congestion and arbitrary-resolution sampling.Each cell predicts a scalar density; its support is locked to the downsampling stride.Source: A Discussion on Mathematical… · p. 14
  • Point-based methods threshold discrete proposals into Dirac-supported masses, which do not explicitly model crowded regions’ spatial extent, anisotropy, or smooth occupancy.Each proposal supplies a confidence and coordinate before thresholding.Source: A Discussion on Mathematical… · p. 14
  • GDSNet models density as a sum of continuous 2D Gaussians, each carrying a learnable center, anisotropic covariance, and non-negative density mass.Gaussian mass is analytically conserved when mass outside the image boundary is negligible or handled by the rasterization domain.Source: A Discussion on Mathematical… · pp. 14–15

B Implementation Details

The implementation uses a pretrained VGG19 backbone and differentiable rendering for density supervision. The authors analyze the representation’s computational scaling and report linear memory growth and image-resolution-bounded rasterization.

B Implementation Details

  • The implementation uses ImageNet-pretrained VGG19 as backbone, gsplat for Gaussian Splatting, and a frozen off-the-shelf structural estimator; control-point sampling stride is s = 10, with dataset-specific crop sizes.At inference, count-only evaluation returns the analytical mass sum; density-map evaluation additionally renders the learned Gaussian field.Source: B Implementation Details · p. 15

C Complexity and Scalability Analysis

  • GDSNet’s Delaunay triangulation and feature aggregation have quasi-linear O(N log N) complexity, while memory grows as O(N) and rasterization is bounded by image resolution O(H × W).The authors report that this scaling avoids OOM failures or exponential inference delays on JHU-CROWD++ with tens of thousands of individuals.Source: C Complexity and Scalability Analysis · p. 15

C.1 Component-wise Efficiency

This block covers component costs, primitive-allocation scaling, and module-ablation implementations. Topology computation is the main added overhead, while the reported primitive-dependent runtime scales approximately linearly.

C.1 Component-wise Efficiency

  • Feature aggregation, constrained Gaussian parameterization, and rasterization together require 0.60 ms and 12.95 MB of memory.The component-wise measurements use a 256×256 input image; “+” values denote additional component costs.Source: §C.1 · pp. 15, 17, Table 7 · p. 17
  • The main additional overhead is non-parametric topology computation, chiefly Delaunay triangulation, which introduces no trainable parameters.The authors identify GPU-accelerated triangulation or learned primitive organization as possible ways to reduce that overhead.Source: §C.1 · p. 17

C.2 Scaling with Primitive Allocation

  • As simplex and Gaussian counts increase, topology time and overall runtime scale approximately linearly; topology dominates primitive-dependent scaling while other costs remain nearly unchanged.The authors characterize the representation’s computational overhead as controllable through the number of allocated primitives.Source: §C.2 · p. 17

D.1 Implementation of Module Ablations

  • The TFA-only removal retains control-point allocation, dithering, and simplex-constrained parameterization, but omits one-hop Delaunay-neighbor context.This variant is used to examine local topological context in scale, rotation, and density-mass prediction.Source: §D.1 · pp. 17–18
  • The module-ablation comparison covers the full model and variants removing TFA, then TFA with SCP, and then CPA as well.The most reduced variant replaces the structural prior with uniform anchors and uses fixed grid-patch receptive fields.Source: D Implementation Details of Ablation… · p. 17, §D.1 · pp. 17–18, Table 9 · p. 18
  • Removing both TFA and SCP replaces the adaptive triangular receptive field with a fixed square patch sampled at its corners and center.Without the simplex constraint, MLP heads predict Gaussian parameter adjustments from the patch features.Source: §D.1 · p. 18

D.2 Implementation and Visualization of Control Point Allocation Strategy Ablations

The section compares control-point allocation priors and Gaussian geometry variants. The structural prior produces more coherent qualitative density fields than Sobel filtering, while geometry ablations progressively vary basis flexibility.

D.2 Implementation and Visualization of Control Point Allocation Strategy Ablations

  • The Sobel-prior variant builds its map from grayscale image gradients, normalizes and squares the response, and applies a lower bound before the unchanged pipeline.The structural complexity prior is the default and uses a frozen, off-the-shelf estimator without target-dataset fine-tuning.Source: §D.2 · pp. 18–19
  • Compared with Sobel filtering, the structural prior yields a more balanced triangulation and smoother ellipsoid-like primitives rather than fragmented meshes and noisy density fields.Figure 6 contrasts the Sobel response prior in the top row with the structural prior in the bottom row.Source: §D.2 · p. 19, Figure 6 · p. 19

D.3 Implementation Details of Gaussian Geometry Ablation

  • The fixed-Gaussian variant freezes center, axis scales, and orientation to the fitted reference ellipse while retaining predicted density mass..Source: §D.3 · p. 20
  • The isotropic variant retains learnable barycentric centers and scalar scale adjustment but removes directional anisotropy by tying the two scale dimensions.Rotation does not affect an isotropic Gaussian’s spatial support.Source: §D.3 · p. 20
  • The geometry ablation compares a triangle basis, fixed Gaussians, isotropic Gaussians, and full anisotropic Gaussians while keeping mass prediction and supervision shared.The passage points to Table 6 for quantitative results but supplies no values.Source: §D.3 · pp. 19–20

E Comparison with MLLMs

On dense crowd-counting scenes, the paper contrasts general-purpose MLLMs with specialized networks such as GDSNet. It reports a substantial performance gap on ShanghaiTech Part A, while counting-specific tuning improves MLLM performance.

E Comparison with MLLMs

  • MLLMs target semantic understanding and object-centric counting in sparse scenes, whereas GDSNet is purpose-built for ultra-dense scenes with scale variation, occlusion, and perspective distortion.FSC-147 averages approximately 56 objects per image; ShanghaiTech Part A averages 501 persons and UCF-QNRF 815 persons per image.Source: E Comparison with MLLMs · p. 20
  • Direct MLLM application to dense benchmarks is challenging because visual tokenization can limit access to small, occluded heads and numeric regression adds difficulty.The paper identifies counting-specific tuning as improving MLLM performance.Source: E Comparison with MLLMs · pp. 20–21
  • The reported ShanghaiTech Part A comparison shows a substantial gap between general-purpose MLLMs and specialized models, and specialized density estimation remains valuable in dense scenes.Counting-specific tuning improves MLLM performance.Source: Table 10 · p. 20, E Comparison with MLLMs · pp. 20–21

F Failure Cases and Limitations

In extremely congested scenes, GDSNet may estimate total count reasonably while rendering smooth regions instead of distinct head peaks. The paper links this limited sharpness to its stability constraint and supervision emphasis.

F Failure Cases and Limitations

  • In extremely congested scenes, GDSNet can produce a reasonably accurate total count while rendering smooth high-response regions rather than sharply localized head peaks.The representative failure case is shown in Fig. 7.Source: F Failure Cases and Limitations · p. 21, Figure 7 · p. 21
  • Constraining primitives against excessively small support improves training stability but limits response sharpness.Source: F Failure Cases and Limitations · p. 21
  • Current supervision emphasizes density alignment and count consistency without a strong individual-head localization constraint.Suggested directions include extracting centers of high-response primitives, sharpening responses at inference, or jointly optimizing a localization-oriented objective.Source: F Failure Cases and Limitations · p. 21
Loading 2610.10396v2…