PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACE: Privacy-Preserving Next-Best-View Selection over Distributed 3D Gaussian-Splat Maps

Amirhossein Mollaei Khass, Athanasios Cosse, Qiyu Sun, Nader Motee

arXiv:2610.00822v1cs.RO

TL;DR

Team NBV selection requires scoring a robot’s view against a pooled 3DGS map even though every robot keeps its own map private. TRACE reconstructs the required ray context from exchanged depth-bin aggregates rather than shared splats. Across 100 simulated decisions, its selected headings were usually close to the centralized choice and retained most of the centralized EIG.

  • Problem

    A robot’s view gain depends on occlusion and radiance from other robots’ maps, but no robot holds the pooled map needed to evaluate that gain.

    Source: §IV · p. 4, §V · p. 4

  • Method

    TRACE exchanges per-bin optical-depth and radiance aggregates with pose derivatives so a robot can reconstruct pooled-map information gain and its gradient without receiving splats.

    Source: §I · p. 1, §VIII · p. 8

  • Results

    Across 100 decisions, TRACE achieved 83.3% agreement within 15◦ and retained 97.9% of the optimal centralized EIG.

    Source: E. Centralized-NBV recovery · p. 8

  • Takeaways & Limitations

    The many-to-one aggregates do not reveal splats or their count, but disclose coarse density and color within the receiver’s masked zone at pixel-angle and δ-depth resolution.

    Source: B. Reconstructing the context from… · p. 5

    Reconstruction is exact when no depth bin behind a splat mixes hits from two robots; otherwise the error is bounded, and mixing requires same-ray hits within δ.

    Source: B. Reconstructing the context from… · p. 6

Abstract

from arXiv · show

Share the light, not the map. We study next-best-view selection for a team of robots, each of which builds its own 3D Gaussian Splatting map and keeps it private. A robot picks the view with the largest expected information gain (EIG) about the splats along its own path. This gain depends on the other maps. Their splats occlude its own and shine behind them, so the gain has to be evaluated against the pooled map. No robot has this map. We show that the coupling passes through only two ray quantities, the transmittance in front of a splat and the radiance behind it, and that both are sums over the hits of the ray. Hence, they decompose across the robots, and each robot sums them over depth bins in its own map, along the rays of a candidate view, and sends the sums with their pose derivatives. The robot planning the view turns them into its EIG and gradient on SO(3). Transmittance and Radiance Aggregates, communicated for the EIG, give the protocol its name: TRACE. No robot shares its splats, and the message size does not grow with a map. We prove that the reconstruction is exact unless a depth bin behind a splat mixes hits of two robots, and we bound the error otherwise. Over 100 next-best-view decisions in Habitat-Sim, TRACE picks a heading within 15 degrees of the centralized one in 83.3% of the cases, and its views reach 97.9% of the centralized EIG.

I. INTRODUCTION

Team next-best-view selection must evaluate each robot’s view against a pooled scene, although the maps remain private. TRACE reconstructs the ray context needed for this evaluation, and the preliminaries define the Gaussian-splat map and safety-relevant region used by the method.

I. INTRODUCTION

  • The value of a view is a team quantity: a robot’s local map can miss useful geometry seen by teammates, while coordination is constrained by bandwidth, energy, and computation.The motivating setting is robots mapping a building after an earthquake to support safe traversal.Source: §I · p. 1

II. PRELIMINARIES AND DEFINITIONS

  • A 3DGS map is a set of anisotropic Gaussian primitives rendered by projection and front-to-back compositing along each ray.The rendering is differentiable in map parameters and camera pose, which NBV methods use to optimize views.Source: §I · p. 1

I. INTRODUCTION

  • A robot rendering only its own map misjudges the team objective because other robots’ splats change the light reaching and leaving its splats.The pooled ray context depends on the joint depth order, which no robot knows from its local map.Source: §I · p. 1
  • TRACE exchanges depth-bin sums of transmittance and radiance, with pose derivatives, so robots can evaluate pooled-map information gain without sharing splats.The method is exact when no depth bin behind a splat mixes robots’ hits, and bounds error otherwise.Source: §I · p. 1

II. PRELIMINARIES AND DEFINITIONS

  • A splat’s sensitivity to the rest of the scene depends only on the transmittance in front of it and the radiance behind it.These are the two quantities that couple the private maps.Source: §II · p. 3
  • The risk-aware masked zone is built from trajectory waypoints using larger radii where point risk is higher, selecting the safety-relevant splats for information objectives.The mask is the union of waypoint-centered balls, and splats are selected when their means lie in that zone.Source: §II · p. 3

III. EXPECTED INFORMATION GAIN OF A VIEW

The section defines a view’s expected information gain from Fisher information accumulated over rendered pixels and map parameters. It uses an additive per-splat gain and a first-order D-optimal approximation, with pose dependence entering through rendering sensitivities.

III. EXPECTED INFORMATION GAIN OF A VIEW

  • The paper measures view value by D-optimal log-determinant gain and uses its first-order trace approximation with a diagonal prior.The prior is stored diagonally because a full map-wide matrix is too large to store or invert; the trace approximation’s error is bounded in terms of the squared trace-scaled information matrix.Source: §III · p. 3
  • The masked EIG sums the information gains of safety-relevant splats, using the view’s Fisher information relative to the prior information already accumulated for each parameter.The view Fisher information is additive over pixels and views, and taking a view updates the prior by adding its Fisher information.Source: §III · p. 3
  • The pose affects the masked EIG through the pose-dependent rendering sensitivities, while the masked set and prior information remain fixed for a candidate pose.The authors also state that the log-determinant gain uses the same information entries with a changed weight; regularity assumes fixed hit sets locally and bounded blending coefficients and a positive-definite prior.Source: §III · p. 3

IV. PROBLEM STATEMENT

Each robot optimizes a view over its own trajectory-conditioned zone, but the objective is computed on a pooled map that no robot possesses. TRACE addresses this mismatch by exchanging only pooled-render context along candidate-view rays.

IV. PROBLEM STATEMENT

  • A local render differs from the pooled render because other robots’ splats occlude local splats and contribute radiance behind them.The privacy-preserving solution exchanges only the render context needed along candidate-view rays, never the splats themselves.Source: §IV · pp. 3–4
  • The centralized EIG for a robot’s candidate view is evaluated on the pooled map and includes gains from safety-relevant splats owned by every robot.The setup assumes maps are disjoint and no two hits share a depth on a ray.Source: §IV · pp. 3–4
  • View decisions are decoupled because each pose appears only in its own objective summand, but every summand still depends on the pooled map.No viewpoint consensus is needed; the challenge is evaluating each robot’s objective while other maps remain private.Source: §IV · p. 4

V. POSE-GRADIENT OF THE EIG ON SO(3)

The section expresses camera rotation as a three-dimensional perturbation about the current orientation and derives the EIG pose-gradient through the rasterizer’s blending coefficients. Radial-depth ordering and fixed hit colors make the pose dependence tractable.

V. POSE-GRADIENT OF THE EIG ON SO(3)

  • The camera rotation is parameterized as R(ξ) = R0Exp(ξ), where ξ is a small rotation about the camera’s own axes and ξ = 0 is the current orientation.The gradient is evaluated at ξ = 0 in the body frame.Source: §V · p. 4
  • Radial depth fixes hit order and depth-bin membership as the camera rotates, while the view-space depth can change.The radial-depth grid is used for the bins.Source: §V · p. 4
  • With hit sets and hit colors fixed under rotation, pose derivatives pass through the blending coefficients by the chain rule.The owner computes the blending-coefficient pose derivative from the rasterizer’s backward-pass quantities or central differences of its own render.Source: §V · p. 4
  • The pooled-scene gradient needs the transmittance in front of each splat and the radiance behind it, which a robot cannot compute from its private map alone.The rasterizer provides the pixel sensitivity for the robot’s own splats; the pooled context must still be obtained from the other maps.Source: §V · p. 4

VI. TRACE: DISTRIBUTED COMPUTATION OF THE POSE-GRADIENT

Rendering only a robot’s own map overestimates transmittance and miscomputes behind-radiance on rays meeting other maps, biasing both EIG and its gradient. TRACE addresses this by sharing depth profiles and their pose derivatives for the masked zone.

VI. TRACE: DISTRIBUTED COMPUTATION OF THE POSE-GRADIENT

  • On rays that meet another map, own-map rendering overestimates transmittance and computes incorrect behind-radiance, biasing both EIG and its gradient.TRACE is introduced to remove this bias.Source: §VI · p. 4
  • Each robot computes two depth profiles from its own map along rays that see the masked zone and shares them with their pose derivatives.The supplied text sketches the proofs due to the page limit.Source: §VI · pp. 4–5
  • The shared quantities are profiles and pose derivatives, rather than the robots’ splats.The section frames these profiles as the remedy for the pooled-scene context unavailable from a robot’s own map.Source: §VI · pp. 4–5

A. Cross-map coupling

The effect of other maps on a splat’s EIG sensitivity is captured by ray context: transmittance before the splat and radiance behind it. This context separates from a local factor computed from the differentiated splat.

A. Cross-map coupling

  • The splat’s color and blending coefficient depend on its own parameters, and the chain rule expresses rendered-color sensitivity through those quantities.The owner computes the local derivatives of color and blending coefficient from its own map.Source: A. Cross-map coupling · p. 5
  • The hit context combines transmittance reaching the splat with radiance behind it, while the paired local factor depends on the differentiated splat alone.The pose derivative of the sensitivity decomposes along the same two pairs.Source: A. Cross-map coupling · p. 5
  • The pose derivative of the local factor includes derivatives of the blending coefficient and its mixed pose-parameter derivative.The mixed derivative vanishes only for harmonic coefficients, where the blending-coefficient parameter derivative is zero.Source: A. Cross-map coupling · p. 5
  • The splat owner cannot compute pooled context and its three pose derivatives on rays meeting other maps; these are 4(1 + C) numbers per masked hit.The context is the only cross-map input needed for the gradient blocks.Source: A. Cross-map coupling · p. 5
  • Once given the context, each robot computes its gradient block from its own map, and no other map information crosses between robots.The overall gradient splits into blocks indexed by robot.Source: A. Cross-map coupling · p. 5

B. Reconstructing the context from depth profiles

TRACE reconstructs each hit’s pooled context from per-robot depth-bin profiles and derivative profiles, without sharing splat parameters or counts. Reconstruction is exact under a per-ray bin-separation condition and has bounded error when relevant bins mix robots’ hits.

B. Reconstructing the context from depth profiles

  • Each robot sums optical depth and composited radiance over its own hits in each radial-depth bin, along the receiver’s masked-zone rays.It also differentiates both profiles and records own-map factors for masked hits, with pose derivatives.Source: B. Reconstructing the context from… · p. 5
  • The exchanged profiles are many-to-one aggregates, not splats or splat counts, but disclose coarse density and color at the pixel-angle and depth-bin resolution within the receiver’s masked-zone footprint.The radiance profile composites each bin against that map’s own transmittance.Source: B. Reconstructing the context from… · p. 5
  • Because bin sums do not locate hits within a bin, the receiver interpolates the other robots’ share of the target hit’s bin when separating transmittance in front from radiance behind.The hit’s fractional position within its bin determines the interpolation.Source: B. Reconstructing the context from… · p. 5
  • The reconstructed context and its pose derivatives are obtained from the binned profiles and received derivative profiles.The derivatives are formed by differentiating the reconstruction term by term and combining them with the receiver’s own derivatives.Source: B. Reconstructing the context from… · pp. 5–6
  • The pooled-context reconstruction is exact for a target hit when every bin at or behind it contains hits from at most one robot.All robots must compute profiles at the same pose and on the same depth grid.Source: B. Reconstructing the context from… · pp. 5–6
  • Mixed bins in front of the target hit are compatible with exact reconstruction because only boundary transmittance values enter there and those factor by owner.The error analysis concerns bins at or behind the hit.Source: B. Reconstructing the context from… · p. 6
  • For nonnegative colors, the log-transmittance error is bounded by other robots’ optical depth in the target bin, while radiance-term errors are bounded relative to their own size by bin optical depth.The interpolated share of the target bin has its own bound; both bounds vanish under the exact-reconstruction condition.Source: B. Reconstructing the context from… · p. 6
  • Two robots mix a bin only when their hits on the same ray lie within δ, so bin width trades message size against a certified error.Optical depth from the other robots in a bin is the quantity used in the error bounds.Source: B. Reconstructing the context from… · p. 6

C. The TRACE protocol

TRACE estimates the ray context needed to evaluate each robot’s masked EIG and gradient, then combines those contributions without forming a pooled map. The reconstruction is exact when depth bins at or behind masked hits do not mix robots; otherwise, the received optical depths bound the error.

C. The TRACE protocol

  • Each robot estimates its masked splats’ EIG and gradient contributions from the received ray profiles, and the view-planning robot sums those contributions before updating its orientation.The profiles replace the exact context in the contribution equations; each robot evaluates its own masked splats.Source: C. The TRACE protocol · p. 6
  • A planning cycle exchanges profile values and pose derivatives over the masked zone’s ray footprint, then returns each robot’s estimated EIG and gradient contributions.The cycle runs for robots’ views simultaneously because the views are decoupled.Source: C. The TRACE protocol · p. 6
  • Computation and communication scale with the masked-zone footprint, not map size or splat count.Each per-step message is specified by footprint pixels and occupied depth bins per pixel.Source: C. The TRACE protocol · p. 6
  • The reconstruction is exact when no depth bin at or behind a masked hit mixes two robots; otherwise, received optical depths bound the error.Source: C. The TRACE protocol · p. 6

VII. EXPERIMENTS

The experiments compare TRACE with decentralized and centralized baselines while holding the simulator, mapper, planning pipeline, and evaluation protocol fixed. The authors avoid end-to-end comparisons with mapping systems because differences in localization and mapping would confound the effect of view selection.

VII. EXPERIMENTS

  • The evaluation compares TRACE with decentralized and centralized baselines across team sizes N ∈ {3, 4, 7, 10}.Source: §VII · p. 6
  • The simulator, incremental mapper, planning pipeline, and evaluation protocol are held fixed to isolate the view-selection policy.Source: §VII · p. 6
  • The authors omit end-to-end comparisons with multi-robot mapping systems because localization and mapping differences would confound the effect of NBV selection.Source: §VII · p. 6

A. Setup and baselines

TRACE improves reconstruction over decentralized planning while remaining competitive with centralized map sharing, under an evaluation that measures maps along each planner’s executed trajectory. The experiments use shared simulation conditions and compare distinct communication and planning baselines.

A. Setup and baselines

  • Experiments use Habitat-Sim and Gibson indoor scenes, with a private incremental 3DGS map per robot except for the Centralized Oracle.All methods share the scene, seed, start and goal, camera, and step limit.Source: A. Setup and baselines · p. 6
  • The baselines range from no-communication decentralized planning to map-sharing and observation-streaming centralized methods; TRACE keeps observations and Gaussian parameters local.TRACE-Sparse sends quantized occupied profiles and yaw derivatives.Source: A. Setup and baselines · p. 6

B. Reconstruction quality

  • TRACE consistently improves reconstruction over Decentralized planning across team sizes and is competitive with Centralized Map Sharing.TRACE outperforms Map Sharing in some settings and remains close in others; Centralized Oracle achieves the highest quality, while TRACE consistently exceeds TRACE-Sparse in PSNR.Source: B. Reconstruction quality · pp. 6–7, Table I · p. 7
  • Reconstruction metrics are averaged over poses around each executed trajectory, so results describe the map each planner produced along its route.Source: B. Reconstruction quality · pp. 6–7
  • Table II covers communication, NBV time, and mean map size during multi-robot exploration.Source: Table II · p. 7

C. Communication, computation, and map scaling

TRACE approaches centralized next-best-view selection while keeping maps private, with results spanning heading recovery, communication and computation, and a trade-off between sparse-gradient and discrete selection. Its reconstruction and gradient are exact unless a depth bin at or behind a splat mixes hits from two robots.

C. Communication, computation, and map scaling

  • Communication falls by approximately three orders of magnitude relative to TRACE and over 90% relative to Centralized Map Sharing.TRACE keeps observations and Gaussian parameters onboard; its payload depends on map overlap and team size. It is faster than both centralized methods, but reported NBV time excludes network transfer.Source: C. Communication, computation, and map… · p. 7

D. Ablation study

  • For N = 2, TRACE-Sparse cuts NBV time from 9.46 to 0.59 s, and for N = 5 from 22.34 to 1.69 s versus TRACE-Discrete.TRACE-Discrete achieves moderately better reconstruction, while both variants keep observations and Gaussian parameters onboard and complete all navigation tasks.Source: D. Ablation study · pp. 7–8

E. Centralized-NBV recovery

  • TRACE matches the centralized heading in 77.8% of decisions, reaches 83.3% agreement within 15°, and retains 97.9% of centralized EIG.Inter-robot map overlap falls from 25.7% to 15.9%; Fig. 3 shows representative selections along two agents’ trajectories.Source: Figure 3 · p. 7, E. Centralized-NBV recovery · p. 8, Table IV · p. 8

VIII. CONCLUSION

  • Reconstruction is exact unless a depth bin at or behind a splat mixes hits from two robots; bin width sets message size and gradient accuracy.Each robot sends optical depth and radiance per depth bin, with pose derivatives, rather than sharing splats.Source: §VIII · p. 8
Loading 2610.00822v1…