RoboShape: Information-Theoretic Point Cloud Representations for Privacy-Aware Robot Perception

Oguzhan Baser; Mirac Sozen; Kaan Kale; Sandeep Chinchali; Sriram Vishwanath

Source-linked AI summary

RoboShape: Information-Theoretic Point Cloud Representations for Privacy-Aware Robot Perception

Oguzhan Baser, Mirac Sozen, Kaan Kale, Sandeep Chinchali, Sriram Vishwanath

arXiv:2608.21380v1Jul 16, 2026cs.ROcs.AIcs.CVcs.IT

TL;DR

Robotic systems share 3D representations that encode both useful object information and sensitive spatial context, while traditional encoders lack principled selective control. RoboShape adds an information-theoretic projection head over a frozen Sonata encoder, shaping voxel embeddings with Donsker–Varadhan mutual-information objectives. Across three real-world indoor LiDAR datasets, it compresses embeddings by 87.5%, retains 98.7% of object classification utility, and reduces private attribute leakage by 39.3%.

Problem
Traditional 3D encoders do not selectively separate task-relevant object attributes from sensitive spatial attributes when representations are shared.
Method
RoboShape trains a lightweight projection over a frozen Sonata encoder using voxel-level Donsker–Varadhan mutual-information optimization to preserve object information and suppress room-type information.
Results
Across ScanNet, Matterport3D, and ARKitScenes, RoboShape achieves 8× compression while retaining 98.7% of object classification utility and reducing private attribute leakage by 39.3%.
Takeaways & Limitations
RoboShape provides a compressed, privacy-shaped drop-in replacement for original encoder outputs in downstream robotic perception pipelines.
Takeaways & Limitations
The approach assumes voxel-level independence for mutual-information estimation, evaluates scene-level room-type privacy labels, and has not been tested on contact-rich manipulation or long-horizon planning.

Abstract

from arXiv · show

With the increased adoption of robotic agents operating in human environments by scanning and sharing 3D representations (e.g., for fleet learning, cloud-based planning, or collaborative mapping), collected point clouds reveal not just the objects in a scene but also sensitive spatial context, such as room function or information that occupants never consented to disclose. Traditional point cloud encoders offer no principled control over this: either all is preserved, or none. Hence, we introduce RoboShape, an information theory guided compression head following the frozen {\tt Sonata} encoder. We project voxel-level embeddings using the Donsker-Varadhan formulation of mutual information (MI). Specifically, we maximize the MI between embeddings and object-level understanding while minimizing it for private attributes. RoboShape leads to 87.5\% smaller embeddings that retain 98.7\% of object classification utility while collapsing sensitive attribute predictions by 39.3\% across the three real-world indoor LiDAR datasets. Its privacy-preserving embeddings are cheaper to transmit over the network or to train a model for any downstream tasks. We release the RoboShape codebase to give the robotics community a practical, encoder-agnostic tool for building perception pipelines that are compact, privacy-aware, and deployment-ready.

1 Introduction

RoboShape addresses the lack of selective information control in 3D encoders by shaping compressed voxel representations to preserve object utility while suppressing private room information. It introduces a lightweight, MI-based projection module and reports strong utility retention with reduced leakage across three real-world datasets.

Motivation: Existing 3D encoders share object geometry, room layout, and functional cues together, while noise, shuffling, and adversarial mitigation lack selective or measurable control.These approaches either degrade useful structure indiscriminately, provide no control over retained information, or lack an information-theoretic leakage objective.
Approach: RoboShape reshapes frozen-encoder voxel embeddings by maximizing mutual information with object labels and minimizing it with sensitive room-type attributes.The method treats information suppression as a constrained representation-design objective rather than a post-hoc correction.
Results: 87.5% embedding compression retains 98.7% of object classification utility while reducing private attribute leakage by 39.3% across ScanNet, Matterport3D, and ARKitScenes.The reported evaluation compares RoboShape with baselines across utility and privacy metrics.
Impact: The released codebase, pretrained models, and evaluation pipeline support reproducible privacy-aware robotic perception research.RoboShape is presented as a practical tool for deployment-oriented perception pipelines.

2 Related Work

Prior 3D encoders provide strong features but do not control which information their representations reveal. RoboShape adapts information-theoretic shaping to variable-count voxel features and unifies compression, utility preservation, and privacy suppression.

3D Representation Learning: Modern point cloud encoders support manipulation and navigation but treat all encoded information equally and provide no control over representation disclosure.RoboShape operates atop a frozen backbone and selectively reshapes its output.
Information-Theoretic Shaping: Earlier information-shaping methods target fixed-dimensional sentence or audio-window embeddings, whereas RoboShape handles variable-count voxel features from sparse, irregular 3D structures.This difference requires distinct batching and estimation strategies for tractable mutual-information optimization.
Privacy-Aware Perception: Existing privacy approaches use heuristics such as differential noise, federated learning, or visual anonymization that can degrade utility indiscriminately and cannot quantify residual sensitive information.RoboShape combines compression, task utility, and privacy suppression under one measurable mutual-information objective.

3 Problem Formulation

The problem formulation asks how to compress indoor 3D scene representations while preserving object information and suppressing sensitive spatial attributes. It defines voxel-level labels and a lower-dimensional projection governed by three competing information goals.

Problem Setting: The central problem is compressing shared 3D scans while preserving task-relevant object information and suppressing sensitive spatial attributes.The setting includes cloud servers or other agents performing object recognition, semantic mapping, or manipulation planning.
Voxel Representation: A frozen Sonata encoder maps each voxelized scene cell to a per-voxel embedding xk ∈ R^d, with d = 512 for Sonata.The point cloud is grouped into K voxels before encoding.
Labels: Each voxel carries a public object or furniture label L(vk) and a private room-type label S(vk).The public label captures geometric understanding for manipulation and navigation, while the private label describes the enclosing scene’s spatial function.
Optimization Objective: A trainable projection maps xk to zk ∈ R^d′ with d′ ≪ d, retaining information about L(vk) while discarding information about S(vk).The formulation explicitly introduces a privacy-utility trade-off absent from standard 3D encoders and distinguishes its per-voxel structure from text and speech shaping.
Optimization Objective: The objective balances information-preserving compression, task utility, and privacy through mutual-information terms weighted by γ, λ, and µ.The ratio d/d′ controls the compression rate.

4 Information-Theoretic Approach

RoboShape makes mutual-information optimization tractable by treating voxel embeddings as independent fixed-dimensional samples and using Donsker–Varadhan estimators during training. A trainable projection compresses frozen Sonata features, while evaluation tests utility and sensitive leakage.

Voxel-Level MI Estimation: Per-voxel embeddings are treated as independent samples, enabling tractable mutual-information optimization over sparse, irregular point clouds.Voxel embeddings are pooled across scenes into batches of fixed-dimensional vectors despite varying voxel counts.
Donsker–Varadhan Estimation: The Donsker–Varadhan representation estimates mutual information from joint and marginal samples using neural networks trained by stochastic gradient descent.Marginal samples are formed by shuffling label assignments within a batch.
Architecture: A frozen Sonata encoder produces 512-dimensional voxel embeddings, and the only trainable component projects them from R512 to R64.At inference, the projection produces representations that are 8× more compact, while the MI estimator is discarded after training.
Optimization: Training alternates between updating MI estimator networks within epochs and updating projection parameters after MI estimates converge.The estimator networks guide differentiable losses during training but are not retained for inference.

5 Experimental Results

Across three indoor datasets and complementary static and embodied evaluations, RoboShape compresses Sonata embeddings while preserving furniture-related utility and reducing room-type information. MI trends, AUROC probing, t-SNE geometry, and PPO navigation consistently support this utility–privacy trade-off, though evaluation remains bounded by stated assumptions and task coverage.

Evaluation setup: RoboShape is compared with Original Sonata, Noisy, and Random Encoder baselines using MI evolution, classification AUROC, t-SNE visualizations, and embodied PPO evaluation.The datasets are ScanNet, Matterport3D, and ARKitScenes; PPO tests use 52 ScanNet-derived Isaac Gym environments.
Static utility and privacy: 87.5% compression retains 98.7% of task-relevant classification utility, with AUROC 0.978 for RoboShape versus 0.990 for Original Sonata.Noisy and Random Encoder achieve AUROCs of 0.955 and 0.943, respectively.
Static utility and privacy: 39.3% reduction lowers room-type AUROC from 0.9998 for Original Sonata to 0.607 for RoboShape.Noisy and Random Encoder retain higher room-type AUROCs of 0.930 and 0.924.
Information shaping: Utility MI increases from 0.293 to 0.535 nats while privacy MI decreases from 0.247 to 0.123 nats over 20 epochs.The curves diverge monotonically, indicating the weighted objective preserves task-relevant information while filtering room-type information without observed oscillation.
Embedding geometry: After RoboShape, furniture-type clusters remain separable in t-SNE while room-type clusters overlap, consistent with the AUROC results.Original Sonata embeddings show distinct clusters for both furniture and room type.
Embodied control: In PPO navigation, RoboShape reaches approximately 69% of Original Sonata’s wall-seeking training success rate while reducing bedroom-conditional successes from approximately 52 to 19, a 63% reduction.The evaluation covers 50 PPO epochs across 52 Isaac Gym environments and tests both utility and privacy behavior under closed-loop control.
Limitations: RoboShape’s evaluation assumes voxel-level independence for MI estimation, uses scene-level room-type labels, and does not yet test contact-rich manipulation or long-horizon planning.These boundaries motivate richer privacy annotations and additional embodied tasks.

6 Conclusion and Future Work

RoboShape shapes compressed 3D point-cloud representations with mutual-information optimization, preserving object utility while suppressing room-type information. The authors identify broader embodied-task, multi-attribute, and optimization extensions for future work.

Conclusion: 87.5% embedding compression retains 98.7% of object classification utility while reducing room-type leakage from near-perfect to near-chance levels.Across ScanNet, Matterport3D, and ARKitScenes, AUROC falls from 0.9998 to 0.607.
Conclusion: RoboShape trains a lightweight projection layer atop a frozen Sonata encoder using Donsker-Varadhan mutual-information optimization.The framework shapes compressed, privacy-aware 3D point-cloud representations for robot perception.
Future Work: Further evaluation is planned on navigation, manipulation, and semantic goal-reaching to validate closed-loop robotic performance.The authors also identify simultaneous protection of room type, building identity, and occupant activity as a future extension.
Future Work: The authors propose nonlinear MI optimization to automate hyperparameter selection across the γ, λ, µ trade-off surface.The code and data are publicly available for reproducibility and future research.
Appendix: The appendix documents information-theoretic foundations, full numerical results, datasets, training details, Isaac Gym experiments, code, and broader impact.These materials cover both the method and its evaluation resources.

A.2 Estimation Error and Finite-Sample Bias

The Donsker-Varadhan estimator provides a differentiable MI lower bound, but finite-sample bias and dimensionality affect estimation. RoboShape uses large batches, moving averages, compression, and bilevel optimization to make privacy minimization informative.

Estimation Error: The DV estimator is consistent as sample size and function-class capacity grow, but it exhibits finite-sample bias.This establishes an asymptotic guarantee without removing practical estimation error.
Finite-Sample Bias: The log-sum-exp term is empirically biased upward, causing the DV estimate to underestimate true mutual information.Large batch sizes and an exponential moving average mitigate this bias.
Dimensionality Dependence: MI-estimation sample complexity scales as O(e^I(A;B)) in the worst case, so reducing embedding dimension improves effective sample efficiency.RoboShape compresses from d = 512 to d′ = 64.
Optimization: RoboShape maximizes compression and utility MI terms while minimizing the privacy MI term.The privacy objective therefore requires interpreting minimization of a lower bound rather than direct minimization of true MI.
Optimization: Bilevel optimization trains the privacy estimator to tighten the DV bound before updating the projection layer to minimize it.This minimax dynamic makes room-type information progressively harder for the estimator to detect and supplies a scalar privacy metric.
Optimization: The data processing inequality supports reducing privacy MI through deterministic projection while preserving task-relevant information through the objective.The projection is designed to reduce the sensitive-attribute bound aggressively without discarding all useful content.

A.4 Data Processing Inequality and Compression

Deterministic compression cannot increase mutual information and necessarily discards some information, so RoboShape steers retained capacity toward task-relevant features and away from privacy attributes. Its results suggest these signals are partially separable in Sonata embeddings.

Data Processing Inequality: For any deterministic mapping, the data processing inequality guarantees that processing can only reduce mutual information.This gives RoboShape a monotonicity basis for privacy suppression.
Compression: Compression necessarily discards some task-relevant information, so RoboShape’s objective steers the projection toward retaining L(x)-relevant features.The same framework reduces, but cannot increase, privacy leakage relative to the original encoder.
Empirical Interpretation: 98.7% task AUROC retention alongside a 39.3% privacy-AUROC reduction suggests partially separable task and privacy subspaces in Sonata embeddings.The MI-guided projection is interpreted as preserving task-relevant features while projecting out privacy-relevant ones.
Voxel-Level Assumption: RoboShape treats per-voxel embeddings as independent MI-estimation samples, pooling voxels across scenes to obtain fixed-dimensional batches.This design addresses variable voxel counts in sparse 3D scenes.
Voxel-Level Assumption: Local receptive fields and cross-scene pooling make the voxel-level i.i.d. approximation more realistic despite local within-scene dependence.The assumption is supported empirically by stable, monotonic MI convergence across all three datasets.
Rate-Distortion Connection: RoboShape’s objective can be viewed as a multi-objective rate-distortion formulation balancing compression rate, object utility, and privacy.The hyperparameters γ, λ, and µ trace a Pareto surface in rate-utility-privacy space.

B Full Numerical Results

Across three datasets, RoboShape maintains task-relevant performance close to uncompressed embeddings while consistently reducing private-label predictability. Its MI objective also reduces privacy MI across datasets.

RoboShape consistently achieves the lowest private-label AUROC across all three datasets while keeping task-relevant performance close to uncompressed Original embeddings.Table 1 compares four embedding types using public furniture labels and private room-type labels.
RoboShape is the only method that simultaneously achieves high public AUROC and near-chance private AUROC on the reported datasets.
Privacy MI drops by 50–60% across all datasets after 20 training epochs.Table 2 reports converged MI values in nats, with utility MI maximized and privacy MI minimized.

C Dataset Details

The evaluation spans controlled, diverse, and deployment-realistic indoor datasets, with ScanNet used to characterize scene, voxel, and label structure. These properties motivate binary utility evaluation, voxel-level independence, and care around furniture–room correlations.

Dataset selection: The three datasets progress from controlled ScanNet scans to diverse Matterport3D environments and deployment-realistic ARKitScenes mobile-LiDAR conditions.
ScanNet characteristics: ScanNet is the primary benchmark for analyses of furniture frequencies, voxel counts, label diversity, and room-type correlations.
Furniture labels: Binary classification is used for utility evaluation because common furniture categories are widespread while bookshelves and curtains occur infrequently.
Voxel structure: ScanNet scenes contain approximately 10K to more than 300K voxels after voxelization, motivating a voxel-level independence assumption for tractable MI estimation.
Privacy–utility structure: Furniture–room correlations make privacy–utility disentanglement nontrivial because suppressing room-type information must preserve co-occurring furniture features.
Voxel labels: Most voxels contain one furniture label, whereas boundary voxels contain 2–3+ labels and receive their majority constituent label.
Encoder baseline: Figure 11 provides a baseline by comparing Sonata segmentation with ScanNet ground-truth annotations for interpreting utility MI values.

D Architecture and Training

RoboShape freezes Sonata, projects its voxel embeddings through a trainable compression layer, and optimizes MI estimators during training. The design targets compact sharing while suppressing sensitive room-type information, with deployment and broader-impact considerations included.

Architecture: The frozen PTv3-based Sonata encoder has 46.1M parameters and produces a 512-dimensional embedding per voxel.Sonata was self-distilled on 140K point clouds from ScanNet, Structured3D, and S3DIS.
Architecture: The trainable projection layer maps 512 dimensions to 64 through Linear(512, 256), Linear(256, 128), and Linear(128, 64) layers with ReLU activations.It contains 172,480 trainable parameters and uses no batch normalization, dropout, or output normalization.
MI estimation: Three Donsker–Varadhan estimators use two hidden layers of 64 and 32 nodes and are discarded after training.Their input dimensions are 576 for I(z; x) and 65 for I(z; L(x)) and I(z; S(x)).
Training: Training runs for 20 epochs with Adam, using learning rates 10^-3 for the projection layer and 10^-4 for MI estimators.The privacy weight µ is selected per dataset via the Pareto frontier between utility MI and privacy MI.
Control evaluation: PPO policies in NVIDIA Isaac Gym test whether MI-based privacy shaping transfers to closed-loop robotic control.
Deployment: RoboShape is intended for privacy-aware sharing in multi-robot and cloud-robotics pipelines, while its codebase provides a repository structure for deployment-oriented use.
Broader impact: RoboShape reduces room-type mutual information but does not provide a formal differential-privacy guarantee against adversaries with auxiliary information.