PaperlayerView exact revision on arXiv

Source-linked AI summary

Seeing Less Is Not Seeing Safely: Privacy Leakage from Task-Scoped Robot Perception Exports

Yuqiao Xu, Erman Ayday

arXiv:2609.03055v1cs.ROcs.CR

TL;DR

Domestic robot exports can reveal household information even when raw sensor data remain local. TFPD keeps rich perception local while profiling task-scoped exports; results show that task-equivalent representations can differ in leakage, with no universal privacy ordering for stronger abstraction.

  • Problem

    Derived representations can disclose household semantics, geometry, and spatial structure even when raw sensor data remain local, raising the question of what may leave local perception.

    Source: §I · p. 1

  • Method

    TFPD keeps rich perception local and evaluates complete task-scoped exports by task utility, direct exposure, and residual inference risks.

    Source: A. Task-Scoped Export Construction · p. 5, §VI · p. 12

  • Results

    Across AI2-THOR and ProcTHOR, task-equivalent navigation exports remained privacy-inequivalent, while AI2-THOR linkability ranged from 0.532 to 0.970.

    Source: §I · p. 2

  • Takeaways & Limitations

    Attribute removal or stronger abstraction alone is not evidence of privacy; exports should be evaluated for task utility, direct exposure, and residual inference.

    Source: §VII · p. 13

    ProcTHOR uses the AI2-THOR execution engine, so the replication does not validate the finding in an independent simulator or on a physical robot.

    Source: §VI · p. 12

Abstract

from arXiv · show

Domestic robots rely on rich perception to operate in private homes, but privacy risk persists even when raw sensor data remain local. Structured representations exported to downstream planners, cloud services, logs, or learning pipelines can still reveal household information through semantics, geometry, spatial structure, and task targets. We introduce Task-Functional Perception Distillation (TFPD), a task-scoped representation-export framework that keeps rich perception local and profiles downstream exports according to task utility, direct exposure, and multiple residual inference risks. Using 120 AI2-THOR scenes with scene-disjoint train/validation/test splits, frozen attacker selection, and representation-aware held-out attacks, we evaluate navigation, collision checking, and object-goal execution. Three navigation exports achieve identical success (1.000) and mean path ratio (0.898), yet representation-level linkability ranges from 0.532 to 0.970. Replacing an explicit target label with a target region reduces target-category macro-F1 from 1.000 to 0.077 while preserving success at 0.995, while geometric coarsening reduces object-category macro-F1 from 0.704 to 0.556 at a measurable collision-utility cost. A ProcTHOR replication preserves the navigation task-equivalence/privacy-inequivalence finding while changing the relative ordering of normalized and topological exports. These results show that neither field removal nor stronger abstraction induces a universal privacy ordering and motivate task-specific, multi-risk evaluation of the complete public representation.

I. INTRODUCTION

Domestic robots need rich perception, but exported representations can disclose household information even when raw sensor data remain local. TFPD evaluates task-scoped exports, and the results show that equal task utility does not guarantee equal privacy risk.

Motivation

  • Rich perception is necessary in private homes, yet derived representations can disclose household semantics, geometry, and spatial structure even when raw images remain local.Downstream systems exchange structured messages, maps, object representations, and scene graphs for planning, collision checking, logging, cloud computation, and learning.Source: §I · p. 1

Approach

  • TFPD places an export boundary after local perception and releases task-scoped representations, while profiling direct exposure and residual inference from the remaining representation.The framework does not presume a universal privacy representation; candidates for a task may preserve equivalent functionality while exposing different structures.Source: §I · pp. 1–2

Evaluation

  • The primary evaluation used 120 AI2-THOR scenes with scene-disjoint train, validation, and held-out test partitions, frozen attacker selection, and representation-aware attacks.Navigation was also evaluated on 120 ProcTHOR houses as a cross-distribution replication, not as an independent-simulator validation.Source: §I · p. 2

Empirical findings

  • Replacing an explicit target label with a target region reduced target-category macro-F1 from 1.000 to 0.077 while preserving object-goal success at 0.995.Geometric coarsening reduced object-category macro-F1 from 0.704 to 0.556, while collision F1 decreased from 1.000 to 0.950.Source: §I · p. 2

Contributions

  • An audit of 33 representation–attack combinations found that restricted attacks sometimes underestimated leakage already present in the public representation.The contribution profiles candidate exports by task utility and multiple privacy risks, rather than treating field removal as evidence of privacy.Source: §I · p. 2

Empirical findings

  • Across AI2-THOR and ProcTHOR, task-equivalent navigation exports remained privacy-inequivalent, while their precise linkability ordering changed across scene distributions.In AI2-THOR, linkability ranged from 0.532 to 0.970; ProcTHOR reversed the relative ordering of normalized and topological exports.Source: §I · p. 2

II. BACKGROUND AND RELATED WORK

Prior robot privacy work has mainly limited exposure at or before visual perception, while transformed representations can retain informative structure. TFPD focuses on evaluating alternative exports for downstream robot tasks.

Privacy in robotics

  • Prior approaches primarily protect information at or before visual perception, including by filtering operator-facing images, reducing visual fidelity, concealing sensitive regions, or discarding raw observations after local processing.These approaches address visual exposure, whereas TFPD focuses on what derived representations reveal after perception.Source: §II · p. 2

Privacy-preserving representations

  • Privacy-preserving representations have been designed for particular functions, including localization, multiview navigation, cloud object detection, and point-cloud utility.TFPD instead considers representations exported to different downstream robot tasks.Source: §II · p. 4

Residual leakage

  • Geometry, topology, and spatial organization can remain informative after explicit semantics or selected attributes are removed.Prior work reports recoverable scene geometry and floor-plan or room information from structured representations and policies.Source: §II · p. 4

Data minimization

  • Minimizing individual fields does not ensure privacy because information removed explicitly may remain inferable from the retained task-facing structure.TFPD’s unit of analysis is the structured semantic, geometric, and spatial representation required by a robot task, rather than a subset of independent attributes.Source: §II · p. 4

Positioning

  • TFPD evaluates metric, normalized, and topological free space; anonymous exact and coarsened geometry; and semantic, spatial, and structural target references.It examines whether task-equivalent representations have comparable privacy risk, whether stronger abstraction reduces leakage, and whether reducing one privacy channel reduces others.Source: §II · p. 4

III. THREAT MODEL AND PROBLEM FORMULATION

The threat model places a trusted local perception stack behind an export boundary and evaluates what downstream adversaries can infer from the complete public representation. TFPD distinguishes direct exposure, residual inference, and task utility.

Scope and trust boundary

  • The scope is environmental-perception privacy in domestic robots: rich sensing and interpretation remain local, while the framework controls which derived representation leaves that boundary.Human–robot interaction channels such as audio, speaker identity, transcripts, and conversational history are outside the stated scope.Source: §III · p. 4
  • The local sensing and perception stack is assumed trusted; downstream consumers may process, store, or reuse exports, so the boundary must precede their release.The adversary observes exported representations, not raw sensor streams or state retained exclusively inside local perception.Source: §III · p. 4

Adversary

  • The adversary is conservatively assumed to know the export schema, downstream task, and public transformation, rather than relying on secrecy of the representation format.This is a threat-model assumption, not a claim that every attacker has that knowledge.Source: §III · p. 4

Evaluation protocol

  • Attack models and representation-specific preprocessing are fitted on training data, selected on validation data, and frozen before held-out testing.The attacker receives no information from inside the protected local boundary.Source: §III · p. 4

Residual privacy risks

  • Residual inference risks include room or environmental context, private-object presence, object categories from geometry, scene linkability, and target-category recovery.These risks concern information recoverable from the export, including through geometry, layout, topology, task context, or other public structure.Source: §III · pp. 4–5
  • Scene linkability measures matching a perturbed export to its source scene in a gallery, not real-world cross-session home re-identification.The query is perturbed from the same underlying export rather than independently recaptured across sessions.Source: §III · p. 5

Privacy and utility goals

  • TFPD aims to reduce direct exposure, characterize residual inference from the complete export, and preserve task utility.These measures distinguish privacy improvement without measured utility loss from privacy–utility tradeoffs or conflicts across privacy channels.Source: §III · p. 5

Non-goals

  • TFPD neither protects a compromised trusted local stack nor provides a formal privacy guarantee; attack results characterize evaluated adversaries, not an upper bound on inference.The work also does not introduce a new planner, collision-checking algorithm, or controller, or adaptively select representations at runtime.Source: §III · p. 5

A. Task-Scoped Export Construction

For each downstream task, TFPD constructs candidate exports from rich local perception by retaining, transforming, or omitting information. The complete task-facing export—not only the transformed field—is the object of utility and privacy analysis.

Candidate construction

  • Each candidate export is constructed from the same local perception state, and candidates for a task may meet the same functional requirement while exposing different structure.The local state can contain observations, semantic information, identities, geometry, free-space structure, localization, and resolved target information.Source: A. Task-Scoped Export Construction · p. 5

Transformation pathways

  • TFPD distinguishes information retained and exposed directly, information used locally to construct a transformed export, and information omitted from the export.Pure field removal is the special case in which no information is used to construct a replacement.Source: A. Task-Scoped Export Construction · p. 5
  • Candidate transformations include normalized coordinates, topological connectivity, coarsened obstacle geometry, and spatial or structural references for locally resolved semantic targets.Locally used information need not itself appear in the export; a target category can be used to derive a target region while remaining inside the protected boundary.Source: A. Task-Scoped Export Construction · p. 5

Public export contract

  • A candidate name identifies the component varied by a transformation, not all information available downstream; task-common interface information remains in the complete export.Utility and residual privacy analysis therefore use the complete task-facing representation.Source: A. Task-Scoped Export Construction · p. 5

Object-goal export

  • For object-goal execution, a target region describes how an already-resolved goal is referenced downstream, and reduced target-category inference does not imply that other household attributes are uninformative.Additional public spatial or planning structure required by the evaluated task remains part of the export.Source: A. Task-Scoped Export Construction · p. 6

B. Task-Specific Instantiations

TFPD instantiates task-scoped exports for navigation, collision checking, and object-goal execution, retaining information needed by each downstream function while transforming or omitting other information.

Navigation

  • Navigation exports preserve reachability while progressively removing metric detail: normalized maps omit global origin and absolute scale, and topological maps retain connectivity without coordinates or metric edge lengths.All three navigation exports omit semantic object labels and identities.Source: B. Task-Specific Instantiations · p. 6

Collision checking

  • Collision checking compares anonymous exact obstacle geometry with coarsened, lower-precision proxies that retain approximate extent for intersection and clearance reasoning.The evaluation specifies the coarsening parameters separately.Source: B. Task-Specific Instantiations · p. 6

Object-goal execution

  • Object-goal execution resolves the target locally, then can export a coordinate, region, anonymous identifier, or planning-graph node instead of the semantic label.The downstream component receives a functional reference to the already-resolved target.Source: B. Task-Specific Instantiations · pp. 6–7

Task-specific export

  • Across tasks, TFPD constructs candidate representations and exports only the selected task-specific representation across the boundary.The evaluated candidate families differ in privacy-relevant information exposed for their downstream functions, while task-common public information remains part of the complete export.Source: B. Task-Specific Instantiations · p. 7, Table I · p. 7

C. Privacy and Utility Profiling

TFPD profiles each candidate export by task utility, direct exposure, and residual inference, treating privacy risks across channels separately rather than assuming that field removal ensures privacy.

Direct exposure and residual inference

  • Direct non-disclosure means selected privacy-bearing fields are unavailable through the interface, but does not establish that their associated information cannot be inferred.Residual inference is measured from the complete public representation using macro-F1 for classification and Top-1 accuracy for linkability.Source: C. Privacy and Utility Profiling · p. 7

Multi-risk profile

  • Candidate exports are characterized by task utility, direct exposure, and a vector of residual-inference risks across privacy channels.The components remain separate because a transformation may reduce one risk while leaving another unchanged or increasing it.Source: C. Privacy and Utility Profiling · p. 7

Utility and privacy

  • Task equivalence does not imply privacy equivalence: representations meeting the same complete set of utility criteria may expose different privacy risks.The navigation candidates are considered equivalent under the evaluated success, path-feasibility, and mean-path-ratio criteria.Source: C. Privacy and Utility Profiling · p. 8

Interpreting changes

  • A channel-specific privacy improvement requires lower measured attack success with preserved task utility; reduced attack success alongside lower task performance is a privacy–utility tradeoff.TFPD does not weight heterogeneous risks or automatically select a globally optimal representation.Source: C. Privacy and Utility Profiling · p. 8

V. EVALUATION

The evaluation uses AI2-THOR as its primary setting, with ProcTHOR navigation as a cross-distribution replication and a synthetic environment for controlled diagnostics. Its questions test direct disclosure, task/privacy equivalence, abstraction, and representation-aware attacks.

Evaluation settings

  • AI2-THOR is the primary evaluation for privacy–utility claims across three tasks; ProcTHOR tests whether the navigation finding persists across a separately generated house distribution.ProcTHOR uses the AI2-THOR execution engine, so it is treated as a cross-distribution replication rather than an independent simulator.Source: §V · p. 8

Evaluation questions

  • The evaluation asks whether field removal prevents inference, whether equal task utility entails equal privacy, whether greater information removal improves privacy, and how representation-aware attackers affect conclusions.Broader use of public-export fields can substantially change leakage estimates and their relative ordering.Source: §V · p. 8

Evaluation results

  • The main results answer the first three evaluation questions negatively: removing explicit fields does not eliminate residual inference, task-equivalent exports can remain privacy-inequivalent, and stronger abstraction does not necessarily lower leakage.Task-specific transformations can still yield channel-specific benefits, sometimes with a measurable utility cost.Source: §V · p. 8

A. Primary AI2-THOR Evaluation Protocol

The protocol uses scene-disjoint data splits and freezes attacker selection before held-out evaluation. It evaluates task utility and privacy attacks on public representations, including an audit of attacks that may use more of the exported structure.

A. Primary AI2-THOR Evaluation Protocol

  • 120 AI2-THOR scenes are split 80/20/20 by scene, with representations from each scene kept in the same partition.Attack models and preprocessing are fitted on training scenes and selected on validation data before held-out testing.Source: A. Primary AI2-THOR Evaluation Protocol · p. 8
  • Navigation utility uses 1,000 trials and measures success, path feasibility, and mean path ratio; collision utility uses collision F1, and object-goal utility uses execution success.Scene-level and episode-level privacy attacks use held-out scenes or episodes, while collision object-category inference uses 848 held-out object records.Source: A. Primary AI2-THOR Evaluation Protocol · pp. 8–9
  • Attacker selection uses validation macro-F1, with classification results compared against a training-set most-frequent-class baseline.The baseline is fixed from training labels and excluded from attacker selection.Source: A. Primary AI2-THOR Evaluation Protocol · p. 9

B. Representation-Aware Attacker Audit

  • Representation-aware attackers use public fields and structure, never information retained inside the protected local boundary; feature mappings and model families are selected on validation data and frozen.The expanded candidates incorporate public spatial, geometric, relational, or target-relative structure, depending on the task.Source: B. Representation-Aware Attacker Audit · p. 9
  • The audit covers 33 representation–attack pairs: 7 already used the relevant public representation, while 26 received broader representation-aware feature variants.Table II summarizes the total pairs and the already-complete versus expanded mappings.Source: B. Representation-Aware Attacker Audit · pp. 9–10, Table II · p. 10

C. Navigation: Equivalent Utility and Different Spatial Leakage

Navigation exports have identical measured utility but substantially different residual privacy risks. The ProcTHOR replication preserves this privacy inequivalence while reversing the normalized-versus-topological linkability ordering.

C. Navigation: Equivalent Utility and Different Spatial Leakage

  • Room-inference macro-F1 is 0.479 for normalized, 0.558 for topological, and 0.736 for metric free space; private-object inference is 0.412 for normalized and 0.567 for topological.Metric free-space private-object inference is 0.928; room and private-object baselines are 0.100 and 0.444, respectively.Source: C. Navigation: Equivalent Utility and… · p. 10
  • Topological free space removes coordinates, absolute metric scale, and metric edge lengths, yet its attack point estimates exceed normalized free space across all three navigation privacy channels in AI2-THOR.The attribute-inference confidence intervals overlap, so this is a point-estimate ordering rather than a formal pairwise significance result.Source: C. Navigation: Equivalent Utility and… · p. 10
  • For metric navigation, private-object macro-F1 rises from 0.405 under the restricted mapping to 0.928 after validation selects expanded public structure.The expanded fields are introduced jointly, so the increase is not attributed to any single field.Source: C. Navigation: Equivalent Utility and… · p. 10
  • Across 1,000 ProcTHOR trials, all three representations achieve success and path feasibility 1.000 and mean path ratio 0.879, while linkability is 1.000 for metric, 0.573 for normalized, and 0.497 for topological free space.Metric remains most linkable, but normalized and topological ordering reverses relative to AI2-THOR; their ProcTHOR confidence intervals overlap.Source: C. Navigation: Equivalent Utility and… · pp. 10–11
  • Top-1 linkability is 0.970 for metric, 0.532 for normalized, and 0.782 for topological free space, despite identical mean path ratio 0.898 and success 1.000 across all three.The privacy–utility results are summarized in Table III.Source: C. Navigation: Equivalent Utility and… · p. 10, Table III · p. 11

D. Collision Checking: Geometry Leakage and Utility Tradeoff

Anonymous geometry can preserve semantic information even without labels or identities. Coarsening reduces object-category inference but costs collision-checking utility, while other measured privacy channels do not improve uniformly.

D. Collision Checking: Geometry Leakage and Utility Tradeoff

  • The coarsened and exact exports use the same ordered obstacle set and differ only in geometric precision.Obstacle geometry is exported anonymously as cuboids after excluding specified structural elements and retaining valid bounding-box centers and sizes.Source: D. Collision Checking: Geometry… · p. 11
  • Anonymous exact geometry yields object-category macro-F1 0.704 despite exporting no semantic labels or object identities.The most-frequent-class baseline is 0.00160.Source: D. Collision Checking: Geometry… · p. 11
  • Coarsening lowers object-category macro-F1 from 0.704 with exact geometry to 0.556, while collision-checking F1 falls from 1.000 to 0.950.Thus, the measured reduction in this semantic leakage channel comes with a collision-utility cost.Source: D. Collision Checking: Geometry… · p. 11
  • Room-inference macro-F1 remains 1.000 for both geometries, while private-object inference is 0.844 for exact and 0.928 for coarsened geometry.The authors therefore do not treat coarsening as a general privacy improvement.Source: D. Collision Checking: Geometry… · p. 11

E. Object Goal: Target-Semantic Reduction and Residual Leakage

Task-functional target references sharply reduce target-category inference while preserving evaluated object-goal success, but broader scene-level attacks remain informative. Controlled diagnostics illustrate that retained geometry and target references can still carry information.

E. Object Goal: Target-Semantic Reduction and Residual Leakage

  • Target-category macro-F1 falls from 1.000 for the explicit label to 0.077 for the target region, while all five target representations achieve object-goal success 0.995.Other functional-reference scores range from 0.053 for the graph node to 0.094 for coordinates and anonymous ID.Source: E. Object Goal: Target-Semantic… · pp. 11–12
  • Across all five target representations, room-inference macro-F1 remains 0.644 and private-object inference remains 0.913 under the representation-aware public-layout attack.The respective baselines are 0.100 and 0.444.Source: E. Object Goal: Target-Semantic… · p. 12
  • The five references are evaluated only for the stated object-goal criterion and are not established as interchangeable for manipulation or grasping, which may require additional pose, orientation, or grasp-relevant geometry.Source: E. Object Goal: Target-Semantic… · p. 12

F. Controlled Synthetic Diagnostics

  • In the controlled geometry diagnostic, anonymous cuboids and evaluated geometric proxies achieve object-category macro-F1 1.000 because dimensions are fixed by type in the generator.This isolates how geometry can encode removed semantics; it is not real-world category-recovery performance.Source: F. Controlled Synthetic Diagnostics · p. 12
  • In the controlled target-reference diagnostic, task-functional references reduce target-category macro-F1 to 0.150–0.240 from 1.000 for an explicit label, while retaining information needed to locate or reference the target.Source: F. Controlled Synthetic Diagnostics · p. 12
  • Strong-perturbation linkability is 0.428 for task-minimal, 0.328 for coarse-grid, and 0.278 for graph; the graph attacker uses only a reduced aggregate summary.The authors caution against interpreting this diagnostic as evidence that topological representations are generally more private.Source: F. Controlled Synthetic Diagnostics · p. 12

VI. DISCUSSION

The discussion and conclusion argue that privacy depends on the task and the complete exported representation, not simply on removing fields or increasing abstraction. The results support multi-risk evaluation while bounding the portability of leakage rankings and the current evaluation scope.

VI. DISCUSSION

  • Privacy effects are channel-specific: a transformation can reduce one leakage channel while leaving another unchanged or increasing it.The discussion therefore motivates task-specific, multi-risk representation design instead of a single sanitization rule.Source: §VI · p. 12
  • TFPD currently profiles candidate representations rather than selecting among them automatically at runtime.Future work includes physical robots, additional task requirements, and adaptive selection accounting for utility, multiple privacy risks, and previously disclosed information.Source: §VI · p. 12
  • ProcTHOR preserves the navigation task-equivalence/privacy-inequivalence finding, but shares AI2-THOR’s execution engine and is not independent-simulator or physical-robot validation.The changed normalized–topological ordering across scene distributions also indicates that precise leakage rankings can depend on environmental structure.Source: §VI · pp. 12–13
  • Linkability uses controlled perturbations of the same export, not independent cross-session observations, and should not be read as real-world home re-identification.The evaluation also excludes repeated or jointly observed exports across tasks or over time and does not attribute leakage changes to individual features without dedicated ablations.Source: §VI · p. 13

VII. CONCLUSION

  • 1.000 success and 0.898 mean path ratio accompany navigation linkability 0.532–0.970 across task-equivalent exports, with topological attack estimates exceeding coordinate normalization.On ProcTHOR, task-equivalence/privacy-inequivalence persists but normalized–topological ordering changes; target-region export reduces target-category macro-F1 from 1.000 to 0.077 at 0.995 success, while coarsening lowers object-category macro-F1 from 0.704 to 0.556 and collision F1 from 1.000 to 0.950.Source: §VII · p. 13
  • Task-oriented privacy assessment should jointly consider utility, direct exposure, and residual inference risks in the complete exported representation.Removing explicit fields or applying stronger abstraction alone does not establish privacy.Source: §VI · p. 12, §VII · p. 13
Loading 2609.03055v1…