PaperlayerView exact revision on arXiv

Source-linked AI summary

Conditional Imaging for Privacy-Preserving Visual Sensing

Hengyu Lian, Debabrata Mandal, Anzhou Wen, Yujie Wang, Xi Peng, Praneeth Chakravarthula

arXiv:2610.28835v1cs.CVphysics.optics

TL;DR

Existing camera safeguards either act after a readable image has formed or suppress information that cannot later be recovered, while static optical systems couple privacy and recoverability. Conditional imaging varies the optical state by frame and conditions reconstruction on access to that state. Experiments report an authorized-versus-unauthorized identity-verification gap, including under evaluation favorable to unauthorized recovery.

  • Problem

    Visual recordings can capture sensitive information, while existing safeguards either act after a readable image forms or prevent later recovery of information suppressed during capture.

    Source: §1 · p. 2

  • Method

    Conditional imaging varies optical modulation across frames and conditions scene reconstruction on access to each frame’s corresponding physical system state.

    Source: §1 · p. 3

  • Results

    0.67 unauthorized face-verification AUC among detectable cases, versus 0.91 for authorized reconstruction, shows an access gap even under observer-favorable evaluation.

    Source: §1 · p. 4

  • Takeaways & Limitations

    Conditional imaging makes privacy compatible with retaining information for later reconstruction by controlling access to frame-specific system states.

    Source: §3 · p. 14

    The prototype demonstrates the principle on human faces; applying the framework elsewhere requires defining the sensitive feature scale and reconstruction requirements for the task.

    Source: §3 · p. 14

Abstract

from arXiv · show

Conventional visual sensing faces a fundamental dilemma: information that should remain private today may need to be recovered tomorrow. This challenge is becoming increasingly important as cameras become pervasive in our daily surroundings. Existing safeguards either protect visual content only after an image is formed or suppress information during capture in ways that prevent its later recovery. Here, we show through an information-theoretic analysis that privacy and recoverability cannot be achieved simultaneously in static optical systems. Information preserved in a measurement for later reconstruction also remains available to an observer of that measurement, limiting the privacy such systems can provide. To overcome this limitation, we introduce conditional imaging, an imaging approach that changes the optical state for every captured frame. The measurement alone conceals the scene, while knowledge of the corresponding physical system state enables its reconstruction. We analyze how the optical parameters determine the information that can be recovered and use this analysis to guide the design of a camera prototype. We evaluate the system on a captured dataset of human faces using increasingly powerful attempts to recover the scene, from direct visualization and computational inversion to learned reconstruction and identity recognition. Our experiments show that conditional imaging preserves identity in authorized reconstructions while withholding visual information from observers without access to the system state. By making the physical state of the camera part of the information needed for reconstruction, conditional imaging enables visual information to be protected at the point of capture while remaining recoverable for future authorized use.

1 Introduction

Cameras must balance privacy with the need to recover recordings later, but static optical systems couple recoverability to information available to observers. Conditional imaging varies the optical state by frame so access to that state governs reconstruction.

  • A camera may need to preserve evidence of unexpected events while protecting people recorded in settings where they expect privacy.The paper uses short-term rentals as an example and notes similar conflicts in autonomous vehicles and robots.Source: §1 · p. 2
  • Static optical systems cannot independently control privacy and recoverability: information retained for reconstruction remains available to an observer who knows the imaging system.Removing that information for privacy also removes it from authorized recovery.Source: §1 · p. 3
  • Conditional imaging changes optical modulation across frames, leaving measurements obfuscated by default and enabling reconstruction when the corresponding physical state is available.Selected frames can be made recoverable by releasing their states while other states remain unavailable.Source: §1 · p. 3
  • The analysis links recoverable information to point-spread-function extent and optical-modulation parameters, providing a design criterion implemented in an SLM prototype evaluated on human faces.Source: §1 · p. 4
  • 0.67 unauthorized face-verification AUC among detectable faces, versus 0.91 for authorized reconstruction, shows an access gap even under observer-favorable evaluation.The unauthorized result is limited to cases where data-driven recovery first succeeded well enough for face detection.Source: §1 · p. 4

2 Results

Conditional imaging makes reconstruction depend on a frame-specific optical state, and the section tests unauthorized recovery against state-aware reconstruction. Across blind, PSF-assisted, and learned attacks, access to the correct state supports stronger reconstruction and face analysis.

2.1 Conditional imaging: principle and implementation

  • Static optics couple privacy to recoverability: preserved information remains available to observers who know or estimate the operator, while suppressed information is lost to authorized users.Conditional imaging changes the operator each frame, making reconstruction depend on access to the corresponding state.Source: Figure 2 · p. 5, §2.1 · p. 6
  • The design criterion bounds PSF width to obscure application-relevant features without suppressing information needed for stable authorized reconstruction.The prototype applies frame-specific phase patterns with a programmable spatial light modulator; its state is withheld by default and released for authorized reconstruction.Source: §2.1 · pp. 6–8

2.2 Conditional imaging: resistance to unauthorized reconstruction

  • Without the modulation state, observers with camera-architecture knowledge face blind, PSF-assisted, and data-driven attacks; the evaluation uses LFW face images and reconstruction and identity metrics.The data-driven attack is trained on paired obfuscated and clean images.Source: §2.2 · p. 8, Resistance to data-driven reconstruction · p. 11

Resistance to blind reconstruction

Resistance to PSF-assisted reconstruction

  • Correct measurement–PSF matches yield 25.2–29.0 dB PSNR, but mismatched pairs degrade substantially; scene-based PSF estimates also leave reconstruction poor across tested intensities.Low-intensity point sources produce contaminated PSF estimates, while brighter sources increasingly dominate the measurement.Source: Resistance to PSF-assisted… · pp. 9–10, Figure 5 · p. 10

Resistance to data-driven reconstruction

  • With the correct modulation state, reconstruction reaches 27.80±2.06 dB PSNR and 0.84±0.04 SSIM, while data-driven attacks remain lower.UNet achieves 19.77 dB PSNR and 0.66 SSIM, SwinIR 18.90 dB and 0.63, and Restormer 21.08 dB and 0.63.Source: Resistance to data-driven reconstruction · p. 11

2.3 Conditional imaging: authorized recovery and downstream utility

  • Authorized reconstructions achieve face-detection rates comparable to clean references and an AUC of 0.91, versus 0.67 for Restormer and 0.63 for UNet.Unauthorized verification is evaluated only on reconstructions that pass face detection.Source: §2.3 · p. 13
  • Obfuscation does not eliminate all scene information: raw measurements can support coarse tasks such as person counting and crowd-density estimation without reconstructing recognizable faces.The paper reports this result in the Supplementary Material.Source: §2.3 · p. 13

3 Discussion

Conditional imaging makes recovery depend on each frame’s physical system state, separating privacy from the permanent loss of recoverable information. The prototype demonstrates this separation, while applying the approach beyond faces requires defining task-specific feature scales and reconstruction requirements.

3 Discussion

  • Static optical systems inherently couple information suppression and recoverability because the same fixed operator governs what reaches the measurement and what can later be recovered.Preserving information for recovery also makes it available to others; suppressing it reduces authorized recovery.Source: §3 · pp. 13–14
  • State-blind unauthorized reconstruction remains substantially degraded, while the correct physical state enables high-fidelity recovery and downstream analysis.Privacy therefore does not require permanently discarding information needed for later reconstruction.Source: §3 · p. 14
  • The physical design criterion bounds PSF extent: privacy requires broadening it beyond a perceptual threshold, while recoverability requires keeping it compact enough for downstream tasks.The prototype is designed within this regime and demonstrates obfuscated state-blind measurements alongside high-fidelity state-aware recovery.Source: §3 · p. 14
  • The current implementation demonstrates the principle on a programmable optical platform, and applying it elsewhere requires specifying sensitive feature scales and reconstruction requirements for each task.The framework is not restricted to faces; integrated photonic platforms are a potential path toward more compact, high-speed implementations.Source: §3 · p. 14
  • Conditional imaging embeds privacy in physical image formation by making recovery depend on a frame-specific state, rather than protecting readable images afterward or irreversibly removing information.Potential application areas include distributed sensing, wearable cameras, augmented reality, robotics, and autonomous systems.Source: §3 · p. 14

4 Methods

The methods implement frame-varying optical modulation, associate each measurement with encrypted modulation coefficients, and use those coefficients for state-aware reconstruction. Experiments use captured LFW face images and evaluate image quality and identity-related outcomes against unauthorized reconstruction methods.

Conditional imaging prototype

  • A programmable Holoeye Leto spatial light modulator applies a different modulation state to each frame; reference images are captured with modulation disabled.Scenes are displayed by a projector and captured through the optical system.Source: Conditional imaging prototype · pp. 14–15

Optical modulation and state management

  • The SLM phase pattern uses 36 Zernike polynomials with independently sampled bounded coefficients, whose ranges limit extreme phase gradients and vary the imaging operator by frame.Modulation strength follows the privacy–recoverability design criterion; theoretical derivation and the coefficient-to-PSF relationship are in the Supplementary Material.Source: Optical modulation and state management · p. 15
  • Each frame is paired with its modulation coefficients, which are encrypted using AES-GCM and transmitted separately for authorized state-aware reconstruction.Decrypted coefficients specify the phase pattern and corresponding PSF.Source: Optical modulation and state management · p. 15

Data acquisition

  • The LFW acquisition produced 10,304 paired training samples and an independent 2,578-image test set; controlled-geometry experiments also use NYU Depth V2 images.Captured measurements are linear intensities recorded under automatic exposure control and standard white balance.Source: Data acquisition · p. 15

Authorized reconstruction

  • For authorized reconstruction, the frame-specific state yields a PSF and forward model, converting recovery into state-aware deconvolution handled with deep unfolding to address physical variation and model mismatch.Reconstruction quality is evaluated with PSNR and SSIM; unauthorized methods use the same 2,578-image test set.Source: Authorized reconstruction · p. 15, Evaluation · p. 15

Evaluation

  • Identity evaluation uses a pretrained InsightFace model without task-specific fine-tuning, ArcFace embedding cosine similarity, and ROC/AUC verification; unauthorized verification is limited to images passing face detection.This conditional evaluation is favorable to the identity information remaining after successful unauthorized recovery.Source: Evaluation · pp. 15–16
  • The study uses publicly available datasets under standard protocols and collects no new human-subject data.Data are available from the corresponding author upon reasonable request; the authors declare no conflict of interest.Source: Evaluation · p. 16, Data availability · p. 16, Conflict of interest · p. 16
Loading 2610.28835v1…