Source-linked AI summary
Offline Ambient-Controlled Latent Diffusion: Architecture, Telemetry, and On-Device Evaluation
Lech Kalinowski, Artur Morys-Magiera, Piotr Miłkowski
TL;DR
Mobile image-generation apps often rely on cloud endpoints, while on-device generative systems lack evidence that outputs remain traceable for offline analysis. This paper presents an ambient-light-controlled Android latent-diffusion app with per-artifact telemetry and finds end-to-end ambient coupling, with log-lux positively associated with output luminance (Pearson r = 0.532, n = 373).
Problem
On-device generative settings lack evidence that each output can be tied to its sensor reading, runtime path, and seed for offline analysis.
Method
The paper builds an on-device ambient-light-controlled latent-diffusion application with per-artifact telemetry linking outputs to sensor readings, runtime paths, and seeds.
Results
Log-lux is positively associated with output luminance (Pearson r = 0.532, n = 373, 95% CI [0.455, 0.601]), while latency spans 552–1334 ms across three quality tiers.
Takeaways & Limitations
The workflow makes ambient-control wiring observable and the resulting artifacts auditable end-to-end on the tested device.
Takeaways & Limitations
Evaluation is limited to one Samsung foldable and an observational capture, so the reported correlations are descriptive rather than causal.
Abstract
from arXiv · showhide
Most mobile image-generation applications are thin clients over cloud services, leaving outputs hard to audit. We present an Android latent-diffusion application that runs entirely on-device and is driven by the ambient-light sensor rather than a text prompt, keeping generation, telemetry, and storage local. The contribution is not a new diffusion method but the surrounding measurement workflow: each output is bound to the sensor reading, runtime path, and seed that produced it, giving a per-artifact audit trail for offline analysis. On a single Samsung foldable, one fixed capture of 373 artifacts shows the controller's log-lux input positively associated with output luminance (Pearson $r=0.532$, 95\% CI $[0.455, 0.601]$), confirming the ambient dependency survives denoising and VAE decoding, while the latent UNet/VAE pipeline runs at 552--1334\,ms mean latency across three quality tiers under the Android Neural Networks API (NNAPI).
1. Introduction
The paper presents an on-device image-generation system driven by ambient light instead of text prompts, with measurement infrastructure that ties each output to its sensor reading, runtime path, and seed. Its primary contribution is the telemetry and offline-analysis workflow rather than a new diffusion algorithm.
- Motivation: The system moves the model, sensing, and instrumentation from cloud-centered mobile applications onto the phone and drives generation with ambient-light input instead of text.This frames the work as an on-device pipeline with sensing and measurement built into the application.
- Measurement problem: The central systems question is whether every generated output remains traceable to its sensor reading, runtime path, and seed for offline analysis.The audit trail must persist long enough to support analysis after generation.
- Contribution: The contribution is a measurement workflow combining per-artifact telemetry, artifact-to-trace joining, and offline analysis, not a new diffusion algorithm.The paper evaluates this workflow through an architecture, instrumentation, and fixed physical-device capture.
2. Related Work
Prior work establishes latent diffusion, on-device image synthesis, environmental control signals, and instrumentation as relevant foundations. This paper’s distinction is applying those foundations to a mobile generative pipeline whose execution, runtime, and measurement workflow are analyzed together.
- Measurement and instrumentation: Instrumentation is treated as a first-class system component because experimental systems research requires explicit measurement definitions and traceable analysis procedures.Reproducibility remains a known weakness in the field.
- Operational scope: Machine-learning products expose an operational surface beyond the model, motivating joint analysis of model execution, runtime selection, trace capture, and per-artifact analysis.The study applies this boundary specifically to on-device latent diffusion.
- Diffusion and mobile execution: Denoising diffusion and latent diffusion provide the algorithmic basis, while latent-space denoising supports realistic mobile deployment through ONNX Runtime and Android NNAPI.Quantization offers a further efficiency path, but the analyzed capture uses 32-bit floating-point models throughout.
- Prior on-device synthesis: On-device latent diffusion is established infrastructure rather than a contribution, following prior fully local Android and mobile-neural-accelerator image-synthesis systems.The paper explicitly does not claim on-device latent diffusion itself as novel.
- Environmental control: Environmental signals were established as control inputs in context-aware and calm-computing systems, while ambient light extends that principle to image synthesis.The claimed novelty lies in the surrounding execution and measurement system.
3. System Architecture
The system couples ambient-light control with on-device latent inference and reproducible artifact capture. Its architecture also exposes runtime state and preserves local telemetry for offline auditing and analysis.
- Ambient control: Ambient readings are smoothed, hysteresis-stabilized, and mapped to a bounded normalized control signal with a 40,000-lux anchor.Readings above the anchor are clipped, and seven semantic profile anchors span the normalized range.
- Architecture overview: The architecture links sensor control, on-device latent inference, artifact capture, telemetry, and reproducible export.The app separates a picture-first ambient view from a diagnostics dashboard; automatic generation runs only on the ambient view.
- Latent denoising: gp = 0.11, gn = 0.08, gr = 0.06, gϕ = 0.04, and gf = 0.07 define the implemented denoising controller’s ambient, conditioning, and perturbation contributions.The controller combines latent carry-over, ambient controls, conditioning features, seeded perturbation, and a dynamic carry-over factor λt = 0.96−0.02 bcontrast,t.
- Instrumentation and audit: Each generation persists latency, runtime path, seed, thermal status, guidance settings, ambient deltas, and rolling-latency context locally for artifact-level analysis.An offline script joins artifacts with successful telemetry and PNG pixel statistics, emits CSV datasets, and preserves unmatched telemetry IDs in a separate audit file.
4. Experimental Methodology
The methodology evaluates one fixed capture from a single Samsung foldable, with artifacts and telemetry exported for offline analysis. It defines artifact matching, native VAE decoding, model precision, and descriptive evaluation measures for the reported runs.
- Experimental setup: A single Samsung SM-F966B foldable running Android 16 supplied the evaluation data, limiting external validity.The device used a 1080×2520 display.
- Experimental setup: 373 generated artifacts, 377 telemetry events, 5 benchmark reports, and 3 reliability reports came from fixed capture 20260303 134809 recorded March 3, 2026.Earlier mixed-variant captures were excluded because they combined latent and procedural outputs.
- Trace joining: Every accepted artifact–telemetry match fell within 58 ms using seed equality and nearest timestamp, despite a 180-second tolerance ceiling.Four successful telemetry events lacked artifact matches and were preserved separately for audit.
- Reported measures: Latency, provider share, pipeline coverage, reliability, trace-join integrity, and Pearson ambient–luminance correlation were reported descriptively for one observational run.Latency aggregates used a 100–10,000 ms inclusion window, and correlations were not causal estimates.
5. Results
The on-device capture produced tier-dependent latency, a positive ambient-light/output-luminance association, and heterogeneous log-lux/latency correlations. Artifact-level joining was complete, with unmatched successful telemetry events retained for audit.
- Latency: 552, 878, and 1334 ms were the robust mean latencies for preview, balanced, and high tiers, respectively, across 373 artifacts.The quality-tier counts were 173 preview, 87 balanced, and 113 high; the robust subset contained 366 rows after excluding seven preview rows above 10,000 ms.
- Latency: 908.40 vs. 907.60 ms were the cold and warm benchmark means, coinciding within roughly 1 ms because sessions were warmed at startup.Cold denotes the first measured pass after startup provisioning, not an uninitialized pipeline.
- Ambient-control results: Pearson r = 0.532, 95% CI [0.455, 0.601], linked log-lux to output luminance across the full latent capture (n = 373).Because ambient terms are explicitly mixed into the latent update, the association serves as a wiring check through denoising and VAE decoding, not causal evidence.
- Ambient-control results: Pearson r = −0.405 (n = 366) linked log-lux and latency overall, while within-tier correlations were −0.057, 0.023, and −0.443 for preview, balanced, and high.The pooled association reflects tier composition and should not be interpreted as a uniform lighting effect within tiers.
- Auditability: The artifact–telemetry join was complete, with 4 unmatched successful telemetry events retained in a separate audit file and listed by ID.The downstream CSV and unmatched list together describe all successful generations recorded during the capture.
6. Discussion
The discussion finds that ambient control remains observable from sensor reading through seed, runtime, latency, and decoded pixels, with log-lux positively associated with output luminance. It also emphasizes tier-dependent latency, limited external validity, and the descriptive—not causal—nature of the observational evaluation.
- End-to-end observability: Sensor readings were traceable to each artifact’s seed, runtime path, latency, and PNG pixel statistics, with positive log-lux–luminance association through denoising and decoding.The observational capture does not establish ambient light as a cause of broader aesthetic effects.
- Latency structure: 552 ms preview, 878 ms balanced, and 1334 ms high latency formed a reproducible tier ladder on the tested hardware.The pooled lux–latency relationship was stronger in the high tier than in preview or balanced.
- Evaluation limits: One Samsung foldable limits external validity, keeping the paper’s framing restricted to tested hardware rather than Android-wide behavior.Broader claims about Android-wide behavior are explicitly out of scope.
- Evaluation limits: The observational design supports descriptive correlations, while causal claims would require paired scenes, fixed seeds, and controlled coefficient variation.The proposed experiment would directly test causality rather than infer it from the capture.
- Control behavior: At 118,519 lux, the normalized control value n(l) saturates at 1 because the reading exceeds the 40,000-lux normalization anchor.The comparison uses native 320×752 VAE-decoded outputs at 60 lux and 118,519 lux.
- Telemetry matching: Every accepted sensor-to-artifact match fell within 58 ms, despite the broader 180 s join tolerance.The tolerance was substantially wider than the observed deltas in the capture.
7. Conclusion
The paper presents an Android latent-diffusion app driven by ambient light, with per-artifact instrumentation for auditable outputs. On one Samsung foldable, ambient input correlates with output luminance, while NNAPI enables on-device generation across three quality tiers.
- Conclusion: Per-artifact instrumentation makes outputs auditable by binding the Android latent-diffusion application to ambient-light sensing.The application is driven by the ambient-light sensor rather than a text prompt.
- Conclusion: 0.532 Pearson r links the controller’s log-lux input to output luminance across 373 artifacts.The 95% CI is [0.455, 0.601] on one Samsung foldable.
- Conclusion: 552–1334 ms mean latency characterizes the latent UNet/VAE pipeline under NNAPI across three quality tiers.The measurements were collected on one Samsung foldable.
Data and Code Availability · Affiliations
The implementation, analysis scripts, and experiment data are publicly available in the project repository, while the authors report independent, unfunded work without competing interests. Automated language editing was used, but human authors authored and verified the scientific content and implementation.
- Data and Code Availability: The implementation, analysis scripts, and experiment data are available in the project repository.Repository: https://github.com/callstackincubator/hyper_ui/tree/codex/ambient-3.
- Data and Code Availability: The reported capture is identified as fold capture 20260303 134809.The manuscript’s artifact-level dataset is provided under paper/csv.
- Financial Disclosure: No external grant funding supported this work.Hardware was acquired and operated independently.
- Financial Disclosure: No party other than the authors’ employer provided funding, hardware, or editorial input.The experiments used independently acquired and operated hardware.
- Conflict of Interest: The authors declare no competing interests.
- AI Assistance Disclosure: Automated tools were used for language editing.Human authors authored and verified all scientific content, implementation details, equations, and reported metrics against the source repository and captured experiment artifacts.