PaperlayerView exact revision on arXiv

Source-linked AI summary

Viser: Imperative, Web-based 3D Visualization in Python

Brent Yi, Chung Min Kim, Justin Kerr, Gina Wu, Rebecca Feng, Anthony Zhang, Jonas Kulhanek, Hongsuk Choi, Yi Ma, Matthew Tancik, Angjoo Kanazawa

arXiv:2507.22885v1cs.CVcs.RO

TL;DR

Visualization tools can be lightweight for simple tasks or richer but domain-specific, leaving a tradeoff for computer-vision and robotics workflows. Viser combines a web viewer with composable 3D scene and 2D GUI primitives through an imperative Python API. The report presents this design as a way to support simple visualization and specialized interfaces, while noting tradeoffs including statefulness and WebSocket transfer overhead.

  • Problem

    Visualization tools range from lightweight libraries for simple tasks to richer domain-specific packages, creating a tradeoff for computer-vision and robotics workflows.

    Source: §1 · p. 1

  • Method

    Viser combines a web-based viewer and composable 3D scene and 2D GUI primitives with an imperative Python API.

    Source: §1 · pp. 1–2, §2 · p. 3, §3 · p. 7

  • Results

    Viser is presented as easy to use for simple visualization while extending to specialized needs through task-specific interfaces.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    The design aims to bring simple visualization and specialized interfaces together for computer vision and robotics.

    Source: §1 · pp. 1–2, §6 · p. 10

    WebSocket-based asset transfer adds overhead and requires interaction-driven updates to round-trip through the Python server, preventing export of those interactions to static webpages.

    Source: §5 · p. 9

Abstract

from arXiv · show

We present Viser, a 3D visualization library for computer vision and robotics. Viser aims to bring easy and extensible 3D visualization to Python: we provide a comprehensive set of 3D scene and 2D GUI primitives, which can be used independently with minimal setup or composed to build specialized interfaces. This technical report describes Viser's features, interface, and implementation. Key design choices include an imperative-style API and a web-based viewer, which improve compatibility with modern programming patterns and workflows.

1 Introduction

Viser addresses the tradeoff between lightweight visualization libraries and richer domain-specific tools by aiming to make simple visualization easy while supporting specialized interfaces. Its web-based viewer and composable scene and GUI primitives are presented across computer vision and robotics examples.

1 Introduction

  • Viser aims to bridge lightweight libraries suited to simple tasks and domain-specific packages with richer features for specialized settings.The library is designed for easy standalone use as well as composition into task-specific interfaces.Source: §1 · pp. 1–2
  • Computer-vision examples span 4D reconstruction, camera poses and point clouds, interactive SMPL controls, and dynamic human motion visualization.Figure 1 includes examples from Shape of Motion [21], COLMAP [23], and VideoMimic [25].Source: Figure 1 · p. 2
  • Published uses span neural rendering, generative models, scene understanding, reconstruction and tracking, and robotics.The cited released works include [11] [26] [32] [36] [38] for neural rendering and [66] [72] [73] [74] for robotics.Source: §1 · p. 2
  • Robotics examples include interactive inverse kinematics, policy rollouts, humanoid control, and batched rendering for parallel simulation.Figure 2 shows examples using PyRoki [68], VideoMimic [25], IsaacGym [7], and MuJoCo Playground [75].Source: Figure 2 · p. 3

2 Features

Viser combines a web-based viewer with 3D scene and 2D GUI primitives to support visualization across computer vision and robotics. The primitives can be used independently or composed into specialized applications.

2 Features

  • Viser’s unified system consists of a web-based viewer, scene primitives, and GUI primitives for computer-vision and robotics tasks.Examples include domain-specific tools built from these primitives.Source: §2 · p. 3

2.1 Web-based Viewer

  • A locally hosted viewer runs in modern browsers, including on headless servers and mobile clients, and supports embedding offline visualizations or sharing active sessions by URL.The browser-based implementation also uses mature libraries including React and three.js.Source: §2.1 · p. 3

2.2 Scene Primitives

  • Scene primitives cover point clouds, meshes, images, Gaussian splats, and geometric objects; batched rendering and level-of-detail optimizations support large scenes.Viser also supports GLB and glTF assets, hierarchical scene organization, real-time data updates, and interactive objects and viewpoints.Source: §2.2 · p. 5
  • Viser’s rendering pipeline supports physically based materials, environment maps, varied lighting, and shadow mapping without exporting to specialized rendering software.The three.js-based pipeline includes ambient, directional, point, area, and spot lighting.Source: §2.2 · p. 5

2.3 GUI Primitives

  • GUI primitives provide standard and specialized inputs, rich information displays, and interface organization through folders, tabs, dialogs, and in-scene containers.Supported displays include text, markdown, HTML, streamed images, and plots; containers can support per-object controls and annotation tools.Source: §2.3 · p. 5

3 API

Viser uses an imperative API that aligns with interactive Python workflows by making object creation, updates, and lifecycle control explicit. This approach prioritizes user control of program flow to support integration into complex programs.

3 API

  • Viser’s imperative API centers on explicit side effects in user-controlled program flow, aligning scene and GUI features with Python programs, notebooks, REPLs, and debuggers.Primitive calls return handles for later interaction and lifecycle control.Source: §3 · p. 7
  • Handle property assignments update visualizations immediately, while browser interactions return to Python for polling or callback-based handling.Users explicitly manage primitive removal, with synchronization handled automatically.Source: §3 · p. 7
  • Unlike declarative APIs such as Gradio [15], Viser retains user control of program flow to ease integration into interactive Python patterns.Declarative approaches hand program-flow details to a runtime, which can simplify use cases but limit integration flexibility.Source: §3 · p. 7, Figure 7 · p. 8

4 System Architecture

Viser separates its user-facing API from its web implementation through a four-layer architecture. The layers coordinate Python interfaces, transport, and browser clients while shielding users from networking and synchronization complexity.

4 System Architecture

  • Viser’s four layers separate user-facing interfaces from the web implementation and shield users from networking and state-synchronization complexity.The layers are Core API, Handles, Transport, and Client.Source: §4 · p. 8, Figure 8 · p. 9
  • The Core API creates scene and GUI objects and supports global sharing or targeted per-client updates.Primitive creation returns handles that manage lifecycle, expose property access, and register event callbacks.Source: §4 · p. 9
  • The transport layer sends Python commands over WebSockets and buffers, batches, and deduplicates updates before serialization.Python message definitions generate TypeScript declarations for static verification across the API boundary.Source: §4 · p. 9
  • Browser clients mirror server state, render updates, and relay user interactions back to Python through the transport layer.Client optimizations include WebWorkers, WebAssembly-based Gaussian-splat sorting, and cascaded shadow maps.Source: §4 · p. 9

5 Limitations

Viser’s simplicity and flexibility come with tradeoffs in data transfer, API state management, language support, and built-in workflows for multi-process and sequence data.

5 Limitations

  • WebSocket delivery adds transfer overhead, requires round trips for interaction-driven updates, blocks static export of those updates, and prevents CPU-to-GPU transfer optimizations.The Python server manages program state.Source: §5 · p. 9
  • Viser’s inherently stateful API can require more duplicated or error-prone state management than declarative or immediate-mode visualization APIs.The authors identify statefulness as an intentional design choice.Source: §5 · p. 10
  • Viser targets Python users and does not provide C++ or Rust bindings, languages common in performance-critical systems.Source: §5 · p. 10
  • Per-script visualization servers may require adaptation for systems with concurrent processes, which are common in robotics.Source: §5 · p. 10
  • Because Viser does not assume temporal structure or provide standard timestamping, visualizing sequence data can require more boilerplate.Source: §5 · p. 10
  • Viser has limited built-in support for serialized-data workflows such as logging and offline playback from formats including rosbag, MCAP, and rrd.Source: §5 · p. 10

6 Conclusion

Viser is a Python library for 3D visualization with scene and GUI primitives designed for independent use or composition. The authors hope it will accelerate research and engineering in computer vision, robotics, and related fields.

6 Conclusion

  • Viser is a Python library for 3D visualization.Source: §6 · p. 10
  • Its 3D scene and 2D GUI primitives aim to be easy to use independently and powerful when composed.Source: §6 · p. 10
  • The authors hope Viser will accelerate research and engineering in computer vision, robotics, and related fields.Source: §6 · p. 10

Related papers

Loading 2507.22885v1…