Source-linked AI summary
Checkpoint, Restore, and Live Migration for Science Platforms
Mario Juric, Steven Stetzler, Colin T. Slater
TL;DR
Remote science platforms can burden providers with computing costs, while terminating inactive sessions can worsen user experience. This paper implements JupyterHub checkpointing, restoration, and migration, and reports cloud costs as low as $200 per typical user per year without degrading experience relative to 24/7 operation.
Problem
Remote science platforms make providers bear storage and computing costs, while terminating inactive instances to reduce costs can mean poor user experience.
Source: §1 · p. 1Method
The implementation checkpoints Jupyter server state and restores it on demand, including on another host, using a custom spawner and container checkpoint support.
Source: §1 · p. 1, §3 · pp. 2–3Results
The authors report a fully functional JupyterHub checkpoint, restore, and migration implementation for analyses as complex as the LSST software stack, with cloud costs as low as $200/yr per typical user and no degradation relative to 24/7 operation.
Source: §4 · p. 4Takeaways & Limitations
The authors identify checkpointing inactive sessions and dynamically migrating users’ instances as options for improving resource use across shared, on-premises, and cloud platforms.
Source: §4 · p. 4The prototype was deployed on DigitalOcean and does not work on other cloud providers out of the box; extensions are planned.
Source: §3 · p. 3
Abstract
from arXiv · showhide
We demonstrate a fully functional implementation of (per-user) checkpoint, restore, and live migration capabilities for JupyterHub platforms. Checkpointing -- the ability to freeze and suspend to disk the running state (contents of memory, registers, open files, etc.) of a set of processes -- enables the system to snapshot a user's Jupyter session to permanent storage. The restore functionality brings a checkpointed session back to a running state, to continue where it left off at a later time and potentially on a different machine. Finally, live migration enables moving running Jupyter notebook servers between different machines, transparent to the analysis code and w/o disconnecting the user. Our implementation of these capabilities works at the system level, with few limitations, and typical checkpoint/restore times of O(10s) with a pathway to O(1s) live migrations. It opens a myriad of interesting use cases, especially for cloud-based deployments: from checkpointing idle sessions w/o interruption of the user's work (achieving cost reductions of 4x or more), execution on spot instances w. transparent migration on eviction (with additional cost reductions up to 3x), to automated migration of workloads to ideally suited instances (e.g. moving an analysis to a machine with more or less RAM or cores based on observed resource utilization). The capabilities we demonstrate can make science platforms fully elastic while retaining excellent user experience.
1. Introduction
Remote science platforms give users access to server-side analysis, but they place storage and computing costs on providers; terminating inactive instances can harm user experience. The paper proposes checkpointing and restoring notebook servers to reduce costs while maintaining that experience.
1. Introduction
- Unmanaged 24/7 service for 100 users can reach $300,000+, while terminating inactive instances risks poor user experience.The provider bears both dataset-storage and computing costs in this model.Source: §1 · p. 1
- Checkpointing freezes a running Jupyter server to disk for on-demand restoration, including on a different host.The authors also describe uninterrupted migration based on resource needs.Source: §1 · p. 1
2. Elsa: A Checkpoint-able JupyterHub Deployment
Elsa is a proof-of-concept JupyterHub deployment that adds checkpoint and restore controls to the notebook workflow. Users can pause to save a session and later resume it on the same or a differently resourced VM.
2. Elsa: A Checkpoint-able JupyterHub Deployment
- Elsa is a fully functional proof-of-concept that adds checkpoint and restore to a cloud JupyterHub deployment.The interface exposes the functionality through notebook controls.Source: §2 · p. 2
- The pause button checkpoints the notebook server’s complete state and releases its computational resources.The saved session can later be restored on the same VM or one with different resources, allowing work to continue as before.Source: §2 · p. 2
- The interface’s fast-forward button initiates migration to a different VM instance type.The two-button interface pairs pause with fast-forward.Source: Figure 1 · p. 2, §2 · p. 2
3. Implementation
Elsa uses a custom JupyterHub spawner and per-user VMs, with containers and shared storage supporting checkpoint restoration across machines. It connects checkpointing to JupyterHub’s start and stop operations and exposes the controls through a small UI.
3. Implementation
- The JupyterHub front end runs on a dedicated node, and pause and fast-forward controls are added through the nbresuse extension.The front end uses an essentially unmodified upstream JupyterHub container image.Source: §3 · pp. 2–3
- A custom Spawner allocates one cloud-provider VM per user because Kubernetes pod checkpoint/restore was unavailable to the authors.The authors note that bare VMs reduce portability while providing additional benefits; their footnote names VM-level isolation, swap, and faster provisioning.Source: §3 · p. 2
- Per-user Jupyter runs in a container, which abstracts VM details, supports the standard notebookserver container, and permits secure VM reuse between users.The passage also notes easier deployment, contrasting a simple pull with an OS-level install.Source: §3 · pp. 2–3
- Podman’s CRIU support is connected to JupyterHub by hijacking the Spawner’s start and stop APIs: start restores a stored checkpoint, while stop checkpoints the session.The authors call this a convenient hack and say a dedicated JupyterHub C/R API should ultimately be added.Source: §3 · p. 3
- A shared NFS-mounted /home preserves users’ data at the inode level across machines, and the same NFS centrally stores checkpoints.The authors report no typical-usage issues, while noting a potential bottleneck and scalable alternatives for larger deployments.Source: §3 · p. 3
- The implementation was prototyped on DigitalOcean and does not work on other providers out of the box; provider extensions of roughly O(50) lines are planned.The spawner uses DigitalOcean APIs to manage VM instances.Source: §3 · p. 3
4. Discussion and Future Work
The authors report a fully functional JupyterHub checkpoint, restore, and migration implementation, including analyses as complex as the LSST software stack. For cloud deployments, they estimate typical-user costs as low as $200 per year without degradation in user experience relative to continuous operation.
4. Discussion and Future Work
- The authors report the first fully functional JupyterHub checkpoint, restore, and migration implementation, supporting analyses as complex as the LSST software stack.Planned work includes other cloud providers, migration of open TCP connections, a dedicated C/R API, and improved performance.Source: §4 · p. 4
- The authors identify shared servers, on-premises platforms, and cloud platforms as application areas, including checkpointing inactive sessions and dynamically migrating users’ instances to optimize resource use.For on-premises platforms, checkpointing rather than terminating inactive instances is described as improving user experience.Source: §4 · p. 4
- As little as $200/yr per typical user is reported for cloud operation with no user-experience degradation relative to running 24/7.Table 1 describes annual on-demand use, checkpointing during inactive time at a typical observed 15% duty cycle, and spot instances with transparent migration.Source: §4 · p. 4, Table 1 · p. 4