LooseLeaf: Enabling Serverless GPUs for Computational Notebooks

1 INTRODUCTION

• Reserved GPUs are idle over 81% of the time because notebooks mix GPU cells with CPU-only work and users pause between executions.Session-pinned services hold the GPU for the notebook's entire lifetime.Source: §1 · p. 1
• Fresh workers need seconds to minutes to reconstruct prior notebook state, putting interactive starts at odds with releasing GPUs between cells.The paper uses one second as the interactive-start threshold because response times around that mark support users' cognitive flow.Source: §1 · p. 2
• Analysis of roughly 5,000 real AI/ML notebooks found that GPU cells typically read a small fraction of session state and import packages from a small common set.Cell code reveals needed variables and packages during the seconds-to-minutes editing window, allowing preparation before Run.Source: §1 · p. 2
• More than 95% of GPU cells begin within one second after Run at 72 sessions, while LooseLeaf saves updated state within seconds so GPUs can be reclaimed after each cell.It prepares a worker with the cell's state and import closures while the user edits the cell.Source: §1 · p. 2
• LooseLeaf provisions 43–70% fewer GPU-hours than session-pinned reservation and 32–55% fewer than NotebookOS across loads.Figure 1 compares provisioned GPU-hours with the fraction of GPU-cell runs missing the one-second interactive-start threshold; LooseLeaf shifts the cost–interactivity frontier toward the lower left.Source: Figure 1 · p. 1, §1 · p. 2

2.1 Interactive Notebooks and Serverless GPUs

• Cells share a persistent kernel namespace, so serverless execution must restore state and packages on a fresh worker before the target cell can run.The paper calls this end-to-end worker readiness cost preparation latency; starting after Run is prohibitively costly.Source: §2.2 · p. 2, §2.1 · p. 2

2.2 State and Import Closures

• Dependency-guided state and import closures collect the minimal variables and packages a target cell requires, including transitive dependencies.The dependency graph follows the most recently executed definition; static analysis is exact except for uncommon dynamic namespace access such as exec or globals().Source: §2.2 · p. 3, Figure 2 · p. 3

2.3 Dataset Collection and Statistics

• Notebook GPU cells account for a minority of cell executions, while median GPU-cell runtime is about eight seconds, despite session-pinned GPU allocation.Among notebooks using a GPU, GPU cells are a median 20% of cells; 37% of notebooks never use a GPU.Source: §2.3 · p. 3

2.4 Dataset Characterization

• The 5,004-notebook corpus characterization identifies three preparation opportunities: small state closures, concentrated package use, and editing time before execution.The corpus was collected from Kaggle and public GitHub repositories, with cell-device labels from two models agreeing on 94.5% of labels.Source: §2.3 · p. 3, §2.4 · p. 4

2.4.1 Observation 1: A GPU Cell’s State Closure Is Small.

• Median state-closure ratios fall from 22% for cells with 5–9 live variables to 11% for 20–49 and 4% for 100 or more.The measured ratio is the required state divided by the live namespace; the trend indicates that required state grows more slowly than accumulated state.Source: §2.4.1 · p. 4

2.4.2 Observation 2: GPU Cells Have a Head-and-Tail Package Dis-

• torch appears in 71% of GPU cells, yet 53.5% use a package outside the top 12 among 717 distinct packages, creating a preload-versus-import tradeoff.A fixed preload either misses packages needed by many cells or loads hundreds that most cells never use.Source: §2.4.2 · p. 4

2.4.3 Observation 3: Editing Provides a Short Preparation Window.

• Editing provides seconds for preparation: median pure-typing intervals were 9.5 seconds in traces, while AI-assisted paste-to-run intervals had a 10.7-second median.The synthetic typing-time estimate had a 9.4-second median and closely matched measured pure-typing intervals.Source: §2.4.3 · p. 5

3.1 LooseLeaf Overview

• A cell’s execution recipe records its state and import closures plus a CPU/GPU label; GPU workers restore and save state through the checkpoint store, allowing GPU reclamation between cells.The frontend derives recipes as the user edits; CPU cells use the session’s persistent worker, while GPU cells run on workers forked from an import cache.Source: §3.1 · pp. 5–6, Figure 5 · p. 5

3.2 Dependency-Guided Checkpoint / Restore

• LooseLeaf preserves cross-variable object sharing with RootRef/ObjRef links, externalizes large payloads, and restores only closure-reachable variables and payloads.After execution, it reserializes reachable structures and refreshes only dirty GPU-tensor regions, tracked at 2 MB granularity.Source: §3.2 · pp. 6–7

3.3 GPU-Aware Import Cache

• CUDA-clean zygotes form a head-and-tail import hierarchy; workers fork from best matches, then import missing packages, while CUDA-unsafe imports stay out.Each host bounds cache nodes, evicting least-recently used leaves while keeping the base resident.Source: §3.3 · pp. 7–8, Figure 7 · p. 8

3.4 Bounded Edit-Time Speculation

• Editing-time prefetch overlaps worker setup with edits; the submitted run refreshes stale closures, while GreedyDual eviction bounds idle residency by previously computed state.Prefetch loads packages and variables without executing the cell; under pressure, completed workers are evicted before speculatively prepared workers.Source: §3.4 · pp. 8–9, Figure 8 · p. 9

4.1 Experimental Setup

• The evaluation uses 24 Kaggle notebooks on AWS p4d.24xlarge hosts with eight A100 GPUs, comparing LooseLeaf with session-pinned reservation, NotebookOS, Modal, and naive serverless.The notebooks cover training, inference, fine-tuning, and inference-only pipelines, with models ranging from CNNs below 300 M parameters to 14 B-parameter LLMs.Source: §4.1 · pp. 9–10

4.2 GPU Provisioning at Scale

• LooseLeaf provisions 43%, 63%, and 70% fewer GPU-hours than session-pinned reservation; 55% and 32% fewer than NotebookOS at 24 and 72 sessions; and 29%, 33%, and 47% fewer than Modal.The first sequence covers 24, 48, and 72 sessions; the Modal sequence does too.Source: Figure 9 · p. 10, §4.2 · pp. 10–11

4.3 End-to-End Interactive Latency

• 98.9% of GPU-cell runs at 24 sessions and 95.9% at 72 sessions start within the one-second interactive-start threshold.At 72 sessions, 86% start immediately and p95 latency is 0.88 s; the residual tail can include node power-on time during demand spikes.Source: Figure 10 · p. 11, §4.3 · p. 11
• NotebookOS reaches 63 s at p99 and 194 s maximum latency, while no naive-serverless run starts within the interactive-start threshold.Modal starts immediately when warm for 78% of GPU cells, but after idle shutdown its upstream-cell recomputation takes 46 s at p90 and exceeds an hour in the tail.Source: §4.3 · p. 11

4.4 Why Preparation Fits the Edit Window

• With all three mechanisms enabled, cold preparation takes 0.71 s at the median and 3.8 s at p95, after a 44× reduction from naive preparation.The full configuration's maximum is 6.9 s versus a 10.7 s median edit window, but the measurements do not pair each cell's preparation time with its realized edit window.Source: Figure 11 · p. 12, §4.4 · p. 12

4.5 Comparison with Transparent GPU Checkpoint/Restore

• LooseLeaf's median preparation is 0.71 s, compared with 2.50 s for GCR-based restore and 4.38 s for CRIU-based restore.The alternatives restore the entire session and pay container startup; LooseLeaf restores only the target cell's state and import closures.Source: Figure 12 · p. 12, §4.5 · pp. 12–13

4.6 Why the Save Fits the Interval

• LooseLeaf's incremental save has a 29 ms median and never exceeds 1.8 s, below the workload's 60 s minimum inter-cell interval.No LooseLeaf save delays the next run in this workload; save completion also makes warm workers evictable for demand-following provisioning.Source: Figure 13 · p. 12, §4.6 · p. 13

• Notebook state systems manage CPU state in a long-lived kernel, with checkpoints read back for failure recovery, checkout, or migration; LooseLeaf moves GPU-resident state between workers on every cell's execution path.Source: §5 · p. 13
• Transparent GPU checkpoint systems capture the whole address space, whereas LooseLeaf restores only the target cell's state closure and preserves sharing across restored variables.Source: §5 · p. 13
• General serverless systems cannot know the next request's required state until it arrives; notebook cell code is available during editing, allowing LooseLeaf to prepare that cell's closure beforehand.Source: §5 · p. 13

6 CONCLUSION

• LooseLeaf derives each cell’s state and import closures from the small part of session state that the cell needs.Source: §6 · p. 13
• It prepares a GPU worker with those closures while the user edits the cell.Source: §6 · p. 13
• After execution, the worker saves updated state within seconds, allowing GPUs to be allocated to cells on demand rather than pinned to sessions.Source: §6 · p. 13