PaperlayerView exact revision on arXiv

Source-linked AI summary

WorldPager: Zero-Stall Tiered Memory for Long-Horizon World Models on a Single Consumer GPU

Bingnan Liu, Ruini Xue

arXiv:2610.35529v1cs.DCcs.CV

TL;DR

Interactive world models retain past-chunk KV to keep revisited places consistent, but long sessions on a consumer machine face capacity and positional limits. WorldPager tiers int4 KV and prefetches from the predicted camera pose while handling positions at inference time. On one consumer machine it generates 37.5 minutes continuously with zero stalls and constant host memory.

  • Problem

    Long-session world-model memory faces a host-capacity wall and a separate model positional horizon on a single consumer machine.

    Source: §I · pp. 1–2

  • Method

    WorldPager predicts spill and prefetch needs, stores int4 KV across GPU, DRAM, and NVMe, prefetches from the extrapolated pose without waiting on misses, and handles positions at inference time.

    Source: §I · pp. 1–2, C. Pose-driven prefetch with a… · p. 5, §VII · p. 10

  • Results

    37.5 minutes of video run continuously with zero stalls and constant host memory, at about 9.5 frames per second.

    Source: §I · p. 2

  • Takeaways & Limitations

    On one RTX 4090 with 32 GiB of host memory and one NVMe drive, the approach sustains continuous long-session generation with constant host memory.

    Source: §VII · p. 10

    Timing evidence comes from one RTX 4090 and one NVMe SSD; slower drives are emulated with a software token bucket.

    Source: §VII · p. 9

Abstract

from arXiv · show

Interactive world models keep the KV of past chunks as retrieval memory so that revisited places stay consistent. On a single consumer machine (one RTX 4090, 32 GiB of host memory, one NVMe SSD), long sessions with such memory hit two walls. The first is capacity: the KV of Matrix-Game 2.0 exhausts host memory after 45 seconds of video in bf16 and after 2.8 minutes in int4. The second is the model's positional horizon of 4.3 minutes. We show that this memory is capacity-bound rather than bandwidth-bound and that its access set is known before compute, because the camera pose is an input. An analytical model built on these observations predicts spill points to within one chunk and classifies zero versus non-zero stalls correctly on held-out configurations. Guided by it, WorldPager stores int4 KV in a GPU/DRAM/NVMe hierarchy instead of recomputing it, prefetches each chunk's memory from the extrapolated pose, never waits on a miss, and writes back in the background; to our knowledge, it is the first to place the KV memory of an interactive world model on an NVMe tier with zero-stall prefetching. With training-free position handling, it generates 37.5 minutes of video continuously with zero stalls and constant host memory, at about 9.5 frames per second. On a sealed test set, int4 storage shows no measurable quality loss; on long-gap revisits it lowers image quality by about one MUSIQ point.

I. INTRODUCTION

Memory-augmented world models must retain past chunks for consistent revisits, but on a single consumer machine capacity and positional limits constrain session length. WorldPager models these constraints and uses tiered storage, pose-driven prefetching, and inference-time position handling to generate longer sessions without stalls.

I. INTRODUCTION

  • Retrieval memory preserves consistency on revisits by retaining past chunks’ keys and values for attention to chunks with matching camera poses.The study asks how long this can run on one RTX 4090, a 32 GiB host-memory budget, and one NVMe SSD.Source: §I · p. 1
  • The workload is capacity-bound rather than bandwidth-bound, and camera-pose inputs make the next chunk’s retrieval set known before compute.These properties ground the analytical model of spill points, prefetch bandwidth, and on-demand stalls.Source: §I · p. 1
  • WorldPager stores int4 KV in a GPU/DRAM/NVMe hierarchy, prefetches from an extrapolated camera pose, falls back without waiting on misses, and writes spilled blocks back in the background.Position handling extends the table at inference time and re-places retrieved memory near the current frame beyond the native horizon.Source: §I · pp. 1–2
  • The model predicts spill points within one chunk and correctly classifies all 15 held-out zero-versus-non-zero stall cases.It also meets 22 of 24 registered criteria on MG2 and int8.Source: §I · p. 2
  • 37.5 minutes of video run continuously with zero stalls and constant host memory, though generation takes 63 minutes (about 9.5 fps).With training-free position handling, long-gap image quality falls by about one MUSIQ point; int4 shows no measurable quality loss on a sealed test set.Source: §I · p. 2

II. TWO WALLS: CHARACTERIZATION AND AN ANALYTICAL MODEL

On the tested consumer platform, retrieved KV exhausts memory well before data movement becomes the constraint. The section characterizes this capacity wall and contrasts it with the compute window available to hide transfers.

A. Setting

  • MG2 has 1.8B parameters, generates 12-frame chunks in four denoising steps, and retrieves three earlier chunks by camera pose.Each chunk’s KV occupies 486.6 MB in bf16 across 30 layers; the platform has one RTX 4090, 32 GiB of host memory, and one NVMe SSD.Source: A. Setting · p. 2

B. The first wall: capacity, not bandwidth

  • At 32 GiB, GPU, host bf16, and host int4 storage stop at chunks 21, 60, and 226, respectively; the figure shows WorldPager reaching 3000 chunks with NVMe.The measured DRAM-only 64-GiB int4 run stops at chunk 496 against a registered prediction of 492, with an extended position table to isolate capacity.Source: B. The first wall: capacity, not… · p. 2, Figure 1 · p. 2
  • A chunk takes about 1.2 seconds to compute, while pipelined transfer of 1.46 GB from NVMe takes 276 ms; the paper contrasts this with bandwidth-limited LLM decoding.The table summarizes the qualitative premises of KV offloading in LLM decoding versus world-model memory.Source: B. The first wall: capacity, not… · p. 2, Table I · p. 3

C. The access set is known before compute

WorldPager exploits pose-driven access to prefetch memory and models when storage will spill or stall. The model’s held-out predictions are largely accurate, while positional limits remain independent of memory capacity.

C. The access set is known before compute

  • Constant-velocity pose extrapolation finds the next retrieval set with 0.66–0.94 hit rate, versus 0.03–0.52 when reusing the current pose; stale-set fallback covers end-to-end misses.End-to-end missed-block fractions were 0.010 on a yaw and 0.020 on b1 ws.Source: C. The access set is known before… · p. 3

D. The second wall: the positional horizon

  • MG2’s 1024-frame rotary position table causes generation to fail at chunk 341, about 4.3 minutes, independently of memory capacity.Recomputing memory at new positions costs 3.07× a denoising step per retrieved chunk on the RTX 4090.Source: D. The second wall: the positional… · p. 3

E. An analytical model

  • The analytical model relates pool size and per-chunk storage to spill points, transfer demand to prefetch stalls, and NVMe bandwidth to background write-back needs.For MG2 int4, it predicts prefetch failure near 0.1 GB/s and write-back needs around 0.11 GB/s; it assumes nominal drive bandwidth and ignores variation in per-chunk reads.Source: E. An analytical model · p. 3
  • All eight held-out spill-point and first-read predictions fell within one chunk, all 15 zero/non-zero stall predictions were correct, and 22 of 24 MG2 criteria were met.The registered predictions were not adjusted; the int8 configuration tested byte accounting, not int8 quality or dequantization.Source: E. An analytical model · p. 3, Table II · p. 4
  • The model projects that a one-hour MG2 session with 64 GiB host memory still needs roughly 300–600 GiB of storage, while bandwidth requirements remain below 0.21 GB/s.The 64 GiB DRAM-only run stopped at chunk 496, near the registered prediction of chunk 492.Source: E. An analytical model · p. 4

III. WORLDPAGER DESIGN

WorldPager is a memory runtime beneath a chunk-wise world model’s retrieval policy: the policy selects past chunks, while WorldPager manages where their keys and values reside and when they move.

III. WORLDPAGER DESIGN

  • WorldPager separates retrieval selection from memory placement and transfer, with each design choice tied to a term in the analytical model.The overview maps the runtime components to the model terms they address.Source: §III · p. 4

A. Store quantized KV instead of recomputing it

WorldPager stores quantized KV rather than recomputing it, then uses a three-tier store and pose-driven prefetch to manage retrieved chunks. When a prefetch misses, generation reuses the previously staged retrieval set instead of waiting.

A. Store quantized KV instead of recomputing it

  • Int4 storage reduces an MG2 chunk from 486.6 MB in bf16 to 129.4 MB, and the 32 GB host pool holds 226 int4 chunks versus 60 bf16 chunks.Only remote retrievable chunks are quantized; the attention sink, local window, and current chunk remain bf16.Source: A. Store quantized KV instead of… · p. 4

B. A three-tier KV store

  • The three-tier store keeps the attention window and staging slots on GPU, int4 blocks in pinned host memory, and overflow on NVMe.The host pool is budgeted after subtracting process baseline and a 2 GiB margin; O_DIRECT avoids page-cache use against that budget.Source: B. A three-tier KV store · p. 4

C. Pose-driven prefetch with a stale-set fallback

  • While chunk c is denoised, WorldPager stages chunk c+1’s retrieval set from an extrapolated pose; a miss reuses chunk c’s staged set rather than blocking generation.This stale-set fallback may delay memory by one chunk; WorldPager extends the store to an unbounded number of chunks on NVMe.Source: C. Pose-driven prefetch with a… · p. 5

D. Write path and bookkeeping

WorldPager writes cold dirty blocks to NVMe asynchronously and uses inference-time position handling beyond the native horizon. The evaluation tests per-chunk cost, read and write stalls, long-session behavior, quality, and a second model.

D. Write path and bookkeeping

  • When reclaimable pool space falls below three chunks, a background thread writes cold dirty blocks to NVMe while blocks being written remain available in memory.The queue approximates coldest-first order; allocation evicts the least recently used clean block.Source: D. Write path and bookkeeping · p. 5

E. Beyond the native horizon: inference-time position handling

  • Position-table extension leaves outputs below chunk 341 bit-identical, while beyond that horizon WorldPager re-rotates retrieved keys to positions near the current window.The re-rotation adds about 27 ms per chunk in paired round-trip runs; its effect on quality depends on the scene, and it is optional beyond chunk 341.Source: E. Beyond the native horizon:… · p. 5

IV. EVALUATION

  • The evaluation asks whether WorldPager controls per-chunk cost, hides NVMe reads, manages write costs, sustains sessions past both walls, preserves quality, and transfers to a second model.It also examines long-gap memory recall and whether int4 storage causes quality loss.Source: §IV · p. 5

A. Setup

The experiments test retrieval-memory systems on a single consumer GPU with bounded host memory and NVMe, using Matrix-Game 2.0 workloads and reimplemented baselines. They compare memory limits and chunk costs, then evaluate NVMe prefetching against on-demand swap-in under bandwidth limits.

A. Setup

  • The setup uses one RTX 4090, a 32 GiB host-memory budget, and NVMe; MG2 generates 12-frame chunks, with retrieval selecting k=3 past chunks by camera pose.Timing uses two 36-chunk synthetic trajectories, five steady-state rollouts after a discarded warm-up, and reimplemented baselines including sliding-window, WorldKV-style, ReWorld-style, synchronous, and on-demand swap-in designs.Source: A. Setup · p. 6

B. Per-chunk cost below the first wall (Q1)

  • At 36 chunks, WorldPager takes 1210.5 ms per chunk at 9.91 fps with zero stalls; GPU, bf16-host, and int4-host designs exhaust memory at chunks 21, 60, and 226.All methods still fit at this scale; int4 without NVMe reaches its memory limit at chunk 226.Source: B. Per-chunk cost below the first wall… · p. 6, Table IV · p. 6

C. The NVMe tier: prefetch versus on-demand swap-in (Q2)

  • With unlimited NVMe bandwidth, prefetching has zero stalls on a yaw, while on-demand swap-in stalls 60.2 ms (P50) per chunk and takes 4.3% longer.A synchronous copy without staging reuse stalls 111.6 ms, so the paper uses on-demand swap-in as the fair reference.Source: C. The NVMe tier: prefetch versus… · p. 6
  • 0 ms median stall persists to 0.25 GB/s with prefetching, while on-demand swap-in stalls 116 ms at 1.0 GB/s and 454 ms at 0.25 GB/s.At 0.1 GB/s, prefetching begins to stall: P50 is 75.9 ms; on-demand swap-in stalls 1130 ms.Source: C. The NVMe tier: prefetch versus… · p. 6, Figure 3 · p. 6
  • At emulated 0.5 GB/s reads and writes, two 2000-chunk round trips have no chunk stalls and median chunk times within 0.1% of the unlimited run.The software token bucket emulates bandwidth; it is not a measurement on a SATA drive.Source: C. The NVMe tier: prefetch versus… · pp. 6–7

D. Cost of spilling to NVMe (Q3)

Background write-back reduces the per-chunk cost of spilling, and the full system sustains a 37.5-minute session within the host-memory budget. Long-gap retrievals arrive without stalls, while int4 storage preserves measured revisit consistency but lowers long-gap image quality by about one MUSIQ point.

D. Cost of spilling to NVMe (Q3)

  • 102.0 ms of per-chunk eviction overhead with synchronous write-back falls to 1.8 ms with background write-back, with zero stalls and bit-identical video.On the return leg, the increments are 135.8 ms synchronous versus 4.9 ms background; write-back runs at about 1.5 GB/s while busy.Source: D. Cost of spilling to NVMe (Q3) · p. 7

E. A 37.5-minute session and long-gap recall (Q4)

  • 37.5 minutes of video run continuously in 63.2 minutes, at about 9.5 fps, with no chunk stalled over 0.5 ms and host memory peaking at 30.7 GiB.The session writes 334 GiB to NVMe; host memory stays within the 32 GiB budget and does not grow, while chunk time remains flat after position remapping.Source: E. A 37.5-minute session and long-gap… · p. 7, Figure 4 · p. 7
  • 1297 NVMe-reading chunks in one MG2 scene and 648 in each of two others had no stall above 0.5 ms; LingBot-World retrievals were all prefetch hits.The longest waits were 0.21 ms on MG2 and 0.131 ms on LingBot-World.Source: E. A 37.5-minute session and long-gap… · p. 7
  • −1.28 MUSIQ points (int4 minus bf16) on long-gap revisits accompanies no measured consistency harm; both differences remain within the registered |∆| ≤2 tolerance.The long-tier LPIPS difference is −0.026 [−0.043, −0.009], while long-tier MUSIQ is −1.28 [−2.25, −0.36]; after drift exclusions, long-tier MUSIQ is −0.97 [−1.76, −0.17].Source: E. A 37.5-minute session and long-gap… · p. 7, Table VI · p. 8

F. Quality and a second model (Q5)

The sealed test finds no measurable int4 quality loss, while LingBot-World provides system metrics only because retrieval did not improve its revisit consistency. The analysis finds that quantization, revisit-triggered retrieval, and position re-mapping have distinct quality and throughput trade-offs, while long-range texture drift is independent of KV precision.

F. Quality and a second model (Q5)

  • Int4 retention is 105% [104%, 106%] and MUSIQ changes by −0.06 [−0.19, +0.07] on the sealed test set, indicating no measurable quality loss.The test used four new scenes and two trajectories; retention slightly above 100% is attributed to the copy effect, not better memory.Source: F. Quality and a second model (Q5) · p. 8
  • LingBot-World retrieval showed no consistency improvement, but prefetching made all 750 retrievals hits with no chunk waiting more than 0.131 ms.On-demand swap-in instead waited a median 37.7 ms and up to 375 ms; retrieval made chunks 35.5% slower than the sliding window.Source: F. Quality and a second model (Q5) · p. 8

V. ANALYSIS AND FINDINGS

  • At about a quarter of the bytes, all-head int4 retained 105% [102%, 108%] of memory benefit, versus 56% [51%, 61%] for 25% of heads in bf16.The 48.1-point [41.0, 55.3] difference held in all 12 development units; online head ranking did not help.Source: §V · p. 8
  • Revisit-triggered restoration raised random-25%-head retention from 56% to 81%, but remote-memory reads only increased throughput by 1.6–3.9%, so the method was not adopted.Its trigger fires more often than needed on long pure rotations.Source: §V · p. 8
  • Int3 retained 128% with MUSIQ −0.90 [−1.50, −0.32], while symmetric int2 retained 120% with MUSIQ −4.05 [−6.44, −2.01], reflecting a copy effect alongside texture loss.Corrected asymmetric int2 retained 102% with MUSIQ +0.30 at 17.2% of bytes, but lengthened attention by 63–124% in synthetic-tensor microbenchmarks.Source: §V · p. 8
  • The long-to-short ratio of memory’s removed revisit inconsistency varied by scene: 0.96 [0.76, 1.15] without re-mapping and 0.86 [0.76, 0.97] with it in scene 0000, versus 0.73 and 0.78 in scene 0003.In scene 0011, where the sliding window barely changed on revisits, absolute LPIPS was reported instead; revisit gap and session position were confounded.Source: §V · pp. 8–9
  • Re-mapping lowered revisit LPIPS by 0.15–0.28 in scene 0000 at all gaps and by 0.04–0.07 in scene 0011 at short and mid gaps, but had no measurable effect in scene 0003.Placing memory 6 or 96 frames before the current chunk was worse than the 30-frame setting; the authors treat re-mapping as optional.Source: §V · p. 9
  • Long-session diagonal texture drift appeared after about 100 chunks in both int4 and bf16 controls, with texture energy ratio 0.97 and MUSIQ difference −0.11.The authors attribute the drift to memory-conditioned generation, not stored-KV precision.Source: §V · p. 9

VI. RELATED WORK

Related work addresses retrieval, memory compression, KV offloading, and generation beyond a model’s native horizon. WorldPager distinguishes itself by managing where pose-retrieved memory resides and when it moves, including an NVMe tier.

VI. RELATED WORK

  • WorldKV, ReWorld, OmniMem, WorldTrace, WorldPlay, Velox, and Inferix address retrieval, memory placement, or attention compute, while WorldPager manages where memory lives and when it moves.The authors state that, to their knowledge, prior systems do not place world-model KV on NVMe; WorldPager can run below any pose-keyed retrieval policy.Source: §VI · p. 9
  • Velox-style routing and WorldKV-style token compression could be combined with WorldPager, but the paper has not tested those combinations.Source: §VI · p. 9
  • KV quantization methods reduce bytes per chunk and delay the capacity wall, while WorldPager adds tiers and prefetching for memory that still does not fit.WorldPager uses a KIVI-style int4 quantizer.Source: §VI · p. 9
  • LLM offloading differs because decoding is short and bandwidth-bound, and selective access sets become known only after query computation; world-model access follows from camera pose before compute.The paper links these workload differences to the feasibility of zero-stall prefetching from NVMe on a consumer machine.Source: §VI · p. 9
  • WorldPager’s position handling follows prior position-reframing approaches but applies them to pose-retrieved long-term memory that may be read from NVMe.It is enabled only beyond the native horizon, leaving the first 341 chunks bit-identical.Source: §VI · p. 9

VII. LIMITATIONS AND CONCLUSION

WorldPager combines int4 tiered storage, prefetching, background write-back, and inference-time position handling to generate long sessions on one consumer machine. The authors bound the evidence to one machine and note limits in timing, quality scope, and long-gap image quality.

VII. LIMITATIONS AND CONCLUSION

  • All timing uses one RTX 4090 and one NVMe SSD; slower-drive cases, including SATA-class, are software-emulated with a token bucket.Source: §VII · p. 9
  • System measurements use one starting image and one seed per configuration, so the paper reports no confidence intervals for them.Source: §VII · p. 9
  • Image-quality and consistency results are for MG2 only; LingBot-World showed no retrieval consistency improvement, so only system metrics are reported for it.Source: §VII · pp. 9–10
  • On long-gap revisits, int4 tiered storage lowers MUSIQ by −1.28 [−2.25, −0.36], within the registered tolerance; scene 0000 retains only two seeds after drift exclusions.Memory-conditioned generation also develops repeated texture after about 100 chunks, independently of KV precision.Source: §VII · p. 10
  • With one RTX 4090, 32 GiB of host memory, and one NVMe drive, WorldPager generated 37.5 minutes of video continuously, with zero stalls and constant host memory.Source: §VII · p. 10

USE OF AI

The paper reports the authors’ use of AI tools and describes how a run is flagged in the long-gap recall analysis.

USE OF AI

  • Claude helped draft and revise the paper, write figure-generation scripts, and write and run parts of the experiment code under author direction.The authors say designs were registered before runs, numbers traced to result files, and the full text reviewed by the authors.Source: USE OF AI · p. 10

BF16 LONG-GAP RECALL

  • A run is flagged when int4 and bf16 arms differ by more than two MUSIQ points before revisits, either on the outbound leg or first-visit frames.Table IX reports paired differences averaged over 13 revisit events per gap tier for each scene and seed, alongside the drift flag.Source: BF16 LONG-GAP RECALL · p. 11
Loading 2610.35529v1…