PaperlayerView exact revision on arXiv

Source-linked AI summary

Memory-Aware Keyframe Management for Dense Maps on Memory-Constrained Robots

Huichang Yun, Jaeyeul Kim, Seungho Yoo

arXiv:2610.33574v1cs.RO

TL;DR

Count-based residency does not control memory when keyframe sizes vary, and sharing device memory with a model makes exceeding the limit an out-of-memory risk. ZEBRA accounts for resident bytes and admits map data by semantic zone; under the budget, it retains more localization coverage than the reported alternatives.

  • Problem

    Keyframe-count bounds do not control resident memory when keyframe sizes vary, complicating map residency on devices where a model and map share limited memory.

    Source: §I · p. 1, §III · p. 3

  • Method

    ZEBRA admits and releases map keyframes by semantic zone, measures resident bytes against a budget, and makes room before reading data.

    Source: C. Zone-Granular Admission Under a… · p. 4, §V · pp. 7–8

  • Results

    39.7 and 42.0 percent of a mission to a designated room were localized, versus 33.4 for a matched-count fixed grid and about 20 for keyframe-level retrieval, at lower ATE.

    Source: §V · p. 8

  • Takeaways & Limitations

    Under the evaluated budget and environments, semantic partitions localized more query frames than the geometric partition and keyframe-level retrieval.

    Source: §V · p. 8

    The current approach is evaluated against a map built beforehand; composition for mapping and localization at once is left for future work.

    Source: §V · p. 8

Abstract

from arXiv · show

Running a language model and a dense 3D map on one device means the two share one memory, and exceeding its limit is an out-of-memory failure. Map systems that keep a bounded working set bound it by a count of keyframes, but the memory one keyframe occupies varies widely with what it carries, so a count does not control what the working set occupies. The unit of admission is as poorly matched, since a radius around the pose, a fixed cell, or the graph neighbors of the last matched keyframe all move parts of the map that are not needed. We therefore propose two decisions. First, the memory the resident map uses is bounded directly, and the cost of every admission is known before any data is read, from per-keyframe resident sizes computed offline and carried with the map. Second, keyframes are admitted and released at a semantic spatial unit, the rooms and corridors of the building, so that the robot holds the zone it occupies together with the zone it is about to enter while the zones it has passed are released. Every admission measures the resident total, makes room, and only then reads, so the budget is never overshot. On two NVIDIA Jetson boards the budget holds in every run and the model gives up at most 0.8 percent of its throughput, whereas without it the operating system kills the model or the map. Under that budget, and over the environments evaluated, admission by semantic zone localizes 6-9 points more of the query frames than a geometric partition of matched zone count and about twice as much as the retrieval of RTAB-Map, at a lower ATE. Bounding memory lets the map and the model coexist, and moving the map in semantic zones makes the bounded map usable.

I. INTRODUCTION

Dense maps provide persistent spatial memory and accurate localization, but their variable keyframe sizes make count-based residency poorly matched to a device’s memory limit. ZEBRA bounds memory directly and moves the map in semantic zones; evaluations report more localized query frames under the budget.

I. INTRODUCTION

  • Keyframe costs range from tens of kilobytes to several megabytes, so a keyframe count does not determine resident memory or place an exact memory boundary.Sharing memory with a model also makes insufficient available memory an out-of-memory risk.Source: §I · p. 1
  • ZEBRA admits and releases rooms and corridors as semantic zones, retaining the occupied and upcoming zones while releasing zones already passed.The resident map is kept within the memory left free by the language model.Source: §I · p. 1
  • Across the evaluated boards and maps, semantic-zone residency localized more mission query frames than RTAB-Map retrieval and more than geometric partitions of matched zone count.The evaluation also weighs zone size and keyframe count alongside the admission unit.Source: §I · p. 2
  • The design bounds the resident map by the device’s memory itself, and the model gives up at most 0.8 percent of its throughput alongside the map.The contribution list also identifies semantic zones as the admission unit because fixed grids and existing retrieval can move unneeded map parts.Source: §I · p. 2

II. RELATED WORK

Prior map-management approaches reduce or compress map contents, or keep only part of an intact map resident. The paper distinguishes its byte-level bound and semantic-zone admission from those approaches, while noting that the closest prior system is not directly comparable.

II. RELATED WORK

  • Related work groups map management into reducing the map, storing its information differently, and limiting how much of an intact map is resident.The review states which of the paper’s two claims each group leaves open.Source: §II · p. 2

A. Reducing or Compressing the Map

  • Pruning and compression change map contents or representation, but neither specifies an enforceable device-memory bound or which keyframes should be resident at a given moment.Pruned information may not return when a later service needs its fidelity.Source: A. Reducing or Compressing the Map · p. 2

B. Bounding What Is Resident

  • Existing residency systems use counts, windows, or visibility-selected chunks, while their admission units include radii and fixed cells rather than building-specific partitions.The paper argues that such units can be inadequate inside buildings.Source: B. Bounding What Is Resident · p. 2
  • ZEBRA differs by budgeting bytes, admitting rooms or corridors, and loading the next route zone before the robot reaches its boundary.DiskChunGS is not directly comparable: it pages rendered Gaussians in online reconstruction, whereas ZEBRA pages localization keyframes against a prebuilt map.Source: B. Bounding What Is Resident · pp. 2–3

C. What Consumes the Dense Map

Some spatial-context systems consume point-cloud geometry directly, while localization can use a dense map whose payload also supports questions about the space. ZEBRA addresses the memory and residency decisions that arise when such a map shares a device with an AI model.

C. What Consumes the Dense Map

  • Point-cloud-based spatial-context systems build semantic or room representations from geometry, and the paper positions dense-map payload as useful for questions about space beyond pose estimation.Sparse features may suffice to produce a pose but not to answer a question about the space.Source: C. What Consumes the Dense Map · p. 3

III. ZEBRA: ZONE-GRANULAR RESIDENCY UNDER A MEMORY BUDGET

  • A robot using a prebuilt dense map cannot keep the whole map resident, and a missing keyframe may prevent pose production until the robot reaches a resident region.The resident set must also fit a budget shared with a running AI model.Source: §III · p. 3
  • ZEBRA addresses both decisions by using semantic zones as the admission unit and bytes as the accounting unit.Source: §III · p. 3

A. Zones, Payload, and the Resident Set

The method represents the environment as spatial zones and assigns each keyframe to one zone. It tracks resident bytes against a fixed budget, using offline size metadata and route information to guide admission and retirement.

A. Zones, Payload, and the Resident Set

  • The environment is assumed to contain a finite set of spatially meaningful zones, such as rooms or corridors, with each keyframe assigned to one by location.Source: A. Zones, Payload, and the Resident Set · p. 3
  • Each keyframe’s resident size includes compressed image and depth data plus the decoded point cloud, so its byte cost is computed offline and stored as map metadata.Zone admission cost is then known before any data is read.Source: A. Zones, Payload, and the Resident Set · p. 3
  • The runtime maps the pose to a zone and uses the planned route to decide which zone to read ahead and which zones not to retire.The route includes the return leg and resolves repeated zones by advancing through its ordered list.Source: A. Zones, Payload, and the Resident Set · p. 3
  • The budget applies to the resident working set in bytes, including resident items that do not belong to any zone, and must hold at the end of every cycle.The fixed byte budget is supplied at start-up.Source: A. Zones, Payload, and the Resident Set · p. 3

B. What a Partition Must Satisfy

A useful partition must keep each zone within the memory budget and align zone boundaries with visibility boundaries. ZEBRA manages admission and retirement at zone granularity, checking available memory and making room before reading data.

B. What a Partition Must Satisfy

  • Each zone must fit the budget, and boundaries should align with visibility so inactive keyframes are outside the robot’s observable region.Rooms and corridors meet the visibility criterion because walls bound both regions and lines of sight.Source: B. What a Partition Must Satisfy · p. 3
  • Fixed-side cubic chunks satisfy fit but align with visibility only when the environment happens to divide at that scale.At matched zone counts, chunks can cut across rooms and corridors.Source: B. What a Partition Must Satisfy · p. 3

C. Zone-Granular Admission Under a Memory Budget

  • The Zone Manager maps pose to zone, while the Memory Manager admits and retires zones to keep resident bytes within the shared-memory budget.The managers exchange lists of active and retired zones.Source: C. Zone-Granular Admission Under a… · p. 4
  • Before reading, ZEBRA computes free room from metadata and releases eligible residents until the requested data fits, preventing even a brief budget overshoot.It releases keyframes outside active zones before retiring zones; reading first could cause an allocation failure.Source: C. Zone-Granular Admission Under a… · p. 4
  • Retirement follows least-recent use, while a supplied route skips the current and upcoming zones when choosing a victim.The route changes retirement order, not the amount of memory recoverable.Source: C. Zone-Granular Admission Under a… · p. 4
  • The current zone must be fully resident; the next zone is prefetched only when both fit together, and an oversized zone is admitted partially as an exception.Prefetch proceeds in batches of at most K = 8 keyframes; the resident working set is kept within Bmax at each cycle.Source: C. Zone-Granular Admission Under a… · p. 4

D. Implementation

The implementation replaces only RTAB-Map’s resident-keyframe decision, leaving pose estimation, loop closure, and graph optimization unchanged. Run behavior is configured through map metadata, and Fig. 3 depicts routes R1–R3 on two maps.

D. Implementation

  • RTAB-Map supplies the interfaces for releasing and reloading keyframes; pose estimation, loop closure, and graph optimization remain unchanged.Only the decision about which keyframes are resident is replaced.Source: D. Implementation · p. 4
  • Fig. 3 shows mission routes R1–R3 on the two maps.Source: Figure 3 · p. 5
  • Table I lists the devices, models, maps, and residual map budgets; model size denotes resident model size, and an asterisk marks text-only models.Source: Table I · p. 5
  • The map file carries the budget, offline keyframe-size metadata, point-cloud settings, route, and prefetch configuration.Plain and route-conditioned runs differ only in the file’s route, prefetch switch, and batch size K.Source: D. Implementation · p. 5

IV. EXPERIMENTS

The experiments examine whether a memory-bounded map can share a device with a language model and which map-admission unit best localizes. The section moves from setup to localization and model delivery, then examines cubic-chunk size and route use.

IV. EXPERIMENTS

  • The experiments ask whether bounding resident map memory permits coexistence with a language model and which admission unit should be used.Source: §IV · p. 5
  • Sections IV-A through IV-D cover setup, localization and model delivery, then cubic-chunk side length and route use.Source: §IV · p. 5
  • Fig. 4 depicts the three partitions, with the hospital on the left and office on the right.Source: Figure 4 · p. 5

1) Environments and maps:

The evaluation uses prebuilt hospital and office maps, three partition types, and device/model pairings with residual memory budgets. It replays mission trajectories and reports localization and trajectory-error measures.

1) Environments and maps:

  • The hospital and office maps contain 1,620 and 1,041 keyframes, with 8,399 and 5,263 MB of decoded resident payload, respectively.Runs replay recorded missions against prebuilt maps with online mapping disabled, using an RGB-D robot at 5 Hz and 0.6 m/s.Source: 1) Environments and maps · p. 5

2) Partitions:

  • The reference partition has 61 hospital zones and 43 office zones; the VLM partition has 92 and 83, while matched-count cubic chunks use 7 m and 8.5 m sides.The VLM partition splits large rooms at furniture boundaries; the reference partition is one interpretation of the layout, not ground truth.Source: 2) Partitions · p. 5

3) Devices and budgets:

  • Table I lists device, model, map, and residual map-budget pairings; Table II reports localization rate and Table III reports payload loaded and ATE RMSE.Source: 3) Devices and budgets · p. 5, Table II · p. 6, Table III · p. 6
  • Each Table II column leaves 50, 100, or 150 MB of device memory free beyond the pairing’s residual budget, reducing the map’s allocation by that amount.Source: 3) Devices and budgets · p. 6
  • Models answer image-description or text-only queries throughout runs and stay resident; the Orin Nano hospital pairing uses text-only Gemma3 1B because Gemma4 E2B leaves no map payload budget.Localization rate is the percentage of query frames with a pose; ATE RMSE is the mean planar ATE over localized frames.Source: 3) Devices and budgets · p. 6

4) Conditions:

The evaluation compares bounded map residency and admission units under shared memory budgets, then tests localization across routes and map partitions. Semantic partitions improve localization over keyframe retrieval and matched-count chunks in most evaluated conditions, with a largest-zone fit exception.

4) Conditions:

  • All conditions used the same memory budget and true starting pose; the Org conditions used RTAB-Map’s graph-neighbor retrieval, while the route variant retrieved along the planned path.The Org conditions were modified to hold the budget strictly and had no zone partition or custom eviction policy.Source: 4) Conditions · p. 6

5) Scenario:

  • Each mission began in the lobby, visited one designated room for 55 seconds of model queries, then returned; localization rate counts query frames with a pose, and ATE is planar RMSE over localized frames.Routes R1–R3 visit three rooms on each floor.Source: 5) Scenario · p. 6

B. Effect of the Memory Budget on Localization

  • No bounded run exceeded its budget or was killed, whereas without the bound the operating system ended one process on every device-and-map pairing.On the Orin Nano with the office map, the budget is below the reference partition’s largest zone, and semantic conditions do not lead.Source: B. Effect of the Memory Budget on… · p. 6
  • 39.7% and 42.0% of query frames were localized by the semantic partitions, 6–9 points more than cubic chunks and about twice the original retrieval’s roughly 20%.ATE separated retrieval from the zone conditions by an order of magnitude on the hospital map and by a factor of three on the office map.Source: B. Effect of the Memory Budget on… · p. 6
  • RTAB-Map produced no pose in the target room in 58 of 72 runs and localized under 4% of room frames in six more; failed matches can leave the same neighborhood resident across cycles.Zone admission instead chooses what to read from the pose and the zone it falls in.Source: B. Effect of the Memory Budget on… · p. 7
  • The ordering in Table II also held when all three routes ran as one continuous mission at the residual budget.The continuous missions replayed for 685 and 643 seconds across the four device-and-map pairings.Source: B. Effect of the Memory Budget on… · p. 7
  • On the hospital map, vision-language zones lead while loading less; on the office map, the largest zones are 164 versus 159 keyframes and results stay within seven points on the AGX Orin.Every admission fits whole on the AGX Orin; on the hospital map, zones gain 3–11 points on the Orin Nano over the AGX Orin, while cubic chunks are last and load the most.Source: B. Effect of the Memory Budget on… · p. 7

C. Effect of the Memory Budget on the Concurrent AI Workload

The concurrent-workload evaluation compares bounded and unbounded map residency. With ZEBRA, both processes completed every pairing and model throughput stayed near its no-map rate; without the bound, the operating system ended a process on every pairing.

C. Effect of the Memory Budget on the Concurrent AI Workload

  • At most 0.8% of the model’s no-map throughput was given up under ZEBRA, and every device-and-map pairing completed with both processes alive.The map’s memory allowance came from the workload running beside it.Source: C. Effect of the Memory Budget on the… · p. 7
  • Without the bound, the operating system ended one process on every pairing: the model on three, and the map on the Orin Nano with the hospital map.On the Orin Nano, throughput fell while the map was resident; on the AGX Orin, the model was killed before enough calls completed to show a decline.Source: C. Effect of the Memory Budget on the… · p. 7

D. Ablation on Chunk Side and Route

Route-conditioned admission improves localization while reducing keyframe admissions and releases, and chunk size trades read volume against localization. The conclusion reports stronger localization from semantic partitions under a byte-enforced memory budget.

D. Ablation on Chunk Side and Route

  • Chunk size trades reads against localization: sides beyond 6 m read more and localize less, while the smallest side reads and localizes least.The best side varies by device and map and is not known before the run.Source: D. Ablation on Chunk Side and Route · p. 7
  • Reading the next zone before reaching its boundary raised localization by 5.6 points and admitted and released less on average than boundary-triggered admission without route conditioning.The authors attribute this to the next zone already being resident at the boundary and to boundary admission following a drifting pose estimate when localization is lost.Source: D. Ablation on Chunk Side and Route · p. 7

V. CONCLUSION

  • The conclusion defines the management units as bytes for accounting and semantic zones for admission, with room made before any map data is read.Measuring the resident total each cycle prevents the budget from being overshot.Source: §V · pp. 7–8
  • 39.7% and 42.0% of a mission to a designated room were localized by semantic partitions, versus 33.4% for a fixed grid and about 20% for keyframe-level retrieval, at lower ATE.With the budget enforced, no run exceeded it or was killed, and model throughput loss was at most 0.8%.Source: §V · p. 8
  • Future work would compose the approach with online room partitioning during mapping and with systems that already use semantic space for navigation or answering.The paper's current map-management contribution addresses residency, and the authors expect the combination to gain most when a robot already reasons over rooms and corridors.Source: §V · p. 8

Related papers

Loading 2610.33574v1…