Source-linked AI summary
Cross-Modal Data Co-Locality for Heterogeneous Storage Systems
Abdullah Al-Mamun
TL;DR
Multimodal queries can require cross-tier movement even when each modality’s ANN search is efficient, because related representations may be physically separated. CoLoc uses cross-modal access, semantic, and temporal signals to guide placement and migration above existing indexes. Its evaluation reports reduced movement and latency, with LPMM approaching periodic reorganization’s locality after a workload shift while moving less data and finishing faster.
Problem
Independently indexed multimodal representations can be physically separated, so queries may incur data movement even when each modality’s ANN search is efficient.
Source: §1 · p. 1Method
CoLoc combines cross-modal co-access, semantic affinity, and temporal evidence to guide topology- and capacity-aware placement and migration above existing ANN indexes.
Source: §1 · p. 2, §3.2 · p. 3Results
Takeaways & Limitations
CoLoc reduces data movement and end-to-end latency while leaving retrieval semantics unchanged, and LPMM recovers locality after workload shifts with less migration than global reorganization.
Source: §1 · p. 2, §5 · p. 6Incremental CAG maintenance is most beneficial under frequent, small workload updates; large, infrequent shifts can trigger a full rebuild.
Source: §3.2 · p. 3
Abstract
from arXiv · showhide
Distributed vector stores increasingly serve multimodal workloads whose representations are jointly accessed but independently partitioned and placed. As a result, cross-modal physical locality remains largely unoptimized. Multimodal queries can incur remote accesses and cross-tier data movement even when each modality's ANN search remains efficient. We present Coloc, a storage-management layer that makes cross-modal access locality a first-class data-layout signal. A Cross-Modal Access Graph combines semantic affinity, co-access, and temporal behavior. A movement-aware optimizer places frequently co-accessed vectors under capacity and topology constraints. A migration manager adapts placement as workloads shift by weighing locality gains against migration cost. A confidence-gated prefetcher hides remaining slow-tier latency. Coloc operates above existing ANN indexes without modifying embeddings, search, or retrieval semantics. We deploy Coloc on a multi-tier AWS testbed spanning GPU, NUMA-DRAM, NVMe, and S3. Coloc substantially reduces data movement and query latency while leaving ANN retrieval unchanged. Under a scheduled workload shift, its migration manager matches the locality of periodic global reorganization while moving 5.9x less data and completing 3.9x faster.
1 Introduction
Multimodal queries can require cross-tier data movement even when each modality’s ANN search is efficient, because independently indexed representations may be physically separated. CoLoc makes their joint consumption a placement signal and evaluates that strategy without changing retrieval quality.
1 Introduction
- Multimodal representations can be independently embedded and indexed, leaving those needed by one query on different machines or storage tiers and requiring data movement to assemble results.Each modality’s ANN search can remain efficient while query-level movement costs remain substantial.Source: §1 · p. 1
- Prior locality-aware systems address index, communication, or resource locality, but do not make cross-modal co-consumption a first-class placement objective.The placement problem combines observed co-access with semantic and temporal evidence while accounting for heterogeneous hardware costs.Source: §1 · p. 1
- CoLoc is a storage-management layer that uses cross-modal access, semantic affinity, and temporal proximity to guide capacity-, load-, topology-, and movement-aware placement.Its LPMM adapts placement under workload drift when expected future locality benefit justifies migration cost.Source: §1 · p. 2
- The paper introduces cross-modal co-locality as a physical data-layout primitive for independently indexed multimodal representations.The objective is where heterogeneous representations required by a query physically meet, rather than where vectors are searched.Source: §1 · p. 2
- 73%→0% of multimodal queries incur cross-tier movement, with up to 25% lower latency; LPMM moves 5.9× less data and finishes 3.9× faster than periodic reorganization.The evaluation also reports an 8.5% latency reduction from CAG-driven prefetching.Source: §1 · p. 2
- Retrieval quality (Recall@k) remains completely unaffected by placement.The placement-management path is separated from ANN search.Source: §1 · p. 2
2 Related Work
Related work spans distributed search, storage and memory locality, multimodal query execution, placement, retrieval fusion, and vector-database engineering. CoLoc is distinguished by its focus on physically co-locating independently indexed multimodal representations across heterogeneous tiers.
2 Related Work
- Recent large-scale and GPU-accelerated ANN systems DistVS, GIDS, and VecFlow continue a search-execution focus and do not reason about placement across independently managed indexes.Source: §2 · p. 2
- ARCADE [30] handles hybrid continuous queries across modalities within one system, complementing CoLoc’s focus on physical co-location across heterogeneous tiers.The two systems address different concerns rather than competing directly.Source: §2 · p. 2
- CoLoc specializes movement-versus-reorganization trade-offs to independently indexed multimodal vectors, where cross-modal query structure shapes placement units and migration triggers.Related examples include learned directory placement, MapReduce locality, and multicloud placement.Source: §2 · p. 2
- Multimodal embedding and fusion work [15], [33], [31], [16] changes retrieval representations or semantics, which CoLoc explicitly leaves untouched.CoLoc addresses the separate physical-placement gap for independently indexed vectors.Source: §2 · p. 2
- Physical placement across heterogeneous tiers remains comparatively underexamined in vector-database engineering work.The paper treats it as a first-class, independently evaluable concern.Source: §2 · p. 3
3 Cross-Modal Co-Locality Design
CoLoc makes joint consumption across independently indexed modalities a placement signal while leaving the ANN retrieval path unchanged. Its graph, placement, migration, and prefetching components adapt physical locality to workload and hardware conditions.
3 Cross-Modal Co-Locality Design
- CoLoc operates above existing modality-specific indexes and changes physical placement according to joint consumption without modifying embeddings, ANN search, distance functions, index structures, or retrieval semantics.Source: §3 · p. 3
3.1 Query-Induced Cross-Modal Locality
- CoLoc defines locality by the physical separation of representations consumed together by multimodal queries, rather than by individual vectors, ANN neighborhoods, or storage blocks.Source: §3.1 · p. 3
3.2 Cross-Modal Access Graph
- CAG combines observed co-access, semantic affinity, and temporal proximity, using co-access as primary evidence, semantic affinity for cold starts, and temporal proximity for short-lived correlations.Source: §3.2 · p. 3
3.3 Movement-Aware Cross-Modal Placement
- CPO forms groups with seeded asynchronous label propagation, orders them by co-locality weight, and assigns feasible resources to maximize movement-cost reduction under capacity constraints.When groups cannot fit, CPO splits them along weak locality edges; optional query frequency can prioritize high-traffic groups.Source: §3.3 · p. 4
3.4 Topology- and Tier-Aware Placement
- A topology-aware cost matrix combines normalized latency, bandwidth, I/O, and GPU-transfer terms, allowing the same CAG relationships to yield different placements across hardware topologies.Source: §3.4 · p. 4
3.5 Locality-Preserving Migration
- LPMM moves units only when expected future movement savings justify migration cost, amortizing the decision over a horizon estimated from observed access rate.Migration uses copy–verify–swap: the placement map changes atomically after verification, and a failed move leaves the prior placement valid.Source: §3.5 · p. 4
3.6 CAG-Driven Prefetching
- The prefetcher uses a Beta-Binomial confidence score to select likely CAG neighbors on slow tiers for a bounded cache, and serves higher-confidence candidates first when capacity is limited.Source: §3.6 · p. 5
3.7 Operational Overhead and Isolation
- All controls run asynchronously off the query path; incremental CAG rebuilds stay drift-bounded only for frequent small updates, while large infrequent shifts can require full rebuilds.Source: §3.7 · p. 5
4 Evaluation
On a multi-tier AWS deployment, CoLoc reduced cross-tier access and latency under constrained placement, while adaptive migration and incremental graph maintenance lowered reorganization costs.
Adaptive Migration
- LPMM moved 5.10 MB and completed migration in 14.03 s, versus 30.19 MB and 54.65 s for periodic reorganization.It reduced post-shift cross-tier access from 42.02% to 0.11% and achieved the lowest post-shift latency, 10.15 ms.Source: Table 5 · p. 6, §4 · p. 6
Control-Plane Overhead
- Incremental CAG maintenance reduced construction time from 5.235 s to 0.756 s, a 6.92× speedup in the many-small-batches regime.It triggered a full rebuild once across 19 batches.Source: Table 6 · p. 6, §4 · p. 6
End-to-End Performance and Movement
- CAG+CPO reduced cross-tier access from 73.04% to 0.00% and end-to-end latency from 12.52 ms to 11.67 ms versus independent placement.CAG+CPO also stayed within its tier-capacity limit, unlike independent placement.Source: §4 · p. 6
- CAG+CPO latency reduction rose from 8.3% at 10% multimodal queries to 16.4% at 30% and 24.8% at 50%.The reported benefit increased with multimodal query intensity.Source: §4 · p. 6
ANN Quality, Retrieval Neutrality, and Prefetch
- Recall@10 (1.0000) and ANN latency (8.72 ms) remained unchanged across configurations; prefetching reduced end-to-end latency from 11.76 ms to 10.75 ms (8.5%).The prefetcher overlaps remote fetches with query processing without affecting retrieval quality.Source: §4 · p. 6
5 Conclusion and Future Work
CoLoc treats cross-modal co-locality as a data-layout objective, using access, semantic, and temporal signals for placement and adapting placement under workload shifts. The AWS evaluation reports reduced data movement and latency, while future work targets online adaptation, larger deployments, and multitenant serving.
Conclusion
- CoLoc reduces data movement and end-to-end latency, while LPMM recovers locality after workload shifts with less migration than global reorganization.A confidence-gated prefetcher complements the system by reducing remaining slow-tier latency.Source: §5 · p. 6
- CoLoc combines CAG locality signals, topology- and capacity-aware placement, and LPMM adaptation under workload shifts without changing ANN retrieval semantics.A confidence-gated prefetcher complements these components by reducing remaining slow-tier latency.Source: §5 · p. 6
Future Work
- Future work includes online adaptation of placement and prefetching parameters, evaluation across larger and geographically distributed deployments, and multitenant vector serving.Source: §5 · p. 6