Cloud-Native Vector Search: A Comprehensive Performance Analysis

Zhaoheng Li; Wei Ding; Silu Huang; Zikang Wang; Yuanjin Lin; Ke Wu; Yongjoo Park; Jianjun Chen

Source-linked AI summary

Cloud-Native Vector Search: A Comprehensive Performance Analysis

Zhaoheng Li, Wei Ding, Silu Huang, Zikang Wang, Yuanjin Lin, Ke Wu, Yongjoo Park, Jianjun Chen

arXiv:2511.14748v2Nov 18, 2025cs.DB

TL;DR

Cloud-native vector search places indexes on remote storage, but providers largely default to cluster indexes despite varied workloads and index options. The paper analyzes cluster and graph indexes and their interaction with caching; it finds workload-dependent index choices and tuning needs, including settings that differ from on-disk search.

Problem
Providers largely default to cluster indexes even though cloud-native workloads and available index classes vary, leaving open which indexes and settings suit each scenario.
Source: §1 · pp. 1–2
Method
The paper models and analyzes SPANN and DiskANN, evaluates them across realistic on-cloud and on-disk setups, and studies how local caching interacts with their indexing choices.
Source: §1 · p. 2, §7 · p. 12
Results
Graph indexes can be favored under high concurrency or recall and for low-dimensional data or smaller datatypes, while cloud-native tuning prioritizes I/O reduction and caching gains can depend on index settings and cache size.
Source: §1 · p. 2
Takeaways & Limitations
Index choice and parameterization should reflect workload characteristics and cache size rather than assume one cluster-index configuration fits cloud-native search.
Source: §1 · pp. 1–2
The study defers distributed setups, since existing cloud-native providers use one compute node per storage bucket.
Source: §2.2 · p. 2

Abstract

from arXiv · show

Vector search has been widely employed in recommender system and retrieval-augmented-generation pipelines, commonly performed with vector indexes to efficiently find similar items in large datasets. Recent growths in both data and task complexity have motivated placing vector indexes onto remote storage -- cloud-native vector search, which cloud providers have recently introduced services for. Yet, despite varying workload characteristics and various available vector index forms, providers default to using cluster-based indexes, which on paper do adapt well to differences between disk and cloud-based environment: their fetch granularities and lack of notable intra-query dependencies aligns with the large optimal fetch sizes and minimizes costly round-trips (i.e., as opposed to graph-based indexes) to remote storage, respectively. This paper systematically studies cloud-native vector search: What and how should indexes be built and used for on-cloud vector search? We analyze bottlenecks of two common index classes, cluster and graph indexes, on remote storage, and show that despite current standardized adoption of cluster indexes on the cloud, graph indexes are favored in workloads requiring high concurrency and recall, or operating on high-dimensional data or large datatypes. We further find that on-cloud search demands significantly different indexing and search parameterizations versus on-disk search for optimal performance. Finally, we incorporate existing cloud-based caching setups into vector search and find that certain index optimizations work against caching, and study how this can be mitigated to maximize gains under various available cache sizes.

1 INTRODUCTION

Cloud-native vector search stores indexes on remote storage and uses compute to fetch index segments, with local resources caching hot segments. The paper asks how indexes should be chosen, parameterized, and combined with caching across differing workloads.

1 INTRODUCTION

Cloud-native services store vector datasets and indexes remotely; compute fetches index segments for approximate-nearest-neighbor queries and may cache hot segments locally.Local cache resources can include memory or SSD.Source: §1 · p. 1, Figure 1 · p. 1
Although providers predominantly offer cluster indexes, their posting-list fetch granularity and parallel fetches appear to fit remote storage, while prior on-disk studies found graph indexes favored in some high-recall or high-throughput workloads.Posting lists are often tens of KB, compared with cost/throughput-optimal remote fetch sizes of 8–16 MiB.Source: §1 · p. 1
Graph indexes can be favored for high concurrency or recall, and low-dimensional data or smaller datatypes can shift the balance toward graphs.The introduction contrasts these cases with ad-hoc recommendations, where cluster indexes may be favored.Source: §1 · p. 2
For cloud-native search, tuning should prioritize reducing I/O even when that substantially raises compute costs, unlike parameterizations described as suboptimal for on-disk indexing.The introduction gives finer-grained graph-index posting lists as an example.Source: §1 · p. 2
Some index optimizations can diminish caching gains, so optimal settings such as SPANN replication may depend on cache size rather than cache hit rate alone.The introduction notes DiskANN multi-neighbor expansion as one such optimization.Source: §1 · p. 2

2 CLOUD VECTOR SEARCH FUNDAMENTALS

Cloud-native vector search stores indexes remotely and fetches segments on demand, reducing storage costs while increasing query latency. Remote I/O constrains cluster and graph indexes differently: data throughput limits cluster search, while roundtrip latency limits graph search.

2.1 Cloud-Native Architecture

Object-storage-first systems fetch index segments from remote storage on demand and use local resources as caches, reducing operating costs while increasing query latency.The passage reports storage costs about 10–100× lower when indexes need not remain in memory.Source: §2.1 · p. 2

2.2 Cloud Environments: Constrained I/O

Remote storage combines lower read throughput, longer read latency, and GET-rate limits that can constrain throughput, especially at high concurrency.The passage contrasts remote throughput with local SSDs and identifies bandwidth and request rate as separate limits.Source: §2.2 · pp. 2–3

2.3.1 Cluster Index Search Cost.

Cluster-index performance is limited by data-fetch I/O or distance-computation capacity; SPANN is I/O-bound, particularly at high recall and concurrency.At 0.995 recall, SPANN reads 256MB per GIST1M query; Figure 3 reports that cloud-native bottlenecks are almost always network-related.Source: §2.3.1 · p. 3, Figure 3 · p. 3, Figure 2 · p. 3

2.3.2 Graph Index Search Cost.

Graph-index search incurs sequential traversal rounds, making remote read latency a bottleneck; DiskANN performs 43 roundtrips per 0.995-recall GIST1M query.Even with unlimited bandwidth and compute, the traversal rounds retain a minimum cost of rt × TTFB.Source: Figure 2 · p. 3, §2.3.2 · p. 4

3 INDEX DESIGN FOR CLOUD STORAGE

The section describes index-design choices that trade I/O against computation to improve cloud-native search. Its cost modeling motivates benchmarking parameter settings for cluster and graph indexes.

3 INDEX DESIGN FOR CLOUD STORAGE

SPANN replication duplicates boundary vectors, increasing index size but lowering nprobe and often reducing total query data read and distance computations.The passage notes index size can increase by up to 3× versus IVF, while remote storage costs make that increase a minor factor.Source: §3 · p. 4
Increasing cluster centroid count creates smaller posting lists, trading more distance computations and BKT-tree search for less I/O.This setting can improve cloud search when fetch cost dominates, although it is discouraged for on-disk use.Source: §3 · p. 4
DiskANN beamwidth W trades more I/O calls per query for fewer roundtrips, but high values can hit IOPS limits and degrade search.The passage says W is commonly set between 4 and 16 and should be tuned to environment and workload.Source: §3 · p. 4
A denser graph trades more distance computations and a larger graph for fewer I/O calls and roundtrips, potentially improving cloud search.Dense graphs are discouraged on disk but can help when distance computation is relatively significant.Source: §3 · p. 4
With a 4GB SLRU cache, query latency correlates with data read for SPANN and roundtrip count for DiskANN.The caption says cache hits can reduce both measures.Source: Figure 5 · p. 5
The section benchmarks cluster and graph index parameterizations to identify settings for optimal on-cloud performance.The cost models vary index parameters to control trade-offs such as I/O versus computation.Source: §3 · p. 5

4 ON-CLOUD INDEX-CACHE INTEGRATION

The section examines caching index data alongside existing metadata caching for cloud-native search, then shows that caching benefits depend on how each index is traversed and parameterized. In particular, index optimizations can reduce caching gains, making cache size relevant to index tuning.

4 ON-CLOUD INDEX-CACHE INTEGRATION

The section extends existing metadata caching by examining whether caching index data can bring further gains in cloud-native vector search.Existing providers cache index metadata locally; the section also reviews caching approaches and their interaction with index optimizations.Source: §4 · p. 5

4.1 Caching Benefits Vector Search

Caching can reduce data read and roundtrips, but most DiskANN queries receive little benefit, unlike SPANN’s more uniform per-query data-read distribution.The initial comparison used a 4GB SLRU cache on GIST1M under high-recall, I/O-constrained conditions.Source: §4.1 · p. 5

4.2 Index Optimizations May Harm Caching

DiskANN’s roundtrips and latency drop substantially only when query-wide cache hit rate exceeds 0.9, because every candidate in a beam-search expansion must be cached to save a roundtrip.Larger beamwidth makes that condition less likely; the relationship applies to the GIST1M workload studied with beamwidth 16.Source: §4.2 · p. 5, Figure 6 · p. 5
SPANN replication and DiskANN beamwidth can diminish caching gains, so optimal index parameterization depends in part on available cache size.The section highlights SPANN’s replication factor and DiskANN’s beamwidth as examples of optimizations whose effects interact with caching.Source: §4.2 · p. 5

5 EXPERIMENTS

Experiments compare cluster- and graph-based indexes across datasets and workloads, examining cloud-specific tuning and caching. They assess how performance and caching gains depend on workload, dataset, index design, parameterization, and cache size.

5.1 Experiment setup

The evaluation uses four benchmark datasets, searches for k=10 neighbors, and sweeps index search parameters and query concurrency.It evaluates DiskANN and SPANN on remote storage and measures throughput, latency, bandwidth, index-specific work, per-request I/O latency, and cache hit rate when caching is enabled.Source: §5.1 · p. 6

5.2 What Index For What Scenario?

DiskANN leads at high recall and/or concurrency, while SPANN leads at lower concurrency; dataset characteristics can shift this balance.On GIST1M, the crossover occurs around 0.8 recall at 16 concurrent queries. SPANN reads more data and can face I/O congestion at high recall and concurrency, while DiskANN’s iterative traversal incurs multiple remote-storage roundtrips at low concurrency.Source: §5.2 · pp. 6–8, Figure 7 · p. 7
Lower dimensionality, smaller datatypes, and larger datasets shift the performance balance toward SPANN.On DEEP10M, SPANN’s smaller posting lists and lower nprobe needs reduced data read by up to 28.4× and improved QPS by up to 7.61× at 0.995 recall; on MSSPACE10M, INT8 reduced reads versus DEEP10M, while BIGANN1B had the highest recall/concurrency cutoff for DiskANN to outperform SPANN.Source: §5.2 · p. 8

5.3 How to Design Indexes?

Higher SPANN centroid counts improve high-recall or high-concurrency performance, but can hurt when I/O congestion is low.On GIST1M, centroid%=32 achieved up to 3.14× QPS over centroid%=16, with 1.47× less data read per query at 0.995 recall; its added distance-computation and tree-search costs can outweigh I/O savings in lighter scenarios.Source: Figure 14 · p. 9, §5.3 · p. 9
Reducing SPANN replication can lower posting-list size but worsen its QPS-recall trade-off by requiring higher nprobe values.The replica=2 index needed 3–4× higher nprobe than replica=8 to reach the same recall; at 0.97 recall it read 2.00× more data and had 1.92× lower QPS.Source: Figure 16 · p. 9, §5.3 · p. 9
A denser DiskANN graph improves QPS across scenarios, while higher beamwidth helps only for high-recall, low-concurrency queries.On GIST1M, R=256 delivered up to 1.85× QPS versus R=64 by reducing roundtrips and requests. Beamwidth 64 improved QPS by 1.85× at 0.995 recall and concurrency=4, but higher request counts can lower QPS or saturate IOPS elsewhere.Source: Figure 17 · p. 9, §5.3 · pp. 9–10, Figure 19 · p. 10

5.4 How to Utilize Caching?

SPANN benefits more from caching at higher concurrency, larger cache sizes, and higher recall.The supplied passage attributes this to congestion relief and larger caches holding more frequently accessed posting lists; at high recall, queries read more overlapping lists, increasing cache locality.Source: §5.4 · pp. 10–11
SPANN cache hits reduce bandwidth pressure from concurrent posting-list fetches and reduce workload data read and request counts, mitigating bandwidth and IOPS congestion.At 0.995 recall, the 1GB cache hit rate falls from 0.19 to 0.17, explaining its performance difference from the 4GB cache; cache hits also reduce posting-list request queuing.Source: §5.4 · p. 11
DiskANN caching improves QPS by up to 1.41× at concurrency=4 for both cache sizes, with a 4GB cache yielding 1.41× at 0.7 recall versus 1.02× at 0.99 recall.Initial graph expansions have near-1 hit rates and account for more traversal at low recall; later expansions for difficult queries have near-zero hit rates. Under IOPS-saturated conditions at concurrency=64 and >0.99 recall, 4GB yields 1.45× QPS versus 1GB’s 1.30× by reducing requests per query.Source: §5.4 · p. 11
At 0.97 recall and concurrency=4, replica=2 reads 3.09× less data per query and delivers a 2.03× QPS increase versus replica=8 with a 4GB cache.The replica=2 index uses nprobe=1024 versus nprobe=256 for replica=8, but its higher cache hit rate (0.778 vs. 0.467) offsets the recall-related increase in nprobe; replica=8 is optimal again with 1GB or 8GB caches.Source: Figure 24 · p. 11, §5.4 · p. 11
For DiskANN, higher beamwidth diminishes caching gains by reducing the chance to save roundtrips, but beamwidth gains can outweigh cache benefits for high-recall ad-hoc queries.At concurrency=1, caching benefits are 1.25× at W=4, while increasing beamwidth from W=4 to 64 yields 7.58×; for non-IOPS-saturated settings, a fixed-size cache covering the entry-point neighborhood may suffice, while standard caching can help under IOPS saturation.Source: Figure 25 · p. 11, §5.4 · pp. 11–12
Reducing SPANN replication shrinks posting lists and can raise cache hit rates, but its performance advantage depends on cache size.With a 1GB cache, the hit-rate gain is insufficient; with an 8GB cache, replica=8 already has a sufficiently high hit rate, so it is optimal again.Source: §5.4 · p. 11, Figure 24 · p. 11

6 RELATED WORK

Related work covers cloud-native search architectures, index and query optimizations, and caching methods. The paper relates its findings to these approaches and identifies settings where their integration or further development may be useful.