Source-linked AI summary
SmartANN: Object Causal Modeling Boosts Approximate Nearest Neighbor Diagnosis and Auto-Design
Yutong Zhou, Guoxin Kang, Lei Wang, Xueya Zhang, Qinwei Yang, Jianfeng Zhan
TL;DR
ANN phase dependencies allow upstream performance loss to propagate, while existing component analyses and end-to-end benchmarks do not trace that loss across phases. SmartANN uses object-level diagnosis and replacement to identify bottlenecks, then composes compatible optimization actions. Across eight real-world datasets, it improves Recall or increases QPS at comparable Recall, with low diagnosis and auto-design overhead.
Problem
Component-level analyses compare isolated design choices and end-to-end benchmarks report aggregate metrics, but neither traces performance-loss propagation across dependent ANN phases.
Source: §1 · pp. 1–2Method
SmartANN represents ANN workflows as eight ordered, replaceable objects, diagnoses bottlenecks through sequential replacement, and composes compatible actions into redesigned algorithms.
Source: §1 · p. 2Results
Across eight real-world datasets, SmartANN improves Recall by 0.24–74.20% or increases QPS by 28.8–256.5% at comparable Recall, with low diagnosis and auto-design overhead.
Source: §1 · p. 2, §9 · p. 12Takeaways & Limitations
For IVF-PQ and HNSW, SmartANN identifies bottleneck objects and failure causes and uses them to generate optimized end-to-end designs.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Approximate Nearest Neighbor (ANN) algorithms achieve high efficiency through interdependent phases across index construction and query execution. This coupling allows upstream performance loss to propagate downstream, affecting execution behavior and measurable outputs. Existing component-level analyses mainly compare isolated design choices, while end-to-end benchmarks report aggregate metrics; neither traces loss propagation across dependent phases, hindering root-cause attribution and automated redesign. We present SmartANN, a framework based on the object causal model (OCM) for ANN bottleneck attribution and automated redesign. SmartANN represents an ANN workflow as eight ordered, replaceable objects and diagnoses them with a sequential diagnose-and-replace loop. At each iteration, it identifies the first object deviating from expected behavior or output as a bottleneck. Because an upstream bottleneck can obscure downstream ones, SmartANN replaces it with a test oracle when available, or with an implementation producing a better outcome, then continues downstream diagnosis. From the diagnosed bottlenecks and failure causes, SmartANN composes compatible actions from a pluggable action library to generate an optimized end-to-end ANN design. We instantiate SmartANN for IVF-PQ and HNSW, covering partition-and-quantization and graph-based ANN families. Experiments on eight real-world datasets show that SmartANN improves Recall by 0.24--74.20%, and increases QPS by 28.8--256.5% at comparable Recall, with low diagnosis and auto-design overhead. The code is available at https://github.com/zhouyutong20/SmartANN.
1 INTRODUCTION
SmartANN addresses the difficulty of tracing ANN performance loss across dependent phases and turning diagnoses into targeted redesigns. It uses sequential object diagnosis and replacement, then composes compatible optimization actions; experiments report improvements across IVF-PQ and HNSW.
1 INTRODUCTION
- Component-level comparisons isolate predefined designs, while end-to-end benchmarks report aggregate metrics without tracing loss across dependent phases.Neither approach identifies which internal phases cause a bottleneck for the current dataset or which optimization can address it.Source: §1 · p. 1
- Upstream performance loss can contaminate downstream inputs and obscure downstream bottlenecks, making independent phase inspection unreliable.Intermediate phases such as data partitioning and graph construction lack theoretically exact outputs, complicating replacement with test oracles.Source: §1 · p. 2
- Existing optimizations often couple changes across phases, making them difficult to modularize and apply as repairs for a diagnosed bottleneck.Because gains on one dataset may not generalize to another, users face costly trial-and-error optimization.Source: §1 · pp. 1–2
- SmartANN models ANN execution as eight ordered, replaceable objects and diagnoses bottlenecks sequentially, replacing each with an oracle or stronger implementation before proceeding downstream.It composes compatible actions from a pluggable library to generate and select end-to-end designs.Source: §1 · p. 2
- Recall improves by 0.24–74.20%, QPS increases by 28.8–256.5% at comparable Recall, and end-to-end latency is 2.9–42.0× faster than VDTuner.Across eight real-world datasets, SmartANN also reports low diagnosis and auto-design overhead.Source: §1 · p. 2
2 PRELIMINARIES
The preliminaries define ANN search outputs and efficiency measures, then introduce object causal modeling as a way to analyze effects across system objects.
2.1 Problem Definition
- An ANN algorithm returns k vectors per query as an approximate result set, while QPS measures search efficiency under fixed hardware and configurations form a Recall–QPS Pareto frontier.The problem definition also frames SmartANN as diagnosing bottleneck phases for a dataset and query set, then generating redesigned algorithms aimed at improving Recall, QPS, or both.Source: §2.1 · p. 2
2.2 Object Causal Model
- The object causal model represents propagation paths among system objects: upstream outputs can affect downstream execution behavior and measurable outputs.Object-level replacement of mechanisms and actions helps isolate these effects.Source: §2.2 · p. 2
- Figure 1 is identified as an overview of ANN object causal modeling, bottleneck diagnosis, and automated design.The caption names the topics but does not specify visual encodings.Source: Figure 1 · p. 3
3 SMARTANN OVERVIEW
SmartANN’s overview organizes ANN construction and query execution into replaceable objects, diagnoses them in execution order, and uses the resulting bottlenecks to select compatible redesign actions.
3 SMARTANN OVERVIEW
- The object causal model represents ANN construction and query execution as eight ordered objects with input-output contracts, expected behavior, and expected outputs.A deterministic expected output can serve as a test oracle; when exact outputs are unavailable, SmartANN uses a stronger replacement implementation under the same input and resource constraints.Source: §3 · p. 3
- The diagnoser records the first object that deviates from expected behavior or output, replaces its output or implementation, and resumes diagnosis downstream.Repeating this process through the workflow returns the complete bottleneck set and failure causes.Source: §3 · p. 3
- The auto-designer uses 42 actions—21 for each ANN family—and composes compatible actions that jointly cover diagnosed bottlenecks before selecting a design.It filters compositions by cause-label matching and selects the highest-Recall design or the highest-QPS design at comparable Recall.Source: §3 · p. 3
4 OBJECT CAUSAL MODEL
SmartANN models ANN execution as eight ordered, replaceable objects to trace performance loss and support diagnosis and redesign. It specifies object-level metrics and replacement methods for IVF-PQ and HNSW, then illustrates diagnosis-driven design for both families.
4 OBJECT CAUSAL MODEL
- The OCM decomposition supports root-cause isolation by making each phase measurable, controllable, and replaceable under a fixed contract.SmartANN uses eight objects because coarser divisions mix causes, while finer divisions lose shared interfaces for IVF-PQ and HNSW.Source: §4 · p. 3, §4.1 · p. 4
4.1 Object-level ANN Decomposition
- The eight objects span representation, unit instantiation, relation generation and materialization, access localization, candidate exploration and scoring, and result decision.Each object has a distinct role in the ANN workflow, from vector representation through final top-k selection.Source: §4.1 · pp. 3–4
4.2 Object Instantiation, Diagnostic Metrics, and Replacement
- For upstream objects, SmartANN pairs family-specific quality metrics with conditional references, using stronger replacements when structures lack a unique correct output.Metrics include representation distortion, unit balance, navigation, and relation completeness; reference conditions include fixed upstream state and resource or construction budgets.Source: §4.2 · p. 4
- For query-time objects, metrics track neighbor-unit coverage, candidate coverage and cost, scoring loss, and final-result loss, with exact or deterministic outputs serving as oracles when available.The references depend on conditions such as selected units, search budget, candidate identifiers, and tie-handling policy.Source: §4.2 · p. 5
4.3 Running example
- On GloVe, diagnosis identifies Access Localization and Candidate Scoring bottlenecks for IVF-PQ, but Candidate Exploration for HNSW, then generates family-specific redesigns.The resulting plans are unit selection and reranking for IVF-PQ, and safe search expansion for HNSW.Source: §4.3 · p. 5
5 REPLACEMENT-GUIDED ATTRIBUTION DIAGNOSIS
SmartANN diagnoses ANN bottlenecks in execution order and replaces upstream failures to reveal downstream ones. Its examples distinguish IVF-PQ’s access-localization and scoring losses from HNSW’s candidate-exploration loss.
5 REPLACEMENT-GUIDED ATTRIBUTION DIAGNOSIS
- The loop diagnoses the first deviating object, replaces it with an oracle or stronger implementation, and resumes downstream until all objects have been examined.The replacement choice uses an oracle for deterministic expected outputs; otherwise, it selects a stronger implementation under the same inputs and resource constraints.Source: §5 · pp. 5–6
- For IVF-PQ, replacing Access Localization raises Candidate Scoring loss from 0.184 to 0.421, exposing a downstream bottleneck that exact scoring then reduces to zero.The replacement increases true-neighbor-unit coverage from 0.644 to 1.000; the scoring-loss change is 0.237.Source: §5 · p. 6
- Replacing HNSW Candidate Exploration raises mean candidate Recall from 0.863 to 0.946 and its 10th-percentile value from 0.570 to 0.860, while below-target queries fall from 35% to 1%.Candidate Scoring adds no loss under the updated input, so the downstream deviation is attributed to propagation from Candidate Exploration rather than an independent scoring bottleneck.Source: §5 · p. 6
- Diagnosis finds IVF-PQ bottlenecks at Access Localization and Candidate Scoring, but only Candidate Exploration for HNSW.The respective causes are insufficient true-neighbor-unit coverage, approximate-scoring loss, and insufficient candidate coverage.Source: §5 · p. 6
6 ATTRIBUTION-GUIDED AUTO-DESIGN
SmartANN builds redesigned ANN configurations from diagnosed bottlenecks and their failure causes, using compatible, modular actions. It evaluates the resulting candidates and selects one according to the specified performance objective.
6.1 Pluggable Action Library
- SmartANN registers an optimization as an action when its object mapping and applicability conditions can be specified, and records its supported family, target objects, failure causes, requirements, compatibility, and output contract.The 42-action library includes 34 adaptations from existing methods and eight actions developed for SmartANN; 21 actions support each ANN family.Source: §6.1 · pp. 6–7
- Standardized object interfaces let heterogeneous actions be interchanged and composed; diagnostic replacements expose downstream behavior, whereas deployable auto-design actions improve the Recall–QPS tradeoff under contracts and resource constraints.Actions use prepare and execute interfaces, and SmartANN validates outputs and resource usage during execution.Source: §6.1 · p. 7
6.2 Design Generation and Selection
- Cause-matched actions or compatible compositions must jointly cover diagnosed bottlenecks; SmartANN evaluates the retained designs and selects the highest-Recall option or highest-QPS option at comparable Recall.Candidates are materialized and tested on a selection query set; construction-phase actions can require rebuilding affected indexes.Source: §6.2 · p. 7
- In the paired example, the selected IVF-PQ composition raises Recall@100 from 0.2670 to 0.6543, while the selected HNSW design raises Recall from 0.6692 to 0.8162.The diagnosed bottlenecks and causes differ between the two ANN families on the same dataset and workload.Source: §6.2 · pp. 7–8
7 EVALUATION
The evaluation tests SmartANN with IVF-PQ and HNSW across eight real-world datasets and compares it with VDTuner and a Random baseline. Separate query splits support tuning, diagnosis, training, selection, and final evaluation.
7.1 Experimental Setup
- The eight datasets span image, word, audio, search, and cross-modal embeddings, with 0.48–9.99 million vectors, 96–1,024 dimensions, and L2, angular, or cosine distances.Four additional datasets emphasize hubness, small nearest-neighbor gaps, heterogeneous local density, and distribution shift.Source: §7.1.1 · p. 8
7.1.1 Datasets and Query Splits.
- Five disjoint query splits assign separate data to tuning, diagnosis and replacement, learned-action training, candidate selection, and final evaluation.Final evaluation queries are not used for tuning, diagnosis, training, or selection.Source: §7.1.1 · p. 8
- Three representative baseline Pareto-frontier points—maximum Recall, maximum Recall×QPS, and the point farthest from both—start diagnosis and redesign.The points are chosen to represent separated operating regions.Source: §7.1.1 · p. 8
7.1.2 Evaluation Methodology.
- SmartANN is compared with VDTuner’s black-box parameter search and a Random baseline that lacks attribution evidence; both follow the training, selection, and final-evaluation protocol.SmartANN filters designs using diagnosed bottlenecks, failure causes, compatibility, and resource constraints.Source: §7.1.2 · p. 8
- The ablations compare attribution-guided selection with Random and sequential Replacement with No-Replacement while holding the specified baseline execution, action library, data, budgets, or diagnostic settings fixed.The replacement comparison tests whether the operator distinguishes propagated loss from independent or masked downstream bottlenecks.Source: §7.1.3 · p. 8
7.2 Performance Comparison
Across the reported operating regions, SmartANN produces more favorable Recall–QPS tradeoffs than VDTuner on most datasets, while diagnosis and redesign expose ANN-family-specific bottlenecks. Tuning is also faster than VDTuner in the reported experiments.
7.2.1 Diagnosis and Replacement Results.
- Sequential replacement exposed an IVF-PQ GIST bottleneck set of O2, O5, and O7: improving O5 coverage from 0.885 to 1.000 exposed O7 scoring loss, which O7 replacement then reduced to zero.Bottlenecks vary in number across workloads.Source: §7.2.1 · p. 9, Figure 2 · p. 9
7.2.2 End-to-End Performance .
- At IVF-PQ GloVe-P1, repairing Access Localization O5 and Candidate Scoring O7 raises Recall from 0.2670 to 0.6543.When candidate Recall meets its requirement but access or distance-computation costs remain redundant, SmartANN can instead select an efficiency-oriented action.Source: §7.2.2 · p. 9
- On high-Recall HNSW Yandex-200, AdSampling preserves Recall at 0.9747 while increasing QPS from 1,506 to 1,939; SmartANN retains the original design when diagnosis finds no bottleneck.Source: §7.2.2 · p. 9
- SmartANN produces more favorable Recall–QPS tradeoffs than VDTuner on most datasets and operating regions.Figure 3 compares HNSW and Table 1 reports IVF-PQ results.Source: §7.2.2 · p. 9, Figure 3 · p. 9, Table 1 · p. 10
7.2.3 Overhead.
- SmartANN requires less tuning time than VDTuner on every dataset, with aggregate speedups of approximately 5.0× for HNSW and 11.6× for IVF-PQ.Per-dataset speedups range from 2.9× to 36.4× for HNSW and 4.7× to 42.0× for IVF-PQ.Source: §7.2.3 · p. 9
7.3 Ablation Study
The ablations test whether attribution-guided action selection and sequential replacement improve redesign and diagnosis. Results show more consistent tradeoffs with attribution and reveal both propagated and independent downstream losses through replacement.
7.3.1 Effect of Attribution-guided Automated Design.
- On IVF-PQ GIST, SmartANN reaches 0.9038 Recall and 124.4 QPS, versus Random’s 0.8195 Recall and 55.6 QPS; SmartANN’s Recall–QPS tradeoffs are more consistent on both ANN families.Random sometimes finds competitive designs but is sensitive to action sampling and often gains QPS while sacrificing Recall.Source: §7.3.1 · p. 10, Figure 5 · p. 10, Figure 6 · p. 10
7.3.2 Effect of Replacement Operator.
- For HNSW, replacing O6 raises GloVe candidate Recall at O6 from 0.642 to 0.790 and at O7 from 0.687 to 0.790, identifying the earlier O7 deviation as propagated loss.The paper reports independent O2 and O6 bottlenecks on Tiny5M and DEEP.Source: §7.3.2 · p. 10, Table 2 · p. 11
- For IVF-PQ, replacing O2 and then O5 exposes downstream O7 scoring loss on GIST, GloVe, and Yandex-200; O7 replacement reduces those losses to zero.No-Replacement can repeatedly assign propagated upstream loss to downstream objects or miss independent bottlenecks masked by upstream loss.Source: §7.3.2 · p. 11
7.4 Key Findings
SmartANN reports gains across eight real-world datasets and finds that sequential replacement improves bottleneck attribution, while diagnosis-guided action composition supports more consistent redesign than a random baseline.
7.4 Key Findings
- No-Replacement misattributes propagated upstream loss to downstream objects and misses independent bottlenecks masked by upstream performance loss.Object-level diagnostic metrics combined with sequential replacement provide the complete bottleneck-object set and failure causes as verifiable attribution evidence.Source: §7.4 · p. 11
- Diagnosis-guided composition of compatible actions produces effective end-to-end ANN redesigns more consistently than the reproducible Random baseline without attribution evidence.Source: §7.4 · p. 11
- Recall improves by 0.24–74.20 percentage points or QPS increases by 28.8–256.5% at comparable Recall across eight real-world datasets, with low diagnosis and auto-design overhead.Against VDTuner, SmartANN achieves more favorable Recall–QPS tradeoffs across most datasets and a 2.9–42.0× speedup in end-to-end latency.Source: §7.4 · p. 11
8 RELATED WORK
Prior work studies ANN indexing, benchmarks, and selected components; SmartANN instead diagnoses mechanisms under a specific workload and selects implementations for targeted replacement.
8.1 ANN Indexing and Optimization
- SmartANN uses IVF-PQ and HNSW for diagnosis, replacement, and auto-design, spanning partitioning with quantization and proximity-graph ANN.Prior work also optimizes representation, partitioning and graph structure, query routing, candidate exploration, and search stopping.Source: §8.1 · p. 12
8.2 ANN Benchmarks
- Existing end-to-end benchmarks compare complete-index construction cost and retrieval performance, while SmartANN identifies bottleneck mechanisms and selects implementations for targeted replacement.Benchmark measures include construction time, memory use, retrieval accuracy, and retrieval speed; some benchmarks also cover compound queries and downstream tasks.Source: §8.2 · p. 12
8.3 Component-Level Analysis of ANN
- Prior component analyses specify a component or problem in advance, whereas SmartANN diagnoses the complete bottleneck set under the current workload without assuming its location.Its diagnosis–replacement–continued diagnosis process measures recoverable performance loss and guides targeted mechanism replacement.Source: §8.3 · p. 12
9 CONCLUSION
SmartANN uses object-causal diagnosis and sequential replacement to identify ANN bottlenecks, then composes compatible actions for redesign. Across eight real-world datasets and IVF-PQ and HNSW, it improves Recall or QPS at comparable Recall with low overhead.
9 CONCLUSION
- SmartANN represents ANN construction and query execution as eight ordered, replaceable objects and sequentially diagnoses them using test-oracle outputs or stronger replacement implementations.Source: §9 · p. 12
- The framework identifies bottleneck objects and failure causes, then composes compatible actions from a pluggable library to generate optimized end-to-end ANN designs.Source: §9 · p. 12
- Across eight real-world datasets spanning IVF-PQ and HNSW, SmartANN improves Recall by 0.24–74.20% or increases QPS by 28.8–256.5% at comparable Recall, with low diagnosis and auto-design overhead.Source: §9 · p. 12