PaperlayerView exact revision on arXiv

Source-linked AI summary

Static Pruning Across Sparse Retrieval Regimes: What Transfers, What Breaks, and What Still Helps

Zirui Song, Yuye Zhu, Yang Yang

arXiv:2608.16309v1cs.IRcs.AI

TL;DR

Prior studies tested static pruning in isolated pipelines, leaving its portability across modern sparse-retrieval engines with dynamic pruning unresolved. This study evaluates pruning across three engines, two benchmarks, and two encoders, finding that index-side pruning transfers while query pruning is often subsumed by built-in mechanisms.

  • Problem

    Existing pruning studies rely on custom exhaustive pipelines, leaving unclear whether their conclusions transfer to engines with built-in dynamic pruning.

  • Method

    The study evaluates query, document, and posting-list pruning across three engines, two datasets, two encoders, and 1,140 configurations.

  • Results

    Index-side pruning transfers across engines with 1.2–6.6× speedup and 18–82% index reduction, while query pruning is subsumed by engine-internal mechanisms.

  • Takeaways & Limitations

    The results support index-side pruning as portable, static pruning as complementary to dynamic pruning, and the NDCG knee as a stopping criterion.

  • Takeaways & Limitations

    The portability claims cover three engine paradigms, two learned-sparse encoders, web-search benchmarks, k=10, and single-threaded execution, not all models or domains.

Abstract

from arXiv · show

Static pruning is widely used to accelerate sparse neural retrieval, yet existing studies each validate their conclusions within a single custom pipeline, leaving it unclear which findings transfer to modern engines with different index organizations and dynamic pruning mechanisms. We present the first cross-engine pruning portability study, evaluating static pruning strategies across three engines - a controlled C++ pipeline (exhaustive inverted index), BMP (block-max pruning), and SEISMIC (clustered inverted indexes) - on two benchmarks (MS MARCO, Natural Questions) with two encoders spanning opposite query-density regimes (SPLADE: 44 avg. query terms; V3-GTE: 7 avg. query terms), totaling 1,140 experimental configurations, with an additional deep-judgment validation on TREC DL 2019/2020. We find that index-side pruning (document and posting-list) is portable: it consistently reduces latency (1.2-6.6$\times$) and index size (18-82%) across all engines because sparse retrieval is memory-bound - a conclusion we support with cache-miss, TLB, and IPC profiling. In contrast, query pruning is already internalized by modern engines: it yields 4-11$\times$ speedup on the exhaustive pipeline but is subsumed by BMP's $β$ and SEISMIC's query_cut. Static pruning complements dynamic pruning: on BMP, combining document and query reduction yields 2.5$\times$ speedup with NDCG@10 within 0.003 of the exact baseline. Finally, NDCG@10 saturates while Recall@10 is still in the ${\sim}$85-95% range across all three engines, providing a portable stopping criterion: practitioners can push pruning to this knee without visible ranking degradation. Together, these findings answer what transfers (index-side pruning), what breaks (query pruning), and what still helps (static atop dynamic pruning).

1 Introduction · 2 Background and Related Work

This paper positions static pruning as a portability question across sparse-retrieval engines, testing whether findings from custom exhaustive pipelines transfer to systems with built-in dynamic pruning. It combines learned sparse retrieval, pruning taxonomies, dynamic engines, and memory-locality analysis to identify cross-engine guidance.

  • 1 Introduction: Existing pruning studies largely validate conclusions within custom exhaustive-scoring pipelines, leaving transfer to dynamically pruned engines unresolved.BMP exposes β as query pruning, while SEISMIC limits query processing through query_cut.
  • 1 Introduction: 1,140 configurations compare query, document, and posting-list pruning across three engines, two benchmarks, and encoders with distinct query-density regimes.The study uses SPLADE with ∼44 query terms and V3-GTE with ∼6.9 query terms, evaluating α-Mass and Max-Ratio criteria.
  • 1 Introduction: Index-side pruning is portable across tested engines because it reduces index size and latency by reducing memory traffic.The paper frames sparse retrieval as fundamentally memory-bound, connecting portability, complementarity with dynamic pruning, and NDCG saturation.
  • 2 Background and Related Work: Learned sparse retrievers represent queries and documents as high-dimensional sparse vectors whose non-zero entries carry learned impact weights.Unlike classic bag-of-words scoring, learned sparse representations often activate more terms through expansion and produce posting weights that deviate from traditional tf-idf distributions.
  • 2 Background and Related Work: Static pruning spans term-centric vocabulary removal, document-centric entry pruning, global posting removal, and probabilistic or information-theoretic approaches.These approaches refine the question of when pruning can preserve retrieval quality in learned sparse retrieval.
  • 2 Background and Related Work: Dynamic pruning avoids scoring documents that cannot enter the top-k through upper bounds, block-level bounds, impact ordering, and early termination.These techniques underpin production systems such as Lucene and PISA, as well as specialized learned-sparse engines; BMP exposes α and β controls.
  • 2 Background and Related Work: Sparse score accumulation is memory-bound because it combines low arithmetic intensity with irregular accesses to postings and accumulator arrays.Consequently, locality-aware traversal and pruning that reduces memory traffic are central to system-level acceleration.
  • 2 Background and Related Work: The study replaces engine-specific pruning heuristics with a portability matrix validated across engines.Prior work examined pruning within single engines, whereas this work evaluates cross-engine transfer.

3 Pruning Strategies

This section defines two score-aware pruning criteria—α-Mass and Max-Ratio—and three pruning families applied to queries, documents, or posting lists. The families reduce different retrieval costs, while built-in query selection can make external query pruning redundant and posting-list pruning trades greater RAM savings for faster Recall degradation.

  • Pruning criteria: α-Mass keeps the smallest descending-weight prefix reaching an α fraction of the ℓ1 mass, yielding adaptive support sizes.High-entropy objects retain more terms.
  • Pruning criteria: Max-Ratio retains terms weighted at least τ times the maximum, producing scale-invariant, continuous trade-off curves.τ ranges over [0, 1).
  • Pruning families: Query pruning traverses only selected posting lists online, reducing list traversals and accumulator updates without modifying the index.External query pruning may be partially or fully redundant on engines with BMP’s β or SEISMIC’s query_cut.
  • Pruning families: Document pruning removes terms before indexing, reducing per-list work and RAM footprint by shrinking the working set, with irreversible deletions.Its reduced memory traffic is hypothesized to transfer across engine architectures.
  • Pruning families: Posting-list pruning removes lower-impact postings offline, shortening lists and reducing per-list work and RAM, especially for frequent terms.It achieves the largest RAM savings but suffers more rapid Recall degradation under aggressive settings than document pruning.

4 Experimental Setup

The study evaluates static pruning across two retrieval benchmarks, two contrasting encoder query-density regimes, and three engines with distinct index and dynamic-pruning designs. It measures ranking quality, oracle fidelity, coverage, latency, and index footprint under a controlled single-threaded protocol.

  • Datasets: Two benchmarks provide cross-dataset validation: MS MARCO contains ∼8.8M passages and 6,980 queries, while Natural Questions contains ∼2.7M passages and 3,452 queries.MS MARCO has shallow relevance judgments; Natural Questions differs in corpus size and judgment characteristics.
  • Encoders: SPLADE represents dense queries through query and document expansion, whereas V3-GTE represents sparse queries with tokenized-only queries and aggressive document expansion.The contrast isolates query density from encoder architecture.
  • Engines: The three engines comprise a controlled exhaustive C++ inverted index, BMP block-max dynamic pruning, and SEISMIC clustered inverted indexes with query-term selection.BMP exposes α and β, while SEISMIC exposes query_cut and heap_factor; β and query_cut already implement query pruning.
  • Metrics: Four metrics cover complementary objectives: Recall@k measures oracle fidelity, NDCG@k measures qrels-based ranking quality, Success@k measures judged-relevant coverage, and latency measures retrieval speed.Index size is reported as the in-memory inverted-index footprint in GB.
  • Protocol: Experiments run single-threaded with CPU core pinning on an AMD EPYC 9R14, using 5 warm-up runs followed by timed runs and mean latency reporting.The default cutoff is k=10 for all metrics unless stated otherwise.

5 Portability of Pruning Strategies

Index-side pruning transfers across engine designs because it reduces memory traffic, while query pruning is largely redundant on engines with built-in query-term selection. Document pruning is the most stable high-recall strategy, whereas posting-list pruning offers comparable speed and larger index savings but degrades Recall faster.

  • Cross-engine synthesis: Index-side pruning transfers across all tested engines by reducing memory traffic, the binding bottleneck, whereas query pruning is regime-dependent or internally subsumed.This synthesis covers document and posting-list pruning alongside BMP’s β and SEISMIC’s query_cut.
  • Query pruning: 4–11× speedups from query α-Mass pruning on the controlled pipeline do not transfer cleanly: SPLADE retains nearperfect NDCG, but V3-GTE Recall falls to 0.775.SPLADE reports NDCG 0.448 vs. 0.449 at α=0.50, while V3-GTE has approximately 7 query terms carrying critical score mass.
  • BMP: −34–36% BMP index reduction and 1.2–1.4× latency improvement from document pruning preserve NDCG@10 within 0.007 of the unpruned baseline.Shorter posting lists tighten block-max upper bounds, improving cache locality and dynamic skipping efficiency across datasets and encoders.
  • SEISMIC: 186 to 161 μs is the SEISMIC MS MARCO latency change from static query MR 0.10 at qc = 5, but Recall@10 drops to 0.924.At qc = 20, the same pruning provides 1.31×, while V3-GTE shows only a 1.05× gain at qc = 5.
  • SEISMIC: 15–34% latency reduction and 16–29% index savings from SEISMIC document pruning show consistent index-side benefits despite its near-memory-bound floor.The primary benefit is index reduction for RAM-constrained deployment.

6 Static–Dynamic Complementarity

Static and dynamic pruning are complementary because they target different bottlenecks: static pruning reduces memory footprint and dynamic pruning avoids wasted traversal. Their combined benefit is strongest on BMP, while SEISMIC’s near-memory-bound floor shifts the main gain toward RAM reduction.

  • Complementarity: Static and dynamic pruning target orthogonal bottlenecks: memory footprint and wasted traversal, respectively.Static pruning shortens lists and improves cache locality, while dynamic pruning skips non-competitive blocks or clusters.
  • BMP: 2.52× speedup with combined BMP pruning keeps NDCG@10 within 0.003 of baseline.The gain is submultiplicative (ρ = 0.92), indicating overlapping but not fully redundant cost components.
  • Cross-regime results: 2.02× combined speedup repeats on NQ+SPLADE, while V3-GTE shows lower but still complementary gains.V3-GTE has less query-pruning headroom, limiting the additional benefit from combining mechanisms.
  • SEISMIC: 1.39× combined speedup on SEISMIC is accompanied by 26% RAM reduction, with less latency benefit because its baseline is near the memory-bound floor.With little memory traffic left to eliminate, pruning’s primary benefit becomes RAM reduction rather than speedup.
  • Mechanism: 2.4× latency reduction in the locality-aware Ψ configuration coincides with cache and TLB misses falling from 30% to 2.7%.Across BMP and SEISMIC, latency tracks absolute cache misses, confirming that smaller indexes reduce memory traffic.

7 How Far to Push: Operating Point Selection

NDCG@10 offers a portable stopping criterion: it saturates before aggressive pruning while Recall@10 remains sensitive, enabling deployment at an engine-dependent NDCG knee. This knee reflects relevance-preserving reshuffles and supports distinct checks for system fidelity, user-facing ranking, and candidate generation.

  • Portable stopping criterion: NDCG@10 remains within 0.008 of baseline while Recall@10 drops by up to 14 points across engines and datasets.The saturation persists under deep, graded TREC DL 2019/2020 judgments.
  • Portable stopping criterion: BMP retains NDCG=0.449 at Recall=0.932 (β=0.5), illustrating operation near the NDCG knee.The knee lies in the high-0.8 to mid-0.9 Recall@10 range before aggressive pruning.
  • Validation metrics: Success@10 remains above 0.96 across the full BMP β sweep, confirming that moderate pruning preserves candidate-generation viability.MRR@10 tracks NDCG@10 across engines and operating points, with degradation only under aggressive pruning.
  • Why saturation occurs: NDCG@10 saturates because near-duplicate documents of equal relevance preserve graded gain over ranks 1–10 despite changing the exact top-k set.A deep-judgment TREC DL query verified that churned top-10 documents were judged equally relevant.
  • Decision framework: The recommended operating point is the NDCG@10 knee at ∼85–95% Recall, with BMP β=0.5 and SEISMIC qc=5 as examples.BMP uses NDCG=0.449 and Recall=0.93; SEISMIC uses NDCG=0.443 and Recall=0.98.
  • Decision framework: Success@10 ≥0.95 suffices for downstream re-ranking at doc AM 0.5 on the controlled pipeline.Use Success@10 to verify coverage of judged-relevant candidate items.

8 Discussion and Practitioner Guidance

The discussion distills the cross-engine findings into a deployment workflow: prioritize document pruning, use engines’ built-in query reduction, add posting-list pruning when RAM limits dominate, and combine methods under tight latency budgets. It also qualifies portability claims by noting pipeline differences and coverage across three engine paradigms and two contrasting query-density regimes.

  • Decision framework: What, How, How Far: BMP 𝛽 and SEISMIC qc should provide query reduction; external query pruning is secondary.The guidance favors built-in engine mechanisms over external query pruning.
  • Decision framework: What, How, How Far: 2.52× is the stated BMP speedup when pruning methods are combined under tight latency budgets.Posting-list pruning is additionally recommended when RAM is the binding constraint, with thresholds tuned under validation constraints because Recall can degrade faster than document pruning in some engine/encoder combinations.
  • Limitations: ∼44 vs. ∼7 query terms span the SPLADE and V3-GTE regimes used to establish portability across three engine paradigms.The paradigms are exhaustive scoring, block-max dynamic pruning, and clustered inverted indexes.
  • Limitations: Two-stage re-ranking in the controlled C++ pipeline differs from the single-stage BMP and SEISMIC setups, so C++† no-rerank reports lower absolute Recall while preserving qualitative portability.The limitation concerns comparability of absolute Recall, not the qualitative portability conclusion.

9 Conclusion

Across 1,140 configurations, index-side pruning transfers across engines, while query pruning is subsumed by engine-internal mechanisms. Static pruning complements dynamic pruning, and the ∼85–95% Recall NDCG knee provides a portable stopping criterion replicated under deep TREC DL judgments.

  • Portability: 1.2–6.6× speedup and 18–82% index reduction make index-side pruning portable across three engines, two datasets, and two encoders.The study covers 1,140 configurations.
  • Portability: Query pruning is subsumed by engine-internal mechanisms, specifically BMP’s 𝛽 and SEISMIC’s query_cut.
  • Portability: 2.52× combined speedup on BMP shows that static pruning complements dynamic pruning.
  • Stopping criterion: ∼85–95% Recall marks the NDCG knee and provides a portable empirical stopping criterion replicated under deep TREC DL judgments.

A Accumulator Variants

The controlled C++ pipeline compares three accumulator strategies that differ primarily in working-set size and SIMD use. Ψ Window-Switch is used throughout because its localized, cache-resident working set targets the memory-bound bottleneck.

  • Accumulator strategies: Φ uses a global N-score array with random access, while Ψ uses localized window accumulators and Ξ adds SIMD vectorization to Φ’s multiply step.Ξ yields <1% improvement, indicating the workload is memory-bound rather than compute-bound.
  • Accumulator strategies: All controlled-pipeline experiments use Ψ unless noted, establishing Window-Switch as the pipeline’s default accumulator.Both accumulator variants score queries against the inverted index and rerank the top-k′ with full dot products.
  • Accumulator strategies: Ψ partitions documents into windows, reducing the accumulator working set from O(N) ≈ 33 MB to O(W) ≈ 400 KB.Its local scores are accumulated per window before positives are pushed to a global heap.

B Detailed Experimental Results · C Re-Ranking Ablation

The detailed results provide full BMP and SEISMIC pruning sweeps, while the re-ranking ablation shows that full-vector re-ranking has limited value at moderate pruning but becomes important under aggressive document pruning. Cross-engine pruning severity also differs, with SEISMIC showing a larger Recall@10 decline under identical thresholds.

  • B Detailed Experimental Results: Tables 7 and 8 report complete BMP and SEISMIC sweeps covering index size, latency, Recall@10, NDCG@10, and MRR@10.These sweeps span all configurations underlying the portability findings; controlled-pipeline operating points appear in Table 2.
  • C Re-Ranking Ablation: The controlled C++ pipeline uses two-stage re-ranking with k′=50, whereas BMP and SEISMIC use single-stage retrieval.This establishes the re-ranking difference tested in the ablation.
  • C Re-Ranking Ablation: At Doc AM 0.90, removing re-ranking costs only 3–6 Recall@10 points and brings C++ in line with BMP.The no-rerank C++ curve closely tracks BMP at the main operating point.
  • B Detailed Experimental Results: Table 7 evaluates BMP document, posting, and query pruning across all four dataset–encoder combinations with α=1.0.Reported speedups are relative to each configuration’s unpruned baseline.
  • C Re-Ranking Ablation: Under the same static pruning threshold, SEISMIC exhibits a larger Recall@10 drop than the other engines because cluster-based traversal changes effective pruning severity.Figure 7 marks Recall@10 ≥0.90 with a gray reference band.
  • B Detailed Experimental Results: Table 8 evaluates SEISMIC document pruning across all configurations with qc=5 and hf=1.0.The table complements the BMP sweep with SEISMIC-specific settings.
  • C Re-Ranking Ablation: Table 9 reports pooled DL19+DL20 mean NDCG@10 using native query-pruning mechanisms: C++ MR 0.10, BMP β=0.5, and SEISMIC query_cut=5.This provides the deep-judgment validation setup for the re-ranking and pruning comparisons.

D TREC DL 2019/2020 Deep-Judgment Validation · GenAI Usage Disclosure

Deep-judgment evaluation on TREC DL 2019/2020 confirms that index-side pruning is lossless and sharpens the query-pruning regime split. The paper also discloses limited GenAI use for polishing author-written text only.

  • D TREC DL 2019/2020 Deep-Judgment Validation: ∼210 documents per query are pooled and graded on a 0–3 relevance scale in TREC DL 2019/2020, unlike MS MARCO’s one or two labeled passages.The validation is reported in the CIKM ’26 paper scheduled for November 07–11, 2026, in Rome, Italy.
  • D TREC DL 2019/2020 Deep-Judgment Validation: Index-side pruning is lossless under the deeper TREC DL 2019/2020 judgments.Table 9 confirms this main-text finding.
  • D TREC DL 2019/2020 Deep-Judgment Validation: The query-pruning regime split becomes sharper under deep judgments: free on SPLADE but catastrophic on V3-GTE.This is reported as confirmation of the main-text findings.
  • D TREC DL 2019/2020 Deep-Judgment Validation: ∼210 judged documents per query make genuine quality loss detectable, providing a stronger validation setting than one or two labels.The passage contrasts TREC DL’s deep pooling and graded judgments with MS MARCO’s sparse labels.
  • GenAI Usage Disclosure: Generative AI tools polished author-written text only; they did not generate ideas, design experiments, analyze results, write code, or create figures or tables.The authors reviewed all affected text.
  • GenAI Usage Disclosure: The authors take full responsibility for the paper and all affected text.This responsibility statement accompanies the GenAI usage disclosure.
Loading 2608.16309v1…