Source-linked AI summary
Embedding Surgery: Localized Updates for Adaptive Ranking Correction in Dense Retrieval
Maddalena Amendola, Antonio Mallia, Raffaele Perego
TL;DR
Dense retrieval struggles to adapt because document embeddings are static after offline indexing. The paper introduces embedding surgery, which makes localized, constrained updates to selected document embeddings using feedback, and reports consistent gains across benchmarks while preserving scalable retrieval behavior.
Problem
Static document embeddings make dense retrieval difficult to adapt to user feedback, evolving topics, and specific ranking errors.
Method
Embedding surgery solves a constrained convex optimization problem that minimally updates selected document embeddings to satisfy feedback-derived ranking constraints.
Results
Embedding surgery consistently improves ranking across seven benchmarks, with up to +60.64% relative nDCG@10 and preserved effectiveness under noisy feedback and distribution shift.
Takeaways & Limitations
Localized updates can propagate corrections to semantically related queries, complement CoRocchio, and support safe in-place updates without costly index reconstruction.
Takeaways & Limitations
Future work must address inconsistent feedback, sensitivity to the margin 𝜖 and surgery cutoff, realistic interaction logs, and additional baselines.
Abstract
from arXiv · showhide
Dense retrieval systems are core components of modern search engines, recommendation platforms, and retrieval-augmented generation pipelines. They encode documents and queries into dense embeddings, enabling efficient semantic search via vector similarity. However, because document embeddings are computed offline and stored in static indexes, these systems struggle to adapt to user feedback or evolving search intent. To address this limitation, we introduce \emph{embedding surgery}, a lightweight approach for adaptive ranking correction in dense retrieval. The method applies localized, minimal updates to selected document embeddings at query time, guided by editorial feedback, user interactions, or pseudo-labels from large language models. We formulate embedding surgery as a convex optimization problem that enforces ranking constraints while minimizing modifications to the affected document representations. We integrate embedding surgery into standard dense retrieval pipelines and evaluate it on TREC Deep Learning, TREC Robust, TREC CAsT, and MS MARCO benchmarks. Results show consistent improvements (e.g., up to +60.64\% relative improvement in nDCG@10 on DL-Hard under editorial feedback), even under noisy or shifting feedback, with low computational cost and without disrupting the global structure of the embedding space. Extensive experiments show that ranking corrections propagate to semantically related queries and that embedding updates can be applied safely and efficiently to scalable Approximate Nearest Neighbor indexes via simple in-place overwriting, without requiring costly index reconstruction. Finally, embedding surgery complements query adaptation methods such as CoRocchio, yielding additional gains while being more robust to noisy feedback.
1 Introduction
Embedding surgery addresses the rigidity of static dense-retrieval embeddings by making localized, feedback-driven updates at query time. It formulates these corrections as constrained optimization and reports broad benchmark gains with low-cost, index-compatible deployment.
- Static document embeddings make it difficult to incorporate feedback or adapt to evolving topics and query-specific ranking errors.
- Embedding surgery directly modifies selected document embeddings to correct ranking errors without retraining models or rebuilding indexes.
- Editorial judgments, user interactions, and LLM-derived supervision guide localized updates for ranking correction.
- A convex quadratic program adjusts selected embeddings to satisfy ranking constraints while keeping them close to their original representations.Single pairwise constraints have closed-form solutions, while batches can be solved with standard optimization solvers.
- +60.64% relative nDCG@10 is achieved at best, with consistent gains across multiple benchmarks and robustness to noisy feedback.Corrections also transfer to semantically related queries, complement CoRocchio, and support in-place updates in HNSW and IVF indexes.
- The work combines localized optimization, heterogeneous feedback, broad evaluation, and large-scale ANN-index analysis into a practical adaptive-ranking framework.The reported evaluation covers four dense retrieval models and seven IR benchmarks, including 56k-query ANN experiments.
2 Related work
Embedding surgery differs from prior work by targeting short-term, query-specific ranking errors through localized document-embedding updates. Related approaches instead emphasize novelty, online ranking adaptation, retriever maintenance, or query-representation adaptation.
- Embedding surgery is presented as a new post-hoc direction based on constrained, minimal modifications to document embeddings.
- Related research spans online learning-to-rank, retriever updating under distribution shift, and user-feedback-based query adaptation.
- Online learning to rank: Online learning-to-rank continuously updates ranking functions through interaction, whereas embedding surgery assumes a trained retriever and corrects residual query-specific errors.Embedding surgery operates directly on document embeddings without exploration or sustained interaction.
- Updating/retraining retrievers: Retriever-maintenance methods address long-term adaptation under distribution shift, while embedding surgery targets short-term correction without retraining or re-indexing.
- Improving query representations: Query-adaptation methods modify query representations from interaction signals, forming a complementary line of work to document-embedding surgery.CoRocchio combines the original query with clicked-document embeddings using click-frequency and position-bias adjustments.
3 Embedding Surgery
Embedding surgery corrects query-specific ranking errors by minimally updating retrieved document embeddings under feedback-derived constraints, without retraining or rebuilding indexes. Its formulation supports multiple update variants and feedback sources, including editorial judgments, user interactions, and LLM-based supervision.
- Embedding Surgery: The method scores documents by query–document similarity and updates retrieved embeddings so preferred documents outrank non-preferred documents by a specified margin.Feedback consists of pairwise preferences among documents in the retrieved set.
- Embedding Surgery: The optimization minimizes L2 movement from original embeddings while enforcing ranking constraints, producing a convex quadratic program with an efficiently solvable global optimum.The margin parameter separates the relevance scores of preferred and non-preferred documents.
- Embedding Surgery: The framework supports symmetric, demotion, and promotion variants that determine which document embeddings may change during a pairwise correction.Symmetric updates move both embeddings, whereas demotion or promotion fixes one side of the pair.
- Feedback Models: Feedback sources include editorial judgments, simulated user interactions, and LLM-based relevance assessments for identifying ranking errors.Editorial feedback supplies expert judgments, while counterfactual click models simulate interaction signals and LLMs help when explicit feedback is sparse.
- Feedback Models: The target ranking is transformed into the model ranking through adjacent swaps, with Kendall Tau distance counting the discordant pairs requiring correction.Each swap induces a local constraint that reverses the relative order of two adjacent documents.
4 Experimental Settings
The experiments evaluate embedding surgery across dense retrieval benchmarks, models, metrics, feedback settings, and deployment configurations. They examine effectiveness, latent-space impact, scalability, and complementarity with query adaptation.
- Feedback Simulation: The click-feedback simulation examines user interactions sequentially through rank 20, with clicks generating pairwise preferences between clicked documents and immediately preceding results.The reported configuration removes position bias by setting examination probability to 1 at all positions.
- Research Questions: The research questions test effectiveness under varied feedback, latent-space effects, ANN scalability without costly re-indexing, and complementarity with CoRocchio.The design also asks whether ranking corrections remain stable across datasets, models, and noisy or conflicting feedback.
- Datasets: The evaluation covers TREC Deep Learning, DL-Hard, Robust ’04, and other dense-retrieval benchmark settings using multiple datasets and relevance regimes.The setup includes DL 2019–2020, DL-Hard, and Robust ’04, with additional benchmark coverage specified in the experimental design.
- Dense Retrieval Models: Experiments compare Contriever, TAS-B, multilingual E5, and Snowflake-Arctic Embed in zero-shot settings without task-specific fine-tuning.The models represent different dense semantic representations and are evaluated from publicly released checkpoints.
- Evaluation Metrics: Retrieval quality is measured with nDCG@k, RR, or MRR, while surgery accuracy and modification effects are assessed with Kendall Tau distance and Swap Accuracy.The metrics separately capture ranking quality, pairwise ordering accuracy, and the impact of embedding modifications.
- Implementation Details: For each query, the implementation retrieves the top-20 results and applies surgery within FAISS-based dense retrieval pipelines, including Flat, IVF, and HNSW indexes.The optimization uses CVXPY, and the ANN experiments test whether in-place updates remain practical at scale.
5 Results and Analysis
Across feedback sources and benchmarks, embedding surgery consistently improves ranking while producing small, localized updates, preserving retrieval structure, transferring benefits to related queries, and remaining computationally practical.
- Overall effectiveness: Symmetric surgery generally outperforms promotion and demotion variants, except on Robust ‘04 where it performs similarly to demotion.
- Feedback robustness: Noisy click feedback remains close to perfect-user performance on DL ‘19 and DL ‘20, while LLM-based feedback improves rankings despite occasional local performance drops.
- Overall effectiveness: Up to +60.64% relative nDCG@10 improvement is obtained for TAS-B on DL-Hard with editorial feedback, while gains remain consistent across models, datasets, and feedback strategies.
- Feedback robustness: +25.98% is the highest gain for perfect-user click feedback, and improvements remain effective on challenging DL-Hard and out-of-domain Robust ‘04 queries.
- Optimization behavior: LLM-generated rankings produce longer constraint lists than editorial judgments, increasing optimization complexity and slightly lowering swap accuracy.
- Latent-space preservation: Update magnitudes are typically below 0.05, indicating minimal localized modifications, while held-out benchmark performance remains statistically stable after large-scale surgery.
- Transfer to related queries: Surgery improves target CAsT queries while usually leaving related queries unchanged, and QSharedRel effects are positive or neutral in almost all cases.
- Practicality and scalability: Top-20 result overlap stays above 19 documents, and average update times are 7.9×10^-3 seconds per click-based constraint and 3.6 × 10^-3 seconds per editorial constraint.
6 Conclusion
Embedding surgery enables real-time ranking correction through localized, feedback-driven updates while preserving the broader embedding space. Experiments report consistent effectiveness gains, low computational overhead, and efficient ANN updates, while identifying several directions for future work.
- Embedding surgery performs real-time ranking correction through lightweight, feedback-driven updates of document representations.The framework supports fine-grained query-time modifications guided by user interactions or human/LLM assessments.
- Experiments on seven IR benchmarks show consistent ranking improvements with negligible computational overhead and generalization under distribution shift.
- In-place overwriting updates scalable ANN indexes with negligible impact on index structure and navigation, avoiding costly reorganization.
- Future work will examine sensitivity to the margin 𝜖 and surgery cutoff, inconsistent feedback, realistic interaction logs, and additional baselines.
GenAI Usage Disclosure
The authors disclose using GenAI tools for language editing and occasional code support, while stating that these tools did not generate core research outputs.
- GenAI tools supported language editing and occasional code development and debugging, with all suggestions reviewed and validated by the authors.
- The authors state that GenAI tools did not generate experimental results, data, citations, or the core scientific contributions.
- The authors retain full responsibility for the paper’s content and results.