Source-linked AI summary
MICE to Meet You: From Re-ranking Only to End-to-End Retrieval
Victor Morand, Yuxuan Zong, Mathias Vast, Josiane Mothe, Benjamin Piwowarski
TL;DR
Late-interaction models enable efficient indexed retrieval, but their constrained interactions can limit effectiveness relative to cross-encoders, and MICE’s document representations lacked a practical storage and search solution. This paper introduces MICE-P, adapting MICE to PLAID storage and evaluating comparable models under matched training conditions. MICE-P reaches a new efficiency–effectiveness trade-off, with latency approaching ColBERT and effectiveness like cross-encoders, while compression reduces effectiveness and increases storage.
Problem
MICE had precomputable document representations but no practical way to store and search them over a collection, leaving its end-to-end retrieval use unexplored.
Source: §1 · p. 2Method
MICE-P adapts MICE with a low-dimensional document bottleneck compatible with PLAID and trains MICE models by transferring from cross-encoders.
Source: §2 · p. 2, §2.1 · p. 3, §2.2 · p. 3Results
MICE-P occupies a new efficiency–effectiveness trade-off region, combining cross-encoder-like effectiveness with latency approaching ColBERT.
Source: §5 · p. 7Takeaways & Limitations
MICE-P is an end-to-end retrieval architecture that precomputes and compresses document representations, then scores SPLADE-retrieved candidates with MICE.
Source: §5 · pp. 6–7PLAID compression reduces MICE-P effectiveness and increases its storage footprint.
Source: §5 · p. 7
Abstract
from arXiv · showhide
Late-interaction models offer state-of-the-art efficiency-effectiveness trade-offs for neural retrieval by enabling document representations to be pre-computed and efficiently indexed. Yet, their relatively simple interaction mechanisms, typically MaxSim, can limit their effectiveness compared to cross-encoders. MICE, a recently introduced re-ranking architecture, aims to combine cross-encoder-like effectiveness through expressive transformer-based scoring with reduced inference-time computation through document-side pre-computation. However, no practical way to store these document representations was explored. We address this limitation here and introduce MICE-P, the first end-to-end retrieval pipeline with MICE, based on PLAID to index the contextualized document token representations. We conduct a controlled comparison of MICE, ColBERT and cross-encoders by training comparable models using the Ettin Family under matched experimental conditions. Our results show that MICE-P defines a new class of retrieval architecture with an efficiency-effectiveness trade-off of its own: it matches late-interaction retrieval throughput while retaining the effectiveness of cross-encoders.
1 Introduction
Late-interaction retrieval can precompute document representations for efficient retrieval, but its interactions are more constrained than cross-encoder scoring. MICE seeks a middle ground, while practical storage and search for its fine-grained representations remained unresolved.
1 Introduction
- Late-interaction models such as ColBERT precompute document token representations and use PLAID for indexed retrieval, but tend to be less effective than cross-encoders [20].The introduction frames the gap as expressive but costly cross-encoder scoring versus efficient but more constrained late-interaction scoring.Source: §1 · p. 1
- MICE separates document/query contextualization from frozen-document interaction, retaining richer interactions than MaxSim while reducing inference-time computation.The introduction also situates MICE among mid-fusion approaches and notes KaLM-Reranker as another precompute-then-interact design.Source: §1 · p. 2
- MICE’s document-side precomputation had not been tested as full-collection storage and search, and PLAID’s transfer from MaxSim to MICE was uncertain.PLAID retrieval is tightly coupled to MaxSim, whereas MICE uses attention-based interactions.Source: §1 · p. 2
- MICE-P adapts MICE to PLAID storage, establishing an architecture intended to improve the efficiency–effectiveness trade-off over late-interaction scorers.The paper asks whether MICE can support efficient end-to-end retrieval and whether ModernBERT-based MICE can match comparable cross-encoders and late-interaction models.Source: §1 · p. 2
2 MICE-P(LAID)
MICE-P adapts MICE for PLAID-compatible document storage and transfers cross-encoder models into the MICE architecture. Its bottleneck is designed to preserve interaction effectiveness while reducing the stored representation dimension.
2 MICE-P(LAID)
- MICE-P combines PLAID storage with a transfer strategy from trained cross-encoders, enabling ModernBERT-based MICE models to retain cross-encoder effectiveness.The bottleneck maps contextualized document representations into a PLAID-compatible space; the authors report no impact on interaction-layer effectiveness.Source: §2 · p. 2
2.1 From MICE to MICE-P: Using PLAID to Store Documents
- A 128-dimensional projection and LayerNorm make MICE document vectors compatible with PLAID storage, and up-projection restores model dimension for interaction.MICE-P-61m reaches 57.8 average BEIR13 nDCG@10, versus 58.0 for MICE-61m; it uses a separate DistilBERT-SPLADEv3 first-stage retriever rather than PLAID retrieval.Source: §2.1 · p. 3
2.2 Transferring Cross-Encoders to MICE
- Cross-encoder transfer uses two fine-tuning phases: adapt weights to MICE, then train a bottleneck with interaction layers while freezing contextualization layers.The authors use the same comparison protocol for ColBERT counterparts and train on 143M teacher-scored query–passage pairs.Source: §2.2 · p. 3
3 Experimental Setup
The experiments compare Ettin-based retrieval models on decontaminated BEIR, Nano-BEIR, and LoTTE-Search, with input lengths and retrieval stages controlled. Quantization is selected to balance ranking effectiveness and residual storage.
Models and tokenization
- Ettin-68m guides design choices, with Ettin-150m extending comparisons to LateOn; queries and documents are capped at 32 and 512 tokens.MICE-P retains the original MICE layer counts for each backbone.Source: §3 · p. 4
Evaluation
- Evaluation uses decontaminated BEIR13 and Nano-BEIR, plus LoTTE-Search for out-of-domain testing; MICE-P reranks DistilBERT-SPLADEv3’s top 100 passages.ColBERT variants can instead use PLAID as a first stage.Source: §3 · p. 4
PLAID Configuration
- At 2 bits, LateOn’s mean nDCG@10 drop is approximately −0.002, while MICE-P’s exceeds −0.03, so the chosen settings are 2 bits and 4 bits, respectively.MICE-P remains close to 8-bit effectiveness at 4 bits, which halves residual storage; its 2-bit drop reaches −0.05 on FiQA and ArguAna.Source: §3 · p. 4
4 Results
MICE-P reaches a comparable efficiency–effectiveness trade-off to leading late-interaction retrieval, while its memory footprint includes both SPLADE and PLAID indexes. Across benchmarks, ModernBERT MICE remains close to cross-encoders and generally exceeds comparable ColBERT models.
Baselines
- ModernBERT MICE is within 0.3 nDCG@10 on BEIR13 and within 1 on LoTTE-Search of cross-encoders, without PLAID storage.MICE models exceed ColBERT counterparts by more than 1 nDCG@10 on BEIR13 and match or beat them on LoTTE-Search.Source: §4 · p. 5, Table 1 · p. 5
- Adding the MICE-P bottleneck and PLAID storage changes effectiveness by less than 0.7 nDCG@10 on BEIR13 and around 1 point on LoTTE-Search.The reported comparison is relative to MICE models without PLAID storage.Source: §4 · p. 5
ColBERT Collapse
- ColBERT checkpoints collapse with PLAID as first stage, while original LateOn loses less than 1.6 nDCG@10 points.The authors suggest Pearson pointwise loss may explain the difference, noting that InfoNCE and hard negatives in LateOn may produce better-separated document vectors.Source: §4 · pp. 5–6
MICE-P is Pareto Optimal
- MICE-P reaches a comparable efficiency–effectiveness trade-off to LateOn when latency includes SPLADE retrieval, PLAID decompression, and MICE scoring.MICE-P-130m is reported as an alternative to LateOn-149m or Ettin-reranker-150m, trading latency or effectiveness depending on the comparison.Source: §4 · p. 6, Figure 2 · p. 6
Storage
- MICE-P’s storage footprint is higher because it combines a SPLADE retrieval index with a PLAID document store, though the final size is described as not excessively higher.LateOn uses PLAID for both storage and first-stage retrieval.Source: §4 · p. 6, Figure 2 · p. 6
5 Conclusion
ModernBERT-based MICE models match Ettin-reranker cross-encoder effectiveness at roughly 60% of their FLOPs, and MICE-P extends MICE to end-to-end retrieval. MICE-P combines cross-encoder-like effectiveness with latency approaching ColBERT, though PLAID compression reduces effectiveness and increases storage footprint.
5 Conclusion
- ModernBERT-based MICE models match Ettin-reranker cross-encoder effectiveness at ∼60% of their FLOPs, while MICE-P reaches latency approaching ColBERT.MICE-P’s resulting latency gains place it in a new region of the Pareto frontier.Source: §5 · pp. 6–7
- MICE-P pre-computes and compresses contextualized document representations with PLAID, then computes query representations to interact with SPLADEv3-retrieved candidates.Source: §5 · p. 7
- PLAID compression reduces MICE-P effectiveness and increases its storage footprint despite the latency gains.Source: §5 · p. 7
- Future work will investigate better document-representation compression or direct retrieval with PLAID to improve MICE-P’s efficiency-effectiveness trade-off.Source: §5 · p. 7