Source-linked AI summary
Working Notes on Late Interaction Dynamics: Analyzing Targeted Behaviors of Late Interaction Models
Antoine Edy, Max Conti, Quentin Macé
TL;DR
Late-interaction retrieval has strong performance but understudied dynamics involving length bias and similarity information beyond MaxSim’s top score. The paper analyzes these behaviors in state-of-the-art models on NanoBEIR and finds empirical causal length bias, extreme-length vulnerability in bi-directional models, and no significant post-top-1 similarity trend.
Problem
Late-interaction retrieval’s length bias and similarity distribution beyond MaxSim’s top score remain insufficiently studied.
Method
The paper analyzes these two behaviors in state-of-the-art late-interaction models using small-scale experiments on NanoBEIR.
Results
Causal multi-vector length bias holds empirically, bi-directional models remain vulnerable at extreme lengths, and no significant similarity trend emerges beyond the top-1 document token.
Takeaways & Limitations
The findings support using MaxSim for current standard retrieval benchmarks while motivating stronger bi-directional models for late interaction.
Takeaways & Limitations
Future analyses should test length bias in controlled settings and token-score distributions across broader tasks because training recipes can affect these behaviors.
Abstract
from arXiv · showhide
While Late Interaction models exhibit strong retrieval performance, many of their underlying dynamics remain understudied, potentially hiding performance bottlenecks. In this work, we focus on two topics in Late Interaction retrieval: a length bias that arises when using multi-vector scoring, and the similarity distribution beyond the best scores pooled by the MaxSim operator. We analyze these behaviors for state-of-the-art models on the NanoBEIR benchmark. Results show that while the theoretical length bias of causal Late Interaction models holds in practice, bi-directional models can also suffer from it in extreme cases. We also note that no significant similarity trend lies beyond the top-1 document token, validating that the MaxSim operator efficiently exploits the token-level similarity scores.
1. Introduction
The paper examines underexplored dynamics behind late-interaction retrieval, focusing on length bias and similarity information discarded beyond MaxSim’s top score. Small-scale NanoBEIR experiments analyze how current state-of-the-art models behave along these axes.
- Late-interaction models use token-level interactions for finer semantic matching, but their underlying retrieval dynamics remain insufficiently studied.
- The analysis studies length bias in multi-vector MaxSim scoring, where causal encoders can favor longer chunks regardless of relevance.
- It also examines whether document-token similarity information beyond each query token’s highest score contains useful retrieval trends.
- Small-scale experiments on the NanoBEIR benchmark evaluate current state-of-the-art models along both behavioral axes.
2. Length Bias In Multi-Vector Retrieval
The section explains why causal multi-vector MaxSim scoring creates length bias and evaluates how architectural choices affect it. Causal multi-vector models show the strongest bias, while bi-directional models reduce but do not eliminate it at extreme lengths.
- Theoretical Motivation: Appending tokens to causal multi-vector chunks creates a strict monotonic bias because each query-token maximum can only increase or remain constant.
- Theoretical Motivation: Bi-directional models theoretically avoid strict length bias because added tokens alter preceding representations, whereas single-vector models use fixed-length aggregation.
- Multi-Vector Architectures Induce A Length Bias: Figure 1 compares false-positive, relevant-document, and corpus-average lengths across quantiles defined by relevant-document length.The global corpus mean is 199 tokens.
- Multi-Vector Architectures Induce A Length Bias: The causal multi-vector model retrieves false positives substantially longer than relevant documents, while the single-vector model tracks relevant-document length more closely.
- Bi-Directional Models Mitigate But Do Not Eliminate Bias: Figure 2 measures expected nDCG loss from adding chunks in length quantile bins against a random baseline and its 90% confidence interval.
- Bi-Directional Models Mitigate But Do Not Eliminate Bias: Causal multi-vector models show near-monotonic bias, single-vector models show no significant bias, and bi-directional multi-vector models remain vulnerable at length extremes.
3. Similarity Distribution: What Happens Beyond The Top-1 Document Token
The analysis tests whether document-token similarity patterns beyond MaxSim’s top-1 token can distinguish failed positive documents from negatives. Apart from a dataset-specific NanoArguAna pattern, no consistent trend appears across NanoBEIR.
- MaxSim keeps only the highest document-token similarity for each query token, discarding token-count and similarity-density information.
- The study compares sorted token scores for unretrieved positives, the top-1 negative, the negative below the positive, and the worst negative on failed queries.
- On NanoArguAna, the positive document surpasses the top-1 negative beyond the first tokens, beginning around 10%.
- Across NanoBEIR, no average similarity trend beyond the top-1 token emerges, and successful retrieval samples show similarly undifferentiated distributions.
4. Conclusion And Future Work
The paper identifies strict length bias in causal multi-vector architectures and finds that bi-directional models mitigate it without fully removing it. It also finds no significant similarity trends beyond the top-1 document token and outlines directions for controlled and broader analyses.
- Causal multi-vector architectures exhibit strict length bias, while bi-directional models mitigate but do not eliminate it.
- Standard retrieval benchmarks show no significant similarity trends beyond the top-1 document token that current models could exploit beyond MaxSim.
- Future work should test length bias in controlled synthetic settings that independently adjust text length and semantic relevance.
- Future analyses should examine token-score distributions across broader tasks and newly released models because training recipes can strongly affect these behaviors.
- Potential mitigation studies could intervene during training, indexing, or similarity-operator design.
A.1. Datasets
The study merges NanoBEIR’s 13 datasets to create a broad chunk-length distribution for evaluating retrieval behavior.
- A.1. Datasets: NanoBEIR contributes 13 diverse datasets with 50 queries each, pooled into one corpus before retrieval.After removing five outlier chunks, the corpus contains 56,718 chunks and 649 queries.
- A.1. Datasets: The merged corpus’s chunk-length distribution is summarized in Figure 4.
A.2. Setup
The evaluation compares four model configurations formed by crossing encoder architecture with pooling strategy, using tokenizer-derived chunk lengths.
- A.2. Setup: Four configurations combine causal or bi-directional encoders with single-vector or multi-vector pooling.
- A.2. Setup: Chunk sizes are computed with the jina-embeddings-v4 Byte-Pair Encoding tokenizer inherited from Qwen2.5-VL-3B-Instruct.
- A.2. Setup: Table 1 summarizes each evaluated model’s pooling strategy, architecture, and size.
B. Retrieval Errors by Chunk Length
Retrieval errors generally increase with chunk length, but the pattern depends on architecture: causal multi-vector models show monotonic growth, whereas bi-directional models peak at length extremes.
- B. Retrieval Errors by Chunk Length: The causal multi-vector model jina-embeddings-v4 is the only configuration with a strictly monotonic increase in errors from zero.This corroborates the length bias predicted for causal multi-vector models.
- B. Retrieval Errors by Chunk Length: Bi-directional models show non-monotonic error distributions with distinct peaks for very short and very long chunks.These peaks indicate sensitivity at the extremes of the length spectrum.
- B. Retrieval Errors by Chunk Length: Figure 5 maps the absolute volume of retrieval errors to document chunk lengths.Its dashed line represents the no-bias expected baseline within a 90% variance interval.
- B. Retrieval Errors by Chunk Length: Both bi-directional models produce many errors despite strong overall nDCG, indicating severe failures despite good average ranking.The contrast is associated with their substantially smaller parameter count than the causal models, 0.15B versus 4B.
C. Similarity Distribution For jina-embeddings-v4
Figure 6 compares jina-embeddings-v4 token similarities for positive and top-1 negative documents on failed queries.
- C. Similarity Distribution For jina-embeddings-v4: On failed queries, the positive document remains worse than the top-1 negative across all document tokens.The positive document also has a larger distance to the top-1 negative.