Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval

• Independent query encoding leaves representations unable to adapt to candidate items that could clarify omitted, implicit, or unevenly weighted cues.Source: §1 · p. 1
• ARR alternates item retrieval with query-embedding updates, then uses the final embedding to rank the collection while keeping item embeddings independently indexable.Source: §1 · p. 2
• SFT teaches feedback interpretation through stepwise contrastive supervision, while RL optimizes selection of feedback items for final retrieval quality.Source: §1 · p. 2
• ARR achieves the highest average performance among compared methods across M-BEIR and seven zero-shot benchmarks.Source: §1 · p. 2
• Feedback improves retrieval during inference, and training with feedback also improves first-step retrieval before any item is observed.Source: §1 · p. 2

• Universal multimodal retrieval work improves representation learning or candidate ranking, whereas ARR focuses on using retrieved content to inform subsequent query representations.Source: §2 · p. 2
• Prior feedback methods refine dense queries or rewrite them, while ARR’s encoder consumes retrieved items and selects the next item for its effect on final ranking.Source: §2 · pp. 2–3
• Reasoning-enhanced retrieval enriches inputs, and prior RL methods optimize rewriting or synthesized representations; ARR instead samples item identities and optimizes selections for downstream retrieval quality.Source: §2 · p. 3

• UMR ranks corpus items for queries that may contain text, images, or both, using a shared retrieval model and annotated relevant items for training.Source: §3 · p. 3
• The MLLM-based encoder processes templated text and image tokens, reads out a fixed-dimensional embedding, and normalizes it for similarity comparison.Source: §3 · p. 3
• InfoNCE training increases the positive item’s similarity relative to negative items in the query’s contrastive set.Source: §3 · p. 3

4.1 Autoregressive Retrieval

• ARR encodes the original query, iteratively selects unselected items by embedding similarity and appends their content, then uses the final query embedding to rank the corpus.Feedback items are not relevance-filtered, and a history mask prevents selecting the same item twice.Source: §4.1 · pp. 3–4

4.2 Supervised Fine-Tuning

• SFT applies relevant-item contrastive supervision at every query state, but only the initial-step loss updates standalone item embeddings.A stepwise degradation penalty discourages relative loss increases; SFT provides relevance supervision without expert labels for feedback selection.Source: §4.2 · p. 4

4.3 Reinforcement Learning for Retrieval

• RL samples current-policy feedback trajectories and trains item selection from terminal reciprocal-rank advantages; final-state contrastive supervision complements the detached reward.The auxiliary loss is gated to trajectories below a reward threshold, concentrating direct supervision on poorer final rankings.Source: §4.3 · pp. 4–6
• RL freezes the SFT backbone and trains a query-side LoRA adapter while standalone item embeddings remain fixed, avoiding index rebuilds during optimization.The adapter operates on query contexts, including feedback items, but not on standalone corpus items.Source: §4.3 · p. 5

5.2 In-Domain Evaluation

• On M-BEIR’s 16 dataset–task combinations, ARR-RL achieves the highest average at both scales: 56.7 at 2B and 61.0 at 8B.At 8B, its average exceeds TRACE’s 58.8 by 2.2 points.Source: §5.1 · p. 6, §5.2 · p. 6, Table 1 · p. 7

5.3 Analysis of the Iterative Retrieval Process

• ARR-SFT_N=1 drops from 55.23 at the initial state to 46.90 at four inference states, while ARR-SFT_N=4 rises from 55.63 to 56.01.Most of the N=4 gain appears after the first feedback item, at Iter-2 (55.95), with smaller changes thereafter.Source: Table 2 · p. 7, §5.3 · p. 7
• Before feedback, ARR-SFT_N=4 outperforms ARR-SFT_N=1 by 0.40 points, showing that feedback-conditioned training also benefits initial retrieval.The reported performance effect does not establish a particular feature-learning mechanism.Source: §5.3 · p. 7
• ARR-RL_N=4 scores 56.36 at Iter-1 and 56.74 at Iter-4, versus 55.63 and 56.01 for its SFT initialization.The authors attribute the gains to RL’s credit assignment for an item’s contribution to eventual retrieval; ARR-RL_N=2 reaches 56.40 at Iter-1 versus 55.44 for ARR-RL_N=1.Source: §5.3 · p. 7
• Four-state training yields the best final score, but its Iter-4 advantage over two-state training is only 0.10 points, with marginal gains beyond Iter-2.The authors describe Iter-2 as a favorable balance between test-time scaling and retrieval performance.Source: §5.3 · pp. 7–8

5.4 Zero-Shot Evaluation

• On seven zero-shot benchmarks, ARR-RL records the highest average, 79.01, versus 74.10 for the strongest baseline average.Its scores include 77.9 on Visual Dialog and 66.0 on Multi-round FashionIQ.Source: Table 3 · p. 8, §5.4 · p. 8

5.5 Ablation Study

• Item gradients lower the final score from 56.01 to 55.71; removing the degradation penalty leaves initial performance nearly unchanged but lowers the final score to 55.68.Initial scores are 55.69 without the penalty and 55.63 with it.Source: §5.5 · p. 8
• Removing final-embedding contrastive loss lowers the final score from 56.74 to 56.43, while removing GRPO yields 55.29, below both the full RL model and its SFT initialization.The authors report that policy training and direct terminal supervision are complementary in this setting.Source: §5.5 · p. 8

• ARR learns to select and use retrieved items as feedback through an autoregressive process while preserving independently indexed item embeddings.SFT teaches feedback interpretation, and RL optimizes item selection for the final ranking outcome.Source: §6 · p. 9
• Experiments show improvements in average retrieval performance on both in-domain and zero-shot evaluations.Source: §6 · p. 9
• Feedback at inference improves subsequent retrieval, while training with feedback improves initial retrieval before any feedback is observed.Source: §6 · p. 9

A.1 Benchmarks

• M-BEIR contains 16 dataset–task combinations from 10 datasets, spanning eight query–item modality configurations and four domains.The domains are news, fashion, Wikipedia, and miscellaneous content; M-BEIR is cited as Wei et al. (2024).Source: §A.1 · p. 12
• The seven zero-shot benchmarks cover image–caption, image-and-text-conditioned, dialogue-to-image, and multi-turn image–text retrieval without task-specific fine-tuning.ShareGPT4V and Urban-1K use images and long captions; Flickr30K evaluates image–caption retrieval; CIRCO and GeneCIS condition on image and text.Source: §A.1 · p. 12
• Several zero-shot benchmarks reuse images from COCO or FashionIQ, so this protocol does not imply that all visual content is unseen.Source: §A.1 · p. 12

A.2 Implementation and Training

• ARR uses 2B and 8B Qwen3-VL-Embedding backbones with frozen visual encoders; SFT fine-tunes remaining backbone parameters, while RL freezes that backbone and trains a query-side LoRA adapter.The adapter is active for queries and feedback contexts, but inactive for standalone corpus items.Source: §A.2 · p. 12
• SFT samples three items without replacement from a pretrained top-10 feedback pool to create four query states, without adding missing positives or filtering feedback by relevance labels.The annotated positive is included separately in contrastive supervision; training lasts two epochs, with the degradation penalty enabled in the second.Source: §A.2 · p. 12
• RL uses 32K queries across 16 M-BEIR dataset–task combinations and a query-specific pool built from the SFT model’s top 100 results plus the annotated positive.The pool supports feedback sampling, reward computation, and final-step contrastive supervision; unless stated otherwise, training and inference use N=4.Source: §A.2 · p. 12
• For multiple-positive queries, SFT randomly selects one positive target and masks the others, whereas RL averages final-embedding losses over positives and max-pools their rewards.SFT takes 24 hours for 2B and 48 hours for 8B; RL takes 8 hours for 2B and 24 hours for 8B on NVIDIA H20 GPUs.Source: §A.2 · p. 12

A.3 Baselines

• MM-Embed adapts an MLLM for universal multimodal retrieval through contrastive learning and modality-aware hard-negative mining.Text retrieval data provides additional supervision.Source: §A.3 · p. 12
• LamRA-Ret learns multimodal embeddings through language-only pretraining and multimodal instruction tuning as LamRA’s retrieval component.LamRA also has a separate reranking component, but the comparison uses its retrieval component.Source: §A.3 · p. 12
• TRACE integrates reasoning and representation learning in an MLLM, adapting its reasoning behavior to retrieval before embedding.Source: §A.3 · p. 12
• ELVA generates a textual synthesis before constructing embeddings and uses ranking and margin rewards for RL.Its policy training targets retrieval-oriented representations, unlike ARR’s policy over feedback items.Source: §A.3 · pp. 12, 14
• Table 5 summarizes M-BEIR tasks and rounded benchmark statistics, with split sizes counting queries and pool sizes counting local evaluation candidates.The statistics are not counts after ARR-specific preprocessing.Source: Table 5 · p. 13
• Table 6 lists zero-shot evaluation tasks, metrics, query counts, and candidate-pool counts, with counts separated when retrieval directions differ.GeneCIS uses query-specific candidate sets.Source: Table 6 · p. 13

Final-step contrastive supervision

• 56.74 average recall at Iter-4 with λret=0.1 exceeds 56.43 at λret=0 and 56.02 at λret=1.0.The intermediate weight performs best among tested settings; policy training with λret=0 reaches 56.43, above the SFT score of 56.01.Source: Table 9 · p. 14, B Hyperparameter Analysis · p. 14

Degradation penalty

• With λdeg=0, average recall is 55.69 at Iter-1 and 55.68 at Iter-4; with λdeg=0.1, it rises from 55.63 to 56.01.The 0.1 setting gives the best final score among tested weights; λdeg=1.0 improves across inference states but reaches a lower final score of 55.89.Source: Table 9 · p. 14, B Hyperparameter Analysis · p. 14

Hyperparameter analysis

• The default configuration uses λdeg=0.1 and λret=0.1.Table 9 examines these two weights using ARR-2B.Source: B Hyperparameter Analysis · p. 14