PaperlayerView exact revision on arXiv

Source-linked AI summary

RenderRank: Learning to Rerank Text with Compressed Visual Tokens

Seongtae Hong, Youngjoon Jang, Jungseob Lee, Hyeonseok Moon, Heuiseok Lim

arXiv:2609.35069v1cs.IR

TL;DR

Reranking many candidate documents makes input length consequential, yet shortening documents can omit relevance evidence. RenderRank scores textual queries against compressed visual document representations, aligning scores with a text-based teacher before refining within-query priorities. It achieves strong ranking quality with fewer tokens across BEIR and long-document datasets, with higher average throughput on the latter.

  • Problem

    Reranking costs accumulate across candidates, but limiting input length to reduce computation can exclude evidence needed to assess relevance.

    Source: §1 · p. 1

  • Method

    RenderRank encodes rendered document images as visual tokens, then trains query-conditioned relevance scoring by matching text-teacher scores and refining positive-versus-negative priorities within each query.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across 11 BEIR datasets, RenderRank averages NDCG@10 of 55.96 with 16.5–35.5% fewer input tokens; across four long-document datasets, it averages NDCG@10 of 88.27 and 1.70× the fastest baseline’s average throughput.

    Source: §1 · p. 2, §5.3 · p. 8

  • Takeaways & Limitations

    Compressed visual representations provide an effective basis for relevance scoring while allowing more document content to contribute within the same sequence-length limit.

    Source: §7 · p. 9

Abstract

from arXiv · show

Rendering document text as images allows vision-language models to encode documents as visual tokens, which can reduce input sequence length compared with text input. This reduction in input length is particularly useful for reranking, where each query involves scoring multiple candidate documents and token savings apply to each candidate evaluation. We introduce RenderRank, a reranker that learns query-dependent relevance scoring from compressed visual document representations instead of the text token sequences used by conventional text-based rerankers. Training first aligns relevance scores from visual inputs with those of a text-based teacher, then refines the relative scores of positive and negative documents for the same query. Across 11 datasets from BEIR, RenderRank uses 16.5-35.5% fewer input tokens while achieving an average NDCG@10 of 55.96, outperforming all evaluated text-based baselines below 4B parameters and some larger models. Across four long-document datasets, it achieves an average NDCG@10 of 88.27 with approximately half the average input token count of the evaluated text-based rerankers. In this setting, RenderRank delivers 1.70x the highest average throughput of the evaluated baselines. These results demonstrate that compressed visual representations can support accurate document relevance scoring, providing an alternative to text token representations for reranking.

1 INTRODUCTION

Reranking in retrieval-augmented generation must prioritize candidate documents, but evaluating long inputs across many candidates is costly and shortening them may remove relevant evidence. RenderRank uses rendered documents encoded as visual tokens and query-conditioned scoring, achieving strong benchmark results with fewer input tokens.

  • In retrieval-augmented generation, rerankers score retrieved query–document pairs to prioritize useful evidence, while longer documents increase computation across candidates.Source: §1 · p. 1
  • Rendering text as images offers a shorter document representation, but visual inputs must remain interpretable and training must adapt relevance scoring to them.Source: §1 · pp. 1–2
  • RenderRank scores textual queries against compressed visual document tokens, first matching a text-based teacher’s scores and then refining positive-versus-negative priorities within each query.Document images are encoded in advance, independently of the query.Source: §1 · p. 2
  • 55.96 average NDCG@10 across 11 BEIR datasets, outperforming evaluated models below 4B parameters and some larger models while using 16.5–35.5% fewer input tokens.Source: §1 · p. 2

2 RELATED WORK

Related work studies visual representations and rendering choices for reducing document-token costs, as well as methods for reranking and improving efficiency. RenderRank focuses on reranking text documents represented by compressed visual tokens.

  • Prior work examines rendering density and layout choices to balance preservation of textual information against the number of visual tokens.Source: §2 · p. 3
  • Reranking research includes teacher-based learning of candidate order or score differences, alongside approaches that precompute or compress visual features to reduce scoring costs.Source: §2 · p. 3
  • RenderRank learns to approximate text-teacher relevance scores from visual inputs, then refines relative scores for candidates associated with the same query.Its compressed document representations are intended to let longer documents fit within the same sequence-length limit.Source: §2 · p. 3

3 RENDERRANK

RenderRank represents text documents as images and trains relevance scoring between visual document representations and textual queries. Rendering settings trade token efficiency against legibility, while training combines score distillation with within-query candidate discrimination.

3.1 RENDERING TEXT FOR VISUAL PROCESSING

  • At line spacing 1.0, 12pt rendering yields higher task performance than 10pt but less token reduction; Qwen3.6 QASPER F1 rises from 41.32 to 48.72.The table compares rendering configurations using QASPER F1 and GovReport ROUGE-L, with token counts relative to text input.Source: Table 1 · p. 3, §3.1 · p. 4
  • Documents are rendered in Roboto Regular at 12pt and 1.0 line spacing, with image dimensions aligned to the visual encoder’s patch and merging sizes.Long documents continue across ordered images processed as one candidate document.Source: §3.1 · p. 4

3.2 LEARNING TO RERANK TEXT THROUGH VISUAL REPRESENTATIONS

  • Cross-modal relevance distillation trains visual-input scores to approximate a text-based teacher’s scores, using mean squared error over query–document training pairs.This aligns relevance scores without requiring token-level correspondence or shared text-image feature alignment.Source: §3.2 · p. 4
  • Query-local relevance discrimination trains higher scores for positive documents than negatives associated with the same query, using candidate-set contrasts.Each relevance score is computed from the query and its corresponding document.Source: §3.2 · pp. 4–5

4 EXPERIMENTAL SETUP

The experiments evaluate RenderRank on BEIR and long-document reranking tasks against text-based baselines. Training uses teacher-scored pairs and query-associated positives and negatives.

  • Distillation uses 6.28M query–document pairs selected from 1.57M records, with one positive and three negatives per query, scored by a text-based teacher.Source: §4 · p. 5
  • Evaluation covers 11 BEIR datasets and four long-document datasets, using nDCG@10 and retrieved candidate sets; document visual embeddings are assumed to be precomputed.Source: §4 · p. 5
  • The comparison includes rerankers spanning varied architectures and model sizes, including models from below 1B to 4B parameters.Source: §4 · p. 5

5 EXPERIMENTAL RESULTS

RenderRank is evaluated for ranking quality and efficiency across BEIR and long-document benchmarks. It combines strong reranking scores with shorter visual-token inputs and measured throughput gains.

5.1 RERANKING PERFORMANCE

  • 55.96 average NDCG@10 on 11 BEIR datasets, with gains of 7.66% over zerank-2-reranker and 7.26% over LightOn-rerank-PW-4B, using 16.5–35.5% fewer tokens.The table reports reranking performance and average input token counts across the benchmarks.Source: §5.1 · pp. 5–6, Table 2 · p. 6

5.2 EFFICIENCY EVALUATION

  • Efficiency evaluation separates estimated computational cost from measured pairs-per-second throughput because shorter inputs do not guarantee proportional throughput gains.The study compares architecture- and input-length-based cost estimates with measured pairs processed per second.Source: §5.2 · p. 6

5.2.1 COMPUTATIONAL EFFICIENCY

  • 0.63 TFLOPs per query–document pair and 20.1 average QPP for RenderRank compare with 1.10 TFLOPs and 12.3 QPP for bge-reranker-v2-gemma.Cross-model cost differences also reflect backbone size and attention architecture.Source: §5.2.1 · p. 7

5.2.2 INFERENCE THROUGHPUT

  • 76.83 average PPS for RenderRank exceeds bge-reranker-v2-gemma by approximately 2× and LAMAR-600m’s 67.55 PPS despite higher estimated TFLOPs per pair.Figure 2 compares average NDCG@10 and GPU forward throughput across 11 BEIR benchmarks.Source: Figure 2 · p. 7, §5.2.2 · p. 8

5.3 LONG DOCUMENT RERANKING

  • 88.27 average NDCG@10 across four long-document datasets accompanies 4.51 PPS versus 2.66 PPS for the fastest baseline, a 1.70× throughput improvement.On MLDR, RenderRank uses 4.20K average input tokens; on LongEmbed datasets it uses 53–57% fewer tokens than the most token-efficient text baseline.Source: §5.3 · p. 8, Table 4 · p. 8

6 ANALYSIS AND ABLATION STUDIES

Across fixed token budgets, RenderRank performs strongly on long-document reranking, while font size and training stages reveal trade-offs in ranking quality and inference efficiency.

6.1 RERANKING UNDER FIXED TOKEN BUDGETS

  • At 2K, RenderRank reaches 97.9 NDCG@10, surpassing gte-reranker and LightOn-PW-4B at 4K; at 4K, its 99.5 is comparable to text-based rerankers at 8K.It achieves the highest NDCG@10 at each tested length, with performance differences narrowing at 8K.Source: §6.1 · pp. 8–9, Table 5 · p. 8

6.2 EFFECT OF RENDERING FONT SIZE

  • Increasing font size from 10pt to 14pt raises average input length from approximately 236 to 370 tokens and reduces throughput from approximately 94 to 63 PPS, while NDCG@10 rises from 55.04 to 56.43.The default 12pt setting scores 55.96 with approximately 290 input tokens and 77 PPS; its score is approximately 0.83% below 14pt.Source: §6.2 · p. 9, Figure 3 · p. 9

6.3 COMPONENT ABLATION

  • Cross-Modal Relevance Distillation raises average NDCG@10 with image inputs from 50.20 to 54.86, and Query-Local Relevance Discrimination raises it further to 55.96.Cached image inputs reach 76.83 PPS, compared with 37.31 PPS when visual encoding is included in the measurement.Source: §6.3 · p. 9

7 CONCLUSION

RenderRank uses compressed visual document representations for query-dependent relevance scoring. The conclusion reports that both training stages improve scoring and that rendering density trades ranking performance against inference efficiency.

7 CONCLUSION

  • RenderRank renders document text as images and encodes it into compressed visual tokens for query-dependent relevance scoring.Cross-modal distillation learns relevance from compressed visual inputs, and query-local discrimination refines relative candidate scores.Source: §7 · p. 9
  • Both training stages improve relevance scoring, while rendering density controls the trade-off between reranking performance and inference efficiency.The conclusion also states that compressed representations allow more document content to contribute within the same sequence-length limit.Source: §7 · p. 9
  • Compressed visual representations are presented as an effective basis for document reranking and as a path toward representing and reranking longer text documents.Source: §7 · p. 9

A FURTHER RELATED WORK

The related-work discussion distinguishes RenderRank from learned compression within text rerankers and positions its contribution as adapting existing visual text representations for query-dependent reranking.

A FURTHER RELATED WORK

  • A learned-compression approach uses a separate document compressor with a fixed number of memory tokens, jointly trains compressor and reranker, and precomputes document representations [D´ejean & Clinchant, 2026].Source: A FURTHER RELATED WORK · p. 16
  • Other methods shorten reranker computation by selecting document key–value states, pooling intermediate token representations, or training across adjustable compression ratios [Lu et al., 2026; Zhuang et al., 2026; Wang et al., 2026].Source: A FURTHER RELATED WORK · p. 16
  • RenderRank instead uses the visual encoder of an existing VLM, freezes the visual encoder and feature mergers, and trains the language model for relevance scoring.Its visual-token count varies with rendering configuration and document length.Source: A FURTHER RELATED WORK · p. 16

B GENERATION EXPERIMENTS FOR RENDERING EVALUATION

The generation experiments vary rendering configurations across two models and three LongBench datasets, measuring task performance alongside document-token counts. Results show that token savings do not consistently track performance.

B GENERATION EXPERIMENTS FOR RENDERING EVALUATION

  • Rendering tests use Gemma-4-26B-A4B-it and Qwen3.6-35B-A3B on QASPER, GovReport, and Multi-News, varying font size and line spacing.Images have fixed width of 896 pixels and height up to 896 pixels, with overflow continuing onto the next image.Source: B GENERATION EXPERIMENTS FOR RENDERING… · p. 16
  • Increasing line spacing from 1.0 to 1.2 at 12pt raises Qwen's Multi-News score from 22.98 to 23.41, while reducing token savings from 42.66% to 27.92%.Gemma at 12pt and 1.2 line spacing uses more document tokens than the text baseline without reaching its performance.Source: Table 7 · p. 16, B GENERATION EXPERIMENTS FOR RENDERING… · p. 17
  • Changes in document-token counts do not consistently translate into corresponding performance changes across models and tasks.Table 7 reports performance alongside average document-token counts; QASPER uses F1, while GovReport and Multi-News use ROUGE-L.Source: Table 7 · p. 16, B GENERATION EXPERIMENTS FOR RENDERING… · p. 17

C TRAINING DETAILS

The section specifies how documents are rendered as images and describes the data, training configuration, and reproducibility setup for RenderRank’s two training stages.

Rendering

  • Document text is rendered as black text on white images in Roboto Regular at 12 pt, after whitespace normalization that removes original paragraph boundaries and manual line breaks.Images are 896 px wide, with text wrapped by measured pixel width.Source: C TRAINING DETAILS · p. 17
  • Documents longer than 56 lines continue on new images without overlap or repeated text, with each image capped at 896 px high.The final image is cropped to its remaining lines rather than padded to a full square.Source: C TRAINING DETAILS · p. 17

Dataset

  • Stage 1 converts 1.57M records into 6.28M query-document pairs, while Stage 2 forms candidate sets from the first positive and first seven negatives per query.Stage 1 uses the positive and first three negatives; Stage 2 uses RLHN-100K, with comparisons restricted to each candidate set.Source: C TRAINING DETAILS · p. 18, Table 9 · p. 18

Hyperparams

  • Both stages train for one epoch with AdamW and LoRA on the text decoder, while the vision encoder and visual feature mergers remain frozen.Stage 1 uses a batch size of 1,024 pairs; Stage 2 uses 128, with contrastive-loss temperature 0.1.Source: C TRAINING DETAILS · p. 18

Reproducibility

  • Each configuration’s reported results come from a single run with seed 42; inference throughput was measured on one GPU.Training used eight NVIDIA RTX A6000 GPUs.Source: C TRAINING DETAILS · p. 18

D DETAILED RESULTS

This section provides detailed BEIR results for rerankers and RenderRank font-size variants, plus training-stage ablations; the supplied passages identify the tables but not their cell values.

Detailed results

Loading 2609.35069v1…