Deep Relevance Ranking Using Enhanced Document-Query Interactions

• DRMM scores query terms using similarity histograms and term-importance gating, but its term interactions ignore the contexts in which words occur.Source: §1 · p. 1
• Differentiable query-document interactions enable end-to-end training of context-sensitive DRMM models, unlike non-differentiable histogram construction.Source: §1 · p. 2
• The extensions include convolutional n-gram matching, context-sensitive encodings, query-focused attention, pooling, and multiple term views.Source: §1 · p. 2
• On BIOASQ and TREC ROBUST 2004, the models outperform strong BM25-based baselines, DRMM, and PACRR.Source: §1 · p. 2

2 Related Work

• Interaction-based models explicitly encode query-document term pairs, enabling exact or near-match modeling, while reranking conventional search results makes their indexing disadvantage less important.Representation-based methods encode queries and documents independently; documents are typically much longer than queries, limiting direct transfer from short-text ranking methods.Source: §2 · p. 2
• BIOASQ methods were mostly traditional IR, while Mohan et al. (2017) used click supervision and context-insensitive interactions; TREC ROBUST 2004 is a common evaluation dataset.Source: §2 · p. 2

2.1 DRMM

• DRMM uses fixed-dimension cosine-similarity histograms that handle varying query and document lengths, but they omit context, word order, and rewards for clustered matches.A histogram bucket counts document terms whose cosine similarity to a query term falls within a specified range.Source: §2.1 · p. 3

2.2 PACRR

• PACRR applies n-gram convolutions to a query-document similarity matrix, then pools strongest matches into query-term encodings for a relevance-scoring MLP.Its context sensitivity is limited because convolutions operate on similarities rather than directly on terms or term embeddings; fixed-size matrices also require padding and length hyperparameters.Source: §2.2 · p. 3

3.1 PACRR-DRMM

• PACRR-DRMM shares a query-term scoring MLP across terms and aggregates scores linearly, outperforming PACRR with the same hidden-layer count and showing less overfitting in early experiments.The authors suggest fewer MLP parameters and shorter inputs may explain the overfitting difference.Source: §3.1 · p. 3

3.2 Context-sensitive Term Encodings

• BILSTM encodings add original embeddings as residuals, placing context in term vectors before similarities rather than applying PACRR's convolutions afterward.Residual addition requires hidden dimensions to match embedding dimensions; tested MLP projection had no performance effect.Source: §3.2 · p. 4

3.3 ABEL-DRMM

• ABEL-DRMM replaces DRMM histograms with attention-weighted document context and a Hadamard product against each query-term encoding, producing fixed-width inputs for end-to-end scoring.Like DRMM, it is document-length agnostic and does not reward denser matches.Source: §3.3 · pp. 4–5

3.4 POSIT-DRMM

• POSIT-DRMM uses cosine attention scores with max and top-k average pooling, retaining strongest and average matches instead of collapsing them into a weighted document-term sum.Its pooling addresses the concern that an attention sum can hide whether one or several document terms matched strongly.Source: §3.4 · p. 5

3.5 Multiple Views of Terms (+MV)

• POSIT-DRMM+MV concatenates three 2D views—contextual encodings, pretrained embeddings, and one-hot exact-match encodings—into six-dimensional query-term representations.For ABEL-DRMM+MV, summing the three views was more effective than concatenating them.Source: Figure 4 · p. 5, §3.5 · p. 6

3.6 Alternative Network Structures

• Development experiments rejected deeper or wider encoders, multi-head attention, and combining model encodings; pointer-network argmax performed similarly to ABEL-DRMM.The authors also describe K-NRM as an end-to-end DRMM using interaction-matrix kernels, and Pang et al.'s matching-context matrices as able to model proximity and exact matches.Source: §3.6 · p. 6

4.1 Methods Compared

• Deep models rerank BM25’s top N documents; systems combine relevance scores with four extra features, train with pairwise loss and sampled negatives, and are evaluated on MAP, Precision@20, and nDCG@20 across five runs.Source: §4 · p. 6, §4.1 · pp. 6–7

4.2 BioASQ Experiments

• BIOASQ’s expert biomedical questions use PubMed articles, with years 1–4 for training, year-5 batch 1 for development, and batches 2–5 for testing.Articles published after 2015 for training and after 2016 for development and testing were removed to match annotation-time availability.Source: §4.2 · p. 7
• On BIOASQ, context-sensitive ABEL-DRMM and POSIT-DRMM outperform other models on average and by significance tests; POSIT-DRMM+MV leads on average but not significantly over POSIT-DRMM.Source: §4.2 · p. 7, Table 1 · p. 8

4.3 TREC Robust 2004 Experiments

• On TREC ROBUST, POSIT-DRMM+MV is best but not significantly different from POSIT-DRMM, and ABEL-DRMM and POSIT-DRMM outperform the deep-learning baselines.Unlike BIOASQ, PACRR-DRMM does not differ significantly from the two deep-learning baselines; ABEL-DRMM+MV differs from ABEL-DRMM on the best run for MAP and nDCG@20, while five-run averages show little difference.Source: Table 2 · p. 8, §4.3 · p. 8

5 Discussion

• Without extra features, POSIT-DRMM+MV’s development MAP was 46.2, above BM25 (43.7) and on par with BM25+EXTRA (46.0).Its MAP with extra features was 48.1.Source: §5 · p. 8
• DRMM without extra features scored MAP 42.5, below BM25; adding extra features raised its MAP to 46.5.The authors qualify DRMM comparisons because splits and preprocessing differ, and the original DRMM reranked Query Likelihood results rather than BM25 results.Source: §5 · pp. 8–9
• Removing context-sensitive encodings reduced development MAP from 47.6 to 46.3 for ABEL-DRMM and from 48.1 to 47.0 for POSIT-DRMM.The models used pre-trained embeddings directly instead of context-sensitive encodings.Source: §5 · p. 9
• For the relevant Vitamin D query, context-sensitive matches support ‘induce’ despite its non-exact match, while repeated ‘vitamin d’ matches yield high average k-max pooling.For another query against the same document, POSIT-DRMM scores the pair low because match density is lower and context-sensitive values are worse despite exact matches.Source: §5 · p. 9
• The authors report a substantial gap between the current best models and oracle reranking scores, leaving room for improvement through more data or better models.Source: §5 · p. 9