Source-linked AI summary
Information Retrieval: Recent Advances and Beyond
Kailash A. Hambarde, Hugo Proenca
TL;DR
Information retrieval research needs a coherent account of models used for retrieval and ranking, including term-based, semantic, and neural approaches. This survey organizes those methods and their learning processes, synthesizes the field’s state of the art, and identifies challenges and future directions. It highlights long-tail and multilingual retrieval as continuing challenges.
Problem
Information retrieval encompasses diverse retrieval and ranking models, while conventional term-based methods face limitations such as lexical gaps between queries and documents.
Method
The survey reviews conventional term-based retrieval, semantic and neural methods, learning approaches, and both stages of the retrieval process.
Results
The survey provides a comprehensive overview of state-of-the-art semantic retrieval models, their connections, major challenges, and promising research directions.
Takeaways & Limitations
The survey is intended to support researchers and practitioners by providing an overview of current information-retrieval models and directions for further research.
Takeaways & Limitations
Long-tail and multilingual retrieval remain major challenges for semantic retrieval systems.
Abstract
from arXiv · showhide
In this paper, we provide a detailed overview of the models used for information retrieval in the first and second stages of the typical processing chain. We discuss the current state-of-the-art models, including methods based on terms, semantic retrieval, and neural. Additionally, we delve into the key topics related to the learning process of these models. This way, this survey offers a comprehensive understanding of the field and is of interest for for researchers and practitioners entering/working in the information retrieval domain.
1. Introduction
Information retrieval seeks relevant information for user queries, but traditional term matching struggles with polysemy, synonymy, and lexical gaps. This survey reviews term-based, semantic, and neural approaches across both retrieval stages.
- Information retrieval locates and ranks information relevant to a user’s query across applications such as web search, question answering, and digital libraries.
- Traditional term-based retrieval is limited by polysemy, synonymy, and lexical gaps between queries and documents.
- Recent IR research also incorporates external knowledge, such as knowledge-graph embeddings, to connect queries and documents with relevant entities and concepts.
- The survey covers models for both retrieval stages, including term-based, semantic, and neural methods, as well as learning-related topics.
2. Information Retrieval: Overview
Modern information retrieval is organized into retrieval and ranking stages. The first stage gathers potentially relevant documents, while the second re-ranks them according to relevance.
- The IR process divides into retrieval, which gathers initial candidate documents, and ranking, which orders them by relevance.
- Retrieval models include vector-space, Boolean, latent-semantic, topic-model, and pretrained approaches such as BERT.
- Figure 2 presents an overview of the modern information-retrieval system.
- Ranking prioritizes effectiveness over efficiency and adjusts the ordering of initially retrieved documents using different models.
3. Conventional Term-based Retrieval
Conventional retrieval developed from direct term matching toward query and document expansion, dependency modeling, topic models, and multilingual approaches. These methods aim to improve representation and matching beyond exact lexical overlap.
- Classical term-based retrieval matches query terms to document terms but cannot adequately handle synonyms, polysemy, or context.
- Query Augmentation: Query expansion uses global concepts, lexical-semantic relations, context, and relevance feedback to improve retrieval.
- Document Augmentation: Document expansion adds related terms to document representations so queries can match relevant documents more effectively.
- Lexical Dependency Model: Lexical dependency models capture relationships among terms through phrase indexing, dependency weighting, language models, and sentence trees.
- Multilingual Retrieval: Statistical translation models treat retrieval as a translation problem and demonstrate potential for improving retrieval systems.
4. First Stage: Retrieval
First-stage retrieval research spans sparse, dense, neural, hybrid, and representation-learning methods designed to improve retrieval effectiveness or efficiency. The literature also explores expansion, approximate search, training strategies, and knowledge transfer.
- The first-stage review covers sparse semantic retrieval, deep-learning semantic retrieval, and hybrid techniques.
- Deep Learning Methods for Semantic Retrieval: Neural semantic retrieval represents queries and documents in continuous vector spaces and ranks them with similarity measures such as cosine similarity or dot product.
- Sparse Retrieval: Sparse methods reduce indexed dimensionality and retrieve only a small subset of documents considered relevant to a query.
- Sparse Retrieval: Document and query expansion, term re-weighting, and sparse representations are used to improve retrieval effectiveness and efficiency.
- Dense Retrieval: Dense retrieval research uses approximate nearest-neighbor search, contrastive learning, negative sampling, and specialized training strategies.
- Knowledge Distillation: Knowledge distillation transfers knowledge from larger pretrained models to smaller models to improve task performance or computational efficiency.
5. Second Stage - Ranker
Second-stage ranking refines initially retrieved documents using models that prioritize relevance effectiveness. The survey covers neural ranking, learning-to-rank, efficiency, supervision, bias, and transferability.
- Neural ranking models learn document relevance from labeled training data and can represent complex query-document relationships.
- Ad-hoc retrieval: Ad-hoc retrieval ranks corpus documents for brief queries whose unclear intent can create vocabulary mismatch.
- Ad-hoc retrieval: Recent ranking approaches address mismatch and relevance diversity through deep relevance matching, topic models, graph neural networks, and hierarchical neural matching.
- Learning-to-rank: Learning-to-rank research includes cascaded models with feature costs, weak supervision, counterfactual correction for position bias, and repeatability studies.
- Training strategies: Curriculum learning improves BERT and ConvKNRM answer ranking by training on easier correct answers before more complex samples.
- Transferability: Pseudo-label training across 5 datasets can produce a competitive or better BERT-based ranking model than transfer learning, while few-shot training may reduce pretrained-model performance.
6. Datasets
The survey catalogs state-of-the-art datasets across diverse information-retrieval and related NLP tasks. Each dataset is organized by task, domain, and corpus size for reference and selection.
- Table 2 covers datasets for passage retrieval, biomedical retrieval, question answering, tweets, news, argument retrieval, community QA, entities, fact checking, and search queries.
- The dataset table records each resource’s task, domain, and corpus size.
- The dataset listing is intended to help researchers select and cite appropriate corpora for NLP tasks.
7. Current Challenges and Further Directions
The survey identifies unresolved challenges in semantic retrieval and points toward task-specific pre-training and more sophisticated document representations. Long-tail and multilingual retrieval remain important challenges.
- Pre-training methods use large document collections and objectives such as Inverse Cloze Task, Body First Selection, and Wiki Link Prediction for retrieval.
- The field’s challenges and future directions provide a general view of open problems in semantic retrieval.
- Long-tail queries, which are infrequent or rare, remain a major challenge for semantic retrieval systems.
- Multilingual retrieval has become an important challenge as multilingual information on the web increases.
- Future work proposes more sophisticated document representations, including deep neural networks and transformer-based models, to improve semantic retrieval effectiveness.
8. Conclusions
The survey synthesizes information-retrieval models across retrieval stages, from conventional and semantic methods to neural approaches. It also connects these models with retrieval learning, challenges, and future research directions.
- The survey focuses on first-stage retrieval, second-stage retrieval, and learning for neural semantic retrieval models.
- The authors present the survey as a resource for researchers interested in semantic retrieval and future developments.