BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negatives

Introduction

• Biomedical retrieval is challenged by low-quality keyword queries and specialized terminology with subtle semantic relationships.Hard negatives are similar to positive examples but ultimately irrelevant, requiring fine-grained distinctions.Source: Introduction · p. 1
• BiCABase and the smaller BiCAsmall achieve state-of-the-art or highly competitive results on biomedical and general-domain benchmarks, often outperforming substantially larger models.The contributions include zero-shot evaluation on 14 BEIR and 4 LoTTE tasks, plus latency analysis highlighting BiCAsmall's practical efficiency.Source: Introduction · p. 1, Our Contributions · p. 2
• BiCA introduces citation-based hard-negative mining from multi-hop PubMed citation chains to train biomedical dense retrievers.The approach is intended to produce challenging negatives for retrieval-model training.Source: Our Contributions · p. 1, Introduction · p. 1
• The pipeline generates a synthetic query, builds a two-hop citation neighborhood, mines negatives by semantic graph traversal, and fine-tunes GTE on query-positive-negative triplets.The figure describes GTE fine-tuning with multiple negative ranking loss.Source: Figure 1 · p. 2

Biomedical Information Retrieval

• Bibliometric Data Fusion uses citation counts and altmetrics, while SL-HyDE generates hypothetical documents [Li et al. 2024] for medical retrieval.Neural Retrievers use template-based question generation and retrieval-aligned pretraining; MedCPT uses PubMed click logs for contrastive retriever-reranker training.Source: Biomedical Information Retrieval · p. 2

Biomedical Language Models

• Biomedical language models include domain-trained models such as SciFive, BioMegatron, PubMedBERT, BioBERT, and BioMedLM.Works by Saxena, Tripathi, and Talukdar (2020), Yasunaga, Leskovec, and Liang (2022), and Yasunaga et al. (2022) use knowledge graphs and multi-hop frameworks for question answering and biomedical text understanding.Source: Biomedical Language Models · p. 3

Methodology

• The methodology converts citation neighborhoods into dense semantic graphs and explores them with diverse, stochastic paths to find documents that are close in meaning but not directly relevant.The pipeline overview shows query generation, citation-neighborhood construction, hard-negative mining, and model fine-tuning.Source: Methodology · p. 3

Data Curation: 2-Hop Citation Neighborhood Construction

• The curated data contains about 20,000 positive documents, each paired with its 1-hop and 2-hop citation abstracts when those abstracts can be retrieved.For each positive document, the process fetches papers it cites, then papers cited by those papers; collection was parallelized across 80 worker processes.Source: Data Curation: 2-Hop Citation… · p. 3

Hard-Negative Mining via Diverse Semantic Traversal

• Mining generates a query from each positive abstract, builds a pairwise cosine-similarity graph over 1-hop and 2-hop abstracts, and selects negatives through multiple stochastic traversals.The algorithm uses multiple query-relevant starts, probabilistic selection among top-K unvisited neighbors, a shared visited set, and one random unvisited negative; the final dataset averages 6.5 negatives per query.Source: Hard-Negative Mining via Diverse… · p. 4

Fine-Tuning

• GTEBase (110M parameters, 768-dim) and GTEsmall (33M parameters, 384-dim) are the two BERT-based embedding models fine-tuned.The models use multi-stage contrastive learning and balance accuracy with efficiency.Source: Fine-Tuning · p. 4
• MNR is the loss function used for fine-tuning.The passage introduces its definition but does not provide a complete formula.Source: Fine-Tuning · p. 5
• The loss uses query and positive-document embeddings alongside negative-document embeddings and the number of negatives.The prose identifies q as the query embedding, d+ as the positive-document embedding, d− as the negative-document embedding, and K as the number of negatives.Source: Fine-Tuning · p. 5

BEIR

• 0.518 average nDCG@10 across 14 BEIR tasks gives BiCABase the highest overall score, ahead of GTR xxl at 0.486.BiCAsmall averages 0.501, ranks second overall, and also exceeds many larger baselines.Source: Evaluation · p. 5

LoTTE

• BiCABase achieves the highest Success@5 across all four LoTTE categories for both Search and Forum queries.BiCAsmall consistently ranks second; the evaluation uses the LoTTE test set and covers four long-tailed subtopics.Source: LOTTE · p. 5

Latency

• 13 ms total latency makes BiCAsmall fastest at batch size 1, while ColBERTv2 has the quickest encoding at 8 ms.ColBERTv2's total is 15 ms; the latency test uses a V100 and retrieves the top 1,000 results.Source: Latency · p. 5
• 19 ms total latency makes BiCAsmall fastest at batch size 10, while ColBERTv2 has the best tail latency at 23 ms.ColBERTv2, RetroMAE, and SpladeV3 encode in 11 ms, compared with 14 ms for BiCAsmall.Source: Latency · p. 5
• 994 ms total latency makes BiCAsmall fastest at batch size 2000, ahead of RetroMAE at 1837 ms.BiCABase is slowest at 1904 ms; BiCAsmall's total comprises 554 ms encoding and 441 ms retrieval.Source: Latency · p. 5

Effect of Traversal Parameters

• 0.2739 overall average performance is highest with Npaths = 3 and Lpath = 3, the selected default configuration.Other settings sometimes lead on an individual dataset but perform worse elsewhere, lowering their overall average.Source: Effect of Traversal Parameters · p. 5

Robustness and Scalability

• Retrieval performance has a clear positive correlation with training-data size across the evaluated subsets.BERT-base-uncased was fine-tuned on subsets from 1,000 to the full 20,000 records and evaluated zero-shot on three biomedical tasks and one BEIR task.Source: Robustness and Scalability · p. 5
• Table 2 reports LoTTE test-set Success@5 for different models on Search and Forum queries.The caption marks the best score in bold and the second-best score with underlining.Source: Table 2 · p. 6
• Table 3 compares V100 latency in milliseconds, with encoding and top-1,000 retrieval timings from a FAISS index of 10,000 MS MARCO passages.Its cell colors mark the lowest-to-highest timing for each metric within a batch size.Source: Table 3 · p. 6

• The study evaluates fine-tuned DistilBERT and e5-base-v2 on five BEIR tasks spanning biomedical and non-biomedical domains.Evaluation follows fine-tuning for up to one epoch with early stopping.Source: Performance of Different Architectures · p. 6
• The reported gains are measured in nDCG@10 after zero-shot evaluation on BEIR.Source: Performance of Different Architectures · p. 6
• DistilBERT improves by 1.56 points on average, while e5-base-v2 improves by 0.84 points.Source: Performance of Different Architectures · p. 6

• BiCA's hard-negative mining exploits multi-hop PubMed citation chains to provide semantically challenging negatives for fine-grained retrieval distinctions.Source: Conclusions · p. 6
• The reported ablations examine traversal counts and path lengths, training-data scale, DistilBERT/E5 comparisons, and fine-tuning-step counts.Source: Table 4 · p. 7, Table 5 · p. 7, Table 6 · p. 7
• BiCABase records the highest average nDCG@10 in both biomedical and non-biomedical BEIR domains, while BiCAsmall offers substantially lower inference latency.BiCABase also outperforms several larger state-of-the-art models.Source: Conclusions · p. 7
• On LoTTE, BiCABase leads across all sub-domains, and BiCAsmall ranks consistently among the top performers.Source: Conclusions · p. 7

• Generating multi-hop citation chains requires iterative PubMed API requests and can take weeks; the current approach is restricted to PubMed.The authors also note that their latency evaluation may not fully reflect ColBERTv2's efficiency advantages.Source: Limitations · p. 7

A Improvement over GTE Models

• BiCAsmall gains an average of ∼5.8 points over GTEsmall, while BiCABase gains ∼6.8 points over GTEBase across fourteen datasets.Source: Table 8 · p. 10, A Improvement over GTE Models · p. 10

B Data Selection

• The 20,000-document corpus was selected to represent the larger PubMed dataset, with embedding distributions plotted for comparison.Source: B Data Selection · p. 10

C Choice of fine tuning steps

• Peak validation performance was consistently reached at 20 training steps, and that checkpoint was used for zero-shot BEIR and LoTTE evaluation.The validation set used an 80%/20% split of the constructed corpus.Source: C Choice of fine tuning steps · p. 10, Figure 2 · p. 10

D.1 BEIR

• The BEIR evaluation dataset listing includes licenses and document and query counts.Source: §D.1 · p. 10

D.2 LoTTE test-set

• The LoTTE test-set description covers test topics, question sets, and sample subtopics, with search and forum query sources identified.Source: §D.2 · p. 10, Table 9 · p. 11