PaperlayerView exact revision on arXiv

Source-linked AI summary

KSE-Web: An Analysis of Hybrid Retrieval and LLM-Assisted Query Expansion for Low-Resource Khmer Semantic Search

Nimol Thuon

arXiv:2608.21365v1cs.CLcs.AIcs.IR

TL;DR

Khmer semantic search lacks robust resources and faces tokenization and representation challenges. KSE-Web constructs and evaluates a Khmer web retrieval dataset across sparse, dense, hybrid, and Qwen2.5-assisted methods, finding that character n-gram BM25 is strongest overall while direct expansion does not surpass non-expanded retrieval. The dataset is a practical foundation, but its scope and partially silver labels motivate stronger human-verified resources.

  • Problem

    Khmer semantic retrieval is constrained by limited annotated resources, unreliable word boundaries, mixed Khmer–English usage, and weak multilingual representation support.

  • Method

    KSE-Web constructs a Khmer web retrieval dataset and compares character n-gram BM25, multilingual dense retrieval, hybrid fusion, and Qwen2.5 query expansion.

  • Results

    Character n-gram BM25 is strongest overall at 0.943 Recall@10 and 0.876 nDCG@10; hybrid retrieval is close, while direct LLM expansion does not outperform non-expanded retrieval.

  • Takeaways & Limitations

    LLM-assisted Khmer retrieval depends on model size and expansion quality, so unrestricted query expansion alone is insufficient.

  • Takeaways & Limitations

    The dataset focuses on web-extracted Khmer text, includes some title-derived queries, and uses silver labels partly based on BM25 candidate retrieval.

Abstract

from arXiv · show

As a low-resource language, Khmer presents several retrieval challenges, including limited annotated data, ambiguous word boundaries, weak support in multilingual embedding models, and frequent mixed Khmer-English usage. This paper presents KSE-Web, an analysis of hybrid retrieval and LLM-assisted query expansion for Khmer semantic search. We construct the dataset from approximately 17K candidate Khmer titles and retain 3K cleaned full-text Khmer documents after filtering, normalization, deduplication, and document-length control. The dataset includes 300 manually reviewed user-style Khmer search queries and silver relevance labels with partial human verification. We evaluate character n-gram BM25, multilingual dense retrieval, hybrid BM25+dense retrieval, and LLM-assisted query expansion using Qwen2.5 models. Experimental results show that BM25 achieves the strongest overall performance, reaching 0.943 Recall and 0.876 nDCG. Hybrid BM25+dense retrieval performs comparably, achieving 0.929 Recall and 0.871 nDCG, while dense retrieval alone performs lower. LLM-assisted query expansion does not outperform non-expanded retrieval; however, Qwen2.5-3B produces substantially stronger expanded-query results than Qwen2.5-0.5B, suggesting that LLM size and expansion quality matter for low-resource Khmer retrieval. Our analysis further shows that direct LLM expansion can introduce topic drift, generic terms, and noisy reformulations, while simple filtering may remove useful semantic cues. These findings highlight both the potential and limitations of LLM-assisted retrieval for Khmer semantic search and provide a foundation for future Khmer retrieval datasets with stronger human-verified annotations and Khmer-aware retrieval models. The dataset and documentation will be made available at github.com/back-kh/KhmerSemantic-Search.

1 Introduction

KSE-Web studies Khmer semantic retrieval under low-resource conditions by combining sparse, dense, hybrid, and LLM-assisted approaches. It introduces a Khmer web dataset and finds character n-gram BM25 strongest overall, while larger-model expansion remains below non-expanded retrieval.

  • Motivation: Khmer retrieval is difficult because resources, tokenization, multilingual representations, and evaluation support are limited.Khmer web content also includes mixed Khmer–English usage and other lexical variation.
  • Framework: KSE-Web separates offline sparse and dense index construction from online query normalization, optional LLM expansion, retrieval, and score fusion.The original or expanded query is passed to BM25, dense retrieval, or hybrid retrieval.
  • Dataset: 3K cleaned full-text Khmer documents, 300 manually reviewed queries, and 5,412 silver relevance labels form the retrieval dataset.The documents were selected from approximately 17K candidate titles through filtering, normalization, deduplication, and document-length control.
  • Results: 0.943 Recall@10 and 0.876 nDCG@10 make character n-gram BM25 the strongest overall method, while hybrid retrieval reaches 0.929 Recall@10 and 0.871 nDCG@10.Dense retrieval alone performs substantially lower.
  • LLM-assisted retrieval: Qwen2.5-3B produces more useful expansions than Qwen2.5-0.5B, but direct expansion remains below non-expanded retrieval.The analysis examines helpful expansion, topic drift, over-expansion, generic-term insertion, and filtering effects.

2 Related Work

Khmer retrieval research addresses a low-resource setting with unreliable word boundaries, mixed-language web text, and limited evaluation resources. Related methods motivate character n-gram, dense, hybrid, and LLM-assisted retrieval, while KSE-Web also recognizes annotation and dataset-scope constraints.

  • Low-resource retrieval: Khmer remains underrepresented in retrieval research and lacks large-scale corpora, standardized query sets, relevance judgments, and language-specific preprocessing tools.These gaps motivate Khmer-specific retrieval datasets and evaluation protocols.
  • Khmer retrieval challenges: Whitespace is not a reliable Khmer word boundary, and web content includes mixed Khmer–English terms, transliterated names, entities, informal expressions, and spelling variations.These properties affect both lexical and dense retrieval methods.
  • Sparse retrieval: Character n-gram BM25 avoids explicit word segmentation and captures partial lexical overlap, named entities, spelling variants, and short Khmer phrases.It provides a practical alternative to word-level BM25 for Khmer.
  • Hybrid retrieval: Hybrid retrieval fuses character n-gram BM25 with multilingual-E5 to test whether dense representations add semantic signals beyond character-level lexical matching.Sparse retrieval supports exact or rare-term matching, while dense retrieval can capture paraphrases and related expressions.
  • LLM-assisted expansion: LLM query expansion can broaden intent, add generic terms, or include nondiscriminative words, risks amplified by weaker language-specific knowledge in low-resource settings.KSE-Web compares Qwen2.5-0.5B and Qwen2.5-3B and finds direct expansion below strong non-expanded baselines.
  • Relevance labeling: Silver labels provide a practical starting point for resource-constrained evaluation but may introduce bias toward systems used for candidate pooling.KSE-Web uses source-document matching, BM25 candidate retrieval, rule-based assignment, and partial human review.

3 KSE-Web Dataset Construction

KSE-Web builds a practical Khmer web retrieval dataset by cleaning web documents, manually reviewing user-style queries, and assigning partially human-verified silver relevance labels. Its scope is deliberately narrower than a large-scale gold-standard benchmark.

  • Document Collection: Approximately 17K candidate Khmer web titles were filtered and cleaned into 3K full-text Khmer web documents.Processing included filtering, normalization, deduplication, and document-length control.
  • Relevance Labels: 5,412 query–document pairs received three-level silver relevance labels with partial human verification.Labels distinguish highly relevant, partially relevant, and non-relevant pairs, supporting binary and graded ranking evaluation.
  • Document Collection: The corpus contains document identifiers, titles, cleaned body text, source URLs, category metadata, and document-length information.Documents are assigned to public service, education, tourism and culture, or general news/information categories.
  • Query Set: 300 user-style Khmer queries were manually reviewed across short, question-style, informal, and mixed Khmer–English search types.Queries were generated from source-document titles and reviewed to remove duplicated, unnatural, overly broad, or noisy examples.
  • Scope and Limitations: KSE-Web is a practical web-extracted resource rather than a final large-scale gold-standard benchmark.It excludes PDFs, scanned pages, and OCR-based documents; some queries derive from titles, and BM25-based candidate retrieval may introduce lexical bias.

4 Overall Retrieval Framework

The retrieval framework separates offline indexing from online query processing and compares character n-gram BM25, dense retrieval, hybrid score fusion, and LLM-assisted expansion. It is designed to test how these signals interact for low-resource Khmer document retrieval.

  • Task Definition: The task ranks KSE-Web documents for a Khmer user query, with earlier-ranked documents expected to be more relevant.All methods use the same corpus and silver relevance labels for document-level evaluation.
  • Sparse Retrieval: Character n-gram BM25 avoids explicit Khmer word segmentation by representing queries and documents with overlapping character n-grams.This representation can capture partial lexical overlap, named entities, spelling variants, and short Khmer phrases.
  • Dense Retrieval: Dense retrieval encodes queries and documents with multilingual-E5-small embeddings and ranks them by cosine similarity.The method is intended to capture semantic similarity beyond exact lexical overlap, although Khmer representation in multilingual training resources remains uncertain.
  • Hybrid Retrieval: Hybrid retrieval min–max normalizes BM25 and dense scores, then combines them with equal weighting using α = 0.5.The resulting documents are ranked by the fused hybrid score.
  • LLM-Assisted Expansion: LLM-assisted expansion uses Qwen2.5-0.5B-Instruct and Qwen2.5-3B-Instruct to add related terms, paraphrases, or Khmer–English expressions before retrieval.The framework evaluates raw and filtered expansions with sparse, dense, and hybrid retrieval; filtering removes repeated, generic, broad, or excessive terms.
  • Evaluation Variants: The framework examines whether LLM expansion improves retrieval and how it interacts with sparse, dense, and hybrid methods in low-resource Khmer search.This comparison includes BM25-char-ngram, dense, hybrid, expanded-query, and filtered-expansion variants.

5 Experiments

The experiments compare sparse, dense, hybrid, and LLM-assisted retrieval on KSE-Web using shared documents, queries, relevance labels, and top-k evaluation metrics.

  • Compared Methods: The evaluation compares character n-gram BM25, multilingual dense retrieval, hybrid BM25+dense retrieval, and Qwen2.5-assisted query expansion.The compared methods search the same document collection and use expanded queries with sparse, dense, and hybrid retrieval.
  • Dataset: KSE-Web contains 3,000 cleaned Khmer documents and 300 manually reviewed user-style queries spanning four query types.The query types are short, question-style, informal, and mixed Khmer–English queries.
  • Evaluation: Relevance uses three-level silver labels, with labels 1 and 2 treated as relevant for Recall, Precision, and MRR, while nDCG uses graded labels.The labels are 2 for highly relevant, 1 for partially relevant, and 0 for non-relevant.
  • Hybrid Retrieval: Hybrid retrieval linearly fuses normalized BM25 and dense scores with α = 0.5, giving equal weight to both signals.The implementation min–max normalizes BM25 and dense scores per query before fusion.
  • LLM-Assisted Retrieval: LLM expansion uses Qwen2.5-0.5B-Instruct and Qwen2.5-3B-Instruct, while filtered expansion removes repeated, generic, broad, and excessive terms.The expansion instruction preserves meaning and named entities while adding retrieval-useful terms.
  • Evaluation: Retrieval performance is measured with Recall@k, Precision@k, MRR@k, and nDCG@k at cutoffs 5 and 10.All metrics are averaged over the full query set.

6 Results and Analysis

Character n-gram BM25 is the strongest retrieval method on KSE-Web, while hybrid retrieval remains close and dense retrieval alone is substantially weaker. Larger LLMs improve expansion quality, but direct expansion and simple filtering do not surpass strong non-expanded baselines.

  • Overall Retrieval Performance: 0.943 Recall@10 and 0.876 nDCG@10 make character n-gram BM25 the strongest overall method on KSE-Web.Character n-grams capture partial lexical overlap, named entities, and short phrases without explicit word segmentation.
  • Overall Retrieval Performance: 0.929 Recall@10 and 0.871 nDCG@10 show that hybrid BM25+dense retrieval performs very closely to BM25.Dense retrieval provides complementary signals, but does not surpass the sparse lexical baseline.
  • Overall Retrieval Performance: 0.563 Recall@10 and 0.523 nDCG@10 show that dense retrieval alone performs substantially lower than BM25.The results indicate that multilingual-E5-small captures some Khmer semantic similarity but remains limited relative to character n-gram BM25.
  • LLM Query Expansion: 0.824 Recall@10 and 0.770 nDCG@10 for BM25 with Qwen2.5-0.5B expansion remain below the original BM25 baseline.Hybrid retrieval with Qwen2.5-0.5B expansion likewise reaches 0.788 Recall@10 and 0.729 nDCG@10, below non-expanded hybrid retrieval.
  • LLM Query Expansion: 0.868 Recall@10 and 0.788 nDCG@10 for hybrid retrieval with Qwen2.5-3B expansion improve over Qwen2.5-0.5B but remain below non-expanded hybrid retrieval.For dense retrieval, Qwen2.5-3B improves Recall@10 from 0.365 to 0.440 and nDCG@10 from 0.318 to 0.372.
  • Filtering Expanded Queries: 0.838 Recall@10 and 0.755 nDCG@10 after filtering reduce hybrid Qwen2.5-3B performance from 0.868 and 0.788.The decrease suggests that simple filtering can remove useful semantic cues as well as noisy terms.
  • Expansion Behavior: Specific entity queries can drift into broad tourism or culture queries, while generic additions can reduce ranking precision.These behaviors explain why expansion helps when it adds precise related terms but hurts when it weakens lexical signal or changes query focus.
  • Discussion and Future Work: The study concludes that character n-gram BM25 is a strong baseline, hybrid retrieval offers limited complementary signals, and unrestricted expansion is insufficient.The discussion identifies entity-preserving, query-type-aware, retrieval-aware, and stronger human-verified approaches as future directions, while noting that silver labels may favor lexical retrieval.

7 Conclusion

KSE-Web provides a Khmer web retrieval dataset for comparing lexical, dense, hybrid, and LLM-assisted methods. Character n-gram BM25 remains strongest, hybrid retrieval is close, and LLM expansion requires control to avoid losing lexical specificity.

  • Dataset: KSE-Web retains 3K cleaned full-text Khmer documents from approximately 17K candidate web titles after filtering, normalization, deduplication, and length control.The dataset also includes 300 manually reviewed user-style Khmer queries and 5,412 silver relevance labels with partial human verification.
  • Findings: 0.943 Recall@10 and 0.876 nDCG@10 make character n-gram BM25 the strongest overall method, while hybrid retrieval reaches 0.929 Recall@10 and 0.871 nDCG@10.Dense retrieval alone performs substantially lower, and Qwen2.5-3B produces stronger expanded-query results than Qwen2.5-0.5B.
  • Implications: LLM-assisted query expansion does not outperform non-expanded retrieval and requires careful control to avoid topic drift and loss of lexical specificity.The conclusion presents the findings as a foundation for future Khmer retrieval and low-resource semantic-search research.
Loading 2608.21365v1…