PaperlayerView exact revision on arXiv

Source-linked AI summary

Cost-Efficient RAG for Entity Matching with LLMs: A Blocking-based Exploration

Chuangtao Ma, Zeyu Zhang, Arijit Khan, Sebastian Schelter, Paul Groth

arXiv:2602.05708v1cs.DBcs.CL

TL;DR

Large-scale entity matching makes conventional per-query RAG retrieval and generation costly, motivating a more efficient design. The paper introduces CE-RAG4EM with blocking-based batch processing and evaluates retrieval granularity and related design choices. Experiments report strong matching performance with reduced cost, while revealing trade-offs between quality and overhead that constrain deployment choices.

  • Problem

    RAG-based entity matching lacks cost-efficient retrieval, unified comparison across variants and knowledge sources, and systematic characterization of accuracy–computational-cost trade-offs.

  • Method

    CE-RAG4EM uses blocking-based batch retrieval and inference within a unified framework covering retrieval granularity, vector search, and graph traversal.

  • Results

    CE-RAG4EM outperforms LLM-EM on all nine datasets in F1, with gains primarily driven by increased recall.

  • Takeaways & Limitations

    Blocking-based RAG can reduce retrieval and inference cost while enabling smaller open-source models to compete with larger LLMs for scalable entity matching.

  • Takeaways & Limitations

    Retrieved context can improve recall while reducing precision on some datasets when noisy or weakly related evidence biases predictions toward matches.

Abstract

from arXiv · show

Retrieval-augmented generation (RAG) enhances LLM reasoning in knowledge-intensive tasks, but existing RAG pipelines incur substantial retrieval and generation overhead when applied to large-scale entity matching. To address this limitation, we introduce CE-RAG4EM, a cost-efficient RAG architecture that reduces computation through blocking-based batch retrieval and generation. We also present a unified framework for analyzing and evaluating RAG systems for entity matching, focusing on blocking-aware optimizations and retrieval granularity. Extensive experiments suggest that CE-RAG4EM can achieve comparable or improved matching quality while substantially reducing end-to-end runtime relative to strong baselines. Our analysis further reveals that key configuration parameters introduce an inherent trade-off between performance and overhead, offering practical guidance for designing efficient and scalable RAG systems for entity matching and data integration.

1 Introduction

Entity matching must balance effectiveness with scalability, while LLM- and RAG-based approaches introduce additional reliability and computational challenges. The paper addresses these gaps with blocking-based, cost-efficient RAG and a unified evaluation framework.

  • Motivation: Entity matching determines whether records refer to the same real-world entity, but comparing m and n records exhaustively costs O(mn).Blocking groups records into candidate sets and restricts comparisons to plausible pairs.
  • Motivation: LLM-based entity matching faces accuracy and reliability challenges from heterogeneous data, imbalanced matches, sparse supervision, bias, limited reasoning depth, and hallucinations.These challenges are especially relevant in large-scale, real-world integration settings.
  • Research gaps: RAG can improve trustworthiness by adding retrieved factual knowledge, but per-query retrieval, embedding, and nearest-neighbor search create substantial overhead for entity matching.RAG-based EM remains comparatively unexplored despite RAG’s use in related data-management applications.
  • Research gaps: Vanilla RAG retrieves context independently for each query, whereas blocking-based batch retrieval shares work across similar entity-matching queries.The paper identifies cost-efficient batch retrieval and generation as an unexplored opportunity for RAG-based EM.
  • Contributions: The paper introduces CE-RAG4EM and a unified framework covering blocking-based retrieval and generation, retrieval granularity, vector search, and graph traversal.Experiments examine performance–overhead trade-offs, blocking methods, backbone LLMs, and key parameters such as maximum block size and Top-k context.

2 Preliminaries

The preliminaries define entity matching, LLM-based matching, knowledge-graph retrieval, and graph-aware serialization. They also formalize RAG inputs and batch inference, where one model call produces decisions for multiple record pairs.

  • Entity matching: Entity matching decides whether two records from source and target tables refer to the same real-world entity.The records belong to a shared record space and are evaluated as a pair.
  • LLM-based entity matching: LLM-based entity matching replaces a task-specific classifier with a generative model that reasons directly over serialized record pairs and produces a binary decision.Prompts expose textual descriptions of the records and request outputs such as yes or no.
  • RAG and KG-RAG: The knowledge retriever maps a record pair to relevant graph content, with entity-, predicate-, and triple-level retrieval providing different context granularities.Triple-level context is obtained by expanding from relevant entities or predicates in the knowledge graph.
  • Serialization: A graph-aware serializer maps record pairs and retrieved knowledge into a textual sequence for the matching model.The serialized input exposes original attributes together with extracted knowledge, such as formatted relevant triples.
  • RAG and KG-RAG: RAG-based entity matching augments serialized record pairs with retrieved knowledge, using textual entity or predicate descriptions in RAG and structured triples in KG-RAG.The retrieved context complements or enriches information contained in the records.
  • Batch inference: Batch RAG constructs one serialized input for multiple record pairs, and a single model call returns B match decisions for B pairs.The generative model’s output text is parsed into the batch’s individual decisions.

3 Methodology

CE-RAG4EM uses blocking to batch related entity-matching queries, retrieve contextual knowledge, expand graph evidence, and generate decisions efficiently. Its design combines blocking-based optimization with multiple retrieval granularities and knowledge refinement.

  • Blocking-based batch construction: CE-RAG4EM begins by blocking similar source and target records into matching pairs while deduplicating redundant pairs across blocks.This creates the batches used by subsequent retrieval and generation stages.
  • Contextual knowledge retrieval: Aggregated block queries are vectorized to retrieve Top-k entities and predicates from an external knowledge graph using embedding similarity.The retrieval pipeline maps queries and graph items into a shared d-dimensional embedding space and ranks them by cosine similarity.
  • Triple search and expansion: Retrieved entities seed breadth-first and neighborhood-based graph expansion to gather structurally relevant triples and subgraphs.The expansion operates on the Top-k retrieved entities and leverages explicit graph structure.
  • Knowledge enrichment and refinement: Knowledge enrichment converts symbolic entity and predicate identifiers into identifier-description representations before LLM reasoning.Triples are likewise represented with textual descriptions for their head, predicate, and tail.
  • Knowledge enrichment and refinement: Instruction-based knowledge refinement directs the LLM to use retrieved information only when it contributes meaningfully to the matching decision.The prompt supports relevance assessment before inference augmentation.
  • Batch generation and design space: Blocking-based batch generation combines entity pairs and shared contextual knowledge in one prompt, while the framework compares block-level optimization and retrieval granularity.The design space includes independent or joint batch retrieval and generation, alongside entity-, predicate-, and triple-based contextual knowledge.

4 Experiments

The experiments evaluate CE-RAG4EM against PLM- and LLM-based entity-matching baselines while analyzing retrieval granularity and blocking-based batching. They measure matching quality and efficiency across diverse benchmark datasets and controlled design choices.

  • Design dimensions: The study analyzes retrieval granularity by comparing entity- and predicate-level retrieval with knowledge-graph triple retrieval.This tests how contextual-evidence granularity affects relevance and downstream generation.
  • Design dimensions: Blocking-based optimization is evaluated through batch retrieval and batch generation, which amortize overhead across related queries.The two batching strategies can be studied as design choices within the CE-RAG4EM framework.
  • Metrics and protocol: Matching quality and efficiency are both measured, with retrieval overhead and end-to-end latency serving as primary efficiency metrics.Latency is treated as a proxy for computational cost under a fixed serving setup, and results are averaged across three random seeds.
  • Datasets: Experiments use nine Magellan and Web Data Commons entity-matching benchmarks spanning diverse domains, schemas, attribute types, and class imbalance.The datasets include textual, numeric, categorical, date, and mixed text-plus-numeric attributes.
  • Evaluation setup: The evaluation compares CE-RAG4EM with Ditto, Unicorn, direct LLM prompting, and vanilla per-query RAG4EM.Vanilla RAG4EM performs retrieval and generation independently for each query without KG traversal or triple augmentation.

4.2 Exp-1: Overall Effectiveness

CE-RAG4EM improves F1 over LLM-EM across all nine datasets and compares favorably with supervised PLM baselines, while its efficiency depends on retrieval and enrichment overhead.

  • CE-RAG4EM outperforms LLM-EM on all nine datasets in F1, with gains primarily driven by increased recall.Precision decreases on some datasets, including AMGO, WAAM, and WDC.
  • Retrieval introduces a precision–recall trade-off: relevant evidence can improve recall, while noisy context can increase false positives on ambiguous record pairs.This motivates analyzing retrieval granularity and context construction.
  • CE-RAG4EM can reduce end-to-end latency relative to LLM-EM because augmented context often shortens LLM outputs enough to offset retrieval and enrichment overhead.Retrieval is usually a small fraction of runtime, whereas enrichment can dominate on some datasets.
  • CE-RAG4EM’s comparison with PLMs uses F1, precision, and recall, with datasets ordered by CE-RAG4EM’s advantage and rows beating both PLMs first.The evaluation includes Ditto and Unicorn as supervised PLM baselines.
  • On six of nine datasets, CE-RAG4EM’s per-pair retrieval-plus-enrichment overhead is lower than the amortized training overhead of Ditto and Unicorn.The contrast is especially pronounced on smaller datasets, where PLM training overhead is large.

4.3 Exp-2: Retrieval Granularity

Exp-2 compares node-level retrieval with KG-triple context construction, showing that relational context can help on ambiguous datasets but increases construction overhead, especially with BFS.

  • The experiment compares PID/QID node-level retrieval with EXP/BFS KG-triple construction in CE-RAG4EM variants.EXP and BFS construct triples through expansion-based traversal or breadth-first search.
  • KG-triple context can improve matching on datasets with sparse or ambiguous attributes because relational information may disambiguate record pairs better than isolated entities or predicates.Examples of relational context include type, brand, artist, venue, and location.
  • EXP is frequently among the top-performing strategies and is best on several datasets, but its gains vary with starting-node quality and its limited expansion budget.EXP typically constructs a smaller, more focused context than BFS, which can reduce noise when initial retrieval is accurate.
  • BFS incurs the highest context-construction cost, while PID and QID add only small extra costs.BFS explores more broadly, increasing API and processing overhead relative to focused EXP expansion.

4.4 Exp-3: Batch vs. Per-Query Execution

Exp-3 evaluates per-query execution against block-based retrieval and generation, finding similar average F1 but different precision–recall balances and overhead profiles.

  • The experiment compares per-query RAG4EM with CE-RAG4EM-BR batch retrieval and CE-RAG4EM-BG batch generation.Timing is decomposed into retrieval, expansion, and generation to identify batching benefits.
  • The three variants achieve similar average F1, indicating that batching primarily changes precision–recall balance and system overhead.CE-RAG4EM-BG tends to increase recall while reducing precision, whereas CE-RAG4EM-BR preserves a more stable balance.
  • Batch generation can increase false positives because presenting multiple pairs together permits cross-pair coupling and shared evidence to influence decisions.This coupling can make the model more willing to predict matches, improving recall but reducing precision when context is weakly related.

4.5 Exp-4: Blocking Strategy Robustness

Blocking choices produce broadly stable performance, with q-gram methods generally outperforming standard token blocking; block size trades retrieval efficiency against matching quality.

  • Q-gram blocking methods outperform StdBlck on most datasets, while performance remains stable across blocking choices.XQGram is often best or near-best; QGram is more efficient in the implementation and is used as the default.
  • QGram and XQGram form batches that better tolerate typos and lexical variation than exact token-based blocking.More coherent batches support block-level retrieval, whereas fragmented blocks reduce shared retrieval context.
  • Larger maximum blocks reduce retrieval calls by sharing one retrieval result across multiple pairs.This improves efficiency because retrieval is performed once per block.
  • Maximum block sizes of 4 or 6 provide the best balance, maintaining near-peak average F1 while substantially reducing retrieval API calls.The trade-off arises because larger blocks dilute query specificity and can provide less targeted context.

4.7 Exp-6: KG-RAG Design Choices

KG-RAG design choices trade relational coverage and matching quality against traversal and expansion overhead. EXP supplies concise local context, whereas BFS can benefit ambiguous cases but becomes costlier as its triple budget grows.

  • Traversal strategy: EXP provides concise one-hop context suited to accurate seeds and near-exact identifiers, while BFS adds multi-hop context for semantically ambiguous records.BFS can also introduce off-topic relations, making its benefit more dataset-dependent.
  • Triple budget: Increasing the triple budget from Top-1 to Top-2 tends to help BFS more than EXP.A second BFS triple can improve evidence coverage, while EXP often shows diminishing returns because it prioritizes salient one-hop relations.
  • Triple budget: Larger triple budgets can improve coverage and recall but require effective refinement to avoid irrelevant knowledge.The result is a quality–noise trade-off in KG-RAG context construction.
  • Traversal cost: BFS can become competitive with or better than EXP on many datasets when its triple budget increases, but larger multi-hop neighborhoods substantially increase expansion overhead.BFS incurs markedly higher expansion and enrichment time than lightweight EXP.

4.8 Exp-7: Backbone LLM Generalization

CE-RAG4EM transfers across API and open-source backbone architectures, with gains varying by model capacity and evidence utilization. Smaller models benefit more strongly but less consistently, while Gemini-2.0-flash-lite shows higher variance.

  • CE-RAG4EM-BR produces predominantly positive ΔF1 across datasets for GPT-4o-mini and both Qwen3 backbones.The evaluation uses the same batch-retrieval pipeline across backbone models.
  • Relevant entity descriptions reduce uncertainty and help API and open-source models align attributes more consistently.The descriptions provide an external anchor when record pairs alone do not fully determine the match.
  • Smaller backbones, notably Qwen3-4B, show more pronounced and less stable gains than larger backbones.Larger models show smaller but generally positive improvements, suggesting diminishing returns when surface cues already support matching.
  • Gemini-2.0-flash-lite exhibits higher ΔF1 variance, including occasional degradations on some datasets.Weakly related retrieved descriptions may be overweighted by some models, causing incorrect matches.

5 Discussion and Recommendation

The discussion identifies recurring performance–cost trade-offs across blocking, retrieval granularity, traversal, and model size. It recommends tuning these choices to record attributes, similarity volume, and model context limits.

  • Summary of Empirical Findings: Blocking size reduces retrieval and inference costs through shared prompts and knowledge, but F1 drops beyond a threshold such as 6.Noisy shared context and embedding or LLM input-length limits contribute to the decline.
  • Summary of Empirical Findings: Triple-based KG-RAG improves recall for ambiguous numeric or mixed-type pairs but adds triple-search and traversal overhead.Its stronger F1 depends on attribute diversity and comes with a clear F1-cost trade-off relative to node-level retrieval.
  • Summary of Empirical Findings: BFS offers deeper multi-hop context and greater coverage, but it introduces more noise and substantially increases triple-search and enrichment time.Local expansion is lighter, so search strategy and depth must balance implicit knowledge against computational efficiency.
  • Recommendations: A maximum block-size range of 4–6 is recommended because it reuses retrieval and reduces retrieval and inference cost without compromising F1.The recommendation is to tune block size dynamically using record attributes and backbone context-window limits.
  • Recommendations: Lightweight backbone models are prioritized because grounding can provide strong gains while maintaining an F1-cost balance.Medium or large models are reserved for highly complex cases requiring intensive internal reasoning.
  • Recommendations: Retrieval granularity should shift from node-level to deeper triple-level contexts as attributes become more diverse and matching more ambiguous.The decision matrix also considers whether the dataset contains small or large volumes of similar records.

6 Related Work

Related work covers PLM- and LLM-based entity matching, blocking methods, and retrieval-augmented approaches using textual or structured knowledge. These strands motivate blocking-based retrieval for efficient RAG-based entity matching.

  • PLM and LLM-based Entity Matching: PLM- and LLM-based entity matching models capture contextual information to classify whether record pairs match.
  • PLM and LLM-based Entity Matching: Selective matching methods combine prompting, ranking, Siamese structures, or contrastive learning to distinguish true matches from similar negatives.
  • Blocking for Entity Matching: Blocking reduces entity-matching complexity by removing likely non-matching pairs or selecting subsets of candidate pairs.
  • RAG: RAG retrieves relevant textual contexts and incorporates them into prompts to guide LLM answers and mitigate hallucinations.
  • GraphRAG and KG-RAG: GraphRAG supports contextual and multi-hop reasoning with relevant subgraphs, but irrelevant or noisy graph context can reduce performance and cause hallucination.
  • GraphRAG and KG-RAG: Graph construction from textual documents is time-consuming because entity and relationship extraction requires substantial computing resources and domain expertise.

7 Conclusion

The paper introduces CE-RAG4EM to address computational inefficiency in RAG-based entity matching through blocking-based batch retrieval and inference. Its unified evaluation framework and experiments support lower costs, competition from smaller models, and practical guidance for scalable RAG systems.

  • CE-RAG4EM addresses computational inefficiency in RAG-based entity matching through blocking-based batch retrieval and inference.
  • The unified framework analyzes and evaluates CE-RAG4EM across diverse retrieval granularities.
  • CE-RAG4EM significantly reduces retrieval and inference cost while allowing smaller open-source models to compete with larger LLMs.
  • The empirical findings provide practical guidance for building scalable, cost-efficient, and reliable RAG systems for real-world data integration.
Loading 2602.05708v1…