Match, Compare, or Select? An Investigation of Large Language Models for Entity Matching

1 Introduction

• Independent pairwise matching ignores global consistency, although ER record relationships are interdependent and LLMs can distinguish similar records when considering them together.In record linkage, a source record typically matches at most one record in the other source; seeing multiple SanDisk models together can help distinguish them.Source: §1 · pp. 1–2
• The paper compares conventional matching with comparing, which picks the likelier match from two records, and selecting, which chooses from a candidate list.Both added strategies use information from other records; the authors report selecting is often the most cost-effective, while its accuracy varies with the matching record's list position.Source: §1 · p. 2
• ComEM first ranks and filters candidates with a medium-sized LLM using matching or comparing, then applies a more powerful LLM's selecting strategy to the top k candidates.The staged pipeline reduces selecting's many-option challenges and lowers invocation costs.Source: §1 · p. 2
• 16.02% average F1 improvement over the current matching strategy was achieved by selecting across eight ER datasets; ComEM further improved average F1 by up to 4% while reducing cost.Experiments also examined 10 LLMs.Source: §1 · p. 2
• The introduction frames the work as investigating the strategies' advantages and shortcomings across scenarios and evaluating a framework that combines strategies and LLMs.The authors report experiments on eight ER datasets.Source: §1 · p. 2

2.1 Entity Resolution

• Entity resolution commonly uses blocking to filter dissimilar records before matching identifies duplicates; prior approaches include rule-based, distance-based, probabilistic, deep-learning, and pretrained-language-model methods.Most deep-learning and LLM methods treat matching as independent binary classification, with GNEM as an exception modeling collective classification on graphs.Source: §2.1 · pp. 2–3

2.2 Large Language Model

• The paper formulates entity matching as comparison or selection and reports cross-model, cross-strategy evaluation to reveal patterns that motivate ComEM.It notes pairwise and listwise strategies in entity matching exhibit patterns different from ranking.Source: §2.2 · p. 3

3.1 Problem Formulation

• The formulation identifies a match between an anchor record and its blocked candidate list, accommodating one-to-one record linkage and single-source deduplication.It uses the widespread assumption that an anchor matches at most one candidate.Source: §3.1 · p. 3

3.2 LLM as a Matcher

• Matching independently classifies each anchor-candidate pair, but misses global consistency and depends on probabilities that may be unavailable or misaligned.The paper uses few-shot prompting for this strategy and leaves exploration of better prompts for other strategies to future work.Source: §3.2 · pp. 3–4

3.3 LLM as a Comparator

• Comparing asks which of two candidates better matches the anchor, supporting ranking and similarity scoring even with black-box LLMs.Comparing all pairs has O(n^2) complexity; bubble sort reduces top-k selection to O(kn).Source: §3.3 · p. 4

3.4 LLM as a Selector

• Selecting chooses a candidate from a shared list, letting the LLM consider record interactions, but long contexts and position bias constrain the strategy.When no candidate matches, the formulation can offer a “none of the above” option.Source: §3.4 · pp. 4–5

3.5 Compound Entity Matching Framework

• COMEM first ranks and filters candidates with matching or comparing, then applies selecting to the top k for fine-grained identification.This pipeline is intended to mitigate selecting’s position bias and long-context demands, and matching’s neglect of consistency.Source: §3.5 · p. 5, Figure 2 · p. 5
• COMEM improves entity-matching performance while reducing practical-application cost by integrating strategies and LLMs of different sizes.The framework delegates ranking to medium-sized LLMs and uses a more powerful LLM to identify a match among the top candidates.Source: §3.5 · pp. 5–6

4.1 Experimental Setup

• The evaluation uses eight clean-clean ER datasets with 10 blocked candidates per record, and all methods use the same post-blocking datasets.Baselines include unsupervised, self-supervised, LLM-based, and supervised methods; performance and LLM invocation cost are reported.Source: §4.1 · p. 6

4.2 Main Results

• Comparing improves average F1 by 10.7% over matching, while selecting adds 5.32% over comparing and beats 6-shot matching by 7.97% in F1.Source: §4.2 · pp. 6–7
• COMEM achieves 2~18% F1 improvements over single comparing and selecting strategies while spending less.Its filtering-and-identifying pipeline improves precision without sacrificing the selecting strategy’s high recall.Source: Table 4 · p. 7, §4.2 · p. 7

4.3 Analysis of Strategies

• Selecting costs less than half as much as matching, whereas comparing costs more than twice as much; comparing and selecting are also position-sensitive, which COMEM alleviates.Selecting avoids repeatedly inputting anchor records and task instructions; an incorrect comparison can prevent a matching record from reaching the top under bubble sort.Source: §4.3 · pp. 7–8, Figure 3 · p. 7

4.4 Effect of LLMs

• No single open-source LLM dominates every strategy: selecting generally outperforms comparing, which generally outperforms matching, while model performance varies by strategy.Matching outperforms comparing for ranking across model sizes, with a small difference for Flan-T5-XL (3B) and Flan-T5-XXL (11B), and a significant difference for smaller models.Source: Figure 4 · p. 8, §4.4 · p. 8, Figure 5 · p. 8

4.5 Ablation Study

• As the number of retained candidates changes, GPT-3.5 Turbo’s performance varies considerably, while GPT-4o Mini’s is relatively stable.The authors suggest COMEM may become more robust to the number of retained candidates as LLMs evolve.Source: Figure 6 · p. 9, §4.5 · p. 9

• The study finds that incorporating record interactions is essential for LLM entity matching and that COMEM’s effectiveness and cost-efficiency highlight task decomposition and LLM composition.Source: §5 · p. 9

Limitations

• The experiments cover two commercial and eight open-source LLMs, selected based on popularity, availability, and cost.The authors propose checking whether findings hold as LLMs evolve.Source: Limitations · p. 9
• Because LLMs may have seen similar records or matching results in web training data, the authors call for evaluation on new or non-public data.They report that strategy performance is relatively consistent across 10 LLMs and varies greatly for the same LLM across strategies, so data exposure is not the determining factor in effectiveness.Source: Limitations · p. 9
• The study investigates strategies with basic zero- or few-shot prompting; the authors identify advanced prompt engineering and fine-tuning as worthwhile directions.They note the potential role of advanced prompt engineering in improving accuracy and robustness.Source: Limitations · p. 9
• The authors demonstrate ComEM's effectiveness for entity matching and propose developing specific modules and extending the paradigm to other stages of entity resolution.Source: Limitations · p. 9

A Global Consistency of ER

• Global consistency means a pair's matching decision is influenced by matching results for other record pairs, rather than being isolated.Source: A Global Consistency of ER · p. 13
• The concept includes reflexivity, symmetry, and transitivity: a record matches itself, matching is reciprocal, and matches carry through chains.Source: A Global Consistency of ER · p. 13
• In clean-clean entity resolution, mutual exclusivity can also apply: if A matches B, it cannot match C.Source: A Global Consistency of ER · p. 13

B Strategy Prompts

• The same manually engineered prompts were used for all experimental LLMs to ensure fairness.Prompt variants were tested and compared to select the prompts used.Source: B Strategy Prompts · p. 13
• Prompts included task descriptions and response instructions such as answering “Yes” or “No” to guide LLM responses.Source: B Strategy Prompts · p. 13
• For in-context learning, prompts and labels were repeatedly supplied before the records, and LLM responses were post-processed into final predicted labels.Source: B Strategy Prompts · p. 13

C Detailed Results of Open-Source LLMs under Different Strategies

• The authors describe the proposed strategies as universally applicable across the evaluated LLMs for entity matching.Source: C Detailed Results of Open-Source LLMs… · p. 13
• Many chat-tuned LLMs are difficult to constrain to the desired output through prompts alone, which may affect their actual performance.The authors identify calibration of label probabilities from long-form responses as important for performance improvement.Source: C Detailed Results of Open-Source LLMs… · p. 13
• Six of eight open-source LLMs achieved their best F1 performance with selecting, while two performed better with comparing.Table 6 presents open-source LLM F1 scores under the strategies and COMEM.Source: C Detailed Results of Open-Source LLMs… · p. 13, Table 6 · p. 14