CaRL-EM: Cost-Aware Reinforcement Learning for Entity Matching with LLMs

• LLM-based EM methods can rival supervised models, but independent pairwise decisions miss candidate interactions and clean-clean exclusivity, while inference at industrial scale makes cost significant.Source: §1 · p. 1
• COMEM enables more candidate interaction, but its pipeline applies the same operations regardless of instance difficulty, treating cost as a fixed byproduct [Wang et al., 2025; Chen et al., 2023; Ong et al., 2025].Source: §1 · p. 1
• For each anchor, CaRL-EM maintains a compact state and chooses MATCH, COMPARE, SELECT, or DECIDE plus model capacity to refine candidates, select from a shortlist, or terminate.It is designed for clean-clean EM, where each anchor has at most one true match and the output is one candidate or NONE.Source: §1 · p. 2
• The controller uses abstract two-level costs and black-box operators, allowing stronger LLM backends to be plugged in at test time without retraining.The policy is trained with cost-aware rewards to reduce unnecessary calls.Source: §1 · p. 2
• CaRL-EM formulates blocked multi-candidate EM as cost-aware sequential RL, with a policy observing matching state and accumulated cost to choose the next operator.The paper presents this as the first work to cast LLM-driven EM as a cost-aware sequential RL process.Source: §1 · pp. 1–2
• On 7 benchmarks, CaRL-EM achieves higher F1 and reduces cost 78% versus the best manually designed composite pipelines; it reaches approximately 89% of supervised EM performance while reducing expense 94%.Source: §1 · p. 2

2.1 Traditional and Pretrained EM

• Earlier EM used string similarity and manual rules, while neural and pretrained models improved semantic matching but remained label-dependent or pairwise and limited in domain transfer.Examples include DeepER, DeepMatcher, DITTO, and dual-encoders (Ebraheem et al., 2017; Mudgal et al., 2018; Li et al., 2020; Shah et al., 2018; Tracz et al., 2020).Source: §2.1 · p. 2

2.2 LLM-Based Strategies

• LLMs support zero- or few-shot pairwise matching with explanations, but Yes/No prompting preserves independent pairwise decisions; COMEM uses a fixed, manually designed multi-candidate pipeline.COMEM re-ranks candidates with cheaper steps before a stronger LLM selects the final match, so pipeline choice determines inference cost (Wang et al., 2025; Peeters et al., 2025).Source: §2.2 · p. 2

2.3 Cost-Aware Control with RL

• Cost-aware policies and RL have been studied for controlling LLM inference in question answering and reasoning, while EM commonly relies on fixed pipelines.CaRL-EM frames multi-candidate EM as a cost-aware sequential decision problem and learns to choose MATCH, COMPARE, SELECT, DECIDE, and LLMs with different costs to navigate the accuracy-cost trade-off.Source: §2.3 · p. 2

3 CaRL-EM: A Cost-Aware RL Controller for LLM-Based EM

• For each anchor, CaRL-EM learns a policy to choose the next action and balance expected matching quality against inference cost.The setting assumes at most one true match in the candidate set, or no match.Source: §3 · p. 3

3.1 LLM-Based Operators and Abstract Cost

• CaRL-EM combines MATCH, COMPARE, and SELECT with a terminal DECIDE action, assigning higher abstract cost to listwise SELECT than to local operations.MATCH updates candidate confidence, COMPARE sharpens relative scores, and SELECT uses listwise reasoning over top-k candidates without making the final decision. Relative cost tiers guide policy learning rather than exactly pricing a deployment, enabling backend swaps without retraining.Source: §3.1 · p. 3

3.2 MDP Formulation

• CaRL-EM models each anchor as an episodic MDP whose state tracks candidate confidence, resource use, and action history, and whose reward favors correct, cost-conscious decisions.State features include candidate eligibility and comparison frequency, accumulated cost, step, and the maximum score; optional semantic embeddings and training-time Gaussian noise provide grounding and robustness. The reward also penalizes premature low-confidence decisions.Source: §3.2 · pp. 3–4

3.3 Policy, Training, and Inference

• A three-layer MLP policy is trained with PPO and, at inference, can use stronger LLM backends without retraining if operator interfaces and relative cost levels are preserved.The controller acts until it chooses DECIDE or reaches the step limit.Source: §3.3 · p. 4

LLMs used in our experiments

• Experiments cover proprietary commercial APIs and open-weight LLM backends, with open-weight model prices reported as ranges from multiple inference providers.Model specifications and costs are summarized in Table 1.Source: §4 · pp. 4–5, Table 1 · p. 5

Datasets and Zero-shot Transfer Setting

• Each dataset’s evaluation set contains 400 anchors, and Sparkly retrieves the top 10 candidates with blocking recall@10 ranging from 94.89% to 99.96%.The 400 anchors include 300 with one true match and the remainder with none; the task requires selecting one candidate or rejecting all, following the COMEM protocol (Wang et al., 2025).Source: §4 · p. 5
• CaRL-EM is trained only on Abt-buy and tested on seven other benchmarks without fine-tuning or adaptation, while supervised DITTO receives 5,000 labeled pairs per target dataset (Li et al., 2020).The benchmarks span e-commerce, academic citations, movies, and restaurants; the transfer setup uses a different training domain and source from the test datasets, with no target-domain training data.Source: §4 · p. 5

Evaluation Metrics

• The evaluation uses instance-level F1id for valid-match anchors, F1none for no-match anchors, and F1macro as their unweighted average.A success requires selecting the exact ground-truth candidate; choosing a wrong candidate counts as both a false positive and a false negative. Under single-match validity, the protocol aligns mathematically with standard pairwise F1.Source: §4 · p. 5

Cost Calculation

• Cost accounting includes LLM token costs and standardized GPU compute costs of $5.98 per hour for CaRL-EM training and DITTO’s full lifecycle.CaRL-EM’s one-time offline training cost is reported separately; its lightweight inference policy is assumed to have zero hardware cost because it runs on a commodity CPU.Source: §4 · pp. 5–6

5.2 Cost-Efficiency Analysis

• CaRL-EM forms a new Pareto frontier, and GPT-4o mini uses only 22% of COMEM’s average total cost at higher F1macro.Swapping in a stronger backend usually improves F1macro but increases cost; the gains mainly come from the controller’s adaptive decisions rather than any single LLM.Source: Figure 3 · p. 6, §5.2 · p. 6

5.1 Main Results

• CaRL-EM averages 76.09 F1macro versus DITTO’s 85.64 while using 5.9% of DITTO’s inference cost.Among zero-shot methods, CaRL-EM has the best average F1macro and outperforms COMEM; it uses about 4.5× less cost. Its global SELECT followed by local MATCH and COMPARE checks can stop with no-match when evidence is insufficient.Source: §5.1 · p. 6, Table 2 · p. 7

5.4 Controller Behavior and Position Bias

• CaRL-EM varies operator use by dataset and takes more steps on harder datasets such as AG and WA, unlike COMEM’s fixed pipeline.It uses fewer MATCH and COMPARE calls than COMEM, while varying tool use across domains and datasets.Source: Figure 4 · p. 7, §5.4 · p. 7
• CaRL-EM stays more stable across gold-candidate positions and has the smallest cross-dataset variation among the compared methods.The controlled test fixes the gold candidate at positions 0–9 and shuffles the remaining candidates; CaRL-EM’s combination of SELECT with MATCH/COMPARE and mutual verification reduces reliance on initial order.Source: Figure 5 · p. 7, §5.4 · p. 7

5.3 Transfer and Robustness to LLM Backends

• Without retraining, CaRL-EM trained once on AB with GPT-4o mini maintains a high F1macro across datasets; DITTO’s F1macro falls from 85.64 to 59.75 when trained on AB and applied to other datasets.At inference, stronger backends usually improve F1macro but increase cost; the controller adjusts MATCH and COMPARE use, learning a decision sequence rather than overfitting to a dataset or LLM.Source: §5.3 · pp. 6–7, Table 3 · p. 8

5.5 Qualitative Analysis: Policy Behavior

• In the easy AG case, CaRL-EM selects the gold candidate and verifies it with two COMPARE calls; in the hard case, it uses repeated MATCH calls to identify it.The easy case avoids MATCH, while the hard case spends more local checks after SELECT initially returns a conflicting candidate, illustrating budget allocation by instance difficulty.Source: §5.5 · pp. 7–8, Figure 6 · p. 8

5.6 Ablation and Design Choices

• Removing the cost term leaves F1macro almost unchanged but raises cost by 15%; removing potential shaping reduces F1id and F1macro and destabilizes performance.Without MATCH, the model is cheaper but less accurate, especially on none cases; without SELECT it is cheapest with high F1none but lower F1id and overall performance, while removing COMPARE also hurts performance.Source: §5.6 · p. 8

6 Conclusion

• CaRL-EM treats blocked entity matching as a sequential decision problem, choosing when to use MATCH, COMPARE, SELECT, or DECIDE and between cheaper and stronger model capacities.The setting gives each anchor a small candidate set, while inference cost quickly becomes a bottleneck at scale.Source: §6 · pp. 8–9
• The authors describe this as the first work to cast LLM-driven entity matching as a cost-aware sequential reinforcement-learning problem.They frame the learned controller as an alternative to fixed pipelines for trading off decision-strategy performance and cost.Source: §6 · p. 9
• Across seven benchmarks under zero-shot transfer, CaRL-EM achieves a better quality-cost trade-off than strong LLM baselines and hand-designed pipelines.It learns to spend less on easy cases and check more on hard ones.Source: §6 · p. 9

Limitations

• The clean-clean assumption excludes multiple valid matches, one-to-many links, and noisy or duplicate-heavy tables, which may require changes to the state and stopping or decision rule.The proposed state could track multiple plausible matches.Source: Limitations · p. 9
• The protocol uses top-10 candidate pools; much larger sets may require multi-stage pruning, hierarchical control, or tighter retrieval coupling to maintain cost and decision quality.Source: Limitations · p. 9
• CaRL-EM's coarse two-level cost model omits prompt length, token billing, latency, batching, and provider pricing, so deployment may require cost recalibration or retraining.The abstraction simplifies backend swapping but may not be cost-optimal under a different cost surface.Source: Limitations · p. 9

Ethical Considerations

• Incorrect matches can cause wrong merges and downstream errors; high-stakes use warrants auditing and human review, while cross-dataset personal-identity linking is discouraged and legal and ethical compliance emphasized.The experiments use public entity-matching benchmarks, and no additional data are released.Source: Ethical Considerations · p. 9

A Hyperparameter Values

• Table 5 lists the concrete hyperparameter values used in the experiments.The supplied passage identifies the table but does not provide its values.Source: A Hyperparameter Values · p. 11

B.1 Match

• The Match template asks whether two records refer to the same real-world entity and requires exactly one first-line token: [YES] or [NO].It supplies an anchor and candidate; [YES] means same and [NO] means different.Source: §B.1 · p. 11

B.2 Compare

• The Compare template asks which of two candidates better matches the anchor and requires one token: [0] for the first candidate or [1] for the second.The prompt presents the anchor and both candidates as indexed options.Source: §B.2 · p. 11

B.3 Select

• The Select template asks for the single best-matching candidate among indexed options, allowing [NONE] when no candidate is selected.The response must be exactly one first-line token from [0..n-1] or [NONE], with no other words or punctuation.Source: §B.3 · p. 11

C Sensitivity to the Abstract Cost Ratio ρ

• With CaRL-EM retrained on Abt-Buy and evaluated zero-shot on seven target benchmarks, raising ρ from 2.5 to 5 kept F1macro nearly unchanged but slightly increased MATCH/COMPARE calls and benchmark variance.The F1macro result is reported in Table 6.Source: C Sensitivity to the Abstract Cost… · p. 11

D Confidence Update Rules

• On WA, CaRL-EM’s performance remained relatively stable across training sizes despite longer fine-tuning, while DITTO fine-tuned faster but dropped sharply at 25% and 50% training sizes.CaRL-EM’s additional wall-clock time is attributed to repeated backend-operator calls during rollout collection.Source: Figure 7 · p. 12
• Matcher YES probabilities are mapped from [0, 1] to [−1, 1] and used to update each candidate’s internal confidence score.The update uses exponential smoothing.Source: D Confidence Update Rules · pp. 11–12
• The matcher uses α_hi when |s − 0.5| exceeds match_margin and α_lo otherwise.Source: D Confidence Update Rules · p. 12
• The selector update distinguishes valid outputs that identify an index in the current top-k shortlist.Source: D Confidence Update Rules · p. 12
• Invalid selector outputs, such as parsing failures, trigger a penalty to the shortlist.Source: D Confidence Update Rules · p. 12
• All confidence scores are clipped to [−1, 1].Source: D Confidence Update Rules · p. 12

E H100 pricing from different cloud service platforms

• H100 server quotes from four cloud providers were used to estimate an average market price.Table 8 lists the collected quotes.Source: E H100 pricing from different cloud… · p. 12

F Blocking Cost and One-time Offline Cost for CaRL-EM

• The cost analysis reports blocking cost per dataset and token and GPU usage during training on AB.The reported H100 GPU server price is $4/h.Source: F Blocking Cost and One-time Offline… · pp. 12–13, Table 9 · p. 13, Table 10 · p. 13
• The end-to-end accounting reports blocking, training, and inference costs separately alongside their total; unlike DITTO, CaRL-EM trains once on AB and transfers without target-dataset retraining.DITTO requires separate training for every target dataset.Source: F Blocking Cost and One-time Offline… · p. 13

Training Scalability and Wall-Clock Analysis

• On WA, CaRL-EM’s wall-clock fine-tuning time is higher but performance remains relatively stable across training sizes, while faster DITTO degrades substantially in low-data settings.The comparison uses 25%, 50%, 75%, and 100% of WA′’s training set and the same WA test set.Source: F Blocking Cost and One-time Offline… · p. 13
• With backend operators replaced by a simulator, tuning at 100k steps takes about 1.6 minutes, indicating backend inference latency—not PPO optimization—is the main wall-clock cost.CaRL-EM’s repeated backend-operator calls during rollout collection mainly explain its higher fine-tuning time.Source: F Blocking Cost and One-time Offline… · p. 13