PaperlayerView exact revision on arXiv

Source-linked AI summary

CaRL-EM: Cost-Aware Reinforcement Learning for Entity Matching with LLMs

Chaohui Guo, Michel Klein, Zhisheng Huang

arXiv:2609.01195v1cs.CL

TL;DR

LLM-based entity matching must handle candidate interactions and inference cost beyond independent pairwise decisions or fixed pipelines. CaRL-EM addresses this with a cost-aware reinforcement-learning controller that adaptively selects operators and model capacities. Across seven benchmarks, it transfers zero-shot and achieves a better quality-cost trade-off than strong baselines and hand-designed pipelines.

  • Problem

    Most LLM-based entity matching methods ignore candidate interactions, set-level exclusivity, adaptive operation choice, and inference cost at scale.

  • Method

    CaRL-EM learns a policy that sequentially selects MATCH, COMPARE, SELECT, or DECIDE operators and model capacities using abstract cost-aware actions independent of specific LLM backends.

  • Results

    Across seven benchmarks, CaRL-EM achieves a better quality-cost trade-off than strong LLM baselines and manually designed pipelines, with 78% lower cost than the best manually designed composite pipelines.

  • Takeaways & Limitations

    The learned controller spends less on easy cases and checks more on hard ones while enabling backend swaps at test time without retraining.

  • Takeaways & Limitations

    The study focuses on clean-clean EM with at most one true match and mainly evaluates small candidate pools of top-10 candidates.

Abstract

from arXiv · show

Entity matching (EM) requires fine-grained contextual understanding and domain knowledge. Recent work shows that large language models (LLMs) can serve as strong matchers across domains, but most methods either make independent pairwise decisions or rely on manually designed composite pipelines, thus lacking flexibility in realistic multi-candidate settings. At the same time, they typically ignore inference cost at scale. We formulate LLM-based EM with candidates as a cost-aware sequential decision problem and propose CaRL-EM, a reinforcement learning controller that manages LLM operations. Given the state of an anchor record, its candidate set, and the cost, CaRL-EM adaptively chooses among different operators (Match/Compare/Select/Decide) and model capacities to maximize a quality-cost objective. The policy interacts with abstract operators, allowing the same controller to be reused with different underlying LLM backends at inference time without retraining. Experiments on 7 benchmarks show that CaRL-EM (i) learns to dynamically plan the usage of inexpensive and expensive operators based on task complexity, (ii) achieves robust zero-shot transfer across diverse datasets and domains, and (iii) consistently achieves a better quality-cost trade-off than strong LLM-based baselines and manually designed pipelines, yielding a lower inference cost at comparable or higher quality.

1 Introduction

CaRL-EM treats multi-candidate LLM-based entity matching as a cost-aware sequential decision problem rather than a fixed or independent pairwise process. Its RL controller adaptively selects operators and model capacities, achieving improved quality-cost trade-offs across seven benchmarks.

  • Motivation: LLM-based EM methods often make independent pairwise decisions, overlooking candidate interactions and clean-clean set-level exclusivity.At industrial scale, millions of anchors and large candidate sets make inference cost a major concern.
  • Motivation: Manually designed pipelines improve candidate interaction but apply the same operations to every instance, treating cost as a fixed architectural byproduct.This prevents operation usage from adapting to instance difficulty.
  • Approach: CaRL-EM formulates LLM-based multi-candidate EM as a cost-aware sequential decision problem controlled by reinforcement learning.The policy observes matching state and accumulated cost information when deciding which operator to apply next.
  • Approach: The controller chooses MATCH, COMPARE, SELECT, or DECIDE together with model capacity, balancing local refinement, listwise selection, and termination.Operators are treated as black-box actions with abstract costs, separating the policy from the underlying LLM backend.
  • Results: 78% lower cost and higher F1 score than the best manually designed composite pipelines are reported across 7 benchmarks.CaRL-EM also reaches approximately 89% of domain-specific supervised EM performance without fine-tuning while reducing 94% of the expense.

2 Related Work

Prior EM methods rely on rules, labeled neural models, or pairwise LLM decisions, while COMEM introduces candidate interaction through a fixed pipeline. CaRL-EM extends this direction by learning cost-aware sequential control over EM operations.

  • Earlier EM methods: Early EM systems used string similarity and manual rules, while neural models improved semantic capture but depended heavily on labeled data.Later pretrained models include cross-encoders such as DITTO and dual-encoders.
  • LLM-based EM: LLMs can perform pairwise matching zero- or few-shot, but prompting them for separate Yes/No decisions preserves the pairwise limitation.These approaches may rival supervised models while providing explanations.
  • LLM-based EM: COMEM adds multi-candidate interaction through MATCH, COMPARE, and SELECT, but its pipeline structure and inference cost remain fixed across anchors.The pipeline first uses cheaper steps to rerank candidates and then invokes a stronger LLM for final selection.
  • Cost-aware inference: Cost-aware policies and reinforcement learning have mainly controlled tool calls or context in question answering and reasoning rather than entity matching.CaRL-EM applies this sequential cost-quality framing directly to multi-candidate EM.

3 CaRL-EM: A Cost-Aware RL Controller for LLM-Based EM

CaRL-EM models multi-candidate entity matching as a cost-aware sequential decision process, using a reinforcement-learning controller to select operators, model capacities, and stopping decisions. Its abstract cost interface separates the controller from specific LLM backends while state features and rewards guide quality-cost trade-offs.

  • MDP Formulation: CaRL-EM casts blocked multi-candidate matching as a cost-aware sequential decision problem that selects one candidate or NONE.The clean-clean setting assumes at most one true match per candidate set.
  • LLM-Based Operators and Abstract Cost: The controller chooses among MATCH, COMPARE, SELECT, and terminal DECIDE actions, with operands specified by each action.MATCH and COMPARE perform local pairwise operations, SELECT performs shortlist-level reasoning, and DECIDE outputs a candidate or None.
  • LLM-Based Operators and Abstract Cost: MATCH and COMPARE refine candidate confidence locally, whereas SELECT applies listwise reasoning to boost the likely match and penalize alternatives.SELECT operates on the top-k candidates ranked by current scores and refines the global ranking without making the final decision.
  • LLM-Based Operators and Abstract Cost: Operators receive low or high abstract cost labels, and the RL objective penalizes action cost to encourage cheap operators whenever they suffice.The default high-to-low cost ratio is ρ=2.5; increasing it to ρ=5 yields nearly the same F1macro but more MATCH/COMPARE usage and higher cross-benchmark variance.
  • LLM-Based Operators and Abstract Cost: Because the policy interacts with abstract operators and relative cost labels, stronger underlying LLMs can be substituted at inference time without retraining.The operator interfaces and relative cost levels must remain preserved for this reuse.
  • MDP Formulation: CaRL-EM learns when to continue or stop through terminal correctness rewards, shaping signals, and penalties for cost or premature low-confidence decisions.A lightweight three-layer MLP policy is trained with PPO and a learned value baseline.
  • MDP Formulation: The controller state combines candidate scores, eligibility masks, comparison frequencies, global cost and time, action history, and optional semantic embeddings.Small Gaussian noise is added during training to improve policy robustness.

4 Experiments

The experiments evaluate CaRL-EM with diverse proprietary and open-weight LLM backends under a zero-shot transfer protocol across entity-resolution benchmarks. Comparisons standardize candidate sets and account for GPU and deployment-cost assumptions.

  • LLMs used in our experiments: The evaluation covers proprietary commercial APIs and open-weight LLMs, with open-weight prices reported as ranges from multiple inference providers.Table 1 summarizes model specifications and costs, including Chain-of-Thought support.
  • Datasets and Zero-shot Transfer Setting: CaRL-EM is trained on Abt-buy and evaluated without fine-tuning or adaptation on seven benchmarks spanning e-commerce, academic citations, movies, and restaurants.The protocol prevents target-domain training and data leakage by using a different source and domain for training.
  • Evaluation Setting: Local computation runs on NVIDIA H100 GPUs, with GPU cost standardized at $5.98 per hour for economic comparison.CaRL-EM policy inference is assumed to have zero hardware cost because its lightweight MLP can run efficiently on a commodity CPU.
  • Evaluation Setting: All compared methods use identical candidate sets from the same blocking stage, whose cost is treated as negligible relative to LLM-based decision cost.Blocking cost is reported separately in Appendix F.

5 Results

Across seven benchmarks, CaRL-EM delivers competitive or higher quality-cost performance through zero-shot transfer, backend swapping, and adaptive operator use. Its learned controller varies computation with dataset and instance difficulty, improving robustness while avoiding unnecessary calls.

  • Main Results: CaRL-EM achieves average F1macro 76.09 versus DITTO’s 85.64 while using 5.9% of DITTO’s cost across seven datasets.CaRL-EM is trained once on AB and transferred without additional fine-tuning, whereas DITTO is trained separately on each target dataset.
  • Cost-Efficiency Analysis: Under the same LLM, CaRL-EM (GPT-4O MINI) achieves higher F1macro than COMEM while using only 22% of its average total cost.Across the evaluated methods, CaRL-EM forms a new Pareto frontier, with gains attributed mainly to adaptive controller decisions.
  • Transfer and Backend Robustness: CaRL-EM transfers across the seven target datasets without retraining, whereas DITTO’s F1macro drops from 85.64 to 59.75 when transferred from AB.The controller also supports swapping stronger backend LLMs at inference time without retraining, with stronger backends usually improving F1macro.
  • Controller Behavior and Position Bias: CaRL-EM uses fewer MATCH and COMPARE calls than COMEM, while varying tool usage across domains and taking more steps on harder datasets such as AG and WA.This contrasts with COMEM’s fixed pipeline, whose call pattern cannot adjust across instances or datasets.
  • Controller Behavior and Position Bias: CaRL-EM remains more stable across candidate positions and shows the smallest cross-dataset variation, while Selecting is more position-sensitive and other baselines vary differently.The figure measures F1macro against the gold candidate’s initial list position, with shaded regions representing variation across datasets.
  • Qualitative Policy Behavior: Easy cases use SELECT and a few COMPARE checks before early termination, whereas hard cases trigger multiple MATCH calls to resolve conflicting candidate scores.In the AG examples, the easy case avoids pairwise MATCH, while the hard case spends more local checks on the ambiguous candidates.
  • Ablation and Design Choices: Removing the cost term increases cost by 15% with nearly unchanged F1macro, while removing potential-based shaping reduces F1id and F1macro and destabilizes performance.Operator ablations further show that removing MATCH, SELECT, or COMPARE lowers different aspects of performance, especially none-case accuracy without MATCH.

6 Conclusion

CaRL-EM treats blocked multi-candidate entity matching as a cost-aware sequential RL problem, dynamically selecting operators and model capacities. Across seven benchmarks, it improves the quality-cost trade-off over strong LLM baselines and hand-designed pipelines.

  • CaRL-EM controls MATCH, COMPARE, SELECT, and DECIDE operations while choosing between cheaper and stronger model capacities.
  • The approach casts LLM-driven entity matching as a cost-aware sequential RL problem rather than a fixed pipeline.
  • The controller learns to allocate less cost to easy cases and more checking to hard cases across seven zero-shot benchmarks.
  • CaRL-EM yields a better quality-cost trade-off than strong LLM baselines and manually designed pipelines.

Limitations

The study's scope is bounded by clean-clean matching, small candidate pools, and a coarse abstract cost model. Larger candidate sets, multiple valid matches, and deployment-specific cost surfaces may require controller or cost-function changes.

  • The clean-clean assumption excludes multiple valid matches, one-to-many links, and noisy or duplicate-heavy tables.Extending to these settings would likely require changes to the state and stopping or decision rule.
  • The experiments mainly use top-10 candidate pools, limiting direct evidence for much larger candidate sets.The paper notes that larger lists may require multi-stage pruning, hierarchical control, or tighter retrieval coupling.
  • The two-level abstract cost model omits prompt length, token billing, latency, batching, and provider-specific pricing.It serves as a relative control signal rather than exact accounting for a particular deployment.
  • The learned policy may not be cost-optimal under a different cost surface and may require recalibration or retraining.

Ethical Considerations

The experiments use public entity-matching benchmarks without releasing additional data. Because incorrect matches can cause wrong merges and downstream errors, high-stakes use should include auditing and human review.

  • The experiments use public entity-matching benchmarks and release no additional data.
  • Incorrect matches can produce wrong merges and downstream errors, so high-stakes applications should include auditing and human review.
  • The paper discourages linking personal identities across datasets and emphasizes legal and ethical compliance.

A Hyperparameter Values

This section lists implementation settings, including exact prompt output constraints and a robustness test that changes the abstract cost ratio. The cost-ratio change leaves F1macro nearly unchanged but increases some operator usage and variance.

  • Table 5 lists the concrete hyperparameter values used in the experiments.
  • MATCH prompts require exactly one first-line token, either [YES] for the same entity or [NO] for different entities.
  • COMPARE prompts require exactly one token identifying which of two candidates better matches the anchor.
  • SELECT prompts require exactly one token from [0..n-1] or [NONE] for the single best-matching candidate.
  • Changing the abstract cost ratio from ρ=2.5 to ρ=5 keeps F1macro nearly unchanged but increases MATCH/COMPARE calls and cross-benchmark variance.

D Confidence Update Rules

CaRL-EM maintains per-candidate confidence scores updated by MATCH, COMPARE, and SELECT operations, with clipping and confidence-dependent update strengths. The supplied passages also frame cost ratios and end-to-end cost as evaluation considerations.

  • Training Scalability: Figure 7 evaluates training scalability using wall-clock fine-tuning time and downstream F1macro on the WA test set.The figure compares CaRL-EM and DITTO across different fine-tuning set sizes.
  • Confidence Update Rules: MATCH updates candidate confidence from matcher probabilities after remapping scores to the interval [−1, 1].The update uses exponential smoothing on the remapped matcher probability.
  • Confidence Update Rules: MATCH uses a higher smoothing rate when matcher confidence exceeds match_margin and a lower rate otherwise.The threshold compares |s − 0.5| with match_margin.
  • Confidence Update Rules: COMPARE updates confidence from pairwise winners and comparator confidence, using separate high and low rates around compare_margin.The comparator confidence q is centered as δ = q − 0.5 before selecting the rate.
  • Confidence Update Rules: SELECT updates the current top-k shortlist when it returns a valid candidate index and penalizes the shortlist when its output is invalid.The invalid-output example is a parsing failure.
  • Confidence Update Rules: All candidate confidence scores are clipped to [−1, 1] after the update procedures.

F Blocking Cost and One-time Offline Cost for CaRL-EM

CaRL-EM’s cost accounting separates blocking, training, and deployment inference, while its controller transfers from AB to new datasets without retraining. Blocking costs are reported per dataset, and scalability analysis compares fine-tuning time and downstream performance under reduced training sizes.

  • Blocking Cost: WA blocking costs range from $1.2e−4 on AG to $1.38e−3 on WA across the seven listed datasets.The table reports costs for AG, DA, DS, IM, IT, TT, and WA.
  • One-time Offline Cost: CaRL-EM is trained once on source dataset AB, then transfers zero-shot to new target datasets without retraining.Supervised baselines such as DITTO require separate training for every target dataset.
  • Training Scalability: CaRL-EM requires more wall-clock fine-tuning time because rollout collection repeatedly invokes backend operators.The controller itself is not the computational bottleneck when operators are replaced by a simulator backend.
  • Training Scalability: CaRL-EM’s downstream performance remains relatively stable across WA training sizes, whereas DITTO degrades substantially in the low-data regime.The comparison uses 25%, 50%, 75%, and 100% WA training subsets evaluated on the same test set.
Loading 2609.01195v1…