Source-linked AI summary
MERGED: Multimodal Entity Resolution via Generated Expert Reasoning Distillation
You-Lin Chen, Kyoungjun Park, Bin Xu, Prithviraj Sen, Pedro Herrero-Vidal
TL;DR
Evolving product relationships make human-labeled adaptation slow and costly, while large VLMs are expensive and can produce unsupported rationales. MERGED distills multi-teacher labels and reasoning into a 7B student using agreement-based SFT and disagreement-based DPO, improving performance and enabling efficient re-adaptation.
Problem
Product relationship definitions evolve, but human annotation is slow, costly, and noisy, while large VLMs are too costly and latent for production-scale use.
Method
MERGED uses multiple teacher VLMs, applying SFT to agreement pairs and meta-judge-curated DPO to disagreement pairs, to train a compact 7B student without human labels.
Results
90.96% PR-AUC is achieved without human labels, 13.79% above the same human-label-trained backbone and 6.32% above Qwen2.5-32B-VL at 6× lower cost.
Takeaways & Limitations
Re-applying MERGED from an existing checkpoint adapts to a new relationship definition with 10K samples, supporting adaptation in days rather than months.
Takeaways & Limitations
Adaptation is evaluated only for exact-to-variant transfer; broader relationship-type transfer, ensemble-size effects, and lighter meta-judge alternatives remain unexplored.
Abstract
from arXiv · showhide
In product entity resolution, relationship definitions constantly evolve with business needs, yet adapting to each change traditionally requires slow, costly human annotation that is often noisy and carries no reasoning. Large vision-language models (VLMs) prompted zero-shot can adapt to a new definition immediately and supply the reasoning that human labels lack, but their cost and latency are prohibitive at production scale. We present MERGED, a distillation framework that transfers not just labels but structured reasoning from large teacher VLMs into a compact 7B-parameter student, requiring no human annotation. Multiple teachers label each product pair and articulate the reasoning behind their decision: agreement pairs supply supervised fine-tuning, while disagreements are resolved by a meta-judge into preference pairs for Direct Preference Optimization. Evaluated against human-labeled ground truth on a multilingual e-commerce dataset, the resulting student improves PR-AUC by 13.79% over the same backbone trained on human labels and surpasses the larger Qwen2.5-32B-VL baseline by 6.32% at 6x lower cost, while also yielding tighter label-reasoning alignment (over 10% above Qwen2.5-32B-VL). Moreover, re-applying MERGED from an existing checkpoint adapts to a new relationship definition with only 10K samples, improving PR-AUC by 6.97% over zero-shot and outperforming from-scratch training. MERGED enables rapid adaptation to evolving relationship definitions, supporting a new one in days rather than months, at a cost and latency suitable for large-scale industrial deployment.
1 Introduction
Product entity resolution must accommodate relationship definitions that evolve, but human labeling is slow, costly, and noisy while large VLMs are too expensive for production. MERGED distills teacher labels and reasoning into a compact student for accurate, efficient adaptation.
- Relationship definitions can change with applications, markets, or revised annotation procedures, creating a new task each time.
- Production pipelines require rapid adaptation, high accuracy, low cost, and low latency, but repeated human annotation typically takes several weeks.
- Large VLMs can adapt to new definitions with little or no labeled data, but their cost, latency, and hallucinated rationales limit production-scale use.
- MERGED uses multiple teacher VLMs to generate labels and reasoning, routing agreement pairs to SFT and disagreements through a meta-judge into DPO preferences.
- 90.96% PR-AUC is achieved without human labels, 13.79% above the same backbone trained on human labels.
- 6.32% higher PR-AUC than Qwen2.5-32B-VL is achieved at 6× lower cost, while re-application reaches a 6.97% gain over zero-shot with 10K samples.
- MERGED improves label-reasoning alignment by over 10% relative to Qwen2.5-32B-VL.
2 Related work
Prior work spans neural and foundation-model approaches, generative cold-start resolution, and reasoning distillation. MERGED combines these directions through consensus supervision, meta-judge preferences, and sequential SFT followed by DPO.
- Generative-model prompting supports cold-start entity resolution, but accuracy trails fine-tuned models and labeling strategies do not flexibly handle evolving relationship definitions.
- Earlier work distills step-by-step rationales, uses model-generated judgments for DPO, and finds staged SFT before preference optimization effective.
- MERGED unifies multi-teacher consensus, meta-judge preference curation, and sequential SFT+DPO for multimodal entity resolution at industrial scale.
3 Problem formulation
The paper formulates product entity resolution as conditional binary relation classification over multimodal product pairs and a relation definition. A generative VLM interface produces both labels and reasoning, enabling token-level training and ranking scores.
- Given multimodal product pair x and relation definition R, the task predicts binary label y indicating whether the pair satisfies R.
- The generative interface prompts a VLM with encoded x and R, then generates a token-by-token completion containing the prediction and its reasoning.
- Token-level modeling supports cross-entropy training and whole-completion preference optimization without relying on internal class logits.
- A scalar score derived from the predicted label and confidence enables ranking metrics such as PR-AUC.
4 Methods
MERGED distills multimodal reasoning from multiple teacher VLMs into a compact student through sequential SFT and DPO. Teacher agreement supplies high-confidence supervised examples, while disagreements become meta-judge-curated preference pairs for harder cases and new definitions.
- Overview: MERGED transfers teacher labels and reasoning into a compact 7B student through supervised fine-tuning followed by Direct Preference Optimization.The recipe targets task-specific product entity resolution without human labeling.
- Teacher generation: Multiple teachers generate labels, reasoning, and confidence scores for each product pair using a shared prompt.Using two teachers partitions examples into agreement and disagreement cases.
- Agreement and SFT: Agreement examples form a high-confidence SFT set that aligns the student with the target relation and reduces label noise.Consensus examples are less likely to reflect idiosyncratic interpretations or unstable predictions.
- Disagreement and DPO: Disagreements identify ambiguous pairs and produce competing completions for preference optimization rather than being discarded.The preferred completion is selected by a meta-judge VLM based on adherence to the relation and grounded evidence.
- Disagreement and DPO: DPO improves difficult-case behavior by favoring discriminative, input-grounded reasoning while discouraging unsupported conclusions and brittle shortcuts.The student remains close to the SFT reference policy during preference training.
- Adapting to new definitions: Re-applying MERGED from an existing checkpoint adapts the model to a new relationship definition without restarting from a base model.The method regenerates a small amount of teacher supervision and performs a fresh SFT+DPO pass.
5 Results
MERGED is evaluated on noisy, multilingual industrial product-matching data against human-labeled ground truth. It improves accuracy and reasoning faithfulness, with gains from distilled reasoning and DPO, and adapts to a new relation using limited data.
- Benchmark: The benchmark contains over 100K multilingual product-listing pairs from 18 countries, with images and heterogeneous text fields.Evaluation uses a disjoint test set of approximately 6,000 balanced product pairs.
- Exact-match results: 90.96% PR-AUC is achieved by MERGED at $600 per million samples, surpassing Qwen2.5-32B-VL by 6.32% PR-AUC at 6× lower cost.The larger zero-shot baseline reaches 84.64% PR-AUC at $3,600 per million samples.
- Ablation: 13.79% PR-AUC improvement over the human-label baseline is decomposed into gains from model-generated labels, distilled reasoning, and DPO.The reported increments are 3.50%, 6.14%, and 4.15%, respectively.
- Preference assignment: The meta-judge outperforms alternative heuristic strategies for assigning DPO preferences.The comparison supports grounded evaluation of reasoning quality over simple heuristics.
- Reasoning faithfulness: 92.82% label-reasoning alignment is reached, with training improving alignment by over 13% and surpassing Qwen2.5-32B-VL.Qualitatively, SFT reduces hallucinated rationales and DPO directs reasoning toward task-relevant attributes.
- New-definition adaptation: 89.48% PR-AUC is reached when MERGED is reapplied to a new relationship using only 10K samples, a 6.97% gain over zero-shot.This transfer setting also outperforms training from scratch on the same data.
6 Conclusion and Industry Impact
MERGED distills teacher-generated labels and reasoning into a compact student without human annotation, achieving production-ready product entity resolution and rapid adaptation to changing definitions.
- MERGED is an automated distillation framework that removes dependence on human labeling while delivering production-ready performance.
- 90.96% PR-AUC exceeds the same backbone trained on human labels by 13.79% and the Qwen2.5-32B-VL baseline by 6.32%.The model runs in under one second per sample at $600 per million predictions, reducing cost 6× versus that baseline.
- MERGED transfers labels and task-specific rationales from diverse teacher VLMs to a compact 7B student through consensus SFT and meta-judge-curated DPO preference pairs.
- The same recipe re-adapts to a new relationship definition with only 10K examples because the student acquires transferable comparison skills.
- In industrial settings, MERGED can support catalog deduplication, relevant product surfacing, search, and discovery.
7 Limitations
The paper reports strong results across two relationship definitions but leaves broader transfer, ensemble-size effects, and lower-cost preference curation for future work.
- MERGED is evaluated across only two relationship definitions, limiting the demonstrated scope.
- Adaptation is tested on only one source-to-target transfer, exact to variant, so broader relationship-type transfer remains future work.
- The study uses two teachers, leaving the effect of ensemble size on consensus quality and downstream performance unexplored.
- The meta-judge depends on a large VLM during data generation, while lighter-weight or self-supervised alternatives remain open.
8 Ethics statement
The study uses publicly visible product catalog data without personal, customer-query, or seller-private information, and anonymizes presented examples.
- Training and evaluation use publicly visible product catalog data, including titles, descriptions, attributes, and images.
- The study uses no personal data, customer queries, or seller-private information.
- Professional auditors collected human annotations under a standardized protocol with informed consent regarding data use.
- All examples presented in the paper are anonymized.
A Prompt examples
The appendix specifies strict JSON outputs, a product-comparison task, example predictions, and a meta-judgment prompt for selecting the better explanation.
- A.1 System prompt: The output must contain only JSON, beginning with {" and ending with "}.
- A.2 Task prompt: The task prompt asks whether two product listings are the same product, treating mismatched color, size, IPQ, brand, or specifications as differences.
- A.2 Task prompt: Missing attributes should not be penalized, and the comparison should use available information holistically.
- A.3 Example entity resolution prediction outputs: Example outputs provide a reason, confidence 100, and a Yes answer when product descriptions and identifiers match.
- A.3 Example entity resolution prediction outputs: A second example cites identical name, brand, EAN, size, and images as evidence for matching sunscreen products.
- A.4 Meta judgment: The meta-judgment prompt presents two labelers’ labels, reasons, and confidences, then asks which explanation is reasonable.