Source-linked AI summary
TRACE: Agentic Catalog Enrichment with Multi-source Evidence Grounding
Rohan Kumar, Steven Xu, Kyle MacDonald, Matthew Long, Bernice Chow, Mac VanRenterghem, Sudeep Das
TL;DR
E-commerce catalogs often lack structured attributes because useful values are buried in unstructured content or missing, making scalable enrichment difficult. TRACE addresses this with ScoutAgent evidence gathering and JudgeAgent verification before writing values. It achieved 98.2% accuracy at 74.7% attribute coverage on a human-labeled dataset, while production deployment and an online experiment improved enrichment coverage and checkout conversion.
Problem
Product catalogs are often attribute-sparse, with important values buried in titles or images or absent from structured fields.
Method
TRACE uses ScoutAgent evidence gathering across catalog, syndicated, image, and identity-matched web sources, followed by JudgeAgent verification before publication.
Results
98.2% extraction accuracy at 74.7% attribute coverage was achieved on the fully human-labeled Grocery and Alcohol dataset.
Takeaways & Limitations
TRACE separates candidate generation from verification and grounds proposed values in product-specific evidence before writing them to the catalog.
Takeaways & Limitations
The online experiment measured the deployed system end to end and did not isolate the JudgeAgent or write-gating policy.
Abstract
from arXiv · showhide
Product catalogs underpin search, discovery, and recommendation in e-commerce, yet they are often attribute-sparse: the attributes shoppers and downstream systems rely on are either buried in unstructured content such as titles and images or missing from the catalog altogether. Manually enriching e-commerce catalogs is impractical given their scale and rapid growth. This paper introduces TRACE, a novel framework for automated catalog attribute enrichment using agentic Large Language Models (LLMs). A ScoutAgent triangulates multimodal evidence across merchant catalogs, syndicated feeds, and identity-matched web search to propose candidate attribute values with supporting evidence, while a JudgeAgent verifies the proposed value for each attribute value against its supporting evidence and decides whether to publish it or route it to human review. On an offline human evaluation dataset, TRACE's proposed attribute values were 98.2% accurate at 74.7% attribute coverage. Deployed in production on an industry-scale catalog, TRACE increased impression-weighted enrichment coverage across four business verticals by 90.4%. An online experiment subsequently showed that surfacing the enriched attributes on the product detail page increased checkout conversion by 0.48%.
1. Introduction
TRACE addresses attribute-sparse product catalogs by combining multimodal, product-specific evidence with agentic verification before publication. This targets missing or unreliable attributes that can mislead shoppers and create risks for safety-sensitive information.
- Motivation: Attribute-sparse catalogs often bury relevant values in titles and images or omit them entirely.Structured attributes support filtering, query matching, and granular recommendations, but seller-provided catalog data is frequently incomplete.
- Motivation: Two practical challenges are sourcing values externally for exact products and maintaining representative accuracy as catalog composition changes.Owned data may be insufficient, while point-in-time audits can become less representative as product mixes evolve.
- Motivation: Missing or inaccurate attributes can mislead shoppers, degrade fulfillment quality, and create especially serious consequences for allergens or dietary restrictions.The risk is greatest for safety-sensitive attributes.
- Approach: TRACE uses a ScoutAgent to gather grounded evidence and a JudgeAgent to verify proposed values before publication.The framework triangulates heterogeneous sources, preserves provenance, and supports identity-matched search grounding.
2. Related Work
Prior work extracts product attributes with task-specific models, foundation models, retrieval, and multi-agent methods, but typically assumes owned data can support the target value. TRACE is positioned against gaps in production-grade per-value verification and exact-product search grounding.
- Attribute extraction: Earlier attribute-extraction systems used task-specific NLP models, while newer approaches use zero-shot or few-shot foundation models and retrieval.These methods reduce reliance on labeled data or task-specific fine-tuning, but commonly assume the value can be inferred from owned sources.
- Multi-agent methods: Multi-agent approaches increasingly refine predictions and cross-check extracted knowledge through iterative collaboration.The related work describes these systems as emerging extensions to generative attribute extraction.
- LLM-as-judge: LLM-as-judge methods evaluate outputs, grounded generations, or multi-model juries, but production catalog pipelines require per-value publication decisions under varying evidence quality.The paper identifies limited prior work integrating judges into changing product and attribute mixes.
- Search grounding: Agentic web search can gather specialized or current evidence, but topical relevance alone cannot establish that a retrieved page describes the exact catalog product or variant.Catalog enrichment therefore requires identity grounding in addition to retrieval relevance.
3. Methodology
TRACE uses category-specific templates and a two-stage verify-before-write pipeline. The ScoutAgent gathers, identity-checks, reconciles, and sometimes abstains on evidence, while the JudgeAgent assigns verdicts that map to catalog actions.
- 3.1. Overview: Category-specific templates map each SKU’s leaf category to meaningful priority attributes.The template collection restricts enrichment to attributes relevant to the product category.
- 3.1. Overview: Each target attribute produces either a grounded candidate value or an explicit abstention, represented with evidence, source types, confidence, and extraction status.Statuses include extracted, not_found, not_applicable, ambiguous, and conflict; outputs become WRITE, BLOCK, or REVIEW actions.
- 3.1. Overview: TRACE separates candidate generation from verification in a two-stage verify-before-write architecture.The ScoutAgent proposes grounded values, and the JudgeAgent re-examines each candidate under a stricter policy before publication.
- 3.2. The ScoutAgent: The ScoutAgent first uses seller and syndicated catalog data, then turns to agentic web search when available evidence cannot reliably determine an attribute.Sources include textual fields, product images, syndicated records, and targeted web queries.
- 3.2. The ScoutAgent: Identity-grounded retrieval discards web pages unless the ScoutAgent determines they describe the target SKU.Matching uses product identifiers and descriptive metadata within a ReAct-style reasoning loop.
- 3.2. The ScoutAgent: The ScoutAgent normalizes and reconciles source values, preserves provenance, and abstains when evidence is insufficient, ambiguous, or conflicting.The resulting candidate record is passed to the JudgeAgent for verification.
- 3.3. The JudgeAgent: The JudgeAgent reassesses product applicability and value support, optionally using additional web search, and evaluates candidates in parallel across SKUs.Its stricter evidence policy focuses on support for the proposed value for the target product.
- 3.3. The JudgeAgent: JudgeAgent verdicts distinguish PASS, FAIL, UNVERIFIED, and UNCERTAIN, separating absent confirmation from active disagreement.This distinction supports different operational outcomes under the write policy.
4. Experiments
TRACE was evaluated with human validation and JudgeAgent adjudication across multiple business verticals, including backbone comparisons under a fixed judge. Results show high extraction quality and coverage, while judge-supported metrics measure operational evidence compliance rather than human-validated accuracy.
- Evaluation metrics: Judge-supported rate measures compliance with the JudgeAgent’s evidence policy and is not interpreted as human-validated accuracy.The judge was fixed to Gemini 2.5 Flash in the backbone comparison, making the results a controlled operational comparison.
- Human-validated evaluation: 98.2% extraction accuracy was achieved at 74.7% attribute coverage on the fully human-labeled Grocery and Alcohol dataset.Human annotators established reference values, while auditors reviewed ScoutAgent proposals.
- JudgeAgent evaluation: 98.4% of PASS values were confirmed correct by human reviewers, but 87.8% of disagreements were false rejections.The binary policy had high precision among approved values but lower recall on correct values because it was overly conservative.
- JudgeAgent evaluation: The current four-verdict taxonomy separates contradicted or unsupported values, plausible but unconfirmed values, and conflicting or ambiguous evidence.FAIL, UNVERIFIED, and UNCERTAIN replace the former undifferentiated rejection outcome, while the PASS evidence requirement remains unchanged.
- Operational evaluation: 87.8% attribute coverage was achieved on Electronics and Home Improvement, with 97.4% of extracted values receiving PASS or UNVERIFIED verdicts.These results use JudgeAgent adjudication rather than exhaustive human labeling.
- Backbone comparison: Gemini 2.5 Flash provided the strongest overall quality–coverage–cost trade-off among compared ScoutAgent backbones.Gemini 3.5 Flash increased extraction coverage by 0.5 percentage points but reduced publication coverage from 85.5% to 81.9% and increased inference cost by more than 7×.
5. Deployment
TRACE was deployed across 31 million SKUs in four business verticals, increasing impression-weighted enrichment coverage by over 90%. A randomized online experiment found that enriched product detail pages increased checkout conversion and reduced missing or incorrect items.
- Deployment: 31 million SKUs were enriched across four business verticals after TRACE was deployed in production.The deployment measured impression-weighted enrichment coverage, defined by customer impressions associated with records carrying enriched attributes.
- Deployment: Over 90% higher impression-weighted enrichment coverage was achieved across the four deployed business verticals.Impression-weighted enrichment coverage is the share of customer impressions associated with product records carrying enriched attributes.
- Online experiment: A five-week randomized A/B test assigned 90% of traffic to enriched PDPs and 10% to the existing PDP experience.The treatment surfaced TRACE-produced attributes, measuring the end-to-end impact of generating, validating, and displaying enriched information.
- Online experiment: 0.48% higher checkout conversion and a 1.08% lower missing/incorrect-item rate were observed with enriched PDPs.Checkout conversion increased by 1.18% among power users, and effects were reported relative to the control group.
6. Limitations
The evaluation has important scope limitations: some operational metrics are not substitutes for human-validated precision, and the online experiment does not isolate individual system components.
- Judge-supported rate in Electronics and Home Improvement is an operational metric, not a substitute for human-validated precision.The JudgeAgent was calibrated using the Grocery and Alcohol human audit, while transfer to other categories received more limited human evaluation.
- Using models from the same family for ScoutAgent and JudgeAgent may produce correlated failure modes.
- The online experiment measures the end-to-end effect of displaying enriched product pages rather than isolating the JudgeAgent or write-gating policy.
7. Conclusion
TRACE separates evidence-grounded candidate generation from stricter verification before catalog publication. Across business-vertical experiments, it achieved high human-audited accuracy and improved checkout conversion when enriched attributes were displayed.
- TRACE separates candidate generation from verification: ScoutAgent gathers and reconciles multi-source evidence, while JudgeAgent applies a stricter publication policy.
- 98.2% accuracy was achieved on the human-annotated dataset.
- 0.48% increased checkout conversion followed surfacing enriched attributes on product detail pages in a randomized online experiment.The experiment also reduced missing or incorrect item reports.
- Evidence-grounded enrichment improved catalog quality and downstream user experience.The findings highlight grounding generated values in product-specific evidence and verifying them before publication.
Declaration on Generative AI
The authors used generative AI tools during preparation for grammar and spelling checks and for paraphrasing and rewording.
- Generative AI tools were used for grammar and spelling checks and for paraphrasing and rewording.
- The authors reviewed and edited the generated content and retained full responsibility for the publication.
- The declaration identifies editing assistance rather than generative AI use as part of the research workflow.
A. Condensed Agent Prompt Templates
The condensed prompt templates define structured ScoutAgent extraction and JudgeAgent verification contracts for one SKU at a time. They enforce evidence grounding, identity checks, abstention, explicit verdicts, and JSON-only outputs.
- Template scope: TRACE makes one ScoutAgent call and one JudgeAgent call per eligible SKU, with each call returning per-attribute outputs.
- ScoutAgent contract: ScoutAgent prioritizes catalog, syndicated, image, and web evidence, while web search resolves unresolved attributes through progressively targeted product queries.
- ScoutAgent contract: Before using image or web evidence, ScoutAgent must match the exact brand, variant, size, and pack count, rejecting nearby variants as identity mismatches.
- ScoutAgent contract: Extracted values require verbatim evidence, provenance, and confidence, while ambiguous or conflicting evidence triggers abstention or explicit conflict handling.
- ScoutAgent contract: ScoutAgent outputs include identity matches and per-attribute status, value, source, source type, reference, quoted evidence, and confidence.Non-extracted statuses use value=null and omit provenance and confidence fields.
- JudgeAgent contract: JudgeAgent returns PASS, FAIL, UNVERIFIED, or UNCERTAIN verdicts, distinguishing contradiction, unsupported inference, partial extraction, and other failure types in valid JSON.