Source-linked AI summary
Correcting to Predict: Pseudo-Value Correction for Multimodal Attribute Value Extraction
Junhao Zhang, Feiran Hu, Xiao Hu, Baoliang Cui, Xiaoyi Zeng
TL;DR
Implicit AVE requires resolving ambiguous values from visual and textual cues, but existing methods can miss or override subtle evidence. C2P trains models to correct pseudo-values using multimodal evidence and refines consistency across pseudo-values. Evaluations report stronger results on ambiguous attributes and efficient retrieval-free, single-pass deployment.
Problem
Method
C2P trains models to retain or correct supplied pseudo-values using multimodal evidence, then applies Self-Consistency Refinement to reduce sensitivity to pseudo-value perturbations.
Source: §1 · p. 2Results
Takeaways & Limitations
A fixed placeholder supports efficient single-pass deployment without online retrieval or iterative refinement.
Source: §1 · p. 2, §6 · p. 7When an attribute is inapplicable or sufficient evidence is absent, the model outputs invalid; other values must come from the canonical vocabulary.
Source: §3 · p. 3
Abstract
from arXiv · showhide
Product attribute value extraction (AVE) is a fundamental task in e-commerce, aiming to identify specific values of predefined attributes from multimodal product profiles such as text and images. While multimodal large language models (MLLMs) have shown promise for AVE, they face challenges in extracting implicit attributes that require joint reasoning over visual and textual cues, often confusing semantically similar values. However, existing methods often fail to resolve such ambiguities because the correct value often depends on subtle multimodal cues that are easy to miss or override. To address this challenge, we propose Correcting to Predict (C2P), a framework that treats attribute extraction as a correction process. Given an initial pseudo-value such as a retrieved candidate or placeholder, the model learns to correct it using multimodal evidence. During training, diverse pseudo-values help the model learn evidence-based correction behavior, and a self-consistency refinement stage further reduces sensitivity to pseudo-value perturbations. At inference, a fixed placeholder triggers the learned correction behavior, enabling efficient single-pass prediction without online retrieval or iterative refinement. We evaluate C2P on a public benchmark and a large-scale industrial dataset. Offline results show that C2P outperforms strong baselines, with notable gains on ambiguous attributes. Online A/B tests on AliExpress further show consistent improvements in seller adoption, attribute completeness, and user engagement, validating C2P's effectiveness and efficiency in real-world deployment.
1 Introduction
Implicit AVE requires inferring values from visual and textual evidence when text alone may not resolve similar attributes. C2P reframes extraction as correcting a pseudo-value, with evaluations reporting stronger results on ambiguous attributes and retrieval-free, single-pass inference.
1 Introduction
- Missing or misaligned seller-provided values leave few products with complete, reliable annotations, motivating automated extraction from titles, descriptions, and images.Product attributes also support e-commerce navigation, algorithms, and business operations.Source: §1 · p. 1
- Implicit AVE requires inferring values not stated in text, where text-only methods may confuse attributes such as whether “golden” denotes color or material.Multimodal methods jointly model textual and visual features, but challenges remain for implicit values.Source: §1 · p. 1
- C2P trains the model to retain or correct a retrieved candidate or placeholder according to multimodal evidence, treating retrieved candidates as hypotheses rather than trusted context.A fixed placeholder supports single-pass inference without online retrieval or iterative refinement.Source: §1 · p. 2
- Self-Consistency Refinement further reduces sensitivity to pseudo-value perturbations, while a fixed placeholder enables retrieval-free, single-pass prediction for production.The framework learns correction behavior from diverse pseudo-values.Source: §1 · p. 2
- C2P consistently outperforms strong baselines across a public benchmark, an industrial dataset, and AliExpress A/B tests, especially on ambiguous attributes.The evaluations include online deployment tests on AliExpress.Source: §1 · p. 2
2 Related Work
Prior AVE approaches use direct generation, retrieval-conditioned context, or post-generation refinement. C2P instead embeds verification in prediction and uses pseudo-value inconsistency as a task-specific refinement signal.
Direct Generation Approaches for AVE
Retrieval-Augmented Generation for AVE
- MVP-RAG conditions generation on retrieved product values, whereas C2P treats retrieved values as hypotheses to verify and correct against product evidence.The passage notes that retrieval errors can propagate and increase inference latency.Source: §2 · p. 2
Post-Generation Refinement in AVE
- Post-generation self-correction and multi-agent debate revise an initial answer, which may already be biased by insufficient multimodal grounding.Self-correction has shown limited accuracy gains despite higher computational cost [3], while MADIAVE [10] uses iterative cross-model critique.Source: §2 · p. 2
Connection to Curriculum Learning
- SCR uses prediction inconsistency under pseudo-value perturbation as a task-specific difficulty signal, rather than generic training loss or confidence.At inference, C2P uses a fixed placeholder for retrieval-free, single-pass prediction.Source: §2 · p. 3
3 Problem Statement
The task maps a product’s image, text, category, and predefined attribute to an attribute value. Values must be supported by multimodal evidence and standardized in the platform’s canonical vocabulary, with invalid used when applicability or evidence is lacking.
3 Problem Statement
- AVE predicts an attribute value from a product image, title, optional description, category, and predefined attribute.The output is a value in the category-attribute output space.Source: §3 · p. 3
- A valid value must be supported by visual and textual evidence; when the attribute is inapplicable or evidence is insufficient, the model outputs invalid.Non-invalid values must come from the platform’s canonical vocabulary for that category-attribute pair.Source: §3 · p. 3
- Implicit attributes are difficult because their values may be absent from text and require joint visual-textual reasoning amid noisy, missing, or conflicting cues.The passage gives inferring a turtleneck from an image as an example when the title says only “winter top.”Source: §3 · p. 3
4 Methods
C2P casts multimodal attribute extraction as correcting an initial pseudo-value using product evidence, then refines unstable predictions and uses a fixed placeholder for efficient single-pass inference.
4.1 C2P as Correction with Pseudo-Values
- C2P predicts a canonical attribute value by correcting an initial pseudo-value according to multimodal evidence.The pseudo-value is a hypothesis to retain or correct, rather than a value to copy unquestioningly.Source: §4.1 · p. 3
4.2 Train-time Pseudo-Values
- Training pairs diverse placeholder and retrieved pseudo-values with the same ground-truth canonical output, teaching evidence-based retention or correction.Retrieved values come from similar products and can be plausible but imperfect; placeholders such as invalid provide little sample-specific information.Source: §4.2 · pp. 3–4
4.3 Self-Consistency Refinement
- Self-Consistency Refinement (SCR) identifies samples whose predictions change across pseudo-values, then retrains on them with replayed base-training data.The instability signal targets cases where predictions remain influenced by the initial hypothesis; replay helps avoid over-specializing to inconsistent cases.Source: §4.3 · p. 4
4.4 Inference with a Fixed Placeholder
- At inference, a fixed placeholder triggers a single-pass prediction without online retrieval or iterative refinement.The main setting uses leastV, the least frequent canonical value, as a practical default rather than a theoretically special choice.Source: §4.4 · pp. 4–5
5 Experiments
Across a public benchmark and an industrial dataset, C2P outperforms strong baselines, with larger gains on hard attributes. Refinement supports fixed-placeholder, retrieval-free inference, while AliExpress tests report improvements across deployment and engagement metrics.
5.1 Datasets and implementation
- The evaluations cover ImplicitAVE and AE-Product, which spans 371 categories, 1,267 attributes, and 3,457 category-attribute pairs and includes cases without a supported canonical value.Micro-F1 is the primary offline metric; training uses Qwen2.5-VL-7B-Instruct with LoRA, and retrieved pseudo-values come from GME top-3 products sharing the target attribute.Source: §5.1 · p. 5
5.3 Why Pseudo-Value Correction Helps
- GME-retrieved pseudo-values outperform fixed invalid placeholders and random values, while C2P also beats RAG-top3 on both datasets with substantially fewer input tokens.Random pseudo-values already improve over direct generation on both datasets; retrieved candidates are treated as hypotheses to verify and correct.Source: §5.3 · pp. 5–6, Table 6 · p. 6
5.2 Main Results
- C2P-SCR beats direct-generation SFT by 1.68 points overall and by 3.29 points on Home, while surpassing GPT-4V and MICE by 2.10 and 0.92 points on the full set.Clothing and Jewelry gains over previous best-performing models are 2.84 and 2.68 points, respectively.Source: §5.2 · p. 5, Table 2 · p. 6
- C2P-SCR preserves performance on easy attributes while gaining more on hard attributes, and SCR further amplifies gains on the hard subset.Hard attributes are defined as those with baseline Micro-F1 below 0.85.Source: §5.2 · p. 5, Table 4 · p. 6
5.4 Self-Consistency Refinement Analysis
- Perturbation-sensitive samples are enriched for hard attributes, and SCR reduces pseudo-value-driven and frequent-value-prior errors without materially changing overall retention.Hard attributes rise from 28.29% to 68.14% on ImplicitAVE and from 33.71% to 58.58% on AE-Product; on ImplicitAVE, PRR changes from 14.05 to 13.93 while E-PRR and E-FVR fall from 14.97/24.06 to 13.97/21.79.Source: §5.4 · p. 6, Table 7 · p. 7
- SCR outperforms curriculum-style refinement on both datasets, with most gains achieved in the first iteration and only marginal improvement from a second.The results characterize SCR as targeted refinement of unstable correction behavior rather than generic second-stage fine-tuning.Source: Table 8 · p. 7, §5.4 · p. 7
5.5 Fixed-Placeholder Inference Analysis
- Fixed placeholders trigger effective correction without retrieval, and after SCR, Micro-F1 varies little across invalid, leastV, and GME-top3 inference inputs.C2P-SCR scores 88.87 on ImplicitAVE with invalid and leastV versus 88.81 with GME-top3; AE-Product scores are 91.26, 91.39, and 91.35, respectively.Source: Table 9 · p. 7, §5.5 · p. 7
5.6 Efficiency and Online Deployment
- C2P maintains near-baseline serving throughput at 4.77 versus 5.18 QPS for direct generation, compared with 1.45 QPS for RAG-top3.In a 7-day AliExpress A/B test with 10% of live traffic, C2P-SCR improved deployable category-attribute pairs by 22.6% and seller adoption from 83% to 85%, alongside gains in other business metrics.Source: §5.6 · p. 7
6 Conclusion
C2P frames multimodal AVE as correcting an initial pseudo-value with evidence, and experiments report consistent gains over strong baselines, especially for ambiguous attributes, with efficient single-pass deployment.
6 Conclusion
- C2P reframes AVE as correcting an initial pseudo-value with multimodal evidence rather than generating an attribute value from scratch.Training with diverse pseudo-values teaches the model when to retain or revise a hypothesis; Self-Consistency Refinement further reduces sensitivity to pseudo-value perturbations.Source: §6 · p. 7
- C2P consistently outperforms strong baselines on ImplicitAVE, an industrial dataset, and AliExpress A/B tests, especially for hard, ambiguous or confusable attributes.It supports single-pass deployment without online retrieval or iterative refinement.Source: §6 · p. 7
GenAI Usage Disclosure
The authors disclose that Generative AI tools were used to improve spelling and grammar in certain passages, in compliance with the ACM Authorship Policy on GenAI usage.
GenAI Usage Disclosure
- Generative AI tools were used only to improve spelling and grammar in certain passages, with use stated to comply with the ACM Authorship Policy on GenAI usage.Source: GenAI Usage Disclosure · p. 8