OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]

Guineng Zheng; Subhabrata Mukherjee; Xin Luna Dong; Feifei Li

Source-linked AI summary

OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]

Guineng Zheng, Subhabrata Mukherjee, Xin Luna Dong, Feifei Li

arXiv:1806.01264v2Jun 1, 2018cs.CLcs.AIcs.IRstat.ML

TL;DR

OpenTag addresses extraction of missing product-attribute values, especially values absent from predefined vocabularies, while seeking limited annotation and interpretable decisions. It formulates the task as sequence tagging with neural context modeling, CRF consistency, attention, and active learning. The system discovers new values from as few as 150 annotated samples, achieving an 83% F-score and reducing annotation effort 3.3x.

Problem
Product catalogs have missing attribute values, while closed-world methods cannot discover new values and the task requires limited annotation and interpretable decisions.
Method
OpenTag is an end-to-end sequence-tagging model combining BiLSTM, CRF, attention, and active-learning sampling without dictionaries or hand-crafted features.
Results
150 annotated samples produce an 83% F-score, reduce annotation effort 3.3x, and outperform state-of-the-art models.
Takeaways & Limitations
OpenTag discovers unseen and multi-word attribute values from unstructured product profiles while providing attention-based explanations and reducing annotation burden.
Takeaways & Limitations
The active-learning strategy can be affected by false certainty in one token’s tag, while careful hyper-parameter tuning might further improve performance.

Abstract

from arXiv · show

Extraction of missing attribute values is to find values describing an attribute of interest from a free text input. Most past related work on extraction of missing attribute values work with a closed world assumption with the possible set of values known beforehand, or use dictionaries of values and hand-crafted features. How can we discover new attribute values that we have never seen before? Can we do this with limited human annotation or supervision? We study this problem in the context of product catalogs that often have missing values for many attributes of interest. In this work, we leverage product profile information such as titles and descriptions to discover missing values of product attributes. We develop a novel deep tagging model OpenTag for this extraction problem with the following contributions: (1) we formalize the problem as a sequence tagging task, and propose a joint model exploiting recurrent neural networks (specifically, bidirectional LSTM) to capture context and semantics, and Conditional Random Fields (CRF) to enforce tagging consistency, (2) we develop a novel attention mechanism to provide interpretable explanation for our model's decisions, (3) we propose a novel sampling strategy exploring active learning to reduce the burden of human annotation. OpenTag does not use any dictionary or hand-crafted features as in prior works. Extensive experiments in real-life datasets in different domains show that OpenTag with our active learning strategy discovers new attribute values from as few as 150 annotated samples (reduction in 3.3x amount of annotation effort) with a high F-score of 83%, outperforming state-of-the-art models.

1 INTRODUCTION

OpenTag addresses missing product-attribute values in noisy, unstructured catalogs, including values unseen during training, while reducing annotation demands and improving interpretability. It formulates extraction as sequence tagging and combines neural context modeling, tag-consistency constraints, attention, and active learning.

Problem: Product catalogs contain noisy, incomplete attribute information, motivating extraction of missing values from product titles, descriptions, and related profile data.
Challenges: OpenTag targets new attribute values under the Open World Assumption, unlike approaches restricted to a predefined vocabulary.
Challenges: Stacked attributes, multi-word values, irregular grammar, and sparse product-profile contexts make conventional parsing and rule-based annotation difficult.Examples include “Fillet Mignon” and “Porterhouse Steak”; over 75% of dataset product titles contain fewer than 15 words.
Approach: The paper models extraction as sequence tagging using LSTM-based context and semantics, CRF-based tagging consistency, and attention for interpretable predictions.
Results: 150 annotated samples yield an 83% F-score while reducing annotation effort 3.3x and outperforming state-of-the-art models.
Scope: OpenTag supports unstructured textual data without dictionaries or hand-crafted features and is presented as an end-to-end framework for open attribute value extraction.

2 OVERVIEW

The paper defines open attribute value extraction over product profiles and recasts it as sequence tagging rather than value classification. This formulation supports unseen, multi-word, and multiple attribute values through token-level tags and joint sequence prediction.

2.1 Problem Definition: A product example extracts “Cesar” as brand, “Fillet Mignon” and “Porterhouse Steak” as flavor, and “6 trays” as size.
2.1 Problem Definition: Open attribute value extraction finds values for predefined target attributes in product profiles while allowing attribute values unseen during training.
2.2 Sequence Tagging Approach: Multi-class classification is limited by label scaling, the closed-world assumption, and label independence issues.
2.2 Sequence Tagging Approach: Sequence tagging assigns a tag to each token and jointly predicts the tag sequence, treating attribute-value extraction as a specific NER case.
2.2.1 Sequence Tagging: BIOE tags mark attribute beginnings, interiors, outside tokens, and ends; UBIOE adds a unit-token tag, while IOB omits the end tag.
2.2.3 Advantages of Sequence Tagging: The approach associates tags with tokens rather than specific values, enabling unseen, multi-word, and multiple attribute values.
2.2.3 Advantages of Sequence Tagging: OpenTag learns a global tagging model that captures relations between tags and models complete product-profile sequences.

3 OPENTAG MODEL: EXTRACTION VIA SEQUENCE TAGGING

OpenTag formulates open attribute-value extraction as sequence tagging and combines BiLSTM context modeling with CRF-based tag consistency. Attribute-conditioned embeddings and attention add semantic representations and interpretable focus without dictionaries or hand-crafted features.

Model overview: OpenTag adapts BiLSTM-CRF named entity recognition to attribute-value extraction without dictionaries or hand-crafted features.The model uses hidden states from BiLSTM as CRF input features.
BiLSTM-CRF: The CRF models label correlations that independent per-token BiLSTM predictions overlook in sequence tagging.The paper motivates CRF because valid attribute boundaries depend on neighboring labels.
BiLSTM-CRF: BiLSTM hidden states combine past and future token context, while CRF dependencies enforce coherence among neighboring output tags.Linear-chain CRF restricts feature functions to adjacent tags, reducing inference complexity to |Y|^2.
Attention: Attention weights hidden states so the model can focus on important concepts and highlight tokens influencing tagging decisions.The attention matrix captures similarity between token hidden states and neighboring contexts.
Word embeddings: Attribute-conditioned word embeddings represent the same token differently for different attributes, such as “duck” as flavor versus brand.The embeddings are learned parameters rather than fixed pretrained representations.

4 OPENTAG: ACTIVE LEARNING

OpenTag uses active learning to reduce annotation costs for sequence tagging, selecting unlabeled product profiles that are informative or uncertain. Its tag-flip strategy addresses weaknesses of whole-sequence confidence by prioritizing unstable token tags across learner epochs.

Active learning lets OpenTag request labels selectively from a large unlabeled pool instead of annotating all product profiles.The process begins with a small labeled set and iteratively adds queried instances according to a strategy until a stopping criterion is reached.
Least confidence: Least-confidence sampling selects the sequence with the lowest CRF confidence, measured from the conditional probability of its best tag sequence.Here, y∗ denotes the best possible tag sequence for x.
Least confidence: Whole-sequence confidence can miss useful queries because one token’s false certainty can reduce the sequence probability, while a corrected token may affect few other tags.These effects motivate a query strategy that evaluates difficulty at the token-tag level.
Method of tag flips: Tag-flip sampling measures uncertainty by counting token-tag changes across successive OpenTag epochs and queries sequences with the most flips.Dropout produces different learner parameters across epochs, enabling a committee-like disagreement signal.
Method of tag flips: In the illustrative comparison, tag-flip sampling chooses a sequence with grossly mistagged tokens whose labels have more impact on parameter tuning than the least-confidence query.The example contrasts S2, selected by tag flips, with S1, selected by overall sequence uncertainty.
Active learning procedure: With batch size B, OpenTag annotates the top B samples by flip count and continues until validation loss converges within a threshold.Algorithm 1 describes this iterative process over labeled and unlabeled sets.

5 EXPERIMENTS

Experiments across product domains evaluate OpenTag for standard extraction, unseen values, joint attributes, interpretability, and annotation-efficient learning. OpenTag performs strongly, including 82.4% F-score on unseen values and substantial gains from attention and active learning.

Experimental Setup: Experiments use Amazon product profiles from dog food, detergents, and camera domains, with domain-specific target attributes.Profiles include titles, descriptions, and bullets, with datasets varying in profile type, target attribute, sample count, and extraction count.
Attribute Value Extraction: OpenTag outperforms BiLSTM-CRF-based NER systems on attribute value extraction across product profiles and datasets.The comparison uses the same training and test data for all models and excludes dictionaries and hand-crafted features from the strongest baseline description.
Open World Assumption: 82.4% F-score is achieved when discovering flavor values absent from training in dog food titles under a disjoint split.Performance is lower than random splitting, where some attribute values are shared between training and test data.
Joint Extraction: 2% F-score improvement is obtained by jointly extracting brand, flavor, and capacity values over the strongest BiLSTM-CRF baseline.The joint model assigns attribute-specific beginning, inside, and ending tags while sharing the outside tag; flavor alone marginally degrades despite gains for brand and capacity.
Interpretability: Attention highlights neighboring words that influence tagging decisions, such as conjunctions connecting adjacent flavor segments.The attention mechanism also projects hidden vectors into a space where similar concepts show better grouping and separability.
Active Learning: Tag-flip active learning raises precision from 59.5% to 91.7% and recall from 70.7% to 91.5% for single-attribute extraction.Tag flip outperforms least confidence by 5.6% F-score for single-attribute extraction and 2.2% for multi-attribute extraction, reaching comparable performance with 150 rather than 500 training instances.

6 RELATED WORK

Prior attribute extraction and sequence-labeling methods commonly rely on dictionaries, syntactic structure, or extensive feature engineering and annotation. OpenTag differs by supporting unseen values without dictionaries or hand-crafted features and by using active learning.

Rule-Based and Linguistic Methods: Rule-based methods rely on domain-specific vocabularies or dictionaries, limiting coverage under closed-world assumptions.Syntactic and linguistic approaches also perform poorly on irregular product-title structures.
Dictionary-Based NER: Existing brand-tagging systems bootstrap from known brand dictionaries, whereas OpenTag discovers new values without dictionaries.One cited system used a seed dictionary containing over 6,000 known brands.
Neural Sequence Tagging: Neural sequence-tagging work includes CRF, BiLSTM-CRF, character CNN, and LSTM approaches, often with engineered features or large labeled datasets.A cited product-attribute LSTM-CRF model used 37,000 manually labeled search queries.
OpenTag’s Distinction: OpenTag avoids hand-crafted features and uses active learning to reduce annotation burden.Its sampling strategy differs from earlier least-confidence and entropy-based approaches for sequence labeling.

7 CONCLUSIONS

OpenTag is an end-to-end BiLSTM-CRF model with attention for extracting missing attribute values from unstructured product profiles. The paper concludes that it supports unseen, multi-word, and multiple attribute values, sparse irregular text, limited annotation, and interpretable explanations.

Contributions: OpenTag combines BiLSTM, CRF, and attention to impute missing attribute values from titles, descriptions, and bullets.It uses no dictionary or hand-crafted features and makes no assumption about input structure.
Supported Extraction: OpenTag discovers unseen attribute values, including multi-word values and multiple attributes, under the Open World Assumption.The conclusion gives emerging brands as an example of values that may not have been encountered before.
Scope and Annotation: OpenTag handles unstructured text with stacked attributes and sparse context while requiring less training data through active learning.The conclusion frames active learning as a way to reduce human annotation burden.
Interpretability: Attention generates explanations for model verdicts, making the system easier to debug.The conclusion identifies interpretability as an additional advantage of the model.