Source-linked AI summary
Learning Latent Vector Spaces for Product Search
Christophe Van Gysel, Maarten de Rijke, Evangelos Kanoulas
TL;DR
Product search suffers from a mismatch between query language and product representations. The paper introduces LSE, which jointly learns word and product representations and their mapping, and reports stronger retrieval than baseline latent models in lower-dimensional spaces, with further gains when used in ranking.
Problem
Product search requires semantic matching because queries and product representations can use different terms for the same concepts.
Method
LSE jointly learns word and product representations and a mapping between them from unstructured product documents without explicit annotations.
Results
LSE outperforms LSI, LDA, and word2vec for product ranking in lower-dimensional vector spaces, and adding it to language-model and popularity features significantly improves retrieval.
Takeaways & Limitations
LSE’s stronger retrieval is associated with better product representations, while errors propagated from products also benefit word representations and the word-to-product mapping.
Takeaways & Limitations
The study focuses on an unsupervised setting with product descriptions and reviews, and identifies substantial room to improve the query projection function.
Abstract
from arXiv · showhide
We introduce a novel latent vector space model that jointly learns the latent representations of words, e-commerce products and a mapping between the two without the need for explicit annotations. The power of the model lies in its ability to directly model the discriminative relation between products and a particular word. We compare our method to existing latent vector space models (LSI, LDA and word2vec) and evaluate it as a feature in a learning to rank setting. Our latent vector space model achieves its enhanced performance as it learns better product representations. Furthermore, the mapping from words to products and the representations of words benefit directly from the errors propagated back from the product representations during parameter estimation. We provide an in-depth analysis of the performance of our model and analyze the structure of the learned representations.
1. INTRODUCTION
The paper addresses query–product vocabulary mismatch by introducing LSE, an unsupervised model that jointly learns word and product representations and their mapping. It evaluates LSE’s parameters, retrieval comparisons, and use in learning-to-rank.
- Query and product representations often use different terms for the same concepts, creating a need for better semantic matching in online product search.
- LSE jointly learns distributed representations of words and entities plus a mapping between them, without predefined entity relationships or explicit annotations.
- The model directly learns discriminative word–entity relationships from mostly unstructured product descriptions and reviews using gradient descent.
- LSE avoids computing the full entity probability distribution during training by using noise-contrastive estimation.
- The evaluation compares LSE with LSI, LDA, word2vec, and lexical language models while testing parameter effects and learning-to-rank integration.
2. RELATED WORK
Related work frames product search as entity finding over unstructured product documents and motivates latent representations for overcoming query–document vocabulary mismatch. LSE follows this direction while learning product-oriented representations and query mappings jointly.
- Product search retrieves products from domain-specific collections using free-form text, unlike broader entity-finding settings with typed or relational queries.
- Latent semantic models support retrieval by conceptual content rather than exact word matching, addressing mismatch between queries and documents.
- Neural language models provide distributed word representations, but the paper focuses on learning representations for the end-to-end product-search task.
- LSE learns entity representations from unstructured product documents and models the connection between query words and entities directly.
- The retrieval pipeline projects a query into entity space and ranks products by cosine similarity to their entity vectors.
3. LATENT VECTOR SPACES FOR ENTITY RETRIEVAL
The paper formulates entity retrieval as mapping word sequences into an entity vector space and ranking entities by cosine similarity. LSE jointly learns word and entity representations, connects their spaces, and uses sampled contrastive training to discriminate associated entities from negatives.
- 3.1 Background: LSE maps word sequences into an entity vector space and ranks entities by cosine similarity with their learned representations.The general framework represents each entity as a vector and projects queries into the same space.
- 3.2 Latent Semantic Entities: Unlike document-aggregation approaches, LSE explicitly learns the word-to-entity mapping and can scale to large entity collections through sampling.Its different-dimensionality word and entity spaces also accommodate potentially different latent structures.
- 3.2 Latent Semantic Entities: LSE represents a string by averaging word embeddings, then applies a learned linear transformation, bias, and tanh nonlinearity.The word projection and entity-space mapping are learned automatically, while tanh regularizes the space and supports nonlinear class boundaries.
- 3.3 Parameter estimation: Training maximizes similarity between each entity and n-grams sampled from its associated documents while minimizing similarity to contrastive entities.The n-gram window remains fixed, and the objective directly distinguishes an associated entity from sampled negatives.
- 3.3 Parameter estimation: Noise-Contrastive Estimation samples uniformly from the entity collection, avoiding iteration over all entities during parameter estimation.The approximation uses z randomly sampled entities as negative training examples.
4. EXPERIMENTAL SETUP
The experiments evaluate LSE on Amazon product retrieval across multiple product domains and collection sizes. They vary key model parameters, compare LSE with latent-vector and language-model baselines, and test LSE as a learning-to-rank feature.
- 4.1 Research questions: The research questions examine parameter effects, comparisons across latent-vector and language-model methods, and the benefit of combining LSE with other ranking features.The proposed feature combinations include popularity-based, exact-matching, and latent-vector features.
- 4.3 Product search benchmarks: The study evaluates LSE on four Amazon departments containing 8,192 to 65,536 products, with descriptions and customer reviews associated with each product.The departments are Home & Kitchen, Clothing, Shoes & Jewelry, Pet Supplies, and Sports & Outdoors across the benchmark overview.
- 4.2 Experimental design: The experiments vary entity-vector dimensionality and context window size to assess how model capacity and contextual evidence affect LSE effectiveness.Both parameters are swept exponentially across predefined ranges for RQ1.
- 4.3 Product search benchmarks: Test topics are extracted from product-category hierarchies, and products associated with a hierarchy are treated as relevant to its category query.Category titles are tokenized into textual queries after removing stopwords and duplicate words.
- 4.5 Methods used in comparisons: LSE is compared with LSI, LDA, word2vec, and smoothed query-likelihood language models, using NDCG and, for learning-to-rank experiments, Precision@5 and Precision@10.The comparisons cover semantic latent-vector baselines, exact lexical matching, and LSE as a feature in machine-learned ranking.
5. RESULTS AND DISCUSSION
LSE performs best with moderate context and low-dimensional entity representations, significantly outperforming latent vector baselines in most test-set comparisons. Its semantic signal complements lexical matching, improving machine-learned ranking when combined with other features.
- RQ1: LSE reaches its highest NDCG with 128- or 256-dimensional entity representations and medium context windows of 4 or 8 grams.Extreme representation sizes or context widths do not achieve the best validation performance.
- RQ2: LSE significantly outperforms LSI, LDA, and word2vec in most test-set comparisons, except one case at 1024 dimensions.The comparison uses window size n = 4, and significance is reported at p < 0.01.
- RQ2: In larger benchmarks, word2vec becomes LSI’s main competitor, but it does not match LSE using representations one or two base-2 orders of magnitude smaller.Word2vec stagnates around 512 dimensions, whereas LSE peaks at 128 or 256 dimensions.
- RQ3: LSE and the Query-likelihood Language Model perform similarly for many topics, while particular topics substantially favor one method over the other.Queries with specific exact lexical matches tend to favor QLM, whereas less specific or less exact matches favor LSE.
- RQ4: Combining query-independent, QLM, and LSE features outperforms smaller feature subsets on every reported metric in RankSVM experiments.The results support LSE as a complementary signal for ranker-fusion systems.
6. ANALYSIS OF REPRESENTATIONS
The representation analysis separates entity-space quality from the query-to-entity projection function by comparing approximately ideal retrieval vectors with projected query vectors. LSE’s ideal vectors outperform the baselines, while projection quality remains an important source of performance differences.
- Analysis method: The analysis treats retrieval performance as depending on entity-space structure, the ideal retrieval vector, and the similarity between that vector and the projected query.The approximately ideal vector is defined through rankings that place relevant entities above non-relevant or unjudged entities.
- Caveat: The approximately ideal vector omits the topic’s textual representation and uses only relevant-entity clustering and relations to other entities.This makes it an entity-space-based upper-bound analysis rather than a complete query representation.
- Analysis method: The analysis estimates how much NDCG could improve by replacing each model’s projected query vector with an approximately ideal retrieval vector.The ideal vector is approximated separately for topics with at least two relevant entities using a pair-wise SVM objective.
- Ideal retrieval vectors: LSE’s approximately ideal retrieval vectors significantly outperform those discovered for every baseline model.The analysis compares absolute NDCG for approximately ideal vectors and projected query vectors.
- Projection quality: LDA’s optimal performance increases with dimensionality while its actual performance stagnates, indicating a widening gap between entity representations and query projection.The projection function fails to produce query vectors sufficiently similar to relevant entity representations as dimensionality grows.
7. CONCLUSIONS
The paper introduces LSE as an unsupervised latent vector space model for product search and finds that it outperforms established baselines in lower-dimensional spaces. Its analyses indicate that product representations are a major strength, while query projection remains an important avenue for improvement.
- LSE outperforms LSI, LDA, and word2vec for product ranking in lower-dimensional vector spaces.
- LSE constructs better product representations than the evaluated baselines, although improving the mapping from words to products could yield further gains.
- Adding LSE to language-model and popularity-based features significantly improves retrieval performance in a machine-learned ranking setting.
- The study focuses on an unsupervised setting where each product has an associated description and a set of reviews.
- Future work could use semi-supervised learning and more complex query projection functions to specialize retrieval for particular settings.
APPENDIX
The appendix derives parameter gradients for the model’s probability computation, covering entity representations, contrastive negatives, and derivatives with respect to biases and matrices.
- The derivation defines a target entity x+ and a sample of z contrastive negative examples X− for a single training instance.
- The model probability is expressed as P(S | e+, f(wj,1, . . . , wj,n)), linking an entity representation to a projected word n-gram.
- The update to an entity representation equals the projected input n-gram multiplied by a scalar, while the product-rule expansion for ∂p/∂θ is omitted for space.
- The appendix gives symbolic derivatives for the bias term, linear map W, and word representations Wv, alongside a general matrix-parameter derivative.