PaperlayerView exact revision on arXiv

Source-linked AI summary

A Vietnamese information retrieval system for product-price

Tien-Thanh Vu, Dat Quoc Nguyen

arXiv:1911.11623v1cs.IR

TL;DR

Product-price retrieval systems either depend on shop-provided data that must be continually updated or require seed product names and target sites for automatic crawling. This paper introduces a Vietnamese system that uses seed names to identify commercial sites and XPath patterns; with 334 initial names, it found 125 sites and collected 47.856 products in 30 hours.

  • Problem

    Vietnamese product-price IR approaches are limited, while shop-dependent systems require continual price updates and automatic crawlers require product names and target websites in advance.

    Source: §I · p. 1

  • Method

    The system starts with seed product names, automatically identifies commercial sites and XPath patterns, then crawls pages to collect and update product names and prices.

    Source: §I · p. 1, §III · p. 2

  • Results

    With 334 initial product names, the system determined 125 commercial sites and collected 47.856 products in 30 hours.

    Source: §V · p. 6

  • Takeaways & Limitations

    The authors conclude that the system can expand its site and XPath-pattern coverage as its seed-product database grows.

    Source: §V · p. 5

Abstract

from arXiv · show

A price information retrieval (IR) system allows users to search and view differences among prices of specific products. Building product-price driven IR system is a challenging and active research area. Approaches entirely depending products information provided by shops via interface environment encounter limitations of database. While automatic systems specifically require product names and commercial websites for their input. For both paradigms, approaches of building product-price IR system for Vietnamese are still very limited. In this paper, we introduce an automatic Vietnamese IR system for product-price by identifying and storing Xpath patterns to extract prices of products from commercial websites. Experiments of our system show promising results.

I. INTRODUCTION

Product-price IR systems either rely on shop-submitted data that must be continually updated, or automatically crawl websites but require product names and target sites in advance. This paper introduces a Vietnamese system that starts from a small set of product names and automatically identifies websites and XPath patterns for collecting and updating product data.

I. INTRODUCTION

  • Shop-submitted product-price databases require constant updating as prices change, while depending entirely on shops limits the database.Source: §I · p. 1
  • Automatic systems crawl commercial websites but require product names and target websites to be specified in advance.Source: §I · p. 1
  • Starting with a small set of seed product names, the proposed system identifies related commercial websites and XPath patterns, then collects and updates product names and prices.Source: §I · p. 1

II. RELATED WORKS

Prior shopping search engines commonly depend on manually maintained product information or merchant submissions, and the Vietnamese examples described here follow that approach. Related work also includes wrapper induction, template-based extraction, bootstrapping, and ontology-based product retrieval.

II. RELATED WORKS

  • The Vietnamese shopping search engines vatgia.com and aha.vn are described as using the shop-submission approach.Source: §II · p. 1
  • PriceScan and Google Product use manually updated databases, while Kelkoo and Yahoo! Shopping rely on merchants submitting products for manual classification.Source: §II · p. 1
  • Wrapper-induction methods learn extraction rules from manually identified target items in training webpages,,,.Source: §II · p. 2
  • Prior extraction work includes storing XPath templates for webpage text blocks and bootstrapping from annotations on 2–5 webpages across 4–6 websites.The bootstrapping method reports extraction accuracy of 83.8% on job-offer sites and 91.1% on vacation-rental sites.Source: §II · p. 2
  • Roadrunner and an ontology-based e-commerce product retrieval framework are also cited as related approaches.Source: §II · p. 2

III. OUR VIETNAMESE IR SYSTEM FOR PRODUCT-PRICE

The system has front-end and back-end components: the front end finds suitable websites and XPath patterns from seed product names, while the back end extracts product names and prices and updates product databases. Its related-webpage module sends templated product-and-currency queries to Google.

III. OUR VIETNAMESE IR SYSTEM FOR PRODUCT-PRICE

  • The front end uses seed product names to identify suitable websites and XPath patterns; the back end crawls those sites, extracts product names and prices, and updates product databases.Source: §III · p. 2

A. The front-end component of websites and Xpath patterns identification

1) Related webpages identification module:

  • The related-webpage module turns seed product names into search queries using templates, such as adding “vnđ or usd,” and submits them to Google.The example reports that all top five returned webpages were from commercial domains.Source: 1) Related webpages identification… · p. 3

2) Xpath patterns extraction module:

The XPath-pattern extraction module takes a product name and a related webpage, identifies the actual price and its XPath pattern, and uses the product-name pattern to select a corresponding price pattern. The method filters candidate numbers with textual and DOM-based rules, then compares their DOM proximity to the product name.

2) Xpath patterns extraction module:

  • The extraction module takes a product name and a related webpage, and outputs an actual price plus XPath patterns for extracting product names and actual prices.Source: 2) Xpath patterns extraction module · p. 3
  • For the Nokia 1200 example, the module returns “VNĐ 540.000” and XPath patterns for the product name and actual price.The same-site pattern reuse relies on webpages having similar structures.Source: 2) Xpath patterns extraction module · p. 3
  • The module finds a leaf DOM node containing the input product name and generates its XPath by tracing the path from the DOM root.Source: 2) Xpath patterns extraction module · p. 3
  • Candidate price strings are found by detecting numbers, then applying prefix, suffix, and exclusion rules.Exclusions include old or market prices, savings amounts, and values inside strike-through tags.Source: 2) Xpath patterns extraction module · p. 3
  • The system selects a candidate price whose DOM path shares the most steps with the product-name path, reflecting their expected proximity in commercial pages.Source: 2) Xpath patterns extraction module · pp. 3–4

3) Websites and corresponding patterns identification module:

The module identifies commercial websites and suitable Xpath patterns by counting webpages that share patterns found earlier. It accepts a website and its patterns when the count exceeds a threshold.

3) Websites and corresponding patterns identification module:

B. The back-end component of product-price information extraction

The back end crawls HTML documents from identified commercial websites, then applies the supplied Xpath patterns to extract product names and prices. Extracted information updates the product and seed-name databases.

B. The back-end component of product-price information extraction

IV. EXPERIMENTS

With 334 seed product names, the system collected 47,856 products, including 34,012 unique products, in 30 hours. The experiments also assessed Google-returned commercial webpages for randomly selected products.

IV. EXPERIMENTS

  • The system run began with 334 seed product names spanning multiple product types and ran for 30 hours.Source: §IV · p. 4
  • 47,856 products, including 34,012 unique products, were collected from 125 commercial websites.The run used 334 seed product names across product types and lasted 30 hours.Source: §IV · p. 4

A. Experiment of “Related webpages identification”

  • The related-webpage experiment evaluated the product-name-plus-currency query template using randomly selected products and counted commercial pages among the top 10, 30, and 100 Google results.The table is titled as reporting numbers of commercial webpages returned by Google.Source: A. Experiment of “Related webpages… · p. 4, Table I · p. 4

B. Experiment of actual price extraction in “Xpath patterns extraction” module

The section evaluates price-extraction accuracy with F-measure and defines precision and recall using extracted actual prices. The supplied table captions identify accuracy reporting for price extraction, product-name and price extraction, and commercial-website identification.

B. Experiment of actual price extraction in “Xpath patterns extraction” module

C. Experiment of “commercial websites identification”

The commercial-website identification component achieved 100% accuracy in both tested webpage settings. Information extraction performed well on one selected site, while category-specific Xpath structures constrained results on the other when seeds covered only mobile phones.

C. Experiment of “commercial websites identification”

  • 100% accuracy was reported for website identification using four seed products and a threshold of 3, for both top 10 and top 100 related webpages.The webpages were returned by Google in the system’s first module.Source: C. Experiment of “commercial websites… · p. 5

D. Experiment of “information extraction” module

  • The extraction evaluation selected two websites and their corresponding Xpath patterns from the front-end component’s output for the four-product seed set.The selected sites were dienthoaididong.com.vn and trananh.vn.Source: D. Experiment of “information… · p. 5
  • Extraction performed well on dienthoaididong.com.vn, but trananh.vn returned 416 mobile-phone products because the four seeds covered only that category.The authors state that seeds from all product categories could improve the result.Source: D. Experiment of “information… · p. 5

V. CONCLUSION

The proposed system identifies Vietnamese commercial sites and Xpath patterns from a small set of seed product names, then extracts product names and actual prices. Experiments collected 47.856 products from 125 sites in 30 hours using 334 initial names.

V. CONCLUSION

  • Extracted product names are added to the seed-products database, expanding it as the system runs.Source: §V · p. 5
  • The authors say the architecture can adapt to a new language by changing its rules.Source: §V · p. 5
  • The system uses a small set of seed product names to detect commercial sites and generate corresponding Xpath patterns.It then uses that information to extract names and actual prices from crawled products.Source: §V · pp. 5–6
  • With 334 initial product names, the system identified 125 commercial sites and collected 47.856 products in 30 hours.Source: §V · p. 6

Related papers

Loading 1911.11623v1…