Source-linked AI summary
CoPhIR: a Test Collection for Content-Based Image Retrieval
Paolo Bolettieri, Andrea Esuli, Fabrizio Falchi, Claudio Lucchese, Raffaele Perego, Tommaso Piccioli, Fausto Rabitti
TL;DR
CBIR research lacked a publicly available collection at the scale needed to evaluate scalable similarity search over Web-scale image data. The paper builds CoPhIR by crawling Flickr, extracting five MPEG-7 visual features with the EGEE GRID, and organizing the resulting descriptors and metadata into a large test collection. It surpassed 100 million processed images and was opened to the research community, with access requested by more than 50 institutions worldwide.
Problem
CBIR research lacked a publicly available image collection at Web scale, limiting experiments with scalable similarity searching and comparisons.
Method
The authors crawled Flickr images, extracted five MPEG-7 visual features using GRID computing, and stored descriptors with metadata and links to the original images.
Results
The project reached and passed 100 million processed images, producing a collection containing visual and textual information for 106 million images.
Takeaways & Limitations
CoPhIR is open for large-scale CBIR experiments and comparisons, with more than 50 research institutions already requesting access.
Takeaways & Limitations
The collection avoids permanently storing original Flickr files and instead maintains hyperlinks, so retrieved originals must remain accessible at the source.
Abstract
from arXiv · showhide
The scalability, as well as the effectiveness, of the different Content-based Image Retrieval (CBIR) approaches proposed in literature, is today an important research issue. Given the wealth of images on the Web, CBIR systems must in fact leap towards Web-scale datasets. In this paper, we report on our experience in building a test collection of 100 million images, with the corresponding descriptive features, to be used in experimenting new scalable techniques for similarity searching, and comparing their results. In the context of the SAPIR (Search on Audio-visual content using Peer-to-peer Information Retrieval) European project, we had to experiment our distributed similarity searching technology on a realistic data set. Therefore, since no large-scale collection was available for research purposes, we had to tackle the non-trivial process of image crawling and descriptive feature extraction (we used five MPEG-7 features) using the European EGEE computer GRID. The result of this effort is CoPhIR, the first CBIR test collection of such scale. CoPhIR is now open to the research community for experiments and comparisons, and access to the collection was already granted to more than 50 research groups worldwide.
1 Introduction
Web-scale image collections challenge CBIR because existing systems typically index far fewer images than the Web provides. The paper therefore targets a 100-million-image collection with descriptive visual features for scalable indexing and searching.
- Data growth: More than 80 billion photographs are taken annually, requiring an estimated 400 petabytes of storage.Much image data remains unstructured because it lacks meaningful associated tags.
- Multimedia search: Google handles over 200 million daily queries across more than 30 billion items, but its search power is typically limited to text and similarity.The paper argues that search must accommodate heterogeneous multimedia data because less than 1% of Web data is textual.
- CBIR scale gap: Typical CBIR databases contain thousands of images, while newer public systems index hundreds of thousands, far below current Web-scale image volumes.The authors investigate shifting these bounds upward by two orders of magnitude.
- Research objective: The SAPIR project sought a distributed architecture for indexing and searching image collections by visual content at the scale of tens of millions.A collection of that size with corresponding descriptive features was required to pursue this goal.
- Computational challenge: Processing 100 million images would require 30 TB to 50 TB of downloads and feature extraction taking about 12 years on a standard PC.A high-end multi-core PC was estimated to reduce the processing time to about 2 years.
2 Building the Image Collection
The collection was built by crawling Flickr to obtain a large, stable, metadata-rich image source, then distributing image crawling and MPEG-7 feature extraction across machines. The resulting CoPhIR collection exceeded 100 million images and was released for large-scale CBIR research.
- Building a 100 million-image collection required addressing source selection, downloading and storage, MPEG-7 metadata extraction, and reliable metadata access.
- Flickr was selected because photo-sharing sites provide relatively high-quality, stable images and rich metadata, while Flickr also offered an efficient content-access API.
- The collection crawl followed Flickr’s user relationships from an initial user, produced a snapshot of about one million users, and used photo IDs to retrieve their public photos.
- For each photo, the system retrieved extensive textual, social, geographic, usage, and camera metadata, alongside MPEG-7 visual descriptors.
- The distributed process used heterogeneous machines and parallel crawling and feature extraction; the largest aggregate contribution came from many machines that each processed relatively few images.
- The resulting CoPhIR collection passed 100 million images and contained visual and textual information for 106 million images, making it available for research outside SAPIR.
3 Conclusions
The paper addresses scalability in digital image management by constructing a large image collection with descriptive features. CoPhIR contains over 100 million images and is open for research experiments and comparisons.
- Scalable CBIR research requires non-trivial image collections with corresponding descriptive features.
- Over 100 million high-quality images were crawled, with five MPEG-7 features extracted using GRID technology.The features represent image colors, shape, and texture and are stored with metadata and links to the original images.
- More than 50 research institutions worldwide requested access to CoPhIR for experiments and comparisons.Access is granted under the CoPhIR Access Agreement, which defines collection-use conditions and terms.