PaperlayerView exact revision on arXiv

Source-linked AI summary

Dataset search: a survey

Adriane Chapman, Elena Simperl, Laura Koesten, George Konstantinidis, Luis-Daniel Ibáñez-Gonzalez, Emilia Kacprzak, Paul Groth

arXiv:1901.00735v1cs.DB

TL;DR

Dataset search supports finding and using data, but current systems largely depend on metadata and face unresolved interaction, ranking, and evaluation challenges. This survey organizes dataset-search research into a framework, connects it to related fields, and identifies open problems and next steps.

  • Problem

    Dataset search matters for finding and using data, yet users’ understanding is limited by metadata quality, comprehensiveness, and nature.

  • Method

    The survey organizes dataset-search literature around the high-level steps needed to implement a search system and examines related research communities.

  • Results

    Dataset search remains in its infancy, with open needs including formal query languages, richer query support, interactive exploration, and benchmark-based evaluation.

  • Takeaways & Limitations

    Dataset search draws on databases, information retrieval, and semantic web search, while Google Dataset Search provides infrastructure for further research.

  • Takeaways & Limitations

    Current dataset searches mainly operate over metadata, whose low quality or absence affects dataset discovery and consumption.

Abstract

from arXiv · show

Generating value from data requires the ability to find, access and make sense of datasets. There are many efforts underway to encourage data sharing and reuse, from scientific publishers asking authors to submit data alongside manuscripts to data marketplaces, open data portals and data communities. Google recently beta released a search service for datasets, which allows users to discover data stored in various online repositories via keyword queries. These developments foreshadow an emerging research field around dataset search or retrieval that broadly encompasses frameworks, methods and tools that help match a user data need against a collection of datasets. Here, we survey the state of the art of research and commercial systems in dataset retrieval. We identify what makes dataset search a research field in its own right, with unique challenges and methods and highlight open problems. We look at approaches and implementations from related areas dataset search is drawing upon, including information retrieval, databases, entity-centric and tabular search in order to identify possible paths to resolve these open problems as well as immediate next steps that will take the field forward.

1 Introduction

Dataset search matters because data increasingly supports decisions and is increasingly shared, yet users struggle to find, trust, and use suitable datasets. This survey frames dataset search as a distinct field and reviews related approaches to identify open problems and next steps.

  • Data increasingly supports public policy, customer analysis, scientific experiments, and innovative services built from sensor-system integration.
  • Datasets are increasingly traded in data markets, shared through open-data portals and scientific repositories, and maintained by data communities.
  • Users face a disconnect between available datasets, their data needs, and datasets they can find, trust, and use.
  • Keyword search over published metadata is limited because metadata may omit task-relevant information and dataset-inappropriate filters.
  • The survey treats dataset search as a field with unique challenges, drawing on information retrieval, databases, semantic web, entity-centric, and tabular search.

2 Background

Dataset search covers discovering, exploring, and returning datasets, with basic search returning released datasets and constructive search helping build datasets. The process can be viewed as querying, query handling, data handling, and results presentation.

  • A dataset is a collection of related observations organized and formatted for a particular purpose.
  • Basic dataset search discovers and returns datasets released for consumption and reuse, including searches within repositories and across global services.
  • The Hurricane Sandy example contrasts an AAA dataset that is complete, clean, accurate, easy to use, and outdated with tweets that are timely but incomplete, dirty, and NLP-dependent.
  • Constructive dataset search builds or enriches datasets by combining sources to support needs such as understanding city flooding.
  • The generic search process comprises querying, query processing, data handling, and results presentation, with implementations drawn from several search communities.
  • Keyword or CQL queries and repository filters search dataset metadata, producing ranked results whose preview pages may include publisher, licensing, descriptions, raw-data samples, or visualizations.

2.2 Common Search Architectures

Dataset search can operate locally within a repository or globally across distributed resources. Query engines select relevant datasets and may aggregate them locally, process them distributively, or use federation.

  • Given a query and dataset sources, a query engine selects relevant datasets before choosing local aggregation, distributed processing, or federation.
  • Local dataset search operates within repositories such as scientific repositories, open-data portals, or organizational data lakes.
  • Global services crawl distributed resources and use schema.org or DCAT tags to structure dataset metadata for search.

2.3 Search sub-communities

Dataset search draws on databases, information retrieval, semantic web, entity-centric search, and tabular search, each contributing representations, retrieval methods, or data-completion tasks.

  • Semantic web and entity-centric search: Entity-centric search identifies named entities in queries and finds matching entities in linked data.
  • Semantic web and entity-centric search: Semantic web systems represent entities, properties, literals, and links in machine-understandable graphs, enabling cross-resource search without additional integration effort.
  • Information retrieval: Information retrieval uses statistical techniques to estimate query-word relevance, while vertical search engines tailor schemas and metadata to specific resources.
  • Databases: Database search typically parses a structured query, creates and optimizes an evaluation plan, then executes it using indexes and catalogues.
  • Tabular search: Tabular search treats a table as the query and includes attribute-name augmentation, attribute discovery, and filling missing values.
  • Tabular search: Table extension commonly discovers similar tables through schema similarity and uses their values to populate the input table.
  • Tabular search: Table completion can be viewed as entity-set completion and relates to imputation of incomplete data using external sources.

3 Current implementations

Current dataset-search systems primarily organize and retrieve datasets through published metadata, spanning enterprise catalogs, web search, open-data portals, domain-specific engines, and commercial data services. Their usefulness depends heavily on metadata quality and publisher maintenance.

  • Enterprise dataset catalogs cluster related datasets by structure or gathering frequency, reducing metadata-generation and schema-computation workload.
  • Google’s web dataset search crawls schema.org Dataset descriptions and ranks them with metadata-derived signals such as page authority.
  • Open-data portals search publisher-provided metadata, with CKAN using Solr/Lucene indexing and DCAT to support catalog interoperability.
  • Domain-specific systems curate dataset discovery with specialized metadata, such as DATS tags used by DataMed to index biomedical datasets.
  • Across current systems, search relies on publishers supplying accurate, correctly formatted, and maintained metadata descriptions.
  • Commercial data providers create value by gathering, organizing, and releasing constructed datasets from scattered public or proprietary sources.

4 Survey of Dataset Search Research

Research on dataset search addresses the full interaction pipeline, from query creation and handling to dataset processing, ranking, presentation, and user interaction. A recurring constraint is that metadata-centered search limits users’ ability to assess and understand datasets before use.

  • Creating queries: Dataset-search users invest more effort than document-search users, and dataset suitability depends more directly on the task being performed.
  • Query Types: Most systems use keyword or metadata-based CQL queries with platform-specific facets, because no formal query language for dataset retrieval yet exists.
  • Query Handling: Low-quality or missing metadata affects both dataset discovery and consumption, making search success dependent on publishers’ knowledge and descriptions.
  • Query Handling: Research extends search beyond metadata through column-aware query mapping, keyword search over columns, semantic search, and keyword queries over whole tables.
  • Data Handling: Additional data handling includes metadata-quality assessment, entity-based cross-validation, table extension through similarity ranking, and generated summaries or annotations.
  • Ranking Datasets: Dataset-ranking research uses table-query ranking, unsupervised topic discovery, and methods specialized for datasets containing continuous information.
  • Results Presentation: Because users often cannot specify exact queries without understanding the data, prototypes support exploration through visualizations, result summarization, navigation, and richer interactions.
  • Interactions: Human-data-interaction studies show that metadata reliance and tool support vary across dataset-search users, alongside social factors affecting search.

5 Open problems

The survey organizes dataset-search research around implementation steps and identifies discoverability and reuse barriers that require further attention. These include missing information, unclear ownership or reuse terms, difficult formats, licensing constraints, and exclusive agreements.

  • The survey organizes prior work into a framework reflecting the high-level steps needed to implement dataset-search systems.
  • Open-data reuse is hindered when users lack information that data exists or clarity about which public authority holds it.
  • Unclear reuse terms and difficult or expensive formats create additional barriers to using available data.
  • Complicated licensing procedures, prohibitive fees, and exclusive or government-restricted reuse agreements constrain data reuse.
  • The survey identifies further dataset-search problems beyond the recognized barriers to open-data discoverability and reuse.

5.1 Query languages: moving beyond keywords

Dataset search needs query mechanisms that move beyond keywords and metadata toward richer representations of dataset content, entities, and tables. Related work suggests structured, programming-oriented, entity-centric, and table-as-query approaches.

  • Existing systems mainly express information needs through keyword queries or metadata filters such as domain, format, and publisher.
  • Searching across multiple domains requires identifying relevant tables, joining them appropriately, and performing subsequent query processing.
  • Research has focused primarily on SQL and SPARQL over metadata, motivating richer query languages and indexing over dataset content.
  • CSV-on-the-Web semantics can describe table columns and values, but typically require manually specified mappings.
  • Entity-centric search could query individual entities, entity sets, or entity types while comparing similarities within datasets rather than only matching entities.
  • Potential directions include native structured dataset-query languages and programming-language concepts applied to databases or MapReduce frameworks.
  • Tabular search treats an input table as the query, enabling interfaces that assist users while creating spreadsheets.

5.2 Query handling: Differentiated access

Differentiated dataset access must support searching across organizational boundaries while respecting legal, privacy, pricing, and security constraints. Related database techniques offer starting points, but dataset search still lacks equivalent query-processing foundations.

  • Differentiated access: Data trusts require search across datasets shared between organizations, including differentiated access to pooled information.Such access is relevant to building cohorts from multiple clinical studies.
  • Differentiated access: Searching hidden datasets must account for differing legal, privacy, and pricing properties to avoid preventing access to much of the available data.Provenance may help inform query-time access decisions.
  • Security: Secure dataset search remains an implementation challenge, including searches over encrypted datasets and digest-based disclosure minimization.Reuse demands may also change query-processing requirements and bottlenecks.
  • Query processing: Dataset retrieval needs targeted approaches beyond general web-search concepts because dataset properties create a distinct search setting.The paper frames dataset retrieval as a unique vertical.
  • Query processing: Dataset search has no established equivalent of relational algebra, although database query-processing ideas and non-relational operators provide possible foundations.Approximate query processing, precomputation, provenance-aware ranking, and data integration are identified as related directions.

5.3 Data handling: extra knowledge

Dataset search engines need richer ingestion, indexing, and cataloging that combine external knowledge with dataset-intrinsic information. Key challenges include automatic metadata and quality understanding, scalable profiling, provenance across granularities, and heterogeneous web resources.

  • Data handling: Advanced dataset search requires better ingestion, indexing, and cataloging through external knowledge and dataset-intrinsic information.These two areas support differentiated access and exploratory interfaces.
  • Extra knowledge: External ontologies, quality indicators, and descriptive papers can make natural-language codebooks usable by supplying information about the codebooks themselves.Without that integration, datasets described through codebooks may be nearly useless.
  • Dataset-intrinsic information: Dataset-intrinsic information should support richer search criteria, including on-the-fly quality assessment, quality indexes, and automatically maintained metadata.The paper identifies metadata identification, creation, maintenance, and ontology linking as open problems.
  • Architectures: Dataset search architectures must determine how to combine information-retrieval systems with lessons from database query engines.The paper identifies effective system architecture as an important open problem.
  • Representation: Converting datasets to RDF can support Linked Data search, but the full cycle may be expensive and original formats may better suit some tools.Geospatial datasets are given as an example where retaining the original format may be preferable.
  • Understanding datasets: Data profiling, outlier detection, summarization, and provenance can help users assess dataset scope, variety, contents, and fitness for use.Provenance remains difficult to aggregate across tuple, table, and dataset granularities, while web and deep-web search must identify dataset resources among other documents.

5.4 Result presentation: interactivity

Result presentation must move beyond ranked lists and facets toward interactive dataset exploration that accommodates exploratory workflows and helps users interpret provenance and result spaces. Dataset search also raises unresolved questions about ranking, entity integration, and constructive query results.

  • Ranking: Current dataset search commonly presents ranked results with faceted search, but ranking should use richer dataset-specific indexes.Learning-to-rank methods are difficult when engines lack sufficient user traffic, and entity-search integration remains open.
  • Interactivity: Conversational search and embedding search within data-use workflows deserve attention because dataset search supports varied tasks.The paper treats dataset retrieval as part of broader data-use processes rather than only standalone lookup.
  • Ranking: Standard web-search ranking is not directly applicable because tables combine structural and related content, lack hyperlink anchor text, and have two-dimensional organization.Tables of widely varying quality may also appear on a single web page.
  • Exploratory search: Dataset search is exploratory, involving repeated queries, information-need refinement, and diverse interaction styles shaped by context, domain, and workflow uncertainty.These characteristics motivate interfaces beyond a single ranked-result interaction.
  • Provenance: Provenance can support dataset choice, but methods are needed to translate provenance graphs into forms users can interpret and use.The paper connects provenance with trust and automated quality estimation while identifying granularity as a challenge.
  • Exploration: Result overviews can improve orientation, while column-, row-, and cell-level inspection may be necessary to match process- and content-oriented requirements.Constructive search also requires appropriate annotation and citation of query results.

6 The Road Forward: Benchmarks

Dataset search lacks benchmarks that can evaluate solutions across its pipeline and account for relationships among users, tasks, datasets, and metadata. Progress requires dataset-specific metrics and result formats that can represent tasks needing multiple relevant datasets.

  • Benchmark gap: The lack of benchmarks is one of dataset search’s most widely recognized problems.Existing efforts such as Bio-CADDIE acknowledge multiple concerns but do not yet establish whether solutions work.
  • Metrics: Evaluation must identify metrics appropriate to dataset search, adapting information-retrieval measures and considering online measures such as session success and zero-result rates.Click-through rate may require adjustment for dataset-search contexts.
  • Metrics: Completeness may be a dataset-specific metric because users often need several datasets stitched together into a fit-for-purpose whole.Benchmarks must assess whether the returned set is appropriate for the task.
  • Result formats: Some queries require multiple equally relevant datasets, making a single-resource-per-rank result list insufficient and leaving the best presentation unresolved.This creates a specific challenge for benchmark design.
  • Benchmark design: Useful benchmarks must test the dataset-search pipeline while representing relationships among the user, task, dataset properties, and metadata.Dataset search therefore requires dimensions beyond finding appropriate information in classic web retrieval.
  • Existing benchmarks: Existing benchmarks cover RDF dataset management, Wikipedia-table retrieval, and semantic labeling of web tables, but further efforts are needed.These benchmarks address related tasks rather than the full underlying dataset-search technology.

7 Conclusions

Dataset search is an emerging field because datasets have properties distinct from documents, tuples, and webpages, while dedicated search remains in its infancy. The paper identifies multiple search mechanisms and open research needs for advancing dataset retrieval.

  • Dataset search treats datasets as entities with properties shared with, but also distinct from, documents, tuples, and webpages.
  • Two mechanisms frame dataset search: issuing a query to return a dataset or issuing a query to build a dataset.
  • Dataset search remains in its infancy despite techniques available from databases, information retrieval, and semantic web search.
  • Open research needs include formal query languages, handling social and organizational restrictions, supporting query processing, and facilitating user exploration of dataset results.
  • Further research is needed to fully understand and support dataset search as demand for datasets and search infrastructure continues to grow.
Loading 1901.00735v1…