Source-linked AI summary
Challenges of Feature Selection for Big Data Analytics
Jundong Li, Huan Liu
TL;DR
The paper examines how big-data characteristics challenge traditional feature selection across data types and performance requirements. It organizes these challenges and presents scikit-feature, an open-source repository intended to facilitate algorithm application, comparison, joint study, and more reliable evaluation.
Problem
Big-data scale, structure, linkage, multiple sources or views, streaming behavior, scalability, and stability create challenges for traditional feature selection.
Method
The paper surveys feature-selection challenges across structured, linked, multi-source, multi-view, streaming, scalability, and stability settings, and introduces the scikit-feature repository.
Results
scikit-feature collects widely used feature-selection algorithms and provides benchmark datasets, evaluation examples, and an interactive FeatureMiner tool.
Takeaways & Limitations
The repository supports feature-selection application, comparison, joint study, and more reliable evaluation of new algorithms.
Takeaways & Limitations
Streaming feature selection remains challenging when data cannot be loaded into memory and only one pass is available.
Abstract
from arXiv · showhide
We are surrounded by huge amounts of large-scale high dimensional data. It is desirable to reduce the dimensionality of data for many learning tasks due to the curse of dimensionality. Feature selection has shown its effectiveness in many applications by building simpler and more comprehensive model, improving learning performance, and preparing clean, understandable data. Recently, some unique characteristics of big data such as data velocity and data variety present challenges to the feature selection problem. In this paper, we envision these challenges of feature selection for big data analytics. In particular, we first give a brief introduction about feature selection and then detail the challenges of feature selection for structured, heterogeneous and streaming data as well as its scalability and stability issues. At last, to facilitate and promote the feature selection research, we present an open-source feature selection repository (scikit-feature), which consists of most of current popular feature selection algorithms.
1 A Brief Introduction of Feature Selection
Feature selection reduces high-dimensional data by selecting relevant original features, helping address dimensionality challenges while preserving interpretability. The paper distinguishes methods by label availability and search strategy, with trade-offs among efficiency, task specificity, and learning performance.
- Motivation: High-dimensional data are growing across domains, intensifying the curse of dimensionality and computational and memory demands.The paper illustrates growth in both samples and features using the UCI machine learning repository.
- Feature selection: Feature selection directly selects a subset of relevant original features for model construction, preserving physical meaning and improving readability and interpretability.It is used in applications including gene analysis and text mining.
- Feature roles: Relevant features separate classes, redundant features add information already provided by another feature, and irrelevant features lack useful class-separating information.Figure 2 illustrates these roles with f1, f2, and f3, respectively.
- Label availability: Supervised feature selection uses known class or regression labels to assess relevance, whereas unsupervised selection operates without labels for clustering.Supervised relevance is normally assessed through correlation with class labels.
- Search strategies: Wrapper methods evaluate subsets through a predefined learner but are slow, while filter methods rank features efficiently without a learner and may be suboptimal for a particular task.The passage also introduces embedded methods but does not provide their description here.
2 Challenges of Feature Selection
Big-data feature selection must address structured and relational features, heterogeneous sources and views, streaming constraints, scalability, and stability. These characteristics challenge methods designed for generic, static data.
- Big-data feature selection challenges arise from structured features, linked data, multi-source and multi-view data, streaming data, scalability, and stability.The paper presents these as six challenge areas for big-data analytics.
- Structured Features: Generic feature-selection methods often assume features have no explicit correlations, although real applications contain groups, trees, graphs, and other structures.Ignoring these structures can make selections insensitive to how features are organized.
- Linked Data: Linked data adds relations among instances, requiring methods to exploit links, use them for selection, and assess feature relevance when labels are unavailable.Links can be represented through an adjacency matrix in addition to conventional attribute-value features.
- Multi-Source Data and Multi-View Data: Multi-source selection uses complementary sources around a target source, whereas multi-view selection selects features simultaneously across dependent feature spaces.Multi-view examples include pixel, tag, and text features describing the same images.
- Streaming Data and Features: Streaming data may be too large, too fast, or unknown in extent for batch selection, making timely one-pass selection a continuing challenge.Streaming feature selection is also motivated by rapidly changing features such as newly popular slang words.
- Scalability and Stability: Terabyte-scale datasets may not fit directly in memory, while feature-selection stability can vary with data characteristics and remains less studied for unsupervised methods.Distributed frameworks such as MapReduce and MPI are cited as attempts to address scale.
3 Feature Selection Repository
The paper introduces scikit-feature as an open-source repository for applying, comparing, and jointly studying feature-selection algorithms. It also organizes popular methods and supplies datasets and evaluation examples.
- Scikit-feature collects widely used feature-selection algorithms to facilitate their application, comparison, and joint study.The repository is intended to support more reliable evaluation when developing new algorithms.
- The repository includes similarity-based, information-theoretical, statistical, sparse-learning, wrapper, structural, and streaming feature-selection categories.These categories cover conventional and big-data-oriented feature-selection settings.
- Structural feature selection and streaming feature selection are represented explicitly among the repository’s method categories.The repository organizes these alongside similarity-based, information-theoretical, statistical, sparse-learning, and wrapper methods.
- Scikit-feature provides benchmark datasets and evaluation examples using classification or clustering tasks, with applicable algorithms and results listed for each dataset.The project website provides the associated datasets and experimental results.