Source-linked AI summary
Feature Engineering for Predictive Modeling using Reinforcement Learning
Udayan Khurana, Horst Samulowitz, Deepak Turaga
TL;DR
Feature engineering is important but lacks a well-defined, efficient basis and often depends on costly human trial-and-error. The paper uses reinforcement learning to guide performance-driven exploration of a transformation graph, reducing error by 24% by median across varied datasets within a relatively small computational budget.
Problem
Feature engineering lacks a well-defined basis and requires costly human-guided trial-and-error, while automated methods face expensive evaluation or limited feature-composition capabilities.
Method
The framework uses a transformation graph and reinforcement learning to learn a performance-driven exploration policy from feature-engineering examples under a limited budget.
Results
24% by median error reduction was achieved across a variety of datasets within a relatively small computational budget.
Takeaways & Limitations
The methodology can potentially save a data analyst hours to weeks of time while producing human-readable features.
Takeaways & Limitations
Exhaustively exploring the transformation graph is computationally infeasible because its size grows massively with composed transformations.
Abstract
from arXiv · showhide
Feature engineering is a crucial step in the process of predictive modeling. It involves the transformation of given feature space, typically using mathematical functions, with the objective of reducing the modeling error for a given target. However, there is no well-defined basis for performing effective feature engineering. It involves domain knowledge, intuition, and most of all, a lengthy process of trial and error. The human attention involved in overseeing this process significantly influences the cost of model generation. We present a new framework to automate feature engineering. It is based on performance driven exploration of a transformation graph, which systematically and compactly enumerates the space of given options. A highly efficient exploration strategy is derived through reinforcement learning on past examples.
Introduction
Feature engineering transforms data representations to improve predictive modeling, but it is typically guided by human intuition and trial-and-error. The paper proposes automating this process with reinforcement learning over a transformation graph.
- Appropriate feature transformations are often essential before constructing predictive models.Replacing a feature with its sine can make otherwise difficult class separation reasonably achievable for common classifiers.
- Feature engineering commonly relies on domain knowledge, intuition, and repeated observation of model performance.This makes the process time-consuming and prone to bias and error.
- Existing automated approaches can be computationally expensive or unable to discover complex features.These limitations make fully automating feature engineering difficult.
- The proposed approach trains a reinforcement-learning agent on feature-engineering examples to explore available transformations under a budget.Its exploration operates over a transformation graph representing relationships among transformed versions of the data.
- The framework produces interpretable features formed from compositions of well-defined mathematical functions.The paper illustrates this interpretability using feature-engineering results on a biking rental prediction dataset.
Related Work
Prior feature-engineering methods use heuristic search, explicit feature expansion, greedy evaluation, or learned transformation choices. Their limitations include surrogate objectives, scalability bottlenecks, expensive evaluations, and limited support for composing transformations.
- FICUS and FCTree guide feature searches with surrogate information-theoretic criteria rather than directly optimizing prediction performance.FCTree also generates only simple features and does not compose transformations.
- Expansion-reduction methods explicitly apply transformations to features, then perform feature selection and model optimization on the enlarged dataset.This creates performance and scalability bottlenecks and does not consider transformation compositions.
- FEADIS and ExploreKit rely on feature generation and selection procedures that require many or costly performance evaluations.ExploreKit reportedly required days on moderately sized datasets and does not consider transformation compositions.
- Cognito uses handcrafted traversal heuristics that do not adapt to budget constraints, while LFE predicts useful transformations independently for each feature.The paper positions its framework as a generalization of tree-like exploration concepts.
- Dimensionality-reduction, kernel, and deep-learning methods perform some feature engineering indirectly but have different scope and data requirements.The related work notes that deep learning requires massive datasets to avoid overfitting and is not suitable for many small or medium-sized problems.
Overview
Feature engineering transforms data representations to improve predictive modeling, but manual and existing automated approaches face costly, biased, or limited exploration. The paper frames feature engineering as selecting original and derived features to maximize model performance and proposes learning exploration decisions from prior examples.
- Challenges: Feature-engineering automation must balance unbounded compositional feature spaces against expensive model training and validation for evaluating candidates.Expansion-reduction methods create scalability bottlenecks and restrict the number of features considered.
- Approach: The proposed approach learns, through reinforcement learning on feature-engineering examples, whether to explore transformations or promising transformation combinations.The learned strategy maps transformation-graph states to actions that apply transformations to graph nodes.
- Problem formulation: The feature-engineering objective is to choose original and derived features that maximize modeling accuracy for a specified learning algorithm and performance measure.The selected feature set combines a subset of original features with derived features.
Transformation Graph
The transformation graph compactly represents datasets produced by transformation paths, with nodes corresponding to candidate feature-engineering solutions. Because exhaustive evaluation is infeasible, the framework ranks graph actions using reward estimates and learns a strategy for exploration.
- Graph structure: A transformation graph is a directed acyclic graph whose nodes represent the original dataset or datasets derived through transformation paths.All graph datasets share the same target and number of rows; edges encode transformations or feature-set unions.
- Graph structure: Hierarchical nodes represent single-transform descendants, while sum nodes represent datasets formed by combining two datasets.The graph also includes a root node for the given dataset and edges for transform or '+' operations.
- Graph structure: Each graph node is a candidate feature-engineering solution, and a complete graph contains a node solving the feature-engineering objective through transformations and feature selection.The operators θ(G) and θh(G) denote all nodes and hierarchical nodes, respectively.
- Search challenge: 3.2 million nodes result from 20 transformations at height 5, making exhaustive exploration prohibitively expensive because each trial implies model training and testing.The paper states that no known deterministic property verifies the optimum from a subset of trials.
- Exploration strategy: At each exploration step, estimated rewards rank available graph actions under a budget measured in steps or another monotonically increasing quantity.The reward function can support different exploration strategies, including handcrafted traversal rules.
- Exploration strategy: Reward estimation can use node accuracy, transformation rewards, reuse counts, accuracy gains, node depth, and remaining budget.Depth penalizes the relative complexity of the transformation sequence.
- Exploration strategy: Handcrafted breadth-first or depth-first strategies may work in specific circumstances, but the paper motivates machine learning for a unified strategy across circumstances.The framework therefore turns to learning the complex exploration strategy rather than relying only on manually designed traversal rules.
Traversal Policy Learning
The framework models transformation-graph exploration as a budget-aware Markov decision process and learns an action-selection policy through Q-learning with function approximation.
- Q-function learning: Because the state space is too large for direct learning, the method uses Q-learning with a linear approximation over state characteristics.The state features include graph, node, transformation, and remaining-budget characteristics.
- MDP formulation: The state combines the transformation graph after node additions with the remaining budget, while an action applies an unused transformation to an existing node.This formulation supports runtime exploration versus exploitation decisions for a given dataset.
- Reward design: Each action creates a new node and dataset, whose model accuracy supplies the immediate reward for exploration.The cumulative objective is defined over rewards obtained from the current state onward.
- Policy objective: The reinforcement-learning objective is to find an optimal policy that maximizes discounted cumulative reward.The discount factor γ ∈[0, 1) prioritizes early rewards over later rewards.
- Q-function learning: The action-dependent approximation learns separate weight vectors for candidate actions, whereas RL2 shares coefficients across actions to reduce learning complexity.The shared-coefficient variant reduces the coefficient space by a factor of c and was somewhat less efficient than RL1 in experiments.
Experiments
Experiments show that reinforcement-learning exploration improves feature-engineering efficiency and predictive performance across varied datasets, while search depth and feature selection affect outcomes.
- Performance comparison: 23.8% median error reduction was achieved across 24 datasets using the proposed feature-engineering routine.Performance was measured using relative absolute error for regression or 1−mean unweighted FScore for classification.
- Traversal policy comparison: RL-based exploration was 4-8 times more efficient than handcrafted policies across 10 datasets with six transformations and hmax = 4.The comparison included breadth-first, depth-first, and global handcrafted strategies.
- Internal system comparisons: Most datasets reached maximum accuracy at hmax = 4 or 5, while a tiny fraction deteriorated at hmax = 6 because of higher exploration cost.
- Internal system comparisons: Feature selection as a transformation improved final performance gain by about 51% across 48 datasets.
Conclusion and Future Work
The paper presents a reinforcement-learning framework that searches a transformation graph to automate feature engineering efficiently. It reports substantial median error reduction with a relatively small computational budget and identifies extensions for improving efficiency and scope.
- The framework combines a transformation graph with performance-driven reinforcement-learning exploration to find valuable features for supervised learning.
- The proposed technique reduced error by 24% at the median across varied datasets using a relatively small computational budget.
- Future work includes nonlinear state modeling, applications to imputation or model selection, and joint optimization of features with learning algorithms.