Source-linked AI summary
Mutual information and sensitivity analysis for feature selection in customer targeting: a comparative study
Nestor Barraza, Sergio Moro, Marcelo Ferreyra, Adolfo de la Peña
TL;DR
Feature selection must identify useful predictors while controlling irrelevant variables and computational complexity, but evidence directly comparing mutual information with data-based sensitivity analysis for customer targeting was limited. The paper compares both methods on bank-telemarketing data, builds logistic-regression models from their selected features, and finds a trade-off between false-positive regimes, feature count, and processing time. Mutual information is favored when avoiding missed successes matters and contact costs are low, while data-based sensitivity analysis delivers good results with fewer features.
Problem
Feature selection must identify influential predictors while discarding irrelevant variables, yet the advantages and disadvantages of mutual information and data-based sensitivity analysis had not been directly compared for customer targeting.
Method
The paper applies mutual information and data-based sensitivity analysis to a bank-telemarketing dataset, then evaluates logistic-regression models built from each method’s selected features.
Results
Data-based sensitivity analysis performs better at lower false-positive values, whereas mutual information is slightly better at higher false-positive ratios and selects 13 features versus 9 for data-based sensitivity analysis.
Takeaways & Limitations
Mutual information is preferable when call costs are low and selling income is high, while data-based sensitivity analysis achieves good prediction results with fewer features.
Takeaways & Limitations
The data-based sensitivity analysis estimate may not reveal dependencies among strongly correlated features, while mutual-information selection can discard variables when too few values provide reliable confidence.
Abstract
from arXiv · showhide
Feature selection is a highly relevant task in a data-driven knowledge discovery project. Several techniques have been developed aiming at finding the features that influence most an outcome to predict, including mutual information and, in recent years, the data-based sensitivity analysis. The present research focus on analyzing the advantages and disadvantages of each of these two techniques, by applying both to a bank telemarketing case. Thereafter, a logistic regression model is built on the tuned set of features identified by each of the two techniques as the most influencing set of features on the success of a telemarketing contact, in a total of 13 features for mutual information and 9 features for the data-based sensitivity analysis. The latter performs better for lower values of false positives while the former is slightly better for a higher false positive ratio. Thus, mutual information becomes a better choice if bank managers intend to reduce slightly the cost of contacts without risking losing a high number of successes. Such results show that mutual information, although not recent, is still a valid method for feature selection. On the other side, the data-based sensitivity analysis selection achieved good prediction results with less features.
1 Introduction
Customer targeting requires selecting informative features from large, complex datasets, yet the advantages and disadvantages of mutual information and sensitivity analysis had not been directly compared in this setting.
- Feature selection identifies influential predictors while discarding irrelevant variables that can mislead data-mining algorithms.More features also increase computational complexity through the growth of possibilities the algorithm must explore.
- Mutual information measures dependence between random features through the information one feature provides about another.It is linked to entropy, which represents the information content of a random feature.
- Sensitivity analysis evaluates feature relevance by measuring how model predictions change when each input feature varies.The paper treats sensitivity analysis as a wrapper method, although it can also be incorporated into model training.
- Earlier studies examined feature-selection methods and customer targeting, but none directly compared mutual information with sensitivity analysis on both methods’ advantages and disadvantages.The paper addresses this gap through a real customer-targeting case and literature-grounded evaluation.
- The study’s contributions include comparing the two methods on a real case, assessing their advantages and disadvantages, and drawing adoption insights for data-driven applications.These contributions are framed around customer targeting and broader real-world problems.
2 Background
The background introduces entropy and mutual information as information-theoretic measures, then relates mutual information to conditional entropy and redundant information in dependent variables.
- Entropy represents uncertainty or information content in a random variable and is bounded by the logarithm of its outcome-set cardinality.Entropy reaches its maximum under a uniform distribution, where events are less predictable.
- Using base-two logarithms measures information in bits.The logarithm base determines the unit of information.
- Joint entropy and conditional entropy quantify information in two variables together and the information remaining about one variable given the other.The section introduces these quantities before defining mutual information.
- Mutual information is the information shared by two random variables and equals entropy minus the corresponding conditional entropy.It can be written symmetrically as H(X) − H(X|Y) or H(Y) − H(Y|X).
- Mutual information extends to sets of random variables and can support removal of redundant information in dependent-variable structures.The paper connects this use to the data processing inequality for a Markov chain.
X1 X2 Y
The paper examines mutual information and data-based sensitivity analysis as feature-selection methods for customer targeting, clarifying their principles and comparative scope. It links these methods to reducing redundant information, assessing model sensitivity, and selecting features for customer-targeting models.
- Mutual information can eliminate redundant information through relationships such as the Markov chain X1 → X2 → Y.The data-processing inequality gives I(X2; Y) ≥ I(X1; Y) in this setting.
- Mutual information selects variables by maximizing information about the output and minimizing conditional entropy.Its feature-addition process continues until a stopping criterion is reached.
- Sensitivity analysis evaluates feature relevance by measuring how predicted outcomes respond to changes in input features.Data-based sensitivity analysis changes input features simultaneously using random samples, thereby considering feature relations.
- The paper addresses a gap by comparing mutual information and sensitivity analysis for feature selection in a real customer-targeting case.Prior work using data-based sensitivity analysis in bank telemarketing had not assessed its advantages and disadvantages against other feature-selection methods.
- Customer targeting involves selecting customers for marketing campaigns, while feature selection supports predictive modeling for that task.The paper positions customer targeting as a setting for testing predictive machine-learning techniques.
- Feature selection seeks a minimum feature set that preserves predictive performance while reducing modeling cost.Fewer features can make frequent model retraining more computationally feasible.
3 Materials and Methods
The study evaluates mutual information and data-based sensitivity analysis for selecting features in a bank-telemarketing prediction task. It uses real Portuguese bank contacts, method-specific feature relevance procedures, cross-validation, and ROC/confusion-matrix comparisons.
- 3.1 Real case: 41,188 phone contacts from a Portuguese bank dataset were used to predict whether customers subscribed to a long-term deposit.The contacts were conducted between 2008 and 2010, and each record contains personal, contact, and previous-campaign features.
- 3.2 Experimental procedure: Mutual information evaluates information contained in each feature, whereas data-based sensitivity analysis evaluates model influence by changing input features.Mutual information uses the dataset directly; sensitivity analysis requires a previously built model.
- 3.2 Experimental procedure: Data-based sensitivity analysis discarded features below 2% individual relevance and retained top-ranked features whose summed relevance reached at least 90%.These thresholds define the feature set used for the subsequent model computation.
- 3.2 Experimental procedure: A ten-fold cross-validation procedure successively divided the dataset into training and testing sets using a sliding-window simulation.The testing window represented 10% of the records and shifted to the next 10% without overlap.
- 3.2 Experimental procedure: Prediction performance was compared using receiver operating characteristic curves and confusion matrices, alongside computational performance.The computational comparison addressed whether a lighter method could support more frequent global learning procedures.
4 Results and Discussion
The experiments compare mutual information (MI) and data-based sensitivity analysis (DSA) for selecting features in bank-telemarketing prediction. MI removes redundant information and is somewhat better at higher false-positive levels, while DSA performs better at lower false-positive levels with fewer selected features.
- Experimental assumptions: The feature-selection comparison was conducted under a predictive setup that excluded features unavailable before contact execution, such as call duration.The paper also excludes campaign-current features because call timing depends on agent and client availability.
- Feature-selection procedures: MI selected features by estimating each attribute’s information content and identifying variables carrying the most information.The procedure calculates mutual information between each attribute and the output variable separately before selecting variables according to the stated criterion.
- Selected features: MI eliminated redundant information because “conf.price.idx” could be totally predicted by “cons.conf.idx” and “nr.employed”.The paper reports similar information-content analyses for the discarded variables “default” and “previous”.
- Feature-selection procedures: DSA selected features by model sensitivity, including features whose summed relevance reached at least 90% within each fold.The selected DSA features remained consistent across the ten folds.
- Predictive comparison: DSA performed better for smaller false-positive counts, whereas MI was slightly better for higher false-positive levels.The ROC curves were similar, with MI slightly worse because of its greater number of false positives.
- Discussion and limitations: The comparison indicates that DSA’s stable selections may overlook dependencies among strongly correlated features, whereas MI selections varied across folds when confidence was insufficient.This is presented as an interpretation of the different selection patterns rather than as a universal limitation of either method.
- Predictive comparison: MI produced confusion matrices with very good specificity but bad sensitivity, contacting many customers who would reject the offer while reaching many willing customers.The study therefore characterizes DSA as more conservative and MI as preferable when call costs are low relative to selling income.
5 Conclusions
The study compares mutual information (MI) and data-based sensitivity analysis (DSA) for feature selection in bank telemarketing. MI enabled a small subset of relevant features for faster, more accurate modeling, while DSA also supported logistic-regression modeling but has a stated disadvantage.
- The study compares mutual information (MI) and data-based sensitivity analysis (DSA) as feature selection methods.
- More than forty thousand bank telemarketing problem instances were used for the empirical procedure.
- MI eliminated redundant attributes and produced a small subset of highly relevant features for modeling the deposit-subscription outcome.
- MI enabled faster and more accurate modeling, and its information content could be obtained easily and rapidly for big data sets.
- Selecting variables carrying most output information allowed logistic regression to achieve good modeling results.